Hallucination Energy on AIBussin — AI applications, systems and books

How Do You Measure a Hallucination?

Sat, 29 Aug 2026 23:14:00 +0100

The first three chapters deliberately avoided building a detector.

Before designing one, we needed to define exactly what it would be expected to detect.

Chapter 1 established that fluent generation can continue after evidential support has weakened or disappeared.

Chapter 2 showed that hallucination covers several different failure relationships.

Chapter 3 then separated the objects a reliability system must not collapse:

truth
≠
evidence
≠
support
≠
attribution
≠
provenance
≠
verification
≠
policy acceptance

Now we can finally ask the engineering question:

Hallucination Energy

Sat, 29 Aug 2026 23:14:00 +0100

Chapter 4 ended with a rule:

A hallucination measurement is a sensor, not a verdict.

This chapter builds one such sensor completely.

It begins with a property we care about:

semantic containment

But the first lesson of Chapter 4 was the proxy gap. We cannot inspect semantic containment directly. We can inspect representations.

So the construction in this chapter has three layers:

    graph TD
    LP[LATENT PROPERTY: semantic containment] --> RA[representation assumption]
    RA --> PP[PROXY PROPERTY: embedding-subspace containment]
    PP --> GC[exact geometric calculation]
    GC --> M[MEASUREMENT: squared projection residual]

That distinction is the chapter’s most important caveat.

How to Evaluate a Hallucination Detector

Sat, 29 Aug 2026 23:14:00 +0100

Chapter 5 gave us something concrete enough to evaluate.

Hallucination Energy has:

a target property
an observable proxy
a mathematical definition
an implementation
continuous outputs
natural-task signal
adversarial failures

That sounds like progress.

It is also the point where it becomes dangerously easy to fool ourselves.

Suppose a detector reports:

AUC = 0.75

Is that good?

Perhaps.

But we still do not know:

which class was called positive
how hard the negatives were
whether train and test shared source documents
whether they shared entities or semantic neighborhoods
what prevalence the deployment actually has
whether the labels were trustworthy
how wide the confidence interval was
what happens at 1% false acceptance
how uncertain that 1% estimate is
whether the threshold transfers to another domain
whether the score is a probability at all
whether the verifier itself failed upstream
whether the generator changed
whether decoding parameters changed
whether the model collapses on structural adversaries

A single number cannot answer those questions.

Breaking the Detector

Sun, 30 Aug 2026 00:17:00 +0100

Chapter 6 changed the standard for evaluation.

A detector is not trustworthy because it has a good AUC.

It becomes trustworthy only after we understand how its errors behave under the conditions in which policy will use it.

That immediately suggests a more aggressive question.

Instead of waiting for those conditions to appear naturally, can we search for them deliberately?

Suppose a containment detector works well on ordinary unsupported claims.

Containment Is Not Truth

Sun, 30 Aug 2026 09:10:00 +0100

Chapter 7 deliberately broke the detector.

The most useful failures had a peculiar shape. The unsupported claim did not introduce a completely new topic. It did not necessarily introduce a new entity. It did not always wander outside the semantic region represented by the evidence.

Instead, the attack often preserved almost everything that a broad semantic representation could reasonably notice:

same entities
same event vocabulary
same topic
same evidence neighborhood
same quantities or dates, sometimes
same semantic ingredients

while changing the thing that made the proposition correct:

Beyond Hallucination: Consistency and Sensitivity

Sun, 30 Aug 2026 10:16:00 +0100

Chapter 8 ended with a rule:

If a downstream decision depends on a distinction, do not discard that distinction before the decision is made.

That rule breaks the idea of one universal hallucination score.

A response can be well contained and still reverse a relation. It can preserve every relation and still ignore the decisive facts of the problem. It can be correct once and unstable under a harmless rephrasing. It can be perfectly repeatable and consistently wrong.

Appendix: The Evidence Ledger

Mon, 31 Aug 2026 09:00:00 +0100

The book argues that a claim should be governed by its provenance, its evidential support, and an explicit statement of what it does not establish. This appendix applies that discipline to the book itself.

Every original empirical number produced for this book is listed here with three things: where it was measured, what its provenance status is, and what it does not establish.

Two facts are visible immediately from the ledger:

Warranted Search: When AI Must Prove Before It Looks

Tue, 02 Jun 2026 18:54:26 +0100

TL;DR

Modern AI systems often retrieve nearby text and generate confident answers, but that is not the same as proof. A citation can be real, the answer can be correct, and the evidence can still fail to support the claim.

This post argues for warranted search: a scoped, claim-driven form of intelligent grep. Instead of asking an AI to rummage through a corpus, we give it a specific claim, a bounded search warrant, and a limited set of safe operations.

Beyond Hallucination Energy: A Three-Dimensional Framework for Reliable AI Outputs

Wed, 22 Apr 2026 10:35:46 +0100

🧩 1. TLDR

AI doesn’t just hallucinate. Sometimes it gives answers that are fluent, safe… and completely useless.

Most discussions about AI failure focus on hallucination:

making things up
getting facts wrong
fabricating sources

That’s real. It matters.

But it’s not the most dangerous failure mode in production systems.

There is a quieter one.

A more subtle one.

And in practice a more pervasive one.

AI systems often fail not by being wrong, but by failing to think at all.

Hallucination Energy: A Geometric Foundation for Policy-Bounded AI

Thu, 05 Feb 2026 13:29:55 +0000

🚀 Summary

This post presents the current research draft and implementation of a geometric framework for bounding stochastic language models through deterministic policy enforcement.

The central contribution is a scalar metric termed Hallucination Energy, defined as the projection residual between a claim embedding and the subspace spanned by its supporting evidence embeddings. This metric operationalizes grounding as a measurable geometric quantity.

We proceed in three stages:

Formal Definition a draft manuscript introducing Hallucination Energy, its mathematical formulation, and its role within a policy-controlled architecture.
Empirical Evaluation structured calibration and adversarial stress testing across multiple domains to assess the robustness and limits of projection-based grounding.
Applied Validation large-scale evaluation on 10,000 samples from the HaluEval summarization benchmark, demonstrating that projection-based containment functions as a strong first-order grounding signal in a real generative setting.

This work does not claim to solve hallucination. Rather, it characterizes the boundary of projection-based grounding, establishes its suitability as a deterministic policy scalar, and documents both its strengths and its structural limitations.

From Evidence to Verifiability: Rebuilding Trust in AI Outputs 🔏

Tue, 03 Feb 2026 12:25:58 +0000

⏰ TLDR

This work shows that the hardest part of using AI in high-trust environments is not the model, but the policy. Once editorial policy is made explicit and executable, AI systems become interchangeable the real challenge is engineering reliable measurements and deterministic enforcement of those policies.

📋 Summary

AI systems are becoming deeply embedded in how we research, write, and reason. At the same time, their use in high-trust environments is under strain not because models are incapable, but because they are being deployed into settings that demand determinism, provenance, and enforceable rules.