Self Consistency on AIBussin — AI applications, systems and books

Candidate Generation and Selection

Sat, 08 Aug 2026 15:54:00 +0100

The previous chapter built a boundary that stops arbitrary model output from acquiring execution authority without explicit checks. It made one class of failure inspectable and enforceable, and it is silent about another.

Suppose the model is asked to solve a coding problem. One run produces the right patch. The next produces a plausible but incomplete one. A third produces something better again. Nothing is malformed, nothing violates the action schema, and every one of them would pass the boundary we just built. Validity and quality are different properties. A proposal can be completely valid and still be a poor choice, which relocates the uncertainty rather than removing it:

How Do You Measure a Hallucination?

Sat, 29 Aug 2026 23:14:00 +0100

The first three chapters deliberately avoided building a detector.

Before designing one, we needed to define exactly what it would be expected to detect.

Chapter 1 established that fluent generation can continue after evidential support has weakened or disappeared.

Chapter 2 showed that hallucination covers several different failure relationships.

Chapter 3 then separated the objects a reliability system must not collapse:

truth
≠
evidence
≠
support
≠
attribution
≠
provenance
≠
verification
≠
policy acceptance

Now we can finally ask the engineering question:

Reasoning Is More Than Architecture — Where Extra Computation Lives

Tue, 01 Sep 2026 10:41:00 +0100

Reasoning Is More Than Architecture — Where Extra Computation Lives

So far in Models From First Principles, we have changed several different things and called all of them model design.

We changed what a model predicts.

MR.Q produced one learned quality score.

EBT added Q, V, policy, and advantage.

SICQL turned those outputs into explicit model components.

Then we changed how computation unfolds.

HRM introduced recurrent state operating at different timescales.

Advanced Agents From First Principles 02: Why Does My Reasoning Agent Give a Different Answer Every Time? Use Self-Consistency Without Confusing Consensus With Truth

Sat, 08 Aug 2026 22:44:00 +0100

A reasoning agent gives you one answer.

You run it again.

It gives you another.

You change nothing important:

same task,
same tools,
same model family,
same broad context.

Yet the result changes.

That is not necessarily a bug.

A probabilistic model is allowed to produce more than one plausible trajectory.

The engineering question is different:

How should an agent system use that variation?

One common answer is self-consistency.

Generate several independent reasoning trajectories.

Agents From First Principles 02: AI Agent Gives Inconsistent Answers? Generate Multiple Candidates and Rank Them

Sat, 08 Aug 2026 15:54:00 +0100

One of the first things you notice when you build anything around a large language model is that the same prompt does not always produce the same quality of answer.

Sometimes the first response is excellent.

Sometimes it is merely acceptable.

Sometimes it misses the point entirely.

That creates a very common agent-engineering question:

If the model is inconsistent, should the agent trust the first answer it gets?

Often, no.