Agentic AI on AIBussin — AI applications, systems and books

Evidence and Verification

Sat, 08 Aug 2026 17:31:00 +0100

The agent says:

Done.

That is a claim.

It is not evidence.

Every mechanism in this book so far has made the agent better at deciding what to do, and none of them establishes that the user’s goal was achieved. A planner can produce a coherent plan for the wrong problem. A tool can return exit code zero without producing the intended effect. A search can select the highest-scoring branch when every branch is wrong. A memory system can retrieve a perfectly relevant fact that stopped being true in March.

Building the Complete Agent

Wed, 26 Aug 2026 10:00:00 +0100

Every mechanism in this book was argued against a problem chosen to isolate it.

That isolation was deliberate, and it was also a form of protection. The memory chapter picked a task where recall was the bottleneck, held everything else still, and measured the one thing it came to measure. The result is a clean explanation and a weak claim. Nothing in it establishes that the same retrieval policy behaves when a search controller is expanding forty nodes, or when a verifier insists that every piece of evidence carry a state identity the search controller has never heard of.

Advanced Agents From First Principles 19: Can Your Agent Explore in Parallel Without Creating Chaos? Use Speculative Execution and Early Cancellation

Sun, 09 Aug 2026 11:10:00 +0100

Advanced Agents From First Principles 19: Can Your Agent Explore in Parallel Without Creating Chaos? Use Speculative Execution and Early Cancellation

A production agent often has more than one useful thing it could do next.

It could:

inspect repository state,
run a targeted test,
retrieve documentation,
ask a second model to critique a candidate,
generate an alternative implementation,
probe an API,
inspect a deployment,
or verify an invariant.

If those actions are independent, executing them one by one can be needlessly slow.

Advanced Agents From First Principles 06: Does One Agent Plan, Execute and Judge Its Own Work? Build a Planner-Executor-Critic Architecture

Sat, 08 Aug 2026 23:49:00 +0100

Does One Agent Plan, Execute and Judge Its Own Work? Build a Planner-Executor-Critic Architecture

A single model can often do all of these things:

understand a task,
decide what to do,
execute a tool call,
inspect the result,
critique its own work,
decide whether it succeeded,
and produce the final answer.

That is convenient.

It is also a dangerous concentration of responsibilities.

If the same component creates the plan, executes it, explains why the result is good, and decides whether the job is complete, then failures become difficult to localize.

Advanced Agents From First Principles 05: Is One Model Doing Everything? Build a Mixture of Experts at the Agent Level

Sat, 08 Aug 2026 23:41:00 +0100

A common agent architecture starts simply:

request
 ↓
model
 ↓
action

That simplicity is valuable.

It should be your default.

But eventually you may notice something strange.

The same model is being asked to do everything:

classify the task,
search documentation,
reason about code,
write SQL,
review a patch,
summarize logs,
judge another model,
decide whether a deployment is safe,
and answer simple questions that did not require an expensive model in the first place.

At that point the problem may no longer be:

Advanced Agents From First Principles 04: Does Your Agent Prune Good Ideas Too Early? Use Monte Carlo Tree Search for Long-Horizon Reasoning

Sat, 08 Aug 2026 23:37:00 +0100

A common failure in search-based agents is easy to miss.

The agent generates several plausible branches.

It scores them.

One branch looks weak.

So the runtime prunes it.

Later, you discover that the discarded branch was the only one that could have reached the correct solution.

The problem was not generation.

The problem was not necessarily the model.

The problem was search allocation.

The agent spent too much compute exploiting what looked good early and too little compute exploring alternatives whose value only became visible later.

Advanced Agents From First Principles 03: Does Your Agent Commit to a Bad Reasoning Path Too Early? Build a Tree of Thoughts

Sat, 08 Aug 2026 23:25:00 +0100

A reasoning agent can fail even when every individual step looks plausible.

The problem is often not that the model cannot produce a good line of reasoning.

The problem is that it commits too early.

It chooses one interpretation, one hypothesis, one plan, or one next step and then spends the rest of the run trying to make that decision work.

That gives us a common failure pattern:

problem
 ↓
first plausible thought
 ↓
second thought conditioned on the first
 ↓
third thought conditioned on both
 ↓
...
 ↓
confident answer built on an early mistake

If the first branch was wrong, every later step inherits the error.

Advanced Agents From First Principles 02: Why Does My Reasoning Agent Give a Different Answer Every Time? Use Self-Consistency Without Confusing Consensus With Truth

Sat, 08 Aug 2026 22:44:00 +0100

A reasoning agent gives you one answer.

You run it again.

It gives you another.

You change nothing important:

same task,
same tools,
same model family,
same broad context.

Yet the result changes.

That is not necessarily a bug.

A probabilistic model is allowed to produce more than one plausible trajectory.

The engineering question is different:

How should an agent system use that variation?

One common answer is self-consistency.

Generate several independent reasoning trajectories.

Advanced Agents From First Principles 01: Does Your AI Agent Fail on Complex Reasoning Tasks? Treat Chain of Thought as Computation, Not Proof

Sat, 08 Aug 2026 22:35:00 +0100

Most developers first encounter chain of thought as a prompting trick:

Think step by step.

That framing is too shallow for agent engineering.

For an advanced agent, the useful idea is not that the model should produce a long explanation. The useful idea is that a difficult task may benefit from intermediate computational state before the system commits to an action or answer.

That is a very different claim.

A reasoning trace can help a system decompose a problem, preserve intermediate conclusions, identify missing information, decide what to verify next, and expose places where search or tools should be used.

Advanced Agents From First Principles 00: When Should You Use an Advanced Agent Architecture?

Sat, 08 Aug 2026 22:27:00 +0100

Advanced Agents From First Principles 00: When Should You Use an Advanced Agent Architecture?

You built an agent.

It can:

call tools,
maintain state,
plan,
revise its own work,
search over alternatives,
remember useful information,
and verify whether the requested outcome actually happened.

Now the temptation begins.

You add another model.

Then a critic.

Then a planner.

Then a judge.

Then a router.

Then three specialist agents.

Then a tree search.

Agents From First Principles 09: AI Agent Says It Worked When It Didn’t? Verify the Result Outside the LLM

Sat, 08 Aug 2026 17:31:00 +0100

An AI agent says:

Done. The task is complete.

That sentence is almost worthless.

The agent may have:

edited the wrong file,
changed the right file incorrectly,
skipped part of the request,
broken another subsystem,
failed to save its work,
misread a tool result,
passed a stale test,
inspected the wrong environment,
or simply decided that its own answer looked convincing.

The central problem is simple:

The system that produced the answer should not be the only system deciding whether the answer is correct.