Applied AI

A model gives you intelligence. Applied AI is the engineering required to make that intelligence participate reliably in a process.

Calling a model is easy.

You send it some text. It sends some text back.

The difficult part is everything around that exchange.

What was the model supposed to accomplish? Which information was it allowed to see? Which version of that information did it receive? What happened if the request failed halfway through? Was the answer merely plausible, or was it checked? Was it permitted to change anything? If it changed something, did the change actually happen? Did it produce the result you wanted? What should happen next?

In a chat window, a person quietly answers all of those questions.

They remember the objective and choose the context. They notice when the model misunderstood and decide whether to believe it. They copy the useful part somewhere else, authorize the effect, and decide whether the work is finished.

The chat interface makes the model visible and the process invisible.

This book is about making the process visible.

That does not mean the chat window goes away. You will still type and talk to a model every day; conversation is the most direct way to say what you want. What leaves the chat box is the process around it.

And once the process is software rather than habit, the chatbot stops being the only way in. A chatbot is an interface, one that somebody else built for everyone at once. Beyond it is an interface you can build yourself, shaped around the way you work, with the model inside it replaceable.

That is where the book ends.

The problem is no longer getting intelligence

For most of the history of software, intelligence was the scarce component.

If a task required understanding an unfamiliar document, proposing a repair, comparing several plausible explanations, or producing something genuinely new, a programmer had two choices:

write rules for it

or

give it to a person

Language models added a third.

software
   ↓
model
   ↓
proposal

That is a profound change.

But it is not yet a system.

The model is stochastic. Its output can change when the apparent inputs have not. Its capabilities move from release to release. The same named service can change underneath an application. It can produce convincing errors rather than exceptions. A successful HTTP request says almost nothing about whether the work succeeded.

And the model is only one component in the process. Around it remain questions of state, evidence, cost, authority, side effects, recovery, verification and human judgment.

Those questions become more important as the model gets better, not less.

A weak model forces you to inspect everything, while a strong model can be right forty times in a row and quietly train you not to inspect the forty-first.

The durable question

This book is therefore not primarily about prompting, agents, or whichever model is strongest when you read it.

Those things will change.

The durable question is:

How do you put intelligence inside a process without making the process itself unreliable?

The answer developed through this book is surprisingly conservative.

Most of the system should be ordinary software.

Use deterministic machinery wherever the correct operation can be specified. Use the model only where you genuinely need a proposal from a space you could not enumerate in advance.

Then surround that stochastic boundary with everything ordinary software already knows how to do well:

intent
  ↓
explicit state
  ↓
selected context
  ↓
model proposal
  ↓
preserved observation
  ↓
evidence
  ↓
authority
  ↓
action
  ↓
independent verification
  ↓
new state
  ↓
next operation

The model supplies cognition without owning the process.

One stochastic box

A central claim of this book is that stochasticity should be confined, not spread through the architecture.

Consider an AI-assisted document review.

The system may need to:

• determine whether a paragraph changed;
• retrieve the current version;
• check a budget;
• identify claims that may require evidence;
• locate the flagged spans;
• verify that cited sources exist;
• decide whether an edit is authorized;
• write a change;
• run checks against the result;
• record what happened.

Only one of those operations clearly benefits from a model: identifying claims whose need for evidence cannot be completely specified in advance.

Everything else has a right answer.

So the rule used throughout this book is:

A component belongs on the deterministic side unless it demonstrably cannot be.

There is an even simpler test:

If you would be annoyed to get a different answer when you rerun it, it probably should not be a model call.

Stochasticity is not something we are trying to eliminate; it is the reason the model is useful.

The model can propose the interpretation you did not anticipate, the repair you did not enumerate, or the connection you had not considered.

The mistake is paying for that variance in places where you wanted certainty.

Verification changes everything

There is a recurring asymmetry behind many dependable AI applications.

Producing an answer can be difficult.

Where an adequate check can be specified, checking that answer is often much easier.

You may not know how to generate the correct code, but you can run the tests.

You may not know which citation belongs in a paragraph, but you can check whether the proposed source actually contains the claim.

You may not know the right patch, but you can compile the result.

This is why software was such fertile ground for AI. It was not because programming was easy; software engineering had spent decades constructing unusually cheap verifiers: compilers, type systems, tests, continuous integration, version control and reversible changes.

The model arrived in terrain that had already been mapped.

That gives us a useful way to evaluate other domains.

Text plus intelligence can produce an impressive demo.

Text plus intelligence plus a cheap verifier can produce a dependable process.

Where the verifier does not yet exist, building it may be more important than choosing a better model.

The human does not disappear

The architecture in this book deliberately leaves four responsibilities outside the model.

Intent — deciding what should happen and what would count as done.

Authority — deciding which effects are permitted.

Verification — obtaining evidence about what actually happened, mechanically where possible and otherwise through independent judgment.

Frontier judgment — deciding whether a model is the right mechanism for this operation at all.

Outside the model does not mean outside software. The point is separation: the model may propose, but it does not get to supply the intent, authorize its own effects, or turn its own claim of success into evidence.

These are not leftovers waiting for the next model generation to absorb them; they are the jobs whose importance grows as generation becomes cheaper.

If a model can produce one hundred times as much work, the system needs more ability to decide what work should exist, more ability to control what may happen, and more ability to tell whether any of it was correct.

That is why this book treats human review as an architectural component rather than a final checkbox.

A person staring at hundreds of mostly-correct outputs eventually becomes a rubber stamp. Decades of automation research suggest that diligence alone does not solve this.

So the process has to make good review cheap:

mechanical checks first
        ↓
evidence attached to claims
        ↓
attention routed by risk
        ↓
human judgment where judgment is actually required

Human-in-the-loop is not a safety mechanism merely because a human appears somewhere in the diagram. The loop has to contain information the human can realistically evaluate.

The book is also a build

The reference system is CodeAI.

We begin with almost nothing: a callable model.

Then we add one engineering obligation at a time.

The model call acquires identity. Attempts become distinct from logical calls. Raw responses survive after the process exits.

Different provider protocols are normalized without pretending their differences do not exist. Context becomes an explicit input instead of whatever happens to be in a transcript, and history becomes durable state.

Claims become separate from the evidence supporting them. A proposed action becomes separate from an authorized action, and an action becomes separate from its observed effect.

Verification becomes independent of the component that generated the proposal. Retries become part of the effect model rather than a convenience hidden inside an HTTP client. Several proposals can be isolated from one another.

Experiments can measure whether more calls, different models, or different prompts actually buy anything. Finally the runtime can answer a question more important than which model should I call?

It can answer:

What operation should happen next?

Sometimes the answer is CALL. Sometimes it is CHECK or ACTION. Sometimes ASK_HUMAN. Sometimes the correct answer is STOP.

Choosing the operation comes before choosing the model.

One runtime, many windows

The process also cannot live inside a chat interface.

If your phone remembers one conversation, your editor another, your terminal a third and your browser a fourth, you do not have one intelligent system.

You have four isolated processes and a person manually synchronizing them.

The architecture developed here reverses that relationship:

             phone
               │
editor ──── runtime ──── browser
               │
            terminal

The surfaces become views.

The durable state behind the runtime becomes the source of truth.

That leads to another distinction that runs through the book:

Memory is durable. Context is selected.

Remembering everything does not mean sending everything to every model call.

The system may retain a rich history while deliberately selecting only the information relevant to the current operation.

The important difference is that exclusion becomes a recorded decision rather than accidental amnesia.

The model is not the foundation

There is another reason to separate the runtime from the model.

The model will change.

The strongest model available next year will not be the strongest one available today. Prices will change. Providers will disappear. Models will regress on individual tasks while improving overall. A prompt tuned for one model may behave differently on another.

So the system should not be built on a model. It should have places where models can be installed.

The book calls these chambers.

fast-classify  → current occupant
draft          → current occupant
deep-review    → current occupant
critic-a       → current occupant
critic-b       → current occupant

The chamber is named for the job.

The model occupying it is temporary.

This gives us a different approach to upgrades:

A new model release is an experiment, not an upgrade.

Replay the work that matters. Compare the candidate with the incumbent per item.

Count the cases that improved and those that regressed, then measure the cost per accepted result.

Change the occupant only where the evidence justifies it.

Finish the frame; turn the chamber.

More intelligence is not automatically better

This book does not stop at proposing that architecture.

It tests one of its most tempting assumptions.

If one model call is useful, surely several are better. If several draws from one model help, surely several different models help more. If different models fail differently, surely diversity buys coverage.

Reasonable ideas, but exactly the kind experiments are for.

A frozen series of experiments in the later book compares repeated sampling, heterogeneous model portfolios, harder tasks, prompt-stance diversity and a preregistered replication.

Some of the attractive results disappear. Some become weaker when the tasks become harder. One promising subgroup survives long enough to justify another experiment and then fails to replicate.

That is not a detour from the book’s argument.

It is the argument.

Discovery is not promotion. A signal is not a result. One successful run is not a measurement.

The runtime should spend intelligence where intelligence has demonstrated value, not where adding another model call merely feels sophisticated.

Intelligence has a price

There is also an economic reason to care about the boundary.

Traditional software has an attractive shape:

expensive to build
       ↓
cheap to run

Put a frontier model permanently into the request path and part of that relationship reverses.

Every user action costs money again — retries, evaluations, and every unnecessary piece of context.

Runtime intelligence is rented capability.

So the book develops a cost ratchet:

remove unnecessary context
        ↓
use the cheapest model that passes
        ↓
escalate failures
        ↓
route by task
        ↓
distil narrow capability where appropriate
        ↓
replace the operation with deterministic software

Where an adequate verifier captures the outcome you care about, model spending has a ceiling.

Once a cheaper system clears that verifier, spending more does not make the verified outcome more correct by that criterion. It may still buy qualities the verifier does not measure, which is why verifier adequacy matters.

Where no adequate verifier exists, spending has no natural stopping point — and that is precisely where it becomes easiest to convince yourself that the more expensive model must be better.

Measurement is therefore part of the cost architecture, not a separate concern.

Build the measurement first

Before the runtime becomes sophisticated, we build the thing that can tell us whether sophistication helped.

A small frozen task set. Mechanical checks. Repeated runs. Recorded model and prompt identities. Error bars rather than anecdotes.

Then every later claim has somewhere to land.

Did the new model improve the task? Did another sample add coverage? Did the critic agree with the actual verifier?

Did the prompt intervention survive another set of draws? Did the system become cheaper per successful outcome? Did the project actually finish faster?

Without measurement, model selection becomes reputation, prompting becomes folklore, and architecture becomes taste.

With measurement, each becomes an engineering decision that can be reversed when the evidence changes.

What you should be able to do after reading this book

The goal is not that you remember thirty chapter titles.

The goal is that you can look at an AI-enabled system and ask better questions.

You should be able to identify:

• which operation genuinely requires a model;
• which operations should be deterministic;
• what exact task a call belongs to;
• what context the model was permitted to see;
• what it actually received;
• which provider and model produced an observation;
• how many attempts occurred;
• which bytes were returned before anyone interpreted them;
• what the model merely asserted;
• what evidence supports those assertions;
• whether an action is available versus authorized;
• what effect actually happened;
• whether verification was independent of generation;
• what state can safely be resumed after a crash;
• whether a retry can duplicate an effect;
• whether another model call would add useful information;
• whether a more expensive model earns its price;
• whether a promising result survived replication;
• which parts of your interface to technology you own, and which you rent;
• what the process should do next;
• and whether it can stop honestly when the objective has not been established.

Most importantly, you should be able to distinguish five roles that are routinely collapsed:

model       → cognition

runtime     → coordination

tools       → action

verifiers   → evidence

human       → intent + authority

That separation is the architecture.

The journey through the book

The six parts follow the construction of that architecture.

Part I — Where You Stand

We begin before the implementation: which work should stay deterministic, why software automated itself first, why human review decays as the model gets better, what intelligence costs, how to measure a stochastic system, and why projects are not finishing dramatically faster.

The conclusion moves the target. Generation is not the bottleneck. Intent, authority and verification are.

Part II — Get the Model Out of the Chat Box

The work moves out of the interface and into a durable runtime: an identity and a record that survive the window closing, a replaceable model underneath, calls and attempts that are recorded process events, several provider protocols behind one contract, and usage numbers that mean something before anyone adds them up. Completion becomes something explicitly established rather than inferred from a successful generation.

By the end of this part, the model is a component rather than a conversation partner.

Part III — Give Intelligence a Runtime

The component acquires a world around it: context compiled and recorded instead of accumulated, working state another process can resume from, raw observations preserved before interpretation, claims that point at evidence, decisions that record what they relied on, and effects that are observed rather than reported.

This is where AI work stops disappearing into scrollback.

Part IV — Make It Safe and Verifiable

Acting on the world raises three questions a prompt cannot answer: who may perform an action, what makes a passing check worth believing, and when trying again is a second effect rather than a returned record. Capability separates from authority, verification is judged on independence, adequacy and binding, and idempotency, stale state and crash windows become explicit engineering problems.

The system learns to say more than “success”. It learns which kind of success has actually been established.

Part V — More Intelligence Is Not Automatically Better

More calls, more models and more prompting strategies all produce more candidates; none of them automatically produces more useful coverage. This part collects proposals blind, measures variety by what gets solved, strengthens a benchmark too easy to tell the methods apart, and refuses to promote an attractive signal until a matched replication earns it.

The recurring question is whether extra models, extra draws or changed prompts earn their added complexity under matched tests. Keeping the incumbent counts as a legitimate experimental outcome here, not as a failed experiment.

Part VI — Put Intelligence Into the Process

The runtime reads explicit state and decides which kind of operation is required next — not which model sounds cleverest. Chapter 1’s stronger ambition, policy as data, did not survive implementation: a declarative table reproduced the decision function exactly while fixing none of its real gaps, so the book keeps the small function and says so. Reporting what did not survive implementation is part of the method.

The capstone then runs one task end to end through intent, context, generation, preserved observation, evidence, authority, action, verification, acceptance and replay, and asks whether the joints between them hold. The last chapter turns from the process to the person: when software can be built around one individual, this architecture is what keeps such a tool improving without losing its discipline.

The larger idea

AI is often presented as a replacement for software.

This book reaches almost the opposite conclusion.

The more capable the model becomes, the more valuable ordinary software engineering becomes around it.

State matters because the model has none you can safely assume.

Schemas matter because free-form boundaries spread uncertainty.

Evidence matters because plausible language is not proof.

Authorization matters because capability is cheap.

Verification matters because generation is abundant.

Persistence matters because an observation may need to be reinterpreted long after the call that produced it.

Measurement matters because a stronger model can be worse on the one case you depended on.

And deterministic control matters because intelligence is most useful when it does not have to govern itself.

The result is not an autonomous blob sitting at the centre of the architecture.

It looks much more like ordinary software with a carefully chosen intelligent boundary.

    flowchart TD
    A["deterministic frame<br/>context · routing · bookkeeping"] --> B(["stochastic proposal<br/>model"])
    B --> C["evidence"]
    C --> D["authority"]
    D --> E["action"]
    E --> F["verification"]
    F --> G["durable state"]
  

The model is powerful.

The system is dependable because the rest of the architecture does not require it to be something it is not.

That is Applied AI.

The interface you build yourself

There is one more step, and it is the one the whole construction points toward.

A chatbot is an interface.

It is a good one, and nothing in this book asks you to give it up. But it was built by someone else, for everyone at once. It holds your intent for as long as the conversation lasts, and everything else stays in your head: the context you chose, the checks you ran, the changes you allowed.

For forty years software has worked this way. Construction was expensive and serving one more user was cheap, so tools were built for an average user and each person adapted themselves to the tool.

As construction gets cheaper, that relationship can start to reverse.

The architecture in this book is ordinary software, so it does not have to be built for everyone. The frame, the tools and the policies can be shaped around one person’s work:

conversation   → what you want               yours
your tools     → the work you repeat         yours
runtime        → state, evidence, authority  yours
model          → proposals                   rented, replaceable

The top three layers are the interface between you and the technology you work with. The bottom layer is replaceable by design, which is why the book works so hard to keep the model in one slot.

So the end result is not a better chatbot. In the end it is not really about programming, or even about AI. AI is what makes this kind of individualized construction increasingly feasible. The subject is how you work with technology, and the chance to build that way of working for yourself instead of accepting the one you were given.

That interface should not quietly rewrite itself. It improves through the same discipline the book builds for everything else:

friction → proposal → authorization → change → verification → measurement → keep or revert

The final chapter makes this argument deliberately without a demonstration. Your version should have the shape of your work, not the author’s. The claims stay bounded: the chapter does not show that software built around a person beats generic software. It argues that the economics have moved far enough to make such tools worth attempting, and that the process built in this book gives those attempts explicit state, authority, verification, and measurement rather than asking the model to govern itself.

Running the code

CodeAI is public at https://github.com/ernanhughes/codeai and needs Python 3.11 or later.

git clone https://github.com/ernanhughes/codeai.git
cd codeai
python -m venv .venv
source .venv/bin/activate    # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
pytest

The test suite runs offline. Fake adapters and recorded fixtures stand in for the providers, so it needs no network and no API keys; at the time of writing all 456 tests pass with no provider keys set.

Live model calls, such as Chapter 1’s example and the live labs, need an OpenCode key in OPENCODE_ZEN_API_KEY. Gateway routes, models and prices change, so treat any live result as an observation dated to its run.

Each chapter that reports an experiment says where its preserved evidence is kept, and each pinned evidence bundle records the exact CodeAI version it ran against.

Experimental status

This is a book about a moving technology, so observations and architectural claims are kept separate.

Specific model behavior, provider protocols, prices and experimental measurements belong to the environment and date in which they were observed. Preserved evidence remains preserved when later models or APIs change.

The architectural contracts are the more durable layer:

• explicit state;
• bounded stochasticity;
• preserved observations;
• independent evidence;
• capability separated from authority;
• effects separated from claims about effects;
• deterministic next-step policy;
• model substitutability;
• measurement before promotion.

All thirty chapters are complete drafts across six parts; the last is an argument chapter rather than a construction stage. The experimental arc, runtime arc and end-to-end capstone are drafted, with preserved run evidence retained separately from the manuscript.

Where an experiment produced a negative result, the negative result remains.

Where an implementation cannot establish something, the book says so.

Where the technology changes, a later edition should change with it.

Applied AI begins where the chat box ends.

Continue with Beyond the Chat Box.