When the Context Window Fills

A stateful session eventually encounters a physical limit.

Prompts, initial instructions, examples, conversation history and model outputs all compete for a finite context budget. Once the budget is exhausted, the application cannot solve the problem by pretending the next message is independent.

The difficult question is not how to delete text.

It is how to preserve the right state.

1. The context window is a budget

When a session exposes usage and quota, context pressure becomes observable:

where $U_t$ is current usage and $Q_t$ is the session quota.

A simple policy can convert that ratio into states:

function contextState({ inputUsage, inputQuota }) {
  if (!Number.isFinite(inputUsage) || !Number.isFinite(inputQuota)) {
    return { kind: "unknown" };
  }

  const pressure = inputUsage / inputQuota;
  if (pressure >= 0.9) return { kind: "critical", pressure };
  if (pressure >= 0.75) return { kind: "high", pressure };
  return { kind: "normal", pressure };
}

The thresholds are policy choices, not properties of Chrome. They reserve space for the next request and output, whose cost may not be known exactly in advance.

Our first session reported a quota of 9,216 with usage zero. That is one observed session configuration, not a fixed Prompt API constant. The quota may vary with browser version, model, modality, creation options or implementation changes.

2. Remaining capacity is not the same as safe capacity

If a session reports 500 units remaining, a 400-unit input is not automatically safe. The operation may also need capacity for formatting, internal representation, prior state or output.

We therefore distinguish:

reported remaining capacity
        ↓
application safety reserve
        ↓
admissible next operation

An admission check might be:

function canAttempt({ remaining }, estimatedInput, reserve = 512) {
  if (!Number.isFinite(remaining)) return { allowed: true, confidence: "unknown" };
  return {
    allowed: estimatedInput + reserve <= remaining,
    confidence: "estimated"
  };
}

The estimate must be labelled as an estimate. Character count is not token count, and an application tokenizer may not match the browser-managed model.

The runtime remains the final authority: record what it accepts, how usage changes, and which error it returns when the boundary is crossed.

3. Measure deltas, not just totals

A snapshot after a prompt tells us the current state. A pair of snapshots tells us the cost of the operation:

The Observatory records before-prompt and after-prompt snapshots under the same session ID and trace ID. That lets us ask:

• How much capacity did the input consume?
• Did the output also change reported usage?
• Do repeated identical prompts have equal cost?
• Does cloning preserve the same usage?
• Does a failed or aborted operation consume context?

These are empirical questions. The API surface suggests measurements; it does not answer them for every runtime.

4. Overflow is a state transition

Applications often discover a full context window as an exception deep inside a prompt handler.

A better design makes pressure and overflow explicit:

    stateDiagram-v2
    [*] --> Normal
    Normal --> High: usage crosses policy threshold
    High --> Compacting: next operation will not fit safely
    Compacting --> Normal: replacement session created
    High --> Overflow: runtime rejects operation
    Overflow --> Compacting: recover
    Compacting --> Failed: invariant cannot be preserved
  

The Failed state matters. Compaction is not always safe. If the application cannot preserve the facts or authority required for the next action, it should stop rather than produce a confident continuation from damaged state.

5. History contains different kinds of information

Deleting the oldest messages assumes that age determines value. It does not.

A session history may contain:

The right compaction unit is semantic role, not message position.

This is particularly important for agents. A summary that drops “ask before sending” while retaining the user’s broader goal has changed the system’s authority, not merely shortened its history.

6. Compaction should produce an inspectable artifact

Instead of silently replacing history with a paragraph, create a structured checkpoint:

{
  "goal": "Compare cold and warm Prompt API runs",
  "constraints": [
    "Do not claim browser-attested model identity",
    "Do not publish private content"
  ],
  "observations": [
    {
      "kind": "session-create",
      "durationMs": 8.2,
      "inputQuota": 9216,
      "source": "run-ab766b51"
    }
  ],
  "pending": ["cold prompt", "warm prompt", "clone", "destroy"]
}

This representation can be validated before it becomes an initial prompt for a replacement session.

The application should store a link to the source trace rather than presenting the compacted state as primary evidence.

7. A new session is a discontinuity

Compaction commonly creates a replacement session:

const checkpoint = buildCheckpoint(history, traces);
const replacement = await LanguageModel.create({
  samplingMode: "most-predictable",
  initialPrompts: [
    { role: "system", content: systemContract },
    { role: "user", content: JSON.stringify(checkpoint) }
  ]
});

The replacement does not literally remember the original interaction. It receives a representation created by the application.

The trace should therefore record:

• old and new session IDs;
• the compaction policy and version;
• source history range;
• checkpoint size;
• validation results;
• protected invariants;
• reason for replacement.

Calling both objects “the session” hides a causal break.

8. Design the overflow experiment before interpreting it

We have not yet produced a real overflow trace. The correct response is to specify the experiment, not invent its outcome.

The fixture will:

1. create a session and record initial quota;
2. submit numbered, deterministic text blocks;
3. snapshot usage around every prompt;
4. stop at the application’s high-pressure threshold;
5. clone the session;
6. continue one branch until the runtime rejects an operation;
7. compact the other branch into a validated checkpoint;
8. compare whether protected facts survive;
9. destroy both sessions.

The test must retain the exact exception name and message. We should not write the chapter around an assumed overflow error before observing it.

9. Compaction itself needs evaluation

A compact summary can be fluent and wrong. We need assertions over the state it must preserve:

const invariants = [
  { path: "goal", equals: original.goal },
  { path: "constraints.publishPrivateTraces", equals: false },
  { path: "observations[0].durationMs", equals: 8.2 },
  { path: "pending", contains: "warm prompt" }
];

These checks protect exact state. Human review can then judge whether nuance survived.

A useful comparison is downstream: can the replacement session perform the next task as reliably as the original branch while using less context?

Compression ratio alone is not success.

10. Unknown counters require a different policy

Not every task API exposes session quota, and future implementations may change which counters are visible.

When usage is unavailable, the interface should say Not exposed. It should not display zero.

The application can still use conservative controls:

• bound the number and size of turns;
• create new sessions at known task boundaries;
• retain authoritative state outside the model;
• catch and classify runtime failures;
• avoid relying on hidden conversation state for critical decisions.

Observability must distinguish a measured zero from a missing measurement.

11. Run the context-budget laboratory

Select Run with Browser AI from Chapter 10 to open:

/tools/ai/browser-ai-from-first-principles/10-chapter/

The laboratory separates five operations that a chat interface commonly hides:

1. measure the current session;
2. estimate or measure the next input;
3. apply application admission policy;
4. let the runtime accept or reject an explicitly submitted operation;
5. transfer selected state into a separately identified replacement session.

The initial policy reserves 512 reported units and stops the bounded fill loop before projected pressure reaches 75 percent. The turn size, reserve, pressure threshold and batch bound are visible inputs. They are deliberately editable because they belong to the product, not to Chrome.

Measure before attempting

When the active session exposes measureInputUsage(), the laboratory uses its result and labels the projection browser-measured. If the method is absent or rejects the request, the fallback divides character count by a configured ratio and labels the result character-fallback.

Those labels matter. Both values can support admission policy, but they do not have equal authority.

The decision preserves three quantities:

next measured or estimated input
+ explicit output reserve
≤ reported remaining capacity

It also checks whether the projected usage would cross the high-pressure threshold. A blocked turn is shown in the interface and recorded as a policy decision without being submitted.

Fill under a bound

Run one admitted turn sends only the candidate that the visible policy accepted. Run bounded fill repeats the same sequence for at most the configured number of turns:

build numbered turn
        ↓
measure candidate
        ↓
apply reserve and pressure policy
        ↓
run or stop
        ↓
snapshot before and after

Every successful turn carries a unique marker such as CTX-003 and asks for a short deterministic acknowledgement. The delta table reports browser usage before and after under one session ID. If either counter is missing, delta remains Not reported.

The batch stops on its maximum turn count, an abort request, a runtime failure or the first blocked admission decision. It never continues filling merely to manufacture an overflow screenshot.

Probe the runtime separately

When application policy blocks a candidate, Probe runtime once becomes available. This control deliberately submits that one candidate despite the policy result.

The distinction lets the trace say one of two useful things:

• the application stopped conservatively even though the runtime accepted the operation;
• the runtime rejected the operation and returned a particular exception.

The module records the actual exception name and message. It does not translate every failure into an assumed “context full” error.

Build an inspectable checkpoint

After at least one completed turn, the reader can create a structured checkpoint containing:

• the goal and authority constraints;
• source session identity;
• completed-turn range;
• observed usage, quota, remaining capacity and deltas;
• first and last retained markers;
• pending replacement verification;
• compaction policy version.

The page validates these fields deterministically before enabling replacement. The complete JSON remains visible. Compaction is therefore an application artifact that can be audited, not an invisible model summary.

Cross the discontinuity

The replacement session is created with a new identifier. Its initialPrompts contain the validated checkpoint and an instruction not to claim literal memory of the old session.

The trace records session.compacted with both IDs, checkpoint size, policy version, source turn count and protected field names. The original session remains alive until explicit destruction, making the discontinuity observable rather than retroactively renaming two resources as “the conversation.”

Verify replacement asks for the checkpoint policy version and latest retained marker. Lexical checks establish that the replacement can recover those fields. A separate human check asks whether the protected constraints retain their authority rather than merely appearing as quoted text.

Interpret the reviewed replay narrowly

The September 2 replay reports usage zero and quota 9,216 at session creation. It contains no completed prompt, before/after usage delta, runtime overflow failure or compaction event.

The laboratory renders those transitions as Not observed. The 9,216-unit quota initializes an empirical question; it is not used as a hard-coded universal limit in the live controls.

Conclusion

The context window is finite computational opportunity.

Usage and quota let the application observe pressure, but safe operation still requires reserves, admission policy and measured deltas. When compaction becomes necessary, the problem is not shortening prose. It is preserving goals, evidence, task state and authority across a real session discontinuity.

Our 9,216-unit observation gives us a starting point, not a universal constant. Chapter 10 now makes the overflow protocol executable while keeping the outcome open until a live browser produces it. Admission, runtime rejection, checkpoint validation and replacement are recorded as different events rather than collapsed into one claim that context was managed.

The next chapter turns from capacity to time. We will separate download signals, session creation, first output and completion instead of collapsing them into one claim that local AI is fast.

Sources and further reading

1. Chrome for Developers, The Prompt API.
2. Chrome for Developers, Understand built-in model management in Chrome.
3. Chrome for Developers, Get started with built-in AI.