Models From First Principles — Working Ledger
Book intention
Understand modern AI models by opening the abstractions and building the mechanisms in layers, while keeping the mathematics, architecture, optimization, inference, training, and working code connected.
The review question is not merely whether each chapter is well written. It is whether each chapter does the job it is supposed to do at this point in the sequence and whether the sequence leaves a useful model or mechanism missing.
Editorial state: ACTIVE — REVISION
Review ladder: UNREVIEWED → STRUCTURE OK → CONTENT OK → PROSE OK → DONE
Current intended arc
The Model Inside the Model → MR.Q → EBT → SICQL → HRM → Tiny → Inside Tiny → PACS → Preference Rankers → Reasoning Is More Than Architecture → comparison / model choice
The new Chapter 10 deliberately zooms out after the architecture, optimization, and preference-learning chapters. Its job is to prevent the book from implying that reasoning capability is primarily an architectural property. It separates where additional computation can live: parameters, recurrent state, inference-time sampling/search, selection/verification, post-training, and distillation.
The final comparison still lives at 15-chapter.md. It should be reviewed after Chapter 10 and then either normalized to 11-chapter.md or deliberately retained as back-loaded numbering. Its current title and comparison set may need widening so the conclusion represents the revised book rather than only MR.Q through PACS.
Chapter ledger
| # | File | Canonical title | Status | Job / durable notes |
|---|---|---|---|---|
| 01 | 01-chapter.md | The Model Inside the Model | UNREVIEWED | Establish the first-principles method: useful model abstractions are made from smaller mechanisms that can be opened, traced, and understood. Set the conceptual contract for the book. |
| 02 | 02-chapter.md | MR.Q — Building a Neural Quality Model From Two Embeddings | UNREVIEWED | Introduce a compact quality model and make scoring/ranking from representations concrete before the book moves into more composite architectures. |
| 03 | 03-chapter.md | EBT — From One Score to Q, V, Policy and Advantage | UNREVIEWED | Expand from a single learned score into related decision quantities and show how Q, V, policy, and advantage fit together. |
| 04 | 04-chapter.md | SICQL — Building a Model From Q, V and Policy Networks | UNREVIEWED | Turn the quantities from EBT into a composed architecture with specialized networks; show how model behaviour emerges from the relationship between components. |
| 05 | 05-chapter.md | HRM — Hierarchical Reasoning With Fast and Slow Recurrent State | UNREVIEWED | Introduce a different reasoning mechanism: recurrent state operating at different timescales. Broaden the reader’s notion of what a reasoning model can be. |
| 06 | 06-chapter.md | Tiny — Recursive Reasoning With a Small Neural Network | UNREVIEWED | Show how repeated computation through a small model can produce reasoning-oriented behaviour; challenge the assumption that capability must come primarily from scale. |
| 07 | 07-chapter.md | Inside Tiny — Residual Blocks, Attention and Sparse Autoencoders | UNREVIEWED | Open Tiny itself and connect its larger behaviour to recognizable internal mechanisms. This chapter must deepen 06 rather than merely repeat it. |
| 08 | 08-chapter.md | PACS — Building an Optimizer From Gradient Statistics | UNREVIEWED | Shift from architecture to learning dynamics and show an optimizer as another stateful mechanism that can be derived and inspected from first principles. |
| 09 | 09-chapter.md | Preference Rankers — Learning Which Answer Is Better | UNREVIEWED | Introduce pairwise preference learning and connect embeddings/quality scoring to ranking, reward modelling, and systems that learn relative rather than absolute targets. |
| 10 | 10-chapter.md | Reasoning Is More Than Architecture — Where Extra Computation Lives | NEW | Synthesize the preceding mechanisms and add the missing system-level axes: sequential compute, multi-sample inference, self-consistency, Best-of-N, selection/verification, post-training, RL, distillation, and compute-aware evaluation. This chapter should widen the reader’s definition of a reasoning system without becoming a second book on RL or search. |
| 11? | 15-chapter.md | Which Model Should You Use? MR.Q, EBT, SICQL, HRM, Tiny and PACS Compared | UNREVIEWED | Finale/comparison. Re-evaluate after Chapter 10. It should compare not only architecture choices but also where additional compute and supervision belong. Decide final number and title after integration review. |
Book-level questions to resolve during review
- Is the book now a coherent progression from model decomposition to system-level reasoning, or still a collection of interesting architectures?
- Does PACS read as a deliberate shift into learning dynamics rather than a detour?
- Does Preference Rankers naturally widen the supervision story before Chapter 10 zooms out to inference and post-training?
- Does Chapter 10 stay at the right abstraction level, or does it drift too far into a separate reasoning-model/RL book?
- Does the finale compare the mechanisms the reader has actually learned: architecture, recurrence, optimization, supervision, inference scaling, selection, verification, and training-time trade-offs?
- Should the final comparison become
11-chapter.mdand receive a broader title?
Current next action
Review 10-chapter.md as a whole-book bridge, then revise the final comparison so it reflects the expanded model-selection framework. Do not add another large mechanism until those two chapters are integrated.