Reinforcement Learning on AIBussin — AI applications, systems and books

Reasoning Is More Than Architecture — Where Extra Computation Lives

Tue, 01 Sep 2026 10:41:00 +0100

Reasoning Is More Than Architecture — Where Extra Computation Lives

So far in Models From First Principles, we have changed several different things and called all of them model design.

We changed what a model predicts.

MR.Q produced one learned quality score.

EBT added Q, V, policy, and advantage.

SICQL turned those outputs into explicit model components.

Then we changed how computation unfolds.

HRM introduced recurrent state operating at different timescales.

EBT — From One Score to Q, V, Policy and Advantage

Sat, 08 Aug 2026 14:39:00 +0100

EBT — From One Score to Q, V, Policy and Advantage

In the previous post we built MR.Q: a small model that takes a context embedding and a response embedding, combines them, and predicts one scalar.

That architecture is useful because it is brutally simple:

context embedding
 +
response embedding
 ↓
 encoder
 ↓
 representation z
 ↓
 predictor
 ↓
 Q value

But one scalar eventually becomes restrictive.

Intelligence Through Execution: The Executable Cognitive Kernel

Tue, 10 Mar 2026 21:58:14 +0000

🧭 Summary

Most modern AI systems treat intelligence as something stored inside a model.

A neural network is trained on massive datasets, its weights are adjusted, and those weights become the system’s knowledge. When the model produces an output, we interpret that output as the result of the intelligence encoded inside those parameters.

But this perspective has a limitation.

Once training is complete, the model is largely static. It does not improve through its own actions, and it does not adapt based on the outcome of its behavior unless we retrain it.

Search–Solve–Prove: building a place for thoughts to develop

Sun, 02 Nov 2025 01:13:06 +0000

🌌 Summary

What if you could see an AI think not just the final answer, but the whole stream of reasoning: every search, every dead end, every moment of insight? We’re building exactly that: a visible, measurable thought process we call the Jitter. This post the first in a series shows how we’re creating the habitat where that digital thought stream can live and grow.

We’ll draw on ideas from:

Self-Improving AI: A System That Learns, Validates, and Retrains Itself

Mon, 30 Jun 2025 10:13:03 +0100

🤖 The Static AI Trap

Today’s AI systems are frozen in time: trained once, deployed forever. Yet the real world never stops evolving. Goals shift overnight. New research upends old truths. Context transforms without warning.

What if your AI could wake up?

In this post, we engineer an intelligence that teaches itself a system that continuously learns from the web, audits its own judgments, and retrains itself when confidence wavers.

Document Intelligence: Turning Documents into Structured Knowledge

Tue, 17 Jun 2025 23:31:13 +0100

📖 Summary

Imagine drowning in a sea of research papers, each holding a fragment of the knowledge you need for your next breakthrough. How does an AI system, striving for self-improvement, navigate this information overload to find precisely what it needs? This is the core challenge our Document Intelligence pipeline addresses, transforming chaotic documents into organized, searchable knowledge.

In this post we combine insights from Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers and Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training to build an AI document profiler that transforms unstructured papers into structured, searchable knowledge graphs.

Adaptive Reasoning with ARM: Teaching AI the Right Way to Think

Wed, 28 May 2025 22:22:46 +0100

Summary

Chain-of-thought is powerful, but which chain? Short explanations work for easy tasks, long reflections help on hard ones, and code sometimes beats them both. What if your model could adaptively pick the best strategy, per task, and improve as it learns?

The Adaptive Reasoning Model (ARM) is a framework for teaching language models how to choose the right reasoning format direct answers, chain-of-thoughts, or code depending on the task. It works by evaluating responses, scoring them based on rarity, conciseness, and difficulty alignment, and then updating model behavior over time.

MR.Q: Model-Based Representations for Model-Free Trading

Tue, 18 Mar 2025 13:05:34 +0000

Introduction

Model-free reinforcement learning learns a policy directly from experience, but it can struggle to discover useful representations from sparse or noisy rewards. Model-based reinforcement learning receives a denser training signal by learning how states, actions, rewards, and termination relate to one another, but it often pays for that knowledge through planning complexity and model error.

MR.Q: MR.Q

Can a model-free agent keep the representation-learning benefits of a learned model without using that model to plan?