Preference Learning on AIBussin — AI applications, systems and books

The Interface Learns You

Fri, 25 Sep 2026 02:45:00 +0100

Chapter 21 gave the reader controls. Chapter 22 forbade turning their settings into a personality. Between those poles sits an unsolved accumulation problem: every interaction produces evidence — a correction here, a repeated choice there, a form that keeps performing better on one task — and with nowhere to accumulate, each encounter starts ignorant. Configuring every dimension manually is burdensome; silently inferring from clicks confuses habit and prominence with desire. This chapter learns from what actually happens, under Chapter 22’s evidentiary rule executed mechanically: no update without conditioning context, no type ever.

What Does a Preference Know About the Future?

Fri, 10 Jul 2026 11:32:58 +0100

We Trained a Model on Editorial Choices to See Whether It Learned What Happened Next

Most preference-learning systems use a choice to change the future.

A model produces two responses. A human selects one. The chosen response becomes positive evidence, the rejected response becomes negative evidence, and training makes outputs resembling the chosen response more likely.

The preference acts as an instruction:

Produce more things like this.

I wanted to know whether the same choice could also function as evidence.

The Preference Was Only the Beginning

Tue, 07 Jul 2026 00:00:00 +0100

A preference is not only a label on what just happened. When the decision belongs to a continuing trajectory, it can also be evidence about what happens next.

Abstract

Most preference-learning systems stop at the choice.

A model produces two responses. A human selects one. The chosen response becomes positive evidence, the rejected response becomes negative evidence, and the training system moves on.

The work itself usually continues.

The selected answer may later be revised, partially retained, contradicted or abandoned. The rejected alternative may reveal a constraint that remains active long after the immediate decision. The preference is therefore not necessarily the outcome. It may be an event inside a longer trajectory.

Document Intelligence: Turning Documents into Structured Knowledge

Tue, 17 Jun 2025 23:31:13 +0100

📖 Summary

Imagine drowning in a sea of research papers, each holding a fragment of the knowledge you need for your next breakthrough. How does an AI system, striving for self-improvement, navigate this information overload to find precisely what it needs? This is the core challenge our Document Intelligence pipeline addresses, transforming chaotic documents into organized, searchable knowledge.

In this post we combine insights from Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers and Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training to build an AI document profiler that transforms unstructured papers into structured, searchable knowledge graphs.

Adaptive Reasoning with ARM: Teaching AI the Right Way to Think

Wed, 28 May 2025 22:22:46 +0100

Summary

Chain-of-thought is powerful, but which chain? Short explanations work for easy tasks, long reflections help on hard ones, and code sometimes beats them both. What if your model could adaptively pick the best strategy, per task, and improve as it learns?

The Adaptive Reasoning Model (ARM) is a framework for teaching language models how to choose the right reasoning format direct answers, chain-of-thoughts, or code depending on the task. It works by evaluating responses, scoring them based on rarity, conciseness, and difficulty alignment, and then updating model behavior over time.