Representation Learning on AIBussin — AI applications, systems and books

What Is an Embedding?

Mon, 07 Sep 2026 09:00:00 +0000

Part I — A Vector Is Not Meaning

Three words and a list of numbers

Take three words:

cat
dog
airplane

Turn them into numbers. Any embedding API will do it. You get three arrays, each maybe 384 or 768 or 1,536 floats long:

cat [ 0.021, -0.114, 0.062, ... ]
dog [ 0.019, -0.098, 0.071, ... ]
airplane [-0.087, 0.203, -0.041, ... ]

Compute the angle between cat and dog. It is small. Compute the angle between cat and airplane. It is larger. Something about “cats and dogs are both pets” appears to have survived the trip into number-space.

Learning an Embedding Space

Mon, 07 Sep 2026 09:20:00 +0000

Part I — A Vector Is Not Meaning

Refusing the handout

For two chapters the vectors were given. Chapter 1 took them from an API and asked what was in them; Chapter 2 placed some by hand and treated the rest as the output of a model we agreed not to open. Chapter 2 closed on the question it left unanswered: where does the geometry come from?

This chapter builds a small space from nothing but a corpus and a counting rule. A usable geometry appears — related words land near each other without anyone labeling an axis “animal” or “drink.” Then comes the more important half: we inspect the construction closely enough to predict, from the mechanism alone, what it cannot preserve. When the measured results arrive, the failures are not surprises. They are consequences of the path the information took.

Alignment

Mon, 07 Sep 2026 14:10:00 +0000

Part VI — Crossing Embedding Spaces · Which map family survives held-out data?

What Chapter 18 left open

Chapter 18 fit the simplest serious bridge — affine ridge regression — from all-MiniLM-L6-v2 into all-mpnet-base-v2, and evaluated it on entity families the map never saw. The result was not a verdict. It was a profile, uneven enough to make one number impossible to trust on its own:

coordinate reconstruction 0.5860
10-NN neighborhood overlap 0.7128
retrieval nDCG@10 ratio 0.7987
rank-triplet agreement 0.7529
calibration-transfer score 0.7882
relation-profile Pearson correlation 0.8673
structured hard-negative margin ratio 0.2342
held-out/train reconstruction ratio 0.6480

The chapter’s lesson was not “ridge works.” It was that ≈ is a preservation contract — a named property, measured on held-out data, not a single number a map either clears or fails. That leaves an open question this chapter exists to answer:

What Does a Preference Know About the Future?

Fri, 10 Jul 2026 11:32:58 +0100

We Trained a Model on Editorial Choices to See Whether It Learned What Happened Next

Most preference-learning systems use a choice to change the future.

A model produces two responses. A human selects one. The chosen response becomes positive evidence, the rejected response becomes negative evidence, and training makes outputs resembling the chosen response more likely.

The preference acts as an instruction:

Produce more things like this.

I wanted to know whether the same choice could also function as evidence.

The Eye That Sees

Fri, 17 Apr 2026 14:27:45 +0100

Using AI to Decode Symbols Without Assuming Meaning

Executive Summary: From Symbol to System

We set out to understand a single image. We ended up building a system that can understand structure itself.

We started with a constraint:

Assume we do not understand the symbol.

No prior knowledge. No accepted interpretations.

Just an image: an eye, a triangle, rays, an unfinished pyramid.

From that starting point, we:

decomposed the symbol into observable parts
grounded those parts in documented history
separated structure from speculation

That produces a defensible analysis.

The Shape of Thought: Exploring Embedding Strategies with Ollama, HF, and H-Net

Sat, 19 Jul 2025 22:06:13 +0100

🔍 Summary

Stephanie, a self-improving system, is built on a powerful belief:

If an AI can evaluate its own understanding, it can reshape itself.

This principle fuels every part of her design from embedding to scoring to tuning.

At the heart of this system is a layered reasoning pipeline:

MRQ offers directional, reinforcement-style feedback.
EBT provides uncertainty-aware judgments and convergence guidance.
SVM delivers fast, efficient evaluations for grounded comparisons.

These models form Stephanie’s subconscious engine the part of her mind that runs beneath explicit thought, constantly shaping her understanding. But like any subconscious, its clarity depends on how raw experience is represented.