Training: Which Link in the Learning Chain Is Broken?

Sat, 29 Aug 2026 14:00:00 +0100

Here is an ordinary piece of transfer-learning code. A small model, a frozen feature extractor, an optimizer, and a new classification head for the new task.

model = TinyClassifier()

for p in model.features.parameters():
 p.requires_grad_(False)

optimizer = torch.optim.AdamW(model.parameters(), lr=1e-2, weight_decay=0.0)

model.head = nn.Linear(32, 2) # adaptation, performed later

Train it for two hundred steps on a balanced, perfectly aligned, entirely learnable binary task:

step= 0 loss=0.683606 acc=0.7144
step= 50 loss=0.683606 acc=0.7144
step= 100 loss=0.683606 acc=0.7144
step= 150 loss=0.683606 acc=0.7144
step= 200 loss=0.683606 acc=0.7144

Not approximately constant. Identical to six decimal places, two hundred times.

PyTorch Model Not Learning? A Systematic Debugging Guide

Sat, 08 Aug 2026 13:51:00 +0100

PyTorch: Zero to Hero — Step 08

Your model runs.

The loss is finite.

Nothing crashes.

And it still does not learn.

This is one of the most frustrating states in machine learning because there is no stack trace telling you what is wrong.

The program is valid Python.

The tensors have legal shapes.

The GPU is busy.

The optimizer is stepping.

And the model is useless.

This post is a systematic way to debug that situation.

Training on AIBussin — AI applications, systems and books

Training: Which Link in the Learning Chain Is Broken?

PyTorch Model Not Learning? A Systematic Debugging Guide

PyTorch: Zero to Hero — Step 08