Tensor Shapes on AIBussin — AI applications, systems and books

The Tensor: What Is Actually Flowing Through the Loop?

Sat, 08 Aug 2026 12:45:00 +0100

The last program in Chapter 1 trained a two-parameter model on four examples. It worked, and we read it as a story about gradients: predict, measure, differentiate, step. But something in it went unexamined. w was a single number and x held four, and w * x + b produced four predictions without anyone specifying how a scalar and a four-element vector should combine. PyTorch had a rule. We never stated it.

CNN Geometry: What Shape Reaches the Next Layer?

Fri, 28 Aug 2026 10:15:00 +0100

Here is a batch of 32 RGB images and an ordinary first convolution.

x = torch.randn(32, 224, 224, 3)
conv = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3)
conv(x)

RuntimeError: Given groups=1, weight of size [16, 3, 3, 3],
expected input[32, 224, 224, 3] to have 3 channels, but got 224 channels instead

The images have three channels. PyTorch says there are 224. The message names a number that appears nowhere in the model definition, so the obvious reading is that the layer was declared wrong, and the obvious repair is to declare it right:

Attention: Which Position Is Comparing With Which?

Sat, 29 Aug 2026 10:00:00 +0100

Here is a tensor of sequence representations and the line that splits it into heads.

B, T, E, Nh = 1, 4, 8, 2
Dh = E // Nh # 4

x = torch.arange(B * T * E).reshape(B, T, E)
heads = x.reshape(B, Nh, T, Dh)

x: (1, 4, 8)
heads: (1, 2, 4, 4)

That is exactly the shape multi-head attention wants: batch, heads, positions, head dimension. Nothing raised. Every shape assertion passes.

Now the same split written the other way:

PyTorch Tensor Shapes: Broadcasting, Reshape, View, Permute and the Errors That Waste Your Time

Sat, 08 Aug 2026 12:45:00 +0100

PyTorch: Zero to Hero — Step 01

Most PyTorch bugs are not really “AI bugs”.

They are shape bugs.

You expected:

[batch, features]

but actually had:

[batch, sequence, features]

You expected two tensors to line up.

They broadcast instead.

You called view() after permute() and got a contiguity error.

You removed a dimension with squeeze() and accidentally removed the batch dimension too.

Or you reached the familiar message:

RuntimeError: The size of tensor a (...) must match the size of tensor b (...)

This article is about becoming dangerous enough with tensors that these errors stop being mysterious.