Transforms: What Does the Model Actually See?

Here are two training runs. Same model, same data, same optimizer, same seed, same number of epochs. The preprocessing pipelines differ by one keyword argument, and neither one raises.

scale=True   range=[   -1.09,    1.45]  first_epoch_loss= 0.5222  last=0.0032  val_acc=1.000
scale=False  range=[  259.89,  886.36]  first_epoch_loss=28.5556  last=0.6938  val_acc=0.400

The second pipeline produced tensors with the right shape, the right dtype, the right device, and no non-finite values. Every check from Chapter 2 passes. Every check from Chapter 5 passes. The batch arrived on time, which is everything Chapter 6 taught us to verify. And the model went from perfect to worse than guessing.

The argument was scale, on ToDtype. Its default is False:

ToDtype(dtype, scale: bool = False)

So the broken pipeline is the one you get by writing the obvious thing. v2.ToDtype(torch.float32) converts 8-bit image values to floating point and leaves them as 0.0 ... 255.0, because converting a dtype and reinterpreting a value scale are two different operations and PyTorch will not guess which one you meant.

Now change one thing that has nothing to do with preprocessing. Swap SGD for Adam and run both pipelines again:

optimizer=adam  lr=0.01
  scale=True   range=[   -1.09,    1.45]  first_epoch_loss=0.2304  last=0.0000  val_acc=1.000
  scale=False  range=[  259.89,  886.36]  first_epoch_loss=0.7134  last=0.0000  val_acc=1.000

The bug is still there. The model-facing values are still hundreds of times outside the intended range. And it is now invisible in the headline metrics: same accuracy, same rounded final loss, nothing obvious in the training curve. In this experiment Adam’s adaptive update normalization makes optimization far less sensitive to the input rescaling than plain SGD was. That is optimizer robustness, not evidence that the representation is correct.

That contrast is the argument for this chapter existing.

The severity of a representation bug depends on things that have nothing to do with representation. You cannot diagnose what the model is seeing by looking at how well it trained.

You have to look at the representation directly.

Where we are

Chapter 6 turned input delivery into an observable system. Something produces samples, something collates them, worker processes may prepare future batches, and the training loop consumes them. You can measure the wait at every boundary and name the stage that cannot keep up.

But Chapter 6 deliberately treated each batch as an object with a size, an arrival time and a production cost. The Producer in that chapter returned torch.randn(128) and a random label, because the contents were irrelevant to the question of when the batch showed up.

Real samples are not random numbers. They are files that were decoded, integers that were converted to floats, arrays that were resized, tensors that were normalized against statistics computed from somewhere, and images that may have been augmented differently on this epoch than the last. Each of those steps is a decision about what the numbers mean.

So the question for this chapter:

What exactly is the model seeing, and where did that representation first stop meaning what we intended?

The underlying fact is simple and easy to forget:

The model never trains on your abstract notion of the raw sample. It trains on whatever the preprocessing pipeline produced.

That makes preprocessing part of the model’s definition rather than scaffolding around it. Change the pipeline and you have changed the learning problem, even when every line of model code is identical.

Chapter 6 owned delivery. Chapter 7 owns representation.

The environment

Every numerical result in this chapter was produced by an executed experiment in the environment below. The listings show the mechanisms under investigation; repetitive training-harness code is omitted where it would obscure the transform being tested.

PyTorch      2.13.0
TorchVision  0.28.0
Python       3.12.3
OS           Linux, CPU only

TorchVision’s transform API has changed more than most of PyTorch, and some behavior below is version-sensitive. Where that matters, the chapter says so and shows how to check rather than asking you to trust a constant.

Five things a transform can change

Strip away the library and a transform is nothing exotic:

def scale_to_unit_interval(x):
    return x.float() / 255.0

Input, output, done. What makes transforms worth a chapter is not their machinery but the number of independent things they can change — and the fact that only one of those things shows up in a shape.

A model-facing sample has five kinds of property, and this chapter is organized around them, because each one fails differently and each one needs a different kind of evidence:

The path a sample travels looks like this:

raw source
    ↓  decode
representation conversion
    ↓  deterministic preprocessing
resize / scale / tokenize
    ↓  stochastic augmentation
crop / flip / jitter
    ↓  normalization
model-facing tensor

Every arrow is a boundary where the sample can stop satisfying the contract you had in mind, in any of the five categories. The rest of this chapter is about finding the first one, and about the fact that no single instrument finds all five.

The technique: trace one sample, find the first divergence

Each chapter of this book has added an investigation method. Chapter 2 asked for the first wrong tensor rather than the first illegal one. Chapter 3 asked for the first broken edge between a parameter and the loss. Chapter 5 asked which of four structures disagreed about ownership. Chapter 6 asked where useful work stopped flowing.

Chapter 7 adds:

Find the first transform boundary where the sample stops satisfying the intended input contract.

The question is never only “which transform raised?” — a raising transform has already told you where it is. The question is where the data first became semantically wrong while remaining perfectly legal.

Making that executable takes about twenty lines.

import torch
from dataclasses import dataclass

@dataclass(frozen=True)
class Stage:
    name: str; kind: str; shape: tuple; dtype: str
    minimum: float; maximum: float; mean: float; std: float; finite: bool

def describe(name, x):
    if not torch.is_tensor(x):
        raise TypeError(
            f"{name}: stage reporter expects a Tensor or TVTensor, got {type(x).__name__}"
        )
    xf = x.float()
    return Stage(
        name, type(x).__name__, tuple(x.shape), str(x.dtype),
        float(xf.min()), float(xf.max()), float(xf.mean()), float(xf.std()),
        bool(torch.isfinite(xf).all()),
    )

def trace(sample, stages):
    """Run a pipeline one stage at a time, describing the sample after each."""
    x, rows = sample, [describe("raw", sample)]
    for name, t in stages:
        x = t(x)
        rows.append(describe(name, x))
    return x, rows

def report(rows):
    print(f"{'stage':<22}{'kind':<8}{'shape':<16}{'dtype':<10}"
          f"{'min':>9}{'max':>10}{'mean':>9}{'std':>8}")
    for r in rows:
        print(f"{r.name:<22}{r.kind:<8}{str(r.shape):<16}"
              f"{r.dtype.replace('torch.',''):<10}"
              f"{r.minimum:9.3f}{r.maximum:10.3f}{r.mean:9.3f}{r.std:8.3f}")

The key move is that trace takes the pipeline apart. Calling a Compose normally gives you the input and the final output without exposing anything between them. The transforms themselves are inspectable; their intermediate tensors are not automatically reported. Passing the stages separately makes each boundary observable.

Run the two pipelines from the opening on the same sample.

from torchvision.transforms import v2

torch.manual_seed(0)
raw = torch.randint(0, 256, (3, 96, 96), dtype=torch.uint8)
MEAN, STD = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]

good = [("ToImage",            v2.ToImage()),
        ("Resize(64)",         v2.Resize((64, 64), antialias=True)),
        ("ToDtype(f32,scale)", v2.ToDtype(torch.float32, scale=True)),
        ("Normalize",          v2.Normalize(MEAN, STD))]

broken = [("ToImage",      v2.ToImage()),
          ("Resize(64)",   v2.Resize((64, 64), antialias=True)),
          ("ToDtype(f32)", v2.ToDtype(torch.float32, scale=False)),
          ("Normalize",    v2.Normalize(MEAN, STD))]

report(trace(raw, good)[1])
report(trace(raw, broken)[1])

=== intended pipeline ===
stage                 kind    shape           dtype           min       max     mean     std
raw                   Tensor  (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
ToImage               Image   (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
Resize(64)            Image   (3, 64, 64)     uint8        23.000   225.000  127.750  32.216
ToDtype(f32,scale)    Image   (3, 64, 64)     float32       0.090     0.882    0.501   0.126
Normalize             Image   (3, 64, 64)     float32      -1.724     2.117    0.231   0.580

=== same pipeline, scale=False ===
stage                 kind    shape           dtype           min       max     mean     std
raw                   Tensor  (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
ToImage               Image   (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
Resize(64)            Image   (3, 64, 64)     uint8        23.000   225.000  127.750  32.216
ToDtype(f32)          Image   (3, 64, 64)     float32      23.000   225.000  127.750  32.216
Normalize             Image   (3, 64, 64)     float32      98.319   998.196  563.338 142.736

The two runs are identical for three rows and diverge at the fourth. That row is the diagnosis. Everything after it — the enormous normalized range, the loss of 28.6 in the opening, the 0.400 accuracy — is downstream symptom.

Note what the divergence looks like. The dtype column is correct in both. float32 is exactly what was requested and exactly what the model wants. The only column that shows the problem is the value range, and only if you know what range that stage was supposed to produce.

A stage report is evidence about representation. It becomes a diagnosis only when you have written down what each stage was supposed to produce.

What this instrument can and cannot see

Now be honest about the tool that was just built, because the rest of the chapter follows from its limits.

1  STRUCTURE                stage report        type, shape, axis count
2  NUMERIC INTERPRETATION   stage report        dtype, min, max, mean, std
                            + a written-down expectation for each

3  SEMANTICS                NOT VISIBLE         needs a probe you write,
                                                or a known sample you look at
4  RELATIONSHIPS            NOT VISIBLE         needs paired types and
                                                explicit target assertions
5  PROVENANCE               NOT VISIBLE         needs a record: which split,
                                                saved with which checkpoint

The stage report covers two of five categories, and even those two only in combination with an expectation. A tensor cannot tell you that its channels are in the wrong order, that the augmented image now contradicts its label, that the mask describes a mirror image of the picture, or that the normalization constants were fitted on the validation set. None of those is a property of the tensor. All of them are properties of the relationship between the tensor and something outside it.

That is not a weakness of this particular reporter. It is the shape of the problem, and it is why the chapter has four more sections of instruments rather than one.

Categories 1 and 2: representation and numeric interpretation

The reason ToImage and ToDtype are two transforms rather than one is that they answer different questions — the first is category 1, the second is category 2.

img = torch.randint(0, 256, (3, 8, 8), dtype=torch.uint8)

a = v2.ToImage()(img)
b = v2.ToDtype(torch.float32, scale=True)(a)
c = v2.ToDtype(torch.float32, scale=False)(a)

raw:                  Tensor torch.uint8    0  252
ToImage:              Image  torch.uint8    0  252   (3, 8, 8)
ToDtype(scale=True):  Image  torch.float32  0.0  0.988
ToDtype(scale=False): Image  torch.float32  0.0  252.0

ToImage changed the type and nothing else — same dtype, same values, same shape. What it produced is a tv_tensors.Image, a tensor subclass that carries the information “these numbers are an image.” That tag is what lets later transforms behave differently for images, masks and bounding boxes, which becomes essential in category 4.

ToDtype changed the numbers. For this uint8 -> float32 conversion, scale=True implements the same [0,255] -> [0,1] scale convention as division by 255, agreeing to floating-point precision rather than bit-for-bit at every intermediate value:

Do not memorize “scaling means divide by 255.” TorchVision defines an expected numeric range for image dtypes and scale=True converts between those conventions. For the conversion used here, uint8 → float32, that maps [0, 255] to [0, 1], which is effectively division by 255. Other dtype pairs follow their own rules — float-to-integer conversion does involve scaling, float-to-float does not. When either dtype changes, inspect the documented conversion rather than extending the /255 rule by analogy.

There is one asymmetry worth knowing, because it explains why this particular bug survives to production. Getting the order wrong is loud:

v2.Normalize(MEAN, STD)(a)      # a is still uint8

TypeError: Input tensor should be a float tensor. Got torch.uint8.

Getting the scale wrong is silent. Normalize can verify the machine-checkable dtype requirement that its input is floating point. It cannot verify the semantic part of the numeric contract — whether those floats are in the scale assumed by its statistics — because nothing in a float32 tensor records which convention produced them.

PyTorch enforces the properties it can see. Value-scale conventions are not among them.

Normalization is a change of coordinates

Normalization has a reputation as a vaguely beneficial step, which is the wrong mental model. It computes exactly this, per channel:

x' = (x - mean) / std

and nothing more. Verified against the transform:

manual = (x - torch.tensor(MEAN)[:, None, None]) / torch.tensor(STD)[:, None, None]
torch.allclose(v2.Normalize(MEAN, STD)(x), manual, atol=1e-6)      # True

Normalization does not make an image better. It presents the image to the model in a different coordinate system, and the two numbers defining that system belong to a specific representation of the data.

That is the opening failure restated. 0.485 is a plausible red-channel mean for images whose values live in [0, 1]. Applied to values living in [0, 255] it subtracts essentially nothing and then divides by 0.229, multiplying everything by about 4.4:

normalize(scaled)     min=  -2.118  max=    2.640  mean=   0.231  std=  1.289
normalize(unscaled)   min=  -2.118  max= 1136.357  mean= 563.325  std=326.478

Both operations are legal. Both produce finite float32. One presents the intended coordinate system and the other produces values hundreds of times larger than intended.

One clarification that trips people up. After normalizing with dataset statistics, a single image will not have zero mean and unit variance. In the trace above, the normalized image has mean 0.231 and std 0.580, not 0 and 1. That is correct behavior: Normalize applies fixed reference statistics, not statistics recomputed per image. Per-image standardization is a different transform answering a different question, and confusing the two leads people to “fix” a pipeline that was never broken.

Category 3: semantics, which no tensor property can hold

The failures so far were numerical — the numbers stopped meaning what the next stage assumed, and the stage report caught them. Now the harder category: the tensor remains perfectly legal, and stops depicting what the label claims.

Order is a program, not a style choice

Because each transform assumes something about what arrived before it, a pipeline is an ordered program. The usual way this is taught — “transforms are not commutative” — is true and does not convey how quietly the difference can hide.

Two pipelines, same operations, opposite order, applied to an image with a small object left of center:

img = tv_tensors.Image(torch.zeros(3, 64, 128, dtype=torch.uint8))
img[:, 20:28, 24:32] = 255                     # small object, left of centre

A = v2.Compose([v2.Resize((48, 48)),      v2.CenterCrop((32, 32))])
B = v2.Compose([v2.CenterCrop((48, 48)),  v2.Resize((32, 32))])

resize -> centercrop   shape=(3, 32, 32) dtype=torch.uint8 max=255 lit_pixels=120
centercrop -> resize   shape=(3, 32, 32) dtype=torch.uint8 max=  0 lit_pixels=0
outputs identical: False

Same shape. Same dtype. One contains the object and the other is a blank image. Cropping first discards the region that resizing would have brought inside the crop window, and the object was in exactly that region.

The stage report cannot flag this, because a blank image is a perfectly legal image with a perfectly reasonable range. What flags it is asking, at each boundary, what the stage assumes it receives and what it must preserve. CenterCrop assumes the interesting content is near the center. On a wide image where it is not, that assumption is the bug.

Read a pipeline top to bottom asking what representation each stage assumes it is receiving. That is what “order matters” actually means.

Augmentation is a claim about invariance

The crop-order example lost content. The sharper version of category 3 is a transform that loses nothing and still makes the example false: the input stays perfectly meaningful on its own, and its relationship to the target becomes a lie.

Consider classifying arrows by the direction they point.

def arrow(direction):
    """A 1x32x32 uint8 arrow. direction: 'left' (class 0) or 'right' (class 1)."""
    a = torch.zeros(1, 32, 32, dtype=torch.uint8)
    a[:, 15:17, 6:26] = 255                       # shaft
    for k in range(8):                            # head: widens toward the tip
        if direction == "right":
            a[:, 16 - k: 17 + k, 25 - k] = 255
        else:
            a[:, 16 - k: 17 + k, 6 + k] = 255
    return a

def arrow_direction(x):
    """Semantic probe: the head is the end with the greater vertical extent."""
    height = (x[0] > 0).float().sum(0)
    lit = height.nonzero().flatten()
    left_mass  = height[lit[:len(lit) // 3]].mean()
    right_mass = height[lit[-(len(lit) // 3):]].mean()
    return "right" if right_mass > left_mass else "left"

probe on known arrows: left right

Now apply the most standard augmentation in computer vision to a left-pointing arrow labelled class 0:

img = tv_tensors.Image(arrow("left"))
out = v2.RandomHorizontalFlip(p=1.0)(img)

after RandomHorizontalFlip(p=1.0):
  type   : Image
  shape  : (1, 32, 32)   dtype: torch.uint8
  min/max: 0 255
  finite : True
  label  : 0 (still 'left')
  image now depicts: right

Every row the stage report prints is correct. The transform succeeded. The training example is now a picture of a right-pointing arrow labelled “left.”

This is not a hypothetical harm. Training the same small classifier three ways:

no flip                       val_acc = 1.000
RandomHorizontalFlip(p=0.5)   val_acc = 0.458
RandomHorizontalFlip(p=1.0)   val_acc = 0.000

The p=0.5 row is the one people expect: roughly half the labels are now wrong, the two classes become nearly indistinguishable, and the model lands around chance.

The p=1.0 row is more interesting, and worth sitting with. Zero percent. Not chance — perfectly wrong. The model learned the task flawlessly; it learned it from a training set in which every image was mirrored, so it learned the exact inverse of the mapping we wanted, and validation (which is not flipped) scores every single prediction wrong. Successful learning of a consistently mislabelled problem looks exactly like this, and the training-loss curve looked wonderful throughout.

The lesson is not “do not use horizontal flips.” Horizontal flipping is genuinely valid for a great deal of object photography. The lesson is what applying it asserts:

Applying augmentation T claims that meaning(x) ≈ meaning(T(x)) for every aspect the target depends on. That claim is about your task, and no transform library can check it for you.

The claim is false whenever the label depends on the property being changed — direction, handedness, text, anatomical orientation, anything where a mirror image is a different thing. It is also false in less obvious ways: an augmentation that removes small objects breaks the claim for a detection task while leaving it true for scene classification on the same images.

Notice what arrow_direction is. It is the instrument for category 3, and building it is the whole move. Some semantic properties can be computed, and when they can, the question “does this augmentation preserve the label?” stops being a judgement call and becomes an assertion you can put in a test suite. When they cannot be computed, the fallback is a small gallery of known samples and your own eyes — which is not an admission of defeat, it is the only remaining instrument.

Category 4: relationships between objects

The arrow was a relationship failure with a scalar target. Two more forms are worth building instruments for.

The target has a contract too

It is easy to treat the target as though its representation were obvious. Datasets commonly expose two hooks, and the split is exactly what it looks like:

dataset = SomeDataset(..., transform=input_transform, target_transform=target_transform)

The useful discipline is that the target representation is dictated by the objective, not by convention or by whichever example you copied. It is easy to reach for one-hot encoding because a target_transform slot exists and one-hot looks tidy. Check what the loss actually wants first.

For cross_entropy, what it wants is more flexible than the folklore suggests:

logits = torch.randn(4, 3)
idx = torch.tensor([0, 2, 1, 0])
one_hot = F.one_hot(idx, num_classes=3).float()
soft_targets = 0.8 * one_hot + 0.2 / 3

class indices   : 1.082031011581421   torch.int64
class probs     : 1.082031011581421   torch.float32
soft targets    : 1.1632474660873413
float 1-D target: RuntimeError expected target dtype to be Long or Byte, but got Float

So “cross entropy requires integer class indices” is not true. It accepts class indices or per-class probabilities, and one-hot probabilities give the same loss as the corresponding indices to within floating-point error. That second form is what makes label smoothing, distillation and mixup expressible at all.

There is a very Chapter-7 catch in the probability-target form. PyTorch does not verify that those floating-point targets are actually probability distributions:

non-distribution float target still returns a loss: 0.6805554032325745

A tensor of negative values and rows that do not sum to one produced a perfectly ordinary number. Correct shape, correct dtype, loss executes, invalid semantics. If you choose the probability-target form, validating the distribution is your responsibility.

Hand it a 1-D float tensor — one-hot encoding you forgot to reshape, or indices you cast by accident — and it raises, because a 1-D float target matches neither contract.

The target transform must produce the representation the chosen objective requires. Read the loss function’s contract; do not infer it from an example.

For the ordinary class-index representation, the machine-checkable contract is two lines:

assert labels.dtype == torch.int64
assert bool(((0 <= labels) & (labels < num_classes)).all())

If the loss deliberately uses an ignore_index, exclude those positions from the range check. The second line catches an entire genre of bug — labels that are 1-indexed, a class map with a gap, a filtered subset whose IDs were never remapped — before it becomes an indexing error thirty layers away.

Objects that share geometry must share the geometric decision

Detection, segmentation and keypoint tasks add a requirement classification does not have: their targets contain geometry themselves. A crop, resize or flip must update both the input and the geometric target using the same sampled transformation.

The failure mode is calling the transform twice:

f = v2.RandomHorizontalFlip(p=0.5)
a, b = f(img), f(mask)        # two independent random draws

independent calls: image and mask disagreed in  99/200 trials
structured call  : image and mask disagreed in   0/200 trials

Half the samples have a mask describing the mirror image of the picture it is attached to. Both tensors are legal. Both have the right shape. The dataset is now half noise, and the disagreement count is the instrument that says so.

The repair is to make the transform aware that these objects belong together, which is what the tv_tensors types are for:

from torchvision import tv_tensors

img = tv_tensors.Image(torch.zeros(3, 32, 32, dtype=torch.uint8))
img[:, 4:12, 20:28] = 255

mask = tv_tensors.Mask(torch.zeros(32, 32, dtype=torch.uint8))
mask[4:12, 20:28] = 1

boxes = tv_tensors.BoundingBoxes(
    torch.tensor([[20, 4, 28, 12]]), format="XYXY", canvas_size=(32, 32))

flipped_img, flipped_mask, flipped_boxes = v2.RandomHorizontalFlip(p=1.0)(img, mask, boxes)

image bright cols: [4, 5, 6, 7, 8, 9, 10, 11]
mask  cols       : [4, 5, 6, 7, 8, 9, 10, 11]
boxes            : [[4, 4, 12, 12]]

One call, one sampled decision, three consistently updated objects. The box moved from x-range [20, 28] to [4, 12], the exact mirror in a 32-wide canvas, and the mask columns match the image columns.

The dispatch is by type, not by position, and the transforms treat the types differently rather than uniformly:

pipe = v2.Compose([v2.RandomHorizontalFlip(p=1.0),
                   v2.ToDtype(torch.float32, scale=True),
                   v2.Normalize([0.5]*3, [0.25]*3)])
i2, m2 = pipe(img, mask)

image: Image torch.float32 range=[-2.00, 2.00]
mask : Mask  torch.uint8   unique=[0, 1]

The image was flipped, converted and normalized. The mask was flipped and left otherwise alone — still uint8 with values {0, 1}, because normalizing a class-index mask would be meaningless. That is why ToImage exists and why the tag it attaches matters: a bare tensor has no way to say “I am a segmentation mask, do not rescale me.”

Anything that shares geometry must share the geometric decision. Express the relationship in the type so the transform can honor it, rather than hoping two random calls agree.

The principle outlives the API. Whatever library you use, the question is the same: is this one sampled transformation applied consistently, or several independent ones applied hopefully?

Stochastic transforms need a distribution, not a sample

Randomness cuts across the categories rather than forming one of its own, and it defeats every instrument built so far in the same way: one execution of a random transform tells you almost nothing. The transform is not a value; it is a distribution over values, and the properties you care about are properties of the distribution.

Take the small object from the order example and ask how often a random crop keeps it:

img = tv_tensors.Image(torch.zeros(3, 64, 128, dtype=torch.uint8))
img[:, 8:14, 10:16] = 255

for scale in [(0.08, 1.0), (0.5, 1.0), (0.9, 1.0)]:
    rrc = v2.RandomResizedCrop(size=(32, 32), scale=scale, antialias=True)
    kept = sum(float((rrc(img) > 0).any()) for _ in range(500))
    print(f"scale={scale} object retained in {kept/500:.1%} of crops")

  scale=(0.08, 1.0)  object retained in  16.8% of crops
  scale=(0.5, 1.0)   object retained in  16.6% of crops
  scale=(0.9, 1.0)   object retained in   0.0% of crops

Two findings, and the second is not the one anybody predicts.

The first: with default settings, more than eighty percent of training views of this image contain no object at all. If the label says “object present,” four out of five augmented examples contradict it — the arrow failure again, arriving through a different door.

The second: the gentlest setting retained the object zero percent of the time. That is backwards, so it needs an explanation rather than a shrug. RandomResizedCrop samples an area fraction and an aspect ratio, retries a bounded number of times if the sampled box does not fit, and falls back to a deterministic center crop when sampling fails. With the default aspect-ratio range (3/4, 4/3), a crop covering at least 90% of a 64×128 image cannot fit inside the 64-pixel height: even the widest allowed ratio is not wide enough. Every random proposal is therefore invalid and the fallback becomes deterministic. Inspecting the sampled parameters rather than only the outputs shows it directly:

for scale in [(0.08, 1.0), (0.5, 1.0), (0.9, 1.0)]:
    rrc = v2.RandomResizedCrop(size=(32, 32), scale=scale)
    boxes = [tuple(rrc.make_params([img]).values()) for _ in range(300)]
    print(scale, "distinct crops:", len(set(boxes)))

(0.08, 1.0)  distinct crops: 300
(0.5, 1.0)   distinct crops: 186
(0.9, 1.0)   distinct crops: 1

At scale=(0.9, 1.0) the “random” transform produced one crop, three hundred times. It is not augmenting anything, and it always excludes the corner where the object lives.

You cannot reason your way to that from the parameter names. You get it by sampling the transform and looking at the distribution — of outputs, and where the API exposes it, of the parameters themselves.

Inspect a stochastic transform by running it many times on a known sample and asking what fraction of the distribution still satisfies the contract. “It ran and looked fine once” is not evidence.

And choose the sample deliberately. Look at known samples picked because you know what must survive: the smallest object, the rare class, the example near the crop boundary, the image whose label depends on text. Random samples from a large dataset will almost never surface the case that breaks.

Category 5: provenance

The last category has no tensor signature at all. Two numbers can be exactly right and still be evidence of a broken experiment, depending entirely on where they came from.

Fitted preprocessing is experiment state

Sometimes normalization statistics come from a pretrained model’s contract. Sometimes you compute them from your own data — and the moment you do, the transform has acquired state derived from data, which changes what it is.

def channel_stats(u8_batch):
    """Population channel mean/std over a [N,C,H,W] uint8 stack, in [0,1] units."""
    x = v2.ToDtype(torch.float32, scale=True)(v2.ToImage()(u8_batch))
    return (x.mean(dim=(0, 2, 3)).tolist(),
            x.std(dim=(0, 2, 3), correction=0).tolist())

Here training data was collected under one lighting condition and validation later under a brighter one — an entirely ordinary situation:

fitted on train only     mean= [0.3706, 0.3709, 0.3707]  std= [0.0792, 0.0792, 0.0793]
fitted on train+val      mean= [0.4488, 0.4491, 0.4490]  std= [0.1468, 0.1470, 0.1470]
shift in channel-0 mean : 0.0782

Fitting on everything, because all the files were sitting there, moved the statistics by nearly eight percentage points. The training procedure now depends on a summary of the validation distribution. That validation set is no longer independent of preprocessing selection: its metric measures a pipeline that was allowed to use information derived from that same evaluation split.

That may be a legitimate transductive design in a deliberately defined experiment. It is not a clean held-out validation protocol. Nothing raised, and the leak is invisible in every tensor the model receives — which is exactly why provenance is its own category.

If preprocessing state is estimated from data, fit it using only data the training procedure is allowed to know.

The same shape appears wherever preprocessing carries fitted state: tabular scalers, categorical vocabularies, token vocabularies, quantization calibration. All are estimated from a split, and all leak if estimated from the wrong one.

The second half of the principle matters as much as the first. A fitted transform is part of the experiment, so it has to survive:

state = {"scale": "unit_interval", "mean": m_tr, "std": s_tr, "fit_split": "train"}

{"scale": "unit_interval", "mean": [0.3706091344356537, ...], "std": [0.0792376846...

Save it with the checkpoint, or record how to reconstruct it exactly. A model trained under one preprocessing state and served under another is not the same system, however cleanly the weights load.

Pretrained weights and preprocessing are one contract

There is an important case where you should not casually replace preprocessing with statistics from your own data: when you begin from pretrained weights. Those weights were optimized under a particular input representation, so the preprocessing shipped with them is the correct contract when you want to reproduce the published behavior.

Fine-tuning can intentionally change that representation, and the model may adapt. But that is an experimental change to the transfer-learning setup, not an interchangeable detail. Measure it rather than assuming dataset-specific statistics are automatically better.

The habit worth building is to stop copying constants out of blog posts. TorchVision ships the preprocessing alongside the weights:

from torchvision.models import ResNet50_Weights, Inception_V3_Weights
print(ResNet50_Weights.IMAGENET1K_V1.transforms())

ResNet50_Weights.IMAGENET1K_V1        ResNet50_Weights.IMAGENET1K_V2
   ImageClassification(                  ImageClassification(
    crop_size=[224]                       crop_size=[224]
    resize_size=[256]                     resize_size=[232]
    mean=[0.485, 0.456, 0.406]            mean=[0.485, 0.456, 0.406]
    std=[0.229, 0.224, 0.225]             std=[0.229, 0.224, 0.225]
    interpolation=BILINEAR                interpolation=BILINEAR
)                                     )

Inception_V3_Weights.IMAGENET1K_V1
   ImageClassification(
    crop_size=[299]  resize_size=[342]
    mean=[0.485, 0.456, 0.406]  std=[0.229, 0.224, 0.225]
)

Read the first two together. Same architecture. Same normalization constants. Different resize_size: 256 for V1, 232 for V2. Two sets of weights for one model, with different preprocessing contracts.

Anyone who memorized “ResNet50 uses mean [0.485, 0.456, 0.406], resize 256, crop 224” gets the constants right and the resize wrong for half the available checkpoints. The consequence is measurable:

V1 preset: (3, 224, 224) torch.float32 mean=0.2286
V2 preset: (3, 224, 224) torch.float32 mean=0.2276
identical: False   max abs diff: 2.941176414489746

Identical shape, identical dtype, nearly identical mean — and individual values differing by nearly three units in normalized space, because the two presets resampled from different source scales.

The Inception row makes a second point: 299 and 342, not 224 and 256. There is no universal image size, just as there is no universal normalization constant.

Model weights and model preprocessing are one compatibility contract. A checkpoint can load with every key matched while inference is quietly broken.

The proof that metadata is not enough

Every category has now been demonstrated. Here is the experiment that shows why the split was necessary, rather than a taxonomy for its own sake.

A model trains to perfect accuracy on a task where the class is determined by which color channel is bright. Serving code then does what serving code does — reads images with a library that returns channels in BGR order:

load_state_dict: <All keys matched successfully>
  serving uses RGB (as trained)  shape=(300, 3, 16, 16) dtype=torch.float32 range=[-0.69, 0.80] val_acc=1.000
  serving uses BGR               shape=(300, 3, 16, 16) dtype=torch.float32 range=[-0.69, 0.80] val_acc=0.000

Identical shape. Identical dtype. Identical range, to two decimals. Identical every field the stage report prints. Accuracy 1.000 and 0.000.

No amount of metadata inspection separates those two tensors, because the difference is which channel index means red, green or blue. That convention is not recorded in an ordinary tensor: both tensors expose the same channel axis, shape, dtype and numeric range. This is Chapter 2’s lesson — legal is not the same as correct — arriving with a much larger blast radius.

Now widen it, because the interesting result is what happens to the other variants of the same drift:

  matching preprocessing    range=[  -0.69,   0.80]  val_acc=1.000
  channel-swapped (BGR)     range=[  -0.69,   0.80]  val_acc=0.000
  forgot to scale           range=[  79.00, 457.00]  val_acc=1.000
  different normalization   range=[  -1.43,   2.19]  val_acc=1.000

Only one of the three mismatches was fatal, and it is the only one the stage report could not see. The two that the range column would have flagged happened to be survivable for this model on this synthetic task. Do not turn that observation into a general invariance claim: different learned weights, nonlinearities, normalization layers or channel-specific transforms can make the same numeric drift destructive.

Do not read that as a ranking. It does not mean scale bugs are harmless — the chapter opened with one costing 60 points of accuracy under SGD. It means the severity of a representation bug and the visibility of a representation bug are unrelated quantities, which is the same point the opening made from the other direction.

So contracts come in two levels, and it is worth being honest about which is which:

Write the machine-checkable half as code, because it is free and it catches an enormous class of errors before the first optimizer step:

def require_model_image(x, size=(3, 224, 224), value_bounds=(-5.0, 5.0)):
    if tuple(x.shape) != size:
        raise ValueError(f"expected {size}, got {tuple(x.shape)}")
    if x.dtype != torch.float32:
        raise TypeError(f"expected float32, got {x.dtype}")
    if not torch.isfinite(x).all():
        raise ValueError("non-finite values in image")

    low, high = value_bounds
    if not (low < float(x.min()) and float(x.max()) < high):
        raise ValueError(
            f"values outside expected range {value_bounds}: "
            f"[{x.min():.2f}, {x.max():.2f}]"
        )

That last check is the one that would have caught the opening failure, and it is the one people leave out. It encodes an expectation about numeric interpretation rather than structure. It is a heuristic, not a proof — but it fires on [98, 998] and stays quiet on [-2.1, 2.6], which is exactly the discrimination we could not otherwise make automatically. The bounds belong to this specific ImageNet-style normalization contract; they are not universal bounds for normalized tensors.

For the semantic half, write probes where the property is computable (arrow_direction is one), and otherwise keep a small gallery of known samples and look at them.

If the pipeline normalizes, build a display copy rather than inspecting the training tensor, and invert the normalization to do it:

def denormalize(x, mean=MEAN, std=STD):
    m = torch.tensor(mean, device=x.device)[:, None, None]
    s = torch.tensor(std, device=x.device)[:, None, None]
    return (x * s + m).clamp(0, 1)

round-trip max error vs pre-normalized: 5.960464477539063e-08

The round trip recovers the pre-normalization image to floating-point precision, so what you are looking at is the actual sample rather than an approximation. Note the clamp: it exists for display only, and applying it to a training tensor would silently discard values the model is supposed to see.

Testing a pipeline

The two-level distinction determines what a test can assert, and the common mistake is testing stochastic transforms for the property deterministic ones have.

A deterministic pipeline should be reproducible, and that is worth asserting because it is the property that breaks when someone adds an augmentation to the wrong list:

a, b = val_tf(raw), val_tf(raw)
torch.testing.assert_close(a, b)          # val_tf deterministic: True

A stochastic pipeline must not be tested that way. Its outputs are supposed to differ; what should not differ are the invariants:

train_tf identical across draws: False
train_tf shape invariant: {(3, 64, 64)}
train_tf dtype invariant: {torch.float32}
train_tf all finite: True

Variation in the values, none in the contract. That is the shape of a good augmentation test: assert over a sample of draws that structure, dtype, finiteness, target validity and paired-geometry alignment all hold, and assert nothing about the values themselves.

The third test is the one people never write: run both pipelines on the same sample and compare their contracts directly.

train_tf (scale=True)  train=((3, 64, 64), torch.float32, -1.8,    2.2)
                       val  =((3, 64, 64), torch.float32, -1.7,    2.1)
                       shape/dtype agree: True   magnitudes comparable: True

train_tf (scale=False) train=((3, 64, 64), torch.float32, 72.1, 1024.8)
                       val  =((3, 64, 64), torch.float32, -1.7,    2.1)
                       shape/dtype agree: True   magnitudes comparable: False

The first assertion passes in both cases. Structure was never the problem, and a test checking only shape and dtype would have signed off on the opening failure without complaint.

The second assertion catches drift between these two pipelines. It does not establish that either pipeline is correct. If both training and validation accidentally used scale=False, their magnitudes would agree and this comparison would pass while both violated the model contract.

So use cross-pipeline comparison for consistency, and compare each pipeline independently against the explicit representation the model expects. Agreement is necessary; it is not sufficient.

Three pipelines, one contract

A project usually has three preprocessing paths, and the discipline is knowing exactly where they may differ.

                       may differ          must agree
training      stochastic augmentation   final model-facing representation
validation    deterministic             final model-facing representation
inference     deterministic             final model-facing representation

Training may inject randomness because variation is part of what it is learning from. Validation is usually deterministic because its job is to be a stable measuring instrument — if the benchmark changes between passes, a metric movement no longer distinguishes a better model from a different measurement. That is a default rather than a law: test-time augmentation is a legitimate technique. The principle is that evaluation randomness must be deliberate and interpreted as part of the evaluation procedure, rather than inherited by accident because someone passed train_transform to both datasets.

What must agree across all three is the final representation: channel count and order, dtype, value scale, normalization state, and the spatial contract the model was built for.

The inference path is the one that rots, because it usually lives in a different file, often a different repository, and sometimes a different language. Chapter 5 showed a checkpoint that loaded with every key matched and returned a worthless model. This is the same failure with the state moved outside the checkpoint: the weights are perfect and the input convention is not.

The remedy is to treat the deterministic preprocessing definition as part of the model artifact — serialized beside the weights, or reconstructed from a recorded configuration — rather than as code that happens to exist in the training script. TorchVision’s weights.transforms() is exactly this idea applied to published models, and it is a good pattern to copy for your own.

Where transforms run, briefly

A transform applied inside Dataset.__getitem__ is part of sample production. With num_workers > 0 it executes inside worker processes, which means transform cost is producer cost in exactly the sense Chapter 6 measured. That is the whole of the connection, and it is deliberately brief because Chapter 6 owns the measurement.

Two consequences are worth stating here.

Deterministic and stochastic work have different placement economics. Deterministic preprocessing whose output depends only on stable input and stable configuration is a candidate for precomputation or caching. If fresh stochastic variation is part of the training design, keep that stage after the cache boundary — you can cache pre-generated augmented views, but doing so changes the training distribution from fresh draws to a fixed finite set:

raw → expensive deterministic preprocessing → CACHE → random augmentation → model

Cache invalidation then becomes a correctness problem, and a category-5 one: change the transform definition and a stale cache silently preserves the old behavior, which is a representation bug that survives a code fix. Version the cache or rebuild it deliberately.

Debug one sample before debugging a pipeline. When something looks wrong, do not start with eight workers, prefetching, random transforms, collation and device transfer all interacting:

raw = dataset_without_transform[index]
x = transform(raw)

One sample, one process, no batching. Chapter 6 made the same argument for a different reason — num_workers=0 to expose a worker traceback — and it applies here for a stronger one: a stochastic transform inside a worker process is harder to isolate and reproduce, while calling it directly on one chosen sample removes several unrelated sources of complexity. And control the randomness enough to reproduce the failure: seed a generator, or replace the random choice with a fixed one (p=1.0 rather than p=0.5, as the arrow experiment did). A transform you cannot reproduce is a transform you cannot debug.

The same model, beyond images

Images dominate this chapter because they make semantic failure visible — you can see that the arrow points the wrong way. The five categories are not specific to images.

Every row has the same columns, and every failure in this chapter has an analogue in every row. A padding token treated as a real token is the text version of a mask that no longer aligns — category 4. A vocabulary fitted on train-plus-test is the text version of leaked normalization statistics — category 5. A tokenizer that differs between training and serving is the text version of BGR at inference — category 3, with token IDs of perfectly valid shape and dtype representing a different mapping from text to model input.

Every model receives a representation produced by an ordered preprocessing program. The domain changes the operations; it does not change the questions.

Using AI on a transform pipeline

Ask an assistant whether a transform pipeline is correct and you may get a fluent answer based only on the code. But code alone is often insufficient here, because the missing facts include what the raw data represents, what the target means, and which preprocessing contract the model was trained under — categories 3, 4 and 5, none of which are in the source.

The move is the one this book keeps using: make it characterize before it rewrites.

Here is a PyTorch transform pipeline and a description of one known raw sample.
Do not rewrite the pipeline and do not propose fixes yet.

First, build a table with one row per transform:
  - the input representation it assumes (type, shape, dtype, value scale);
  - the output representation it should produce;
  - which properties it intentionally changes;
  - which properties must remain invariant;
  - whether it introduces randomness;
  - whether it changes geometry;
  - whether the target must change with it.

I will then give you the observed type, shape, dtype, min, max, mean and std
after every stage, plus the label and what the sample depicts.

Identify the FIRST stage at which the observed representation becomes
inconsistent with the contract in your table.

Then answer separately:
  1. Is the failure structural, numerical, semantic, relational or provenance?
  2. Which of those can my stage report detect, and which cannot?
  3. What is the smallest experiment that would confirm that diagnosis rather
     than the two most likely alternatives?
  4. What measurement should change after the repair?
  5. What measurement must NOT change after the repair?

Only after I report the result of that experiment should you propose a change.

Question 5 is the one to keep. It is what stops a repair from becoming a rewrite: if fixing a value-scale bug also changes the output spatial size, something was replaced that should not have been.

There is a specific weakness worth exploiting. Assistants are good at the transform catalogue and much weaker at your task semantics, because the semantics are not in the code. So do not ask “is RandomHorizontalFlip appropriate here?” Ask:

My labels are: 0 = arrow points left, 1 = arrow points right.
For each transform in this pipeline, state whether it can change the property
that determines the label, and give me a probe I can run on one sample to
check whether it did.

That question has a checkable answer, and the probe is worth more than the opinion.

Ask AI to characterize the pipeline before asking it to rewrite the pipeline.

The transform debugging sequence

 1. Choose ONE known raw sample — one where you know what the answer
    should be. Prefer an edge case: the small object, the rare class,
    the directional one.

 2. Bypass concurrency. No DataLoader, no workers, no batching.
    raw = dataset_without_transform[i]; x = transform(raw)

 3. Write down the intended final contract BEFORE looking at anything:
    shape, dtype, channel order and meaning, value scale,
    normalization state.

 4. Run the pipeline one stage at a time and describe each output:
    type, shape, dtype, min, max, mean, std, finite.                    cat 1-2

 5. Compare each stage against what it was supposed to produce.
    The FIRST divergence is the diagnosis; everything after is symptom.

 6. If nothing diverged, the failure is in a category the report
    cannot see. Go to steps 7-10 rather than concluding it is fine.

 7. Verify the target still describes the transformed input.
    Use a probe if the property is computable; look at it if not.       cat 3

 8. Check paired objects: mask, boxes, keypoints, second modality.
    Measure the disagreement rate over many draws.                      cat 4

 9. If the stage is stochastic, run it many times on the same sample
    and inspect the DISTRIBUTION, including sampled parameters.         cat 3-4

10. Check the provenance of any fitted state: which split produced it,
    and is it saved with the model?                                     cat 5

11. Compare the final contracts of train, validation and inference.
    They may differ in randomness. They must agree in representation.

12. Only now reintroduce batching, workers and performance measurement,
    and hand any remaining slowness back to Chapter 6.

Steps 1 through 5 are deliberately cheap. Step 6 is the one that matters most: an all-green stage report is not a clean bill of health, it is a clean bill of health for two of the five categories.

What you should now be able to answer

Here is a pipeline of the kind you will be handed. It runs. Work through it before reading on.

MEAN, STD = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]

train_tf = v2.Compose([
    v2.ToImage(),
    v2.RandomResizedCrop((224, 224), scale=(0.8, 1.0)),
    v2.RandomHorizontalFlip(p=0.5),
    v2.ToDtype(torch.float32),
    v2.Normalize(MEAN, STD),
])

val_tf = v2.Compose([
    v2.ToImage(),
    v2.Resize((256, 256)),
    v2.CenterCrop((224, 224)),
    v2.ToDtype(torch.float32, scale=True),
    v2.Normalize(MEAN, STD),
])

weights = ResNet50_Weights.IMAGENET1K_V2

# ArrowDataset returns three-channel RGB images and labels: 0 = left, 1 = right.
train_ds = ArrowDataset(train_paths, transform=train_tf)
val_ds   = ArrowDataset(val_paths,   transform=val_tf)

model = resnet50(weights=weights)
model.fc = nn.Linear(model.fc.in_features, 2)

Where is the first divergence in train_tf, and which category is it? At ToDtype(torch.float32). scale defaults to False, so this produces float32 values in [0, 255] and the subsequent Normalize applies the wrong coordinate system. Category 2, and the stage report shows it as a value range of roughly [98, 998] where [-2.1, 2.6] was intended. The dtype column is correct throughout, which is why nobody notices.

Is that the only problem? No, and this is the point of the technique. Fixing the first divergence does not end the investigation; it lets you inspect the next contract. The remaining questions are not all visible to the stage report: the flip is semantic, the pretrained resize is provenance/compatibility, and the stochastic crop still needs a distribution-level test before we can call it safe.

What is wrong with the flip? The labels encode direction, so meaning(x) and meaning(flip(x)) are different classes. Category 3. At p=0.5 roughly half the training examples become contradictory; the measured consequence in this chapter was validation accuracy falling from 1.000 to 0.458. Nothing raises, and the training loss curve looks unremarkable.

How does the validation preprocessing differ from the pretrained weights’ contract? ResNet50_Weights.IMAGENET1K_V2 uses resize_size=232, while val_tf uses 256. Category 5. If the intention is to reproduce the published evaluation behavior, use weights.transforms() rather than reconstructing the contract by hand. If the model has been fine-tuned under a deliberately modified regime, the difference is not automatically a bug — it is an experimental choice whose validation and inference pipelines must agree and whose effect should be measured.

Do the two pipelines end at a compatible representation? After the scale bug is fixed, yes: both produce [3, 224, 224] float32 normalized with the same statistics. Before the fix, no — training sees roughly [98, 998] and validation sees [-2.1, 2.6], so the validation metric measures the model on inputs it was never trained on. Checkable with a two-line assertion on one batch from each loader.

Is scale=(0.8, 1.0) a safe augmentation here? Unknown from the code, and it needs a distribution check rather than an opinion. Run the crop several hundred times on a known sample and count how often the arrow head survives. This chapter found a case where scale=(0.9, 1.0) collapsed to a single deterministic crop, so inspect the sampled parameters too, not just the outputs.

If the targets were masks or boxes rather than class labels, what else breaks? The geometric transforms would have to be applied to the target with the same sampled parameters. Category 4: calling the transform separately on image and target produced disagreement in 99 of 200 trials under p=0.5. The repair is to wrap the target in the appropriate tv_tensors type and pass both to one call.

The model trains to 99% and production predictions are nonsense. Where do you look first? Not the model. Compare the inference preprocessing against the training contract field by field: channel order, resize policy, value scale, normalization statistics. The chapter’s BGR experiment produced identical shape, dtype and value range with accuracy going from 1.000 to 0.000, so the comparison has to be of the contract, not of the tensor metadata.

What evidence proves a preprocessing repair worked? Not that the loss fell — the opening showed a broken pipeline reaching zero loss under Adam. A stage report whose every row matches the intended contract, a validation batch and a training batch agreeing on the final representation, a probe confirming the target still describes the transformed input, and end-to-end accuracy on a held-out set that was never used to fit any preprocessing state.

Exercises

1. Trace and diverge. Build the stage reporter and run it on a four-stage pipeline. Introduce each of these one at a time and record which row first diverges: dropping scale=True, normalizing twice, resizing after normalizing, converting to float before resizing. For each, say which of the five categories it belongs to, and whether the report could see it at all.

2. Optimizer masks the bug. Reproduce the opening comparison under both SGD and Adam. Record the first-epoch loss, final loss and validation accuracy for all four combinations, and explain why the same representation bug has such different visibility.

3. Verify the scaling. Confirm that ToDtype(torch.float32, scale=True) on uint8 implements the same [0,255] -> [0,1] convention as division by 255 to floating-point precision, then find out what it does to int16 and to a float32 input already in [0, 1]. Predict each before running.

4. Build a semantic probe. Write arrow_direction, verify it on known left and right arrows, then use it as a test: assert that a proposed augmentation preserves direction. Find one augmentation it passes and one it fails.

5. Distribution, not sample. For a stochastic transform of your choice, run 500 draws on a known sample and report the fraction satisfying your contract. Then inspect the sampled parameters and find a setting where the transform is effectively deterministic.

6. Break the pairing. Transform an image and its mask with two independent calls and measure the disagreement rate. Repair with tv_tensors and confirm it goes to zero. Then do the same with bounding boxes and verify the coordinates against hand-computed mirrors.

7. Fit on the wrong split. Compute normalization statistics from train-only and from train-plus-validation on data where the two splits differ. Report the shift, then argue what the validation metric now means.

8. Sort the drift by category. Train a small model on a color-dependent task, save it, and evaluate under four preprocessing variants: correct, channel-swapped, unscaled, differently normalized. Record shape, dtype, range and accuracy for each. Identify which mismatches the stage report could have caught, which were actually fatal, and note that these are not the same set.

Next: inside the model

The representation is now something you can interrogate rather than assume. A transform is a function with a contract covering five categories, a pipeline is an ordered program in which each stage assumes something about what the previous one produced, and when a pipeline misbehaves the move is to trace one known sample stage by stage and find the first boundary whose output stops satisfying the contract.

The five categories are worth carrying forward as a set, because each one needed a different instrument:

1  STRUCTURE                stage report              shape, type, axis count
2  NUMERIC INTERPRETATION   stage report + expected   the scale bug
                            range
3  SEMANTICS                a probe, or your eyes     the arrow, the crop
                                                      order, BGR at serving
4  RELATIONSHIPS            paired types, target      mask and image
                            assertions, disagreement  disagreeing 99/200
                            counts over many draws
5  PROVENANCE               a record of which split   statistics fitted on
                            and which checkpoint      train+val; resize 232
                                                      vs 256

Only the first two leave directly inspectable evidence in the tensor itself, and even category 2 becomes useful only when you compare the observed numbers with an expected numeric contract. That distinction explains the two results that bracket the chapter: a scale bug that was catastrophic under SGD and hidden by Adam, and a serving drift where the fatal variant was the one ordinary tensor metadata could not distinguish.

A stage report can verify the tensor properties you chose to measure. It cannot prove the tensor means what you intended. Semantics, relationships and provenance need an instrument you built on purpose.

That completes the path from stored bytes to the model boundary:

raw source → decode → representation → augmentation → normalization → batch → model

Chapter 6 verified that the batch arrives on time. Chapter 7 verified that the tensor inside it means what we intended. Both stop at the same place — the moment model(x) is called.

So we now have much stronger evidence for the tensor at the model boundary: [B, 3, 224, 224], float32, normalized under a contract we can state and test, with axis meaning made explicit and semantic checks applied where metadata alone was insufficient. Chapter 4 built a network that consumed a flat vector of features, and Chapter 5 organized it. Neither did anything with the fact that two of those axes are spatial — that pixel (h, w) is adjacent to (h, w+1) in a way that pixel 300 is not adjacent to pixel 301 in a flattened vector.

A convolutional network is the architecture that takes that adjacency seriously, and it does so by transforming the channel and spatial axes in ways that are entirely mechanical and entirely easy to get wrong. The next chapter asks what happens to [B, C, H, W] once it crosses the model boundary: what convolution does to channels and geometry, how output sizes are determined rather than guessed, and why a channel-count error can originate in a mistaken spatial or axis interpretation.

Into the network.