Gpu on AIBussin — AI applications, systems and books

Run Cellular Automata on the GPU

Mon, 10 Aug 2026 20:54:00 +0100

Once the update is expressed as tensor operations, moving to a GPU becomes straightforward.

But a GPU is not automatically faster.

It wins when there is enough parallel work to amortize transfer and launch overhead.

A tensor implementation of Life

import torch
import torch.nn.functional as F

LIFE_KERNEL = torch.tensor(
 [[1.0, 1.0, 1.0],
 [1.0, 0.0, 1.0],
 [1.0, 1.0, 1.0]]
).view(1, 1, 3, 3)


def life_step(x):
 padded = F.pad(x, (1, 1, 1, 1), mode="circular")
 neighbors = F.conv2d(padded, LIFE_KERNEL.to(x.device))
 alive = x > 0.5
 born = neighbors == 3
 survive = alive & (neighbors == 2)
 return (born | survive).float()

Two things matter here beyond the convolution. First, this is the third life_step in the book (after Chapters 6 and 51) — same B3/S23 rule, new backend, bridged explicitly rather than silently redefined. Second, the padding mode is load-bearing: plain padding=1 zero-pads, which silently changes periodic boundaries into dead ones. Circular padding preserves the torus semantics every roll-based helper in this book assumes — verified bit-identical against the NumPy version on 20 random grids, where the zero-padded variant mismatched all 20 at the edges.

PyTorch Performance Debugging: CUDA OOM, Slow Training, GPU Utilization and torch.compile

Sat, 08 Aug 2026 13:56:00 +0100

PyTorch: Zero to Hero — Step 09

At this point in the series, the model runs.

That does not mean it runs well.

A training loop can be correct and still waste most of the machine.

A model can fit in memory and still spend half its time waiting on synchronization.

A torch.compile call can make code faster, slower, or simply move the bottleneck somewhere else.

A CUDA out-of-memory error can be caused by the model, the optimizer, activations, fragmentation, a leaked reference, a larger batch, a longer sequence, or an innocent-looking tensor that was kept alive by Python.

PyTorch DataLoader Performance: num_workers, pin_memory, Prefetching and Why Your GPU Is Waiting

Sat, 08 Aug 2026 13:21:00 +0100

PyTorch: Zero to Hero — Step 05

A fast model with a slow input pipeline is still a slow training system.

One of the most common PyTorch performance failures looks like this:

GPU utilization: 20% → 95% → 10% → 90% → 15%

The model is not necessarily slow.

The GPU may simply be waiting for the next batch.

This article is about finding out where the wait is happening.