TPU and GPU architecture, side by side

Tick each objective you can deliver out loud without notes. Then write the one-paragraph explanation you would give a room of GPU engineers. Saved attempts stay below, so you can compare them over the week.

• I can state the design bet behind GPUs, TPUs and Trainium, and what each gives up.
• I can compare current chips quantitatively — compute per precision, HBM capacity and bandwidth, ridge point and on-chip memory — for Ironwood, TPU 8t and 8i, H100, B200, GB200, Rubin, Trainium2 and Trainium3.
• I can compare scale-up and scale-out designs, and derive their consequences for AllGather and AllToAll and for training thresholds.
• I can compare the programming models — kernels, compilation, PyTorch paths and partitioning.
• I can say, for a given workload, which design has the structural advantage, and why.
• I can name the 2025–26 convergence trends and the evidence for each.

Prompts to answer out loud

1. In five minutes, explain to a room of CUDA engineers what a TPU gives up relative to a GPU, what it gets in return, and how to tell which side of that trade their workload is on.
2. Why can a TPU pod reach about 9,000 chips without packet switches while NVLink stops at 72, and what does each topology make cheap or expensive?
3. Blackwell added Tensor Memory and single-thread MMA issue. Argue whether GPUs are converging on TPU design, with the evidence for and against.
4. For an MoE model with 256 experts, which interconnect would you want for training, and why?