TPU lineage and roofline thinking

Tick each objective you can deliver out loud without notes. Then write the one-paragraph explanation you would give a room of GPU engineers. Saved attempts stay below, so you can compare them over the week.

• I can explain each TPU generation, from v1 to 8t and 8i, as a response to a named bottleneck.
• I can compute the math time, the communication time and the ridge point for any chip from its spec sheet, and state which regime a kernel is in.
• I can derive that a bf16 matmul's arithmetic intensity is about its token batch, and give the compute-bound batch for v5e, Ironwood, H100 and TPU 8i.
• I can show why int8 or fp8 weights with bf16 math halve the compute-bound batch, and why fp8 math doubles it again.
• I can apply the same roofline to an interconnect link and derive when a sharded matmul is communication-bound.
• I can name what the HBM roofline leaves out (VPU-bound ops, latency-bound transfers, achievable fraction of peak) and when each matters.

Prompts to answer out loud

1. Your H100 intuition says a decode step at batch 64 is memory-bound. Show on a whiteboard whether that holds on Ironwood, and at what batch it stops holding.
2. Why did Google build an inference-only chip first, then a training chip, then split into two chips again in 2026? Name the bottleneck each time.
3. Why does int8 weights with bf16 math lower the compute-bound batch, while int8 math does not?
4. From TPUv2 to Ironwood, FLOPs per chip grew 50× and HBM bandwidth 10.5×. What does that do to the batch you need, and what did Google build to compensate?