Sources
Every source cited in the course, all accessed 2026-10-23. Generated from research/sources.md, which also records the precedence rule for conflicting numbers and each source's verification status.
. Spine: the Scaling Book
Papers
- P-v1Jouppi et al., "In-Datacenter Performance Analysis of a Tensor Processing Unit"
- P-v23Jouppi et al., "A Domain-Specific Supercomputer for Training Deep Neural Networks"
- P-NorrieNorrie et al., "The Design Process for Google's Training Chips: TPUv2 and TPUv3"
- P-v4iJouppi et al., "Ten Lessons From Three Generations Shaped Google's TPUv4i"
- P-v4Jouppi et al., "TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings"
- P-Micro26Jouppi, Patterson, Lakshmanamurthy, Young, "Google's Training Supercomputers from TPU v2 to Ironwood…"
- P-LightwaveLiu et al., "Lightwave Fabrics: At-Scale Optical Circuit Switching for Datacenter and ML Systems"
- P-ApolloPoutievski et al., "Mission Apollo: Landing Optical Circuit Switching at Datacenter Scale"
- P-ResilZu et al., "Resiliency at Scale: Managing Google's TPUv4 Machine Learning Supercomputer"
- P-PathwaysBarham et al., "Pathways: Asynchronous Distributed Dataflow for ML"
- P-GSPMDXu et al., "GSPMD: General and Scalable Parallelization for ML Computation Graphs"
- P-CollMMWang et al., "Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models"
- P-RPAJiang, Chen, Hechtman, Zhang, Mu, "Ragged Paged Attention"
- P-ESTIPope et al., "Efficiently Scaling Transformer Inference"
Google Cloud TPU documentation
JAX/Pallas hardware table
Secondary hardware material
Reference implementations and serving
GPU parallelism and collectives literature