TPU Deep Dive
Course
Review
Slides
Exams
Sources
Theme: system
Sources
SB2
How to Think About TPUs
What is a TPU
SB2
How to Think About TPUs
Appendix A: VPU, scalar core
SB2
How to Think About TPUs
Appendix B: systolic arrays
SB2
How to Think About TPUs
Worked problems
P-Norrie
Norrie et al., "The Design Process for Google's Training Chips: TPUv2 and TPUv3"
scalar, vector and matrix units
P-v4
Jouppi et al., "TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings"
§2.2 chip, §3 SparseCore
D-arch
System architecture
D-7x
TPU7x (Ironwood)
J-hw
Pallas "TPU Hardware Reference"
X-tiled
Tiled layout
PL-tpu
Writing TPU kernels
PL-sc
SparseCore kernels
G-8dd
Gupta & Mugazambi, "TPU 8t and TPU 8i technical deep dive"
PL-pipe
TPU pipelining
megacore dimension semantics
T-exec
TorchTPU docs: Execution model
compile cost
SB12
How to Think About GPUs
GPUs vs TPUs at the chip level
N-tune
Hopper tuning guide
§1.4.1, occupancy and SMEM
A-trn2-nki
Trainium2 architecture guide for NKI
Chapter 2. Day 1, second block, 55 minutes
Anatomy of a TensorCore
Read
Quiz
12
Flashcards
20
Teach-back
8
Question 1 of 12
Recall
On a GPU, memory latency is hidden by switching among resident warps. What hides it on a TPU TensorCore?
1
A compile-time schedule that issues the DMA for the next tile while the current tile is computed.
2
Hardware switching among several resident threads per TensorCore.
3
A large L2 cache that keeps recently used tiles.
4
Nothing; TPUs are latency-bound on every load.
Check answer
Enter
Press 1–4 to choose.