TPU Deep Dive
Course
Review
Slides
Exams
Sources
Theme: system
Sources
B-vllmtpu
"vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU"
R-tpuinf
tpu-inference (being renamed vLLM JAX)
README, runner/, layers/common/sharding.py, kernels/
R-torchax-guide
tpu-inference torchax model development guide
R-torchax
torchax (moved here 2025-10-06)
P-RPA
Jiang, Chen, Hechtman, Zhang, Mu, "Ragged Paged Attention"
R-vllm-torchtpu
vllm-torchtpu ("recommended path for serving PyTorch model definitions on TPU")
T-engines
"Announcing New vLLM and SGLang Inference Engines on TPU"
T-xla
PyTorch/XLA repo
R-jetstream
JetStream
R-qwix
Qwix
X-tiled
Tiled layout
SB7
All About Transformer Inference
JetStream
SB7
All About Transformer Inference
Continuous batching
SB8
Serving LLaMA 3-70B on TPUs
LLaMA serving story
SB8
Serving LLaMA 3-70B on TPUs
Latency against throughput
SB8
Serving LLaMA 3-70B on TPUs
Problems
Chapter 8. Day 4, second block, 55 minutes
The TPU serving stack
Read
Quiz
12
Flashcards
20
Teach-back
6
20
left in this session,
20
of them new.
Reviewed
0
Question
TPU serving stack timeline?
Show answer
Space