TPU Deep Dive
Course
Review
Slides
Exams
Sources
Theme: system
Sources
B-vllmtpu
"vLLM TPU: A New Unified Backend Supporting PyTorch and JAX on TPU"
R-tpuinf
tpu-inference (being renamed vLLM JAX)
README, runner/, layers/common/sharding.py, kernels/
R-torchax-guide
tpu-inference torchax model development guide
R-torchax
torchax (moved here 2025-10-06)
P-RPA
Jiang, Chen, Hechtman, Zhang, Mu, "Ragged Paged Attention"
R-vllm-torchtpu
vllm-torchtpu ("recommended path for serving PyTorch model definitions on TPU")
T-engines
"Announcing New vLLM and SGLang Inference Engines on TPU"
T-xla
PyTorch/XLA repo
R-jetstream
JetStream
R-qwix
Qwix
X-tiled
Tiled layout
SB7
All About Transformer Inference
JetStream
SB7
All About Transformer Inference
Continuous batching
SB8
Serving LLaMA 3-70B on TPUs
LLaMA serving story
SB8
Serving LLaMA 3-70B on TPUs
Latency against throughput
SB8
Serving LLaMA 3-70B on TPUs
Problems
Chapter 8. Day 4, second block, 55 minutes
The TPU serving stack
Read
Quiz
12
Flashcards
20
Teach-back
6
Question 1 of 12
Recall
In October 2026 you start porting a new PyTorch model definition to TPU for vLLM serving. Which path do the projects themselves recommend?
1
vllm-torchtpu, which runs upstream vLLM PyTorch models through torch.compile on TorchTPU.
2
tpu-inference's torchax path.
3
vLLM on PyTorch/XLA.
4
JetStream.
Check answer
Enter
Press 1–4 to choose.