Chapter 7. Day 4, first block, 55 minutes
Inference rooflines
Question 1 of 12
Derivation: enter a number
On Ironwood (313 FLOP/B BF16 ridge), weights are stored in fp8 and the math stays in bf16. Above what decode batch are the linear layers compute-bound?
sequences