Attention under real constraints.

This course is upcoming. The outline below describes the intended progression; its lessons are not yet published.

The planned destination

Start from a working causal decoder, build a measured inference baseline, and change one source of cost at a time. Every optimization should come with a correctness comparison, a memory estimate whose terms are explained, and an experiment with a stated workload.

Planned learning sequence

1. Prefill versus decode. Follow the available tokens and tensor shapes during each phase; distinguish latency to the first token from time per subsequent token.
2. A correct KV cache. Reuse past projections, preserve positional offsets, and test cached output against full-prefix recomputation.
3. Memory accounting. Count layers, sequences, cached positions, KV heads, head width, and bytes per value. Separate model weights, cache, activations, and allocator overhead.
4. Multi-query and grouped-query attention. Change which heads share keys and values; identify the architectural assumptions and quality trade-offs.
5. Exact attention implementations. Distinguish mathematical attention from how an implementation schedules computation and memory access.
6. Linear and recurrent attention. Derive the state update from a kernel definition, then show exactly how that model differs from ordinary softmax attention.
7. A controlled comparison. Report sequence lengths, batch sizes, precision, hardware, quality, and warm-up conditions before drawing a speed conclusion.

What you can read today

The existing KV cache note and linear attention note provide short previews. The prerequisite Attention & Transformers course is available now.