The foundations, without the missing steps.

A Transformer diagram can be beautifully drawn and still feel like a map with no legend. Before attention, we need to know what is flowing through those arrows, what an operation changes, and what “learning” actually updates.

This course supplies that legend. We start with two small rows of numbers. We project their features, mix information with weights, turn scores into probabilities, and make one parameter learn. Each lesson ends by identifying the piece it contributes to attention.

Start where you are

You need basic Python: assigning a variable, indexing a list, and calling a function. You do not need calculus, linear algebra, or an ML background. We introduce the notation at the point where it becomes useful. There is no timed assessment.

If you already work comfortably with tensors and gradients, continue to Attention & Transformers . Otherwise, work through these four lessons in order. The numbers deliberately recur so that each new idea has something familiar to attach to.

How to read an unfamiliar equation

First find the output, usually on the left. Then ask what each input represents and which axes it has. Read the operation in words before trying to calculate it. Finally, substitute small numbers. An equation is a precise instruction, not a test of how quickly you recognize symbols.

Every displayed equation has a reading guide: its terms, their roles, what they control, and a numerical check. You can return to the same guide as often as you need.

How to use the examples

Every function declares input and output types. float means a scalar floating-point number; FloatArray means a NumPy array of float64 values; Tensor means a PyTorch tensor. Type hints help an editor check usage, but they do not automatically validate shapes. The docstrings, equations, and explicit checks supply that part of the contract.

Use Open complete file to see the imports and surrounding implementation. It highlights the exact region used in the lesson. Visualize turns the code block into a step reader: the UI follows the source, inputs, and changed values for you. Use Previous and Next, or choose a recorded event. Explore file lets you inspect the surrounding source; Exit restores the original snippet. Large arrays are explicitly sampled; no Python is executed in your browser.

The NumPy source and PyTorch source are also included in the complete course code bundle . Both files have a check() entry point you can run locally.

Where this leads

After these lessons, attention becomes a composition of operations you already know: project features, calculate matching scores, normalize them, and mix values. The intermediate course adds tokens, positions, masks, heads, and the architecture around that calculation.

The later advanced attention course and AI systems course are upcoming outlines. They make the longer learning path visible without pretending unpublished lessons are ready.