cd .. / tbash.xyz
Interactive deep-dive · Part 1

How LLMs Work — Meaning & Attention

This track opens the two hardest boxes in an LLM — where a token's meaning-vector comes from and how attention actually computes — and makes you trace real numbers, not analogies. Fifteen short pages, one idea and one live widget each — from tokens through attention to the next-word prediction — and every number on screen is computed in front of you. Then Part 2 · How LLMs Run Fast takes the same model and shows how it's served in milliseconds — the KV cache, quantization, paging, batching, speculative decoding, and the roofline that explains why they all help. Then Part 3 · Beyond the Basics goes past the core loop — the GPU it runs on, the attention and memory tricks (GQA, FlashAttention), and the capability patterns every assistant uses (mixture-of-experts, tool use, reasoning, RAG, multimodal).

📚 Part 1 · 15 lessons ⚡ Part 2 · 8 lessons 🧩 Part 3 · 8 lessons 🎚 interactive math: traceable

Part 1 · How LLMs Work

From a token ID to the next word — meaning, attention, and the full forward pass, every number computed live.

  1. 01 Where does the vector come from? Read a token's vector straight out of the learned lookup table, by row.
  2. 02 Nearby vectors mean similar things Rank every token by dot-product similarity to an anchor you pick.
  3. 03 Meaning has directions you can do arithmetic with Compute A − B + C on real vectors and watch the nearest word fall out.
  4. 04 One neuron: multiply, add, squish Slide the inputs and watch each multiply, the running sum, and the tanh squish.
  5. 05 Query, Key, Value: three vectors from one token Turn one embedding into its Q, K and V through three different weight matrices.
  6. 06 How attention actually computesflagship Watch Q·K dot products → softmax weights that sum to 1 → a blended Value vector, live.
  7. 07 Many heads, many kinds of attention Toggle heads and watch the same token get routed three different ways.
  8. 08 Where does the blended vector go? Add attention's blend back onto the token's own vector — refine, don't replace.
  9. 09 The feed-forward step: each token thinks alone Run the lesson-4 neuron across a real post-attention token vector.
  10. 10 From vector to a next-word guess Dot the final vector against every embedding row to rank the next word.
  11. 11 Three dials on the next-word distribution Reshape the probabilities with temperature, top-k, and top-p.
  12. 12 One block, stacked N times Stack attention + FFN + residual and watch a token's vector drift over layers.
  13. 13 Meaning + position, literally added Add position to meaning, element-wise: "dog bites man" vs "man bites dog".
  14. 14 Why models hallucinate See a confident, false token outrank the true one — no truth-check anywhere.
  15. 15 Why it can't count the r's in "strawberry" Watch "strawberry" become opaque IDs the letter-count can't survive.

Part 2 · How LLMs Run Fast

Same model, now served in milliseconds. Eight pages, one lever each — and every FLOP, byte, and token-per-second is computed live from one 7B-class model preset.

  1. 01 Reading the prompt is fast; writing is slow Compute prefill vs decode time and watch decode dominate (~156×).
  2. 02 The KV cache: don't recompute the past Count the triangular blow-up of recomputing every past token's K/V.
  3. 03 The cache grows and eats VRAM Size the KV cache live: layers × heads × dim × seq × 2 × bytes.
  4. 04 Store each weight in fewer bits Snap weights onto int8/int4 levels; watch memory drop 4×.
  5. 05 Paged attention: stop reserving unused memory Compute padding waste for reserve-max vs paged (~75% → ~1%).
  6. 06 Read the weights once, serve many Compute GPU utilization and throughput for static vs continuous.
  7. 07 Guess ahead, verify in one pass Compute expected tokens/step and speedup from an acceptance rate.
  8. 08 The roofline: moving weights, not maththe why Compute arithmetic intensity and land below the ridge point.

Part 3 · Beyond the Basics

Past the core loop: the GPU it runs on, the attention and memory tricks that scale it, and the capability patterns every assistant is built from — each one a live widget, every number still computed in front of you.

  1. 01 What a GPU actually isthe stage Place weights, KV, and a Flash tile in the VRAM→SRAM hierarchy and see what fits.
  2. 02 Share the Keys and Values (GQA) Dial 32→8→1 KV heads and watch the cache shrink 4× → 32×.
  3. 03 Keep the score matrix on-chip Tile the n×n scores into SRAM; watch 32 MiB of VRAM traffic vanish.
  4. 04 Many experts, a few per token Route a real token to its top-2 experts; 47B stored, 13.6B active.
  5. 05 It's still just predicting tokens Step a token stream that emits a JSON call, runs it, injects the result.
  6. 06 Buying accuracy with tokens Slide thinking-tokens and watch accuracy climb, then plateau.
  7. 07 Retrieve the facts, then prompt Cosine-retrieve the top passages for a query and watch them prepend.
  8. 08 An image is just more tokens Click a patch and watch it project into an embedding in the token stream.

The classic walkthrough (overview pages)

Want the wide tour first? These are the full narrated overviews the deep-dive above zooms in on.

How LLMs Work the original narrated version of Part 1 How LLMs Run Fast the original narrated version — Part 2 is now the interactive track above
MythLLMs understand words the way we do.
RealityThey turn every token into a list of numbers and do arithmetic on those lists — this track shows the arithmetic.
What will you actually do on the Attention II page?
Why: every widget on this track computes its answer live from the data — no answer is typed in as a constant.