Every step so far ranks next-token candidates by learned plausibility — how well a continuation fits the training patterns. Nowhere in the arithmetic is there a step that asks "is this actually true?" So a confident, plausible, false token can outrank the correct one.
Lessons 10 and 11 turned a final vector into vocab-logits and then, through softmax, into next-token probabilities. That whole pipeline ranks tokens by one thing only: how plausible each continuation is, given the patterns the model saw in training.
So when a false answer appeared more frequently in the training text (a famous city said far more often than a quiet capital), it can carry the higher logit — and win. That's hallucination, mechanically: not the model "lying," just plausibility beating truth, because truth was never a term in the computation.
Below is a toy distribution for "The capital of Australia is ___". Canberra is the real capital; Sydney is the larger, more-talked-about city — the common wrong guess. Watch which one softmax picks.
the model's ranking — argmax is whatever has the highest probability, full stop
Hit Mark the true answer. The model's pick doesn't move — because no "is this true?" step exists anywhere in this computation. Only you, from outside, can flag which bar is correct.
TRY → Hit Recompute softmax and the five bars are rebuilt from the raw logits every time (never replayed from a stored answer) — the winner is Sydney at ~50%. Then hit Mark the true answer: Canberra lights up green at rank 3, and the confident, wrong Sydney is flagged red. The ranking never changes — truth was never part of it.
Five candidate logits for "The capital of Australia is ___", run through the same softmax(z) = e^z / Σe^z from every other page. Subtracting the max (3.0) for numerical stability, the denominator is Σ e^(z−3.0) = 1.997:
| token | logit z | e^(z−3.0) | probability |
|---|---|---|---|
| Sydney | 3.0 | 1.000 | 50.1% |
| Melbourne | 2.2 | 0.449 | 22.5% |
| Canberra (true) | 1.8 | 0.301 | 15.1% |
| Perth | 1.0 | 0.135 | 6.8% |
| Brisbane | 0.8 | 0.111 | 5.5% |
The false-but-famous Sydney (50.1%) beats the true Canberra (15.1%, rank 3). Canberra is Australia's capital; Sydney is its largest, most-talked-about city, so training made Sydney the stronger association here. The softmax ranks by that association — it has no fact table to consult.
Turning up temperature (lesson 11) doesn't fix this — it only changes how often the wrong-but-confident token is sampled; Sydney is still the argmax. (The logits here are illustrative; the mechanism — plausibility with no truth signal — is the real point.)
Prediction picks the most plausible continuation, never the verified-true one — because nothing in the softmax computes truth. A confident, fluent, false token can outrank the correct answer.