Once every token is a vector, "similar meaning" becomes "vectors that point the same way" — and you measure it with arithmetic, not a dictionary.
On the last page, every token became a row of numbers — a vector. Now the payoff: once meaning lives in vectors, two tokens are similar exactly when their vectors point the same way.
We measure that with a dot product — multiply the two vectors slot-by-slot and add up the products (a·b = Σ aᵢbᵢ). Bigger means more alike. To keep long vectors from winning just for being long, we normalize it into cosine similarity cos(a,b) = (a·b)/(|a||b|), a score from −1 to 1. Same operation returns as the attention score four pages from now.
Our 4 dimensions can't be drawn, so the scatter below is a flattened 2-D map for intuition only — but every similarity bar is computed from the full 4-dim vectors.
TRY → Click any dot (or chip) to make it the anchor. The bars re-rank live — each is cos(anchor, token) computed over the 4-dim vectors, longest bar = most similar, with the raw dot product shown on the left. Anchor it ranks cat highest.
Take the anchor it = [0.8, 0.1, 0.2, 0.3] and the token cat = [1.0, 0.2, 0.1, 0.0]. The dot product multiplies slot-by-slot and adds:
Then divide by the two lengths to normalize out magnitude:
|it| = √(0.8²+0.1²+0.2²+0.3²) = √0.78 = 0.883 |cat| = √(1.0²+0.2²+0.1²+0.0²) = √1.05 = 1.025 cos(it,cat) = 0.84 / (0.883 · 1.025) = 0.93Compare the others from the same anchor: cos(it,tired) = 0.49, cos(it,was) = 0.45, cos(it,because) = 0.39, cos(it,The) = 0.34, cos(it,sat) = 0.31. So it sits closest to cat — the same reason it will attend to cat two pages from now. None of these numbers are stored: the widget runs this loop over E on every click, so editing a vector would reorder the bars.
Similarity of meaning is just vectors pointing the same way, scored by the dot product — one operation that returns as the attention score later.