A token starts life as a plain integer ID. Turning it into a list of numbers isn't a calculation — it's a lookup: go to that row of a learned table and read it off.
A token is a chunk of text the model reads as one unit, and each distinct token has a fixed integer ID — its row number. That ID carries zero meaning on its own: cat might be ID 2543, but 2543 is just where the row lives, not what it means.
So the model keeps one giant lookup table: one row per token, and each row is a short list of numbers — the token's embedding vector. "Turning a token into a vector" is literally go to that row and read it off. No arithmetic — just a lookup.
Those numbers were learned during training and are now frozen; we only read them. Each dimension is one slot — a learned axis of meaning. No human labels these axes, but we pretend-label ours (animacy · action · state · function) so the numbers are readable — these four names are our invention for readability, not labels the model learned; the real axes have no names. Real models use hundreds to thousands of dimensions; ours has 4.
| ID | token | animacy | action | state | function |
|---|
TRY → Click a token chip (or type an ID 0–6). Watch the matching row light up and its four numbers get copied — read straight out of the table — into the big embedding( ) readout. This is the exact same table the attention pages later read from.
A real embedding table is vocab_size × d_model — e.g. 50,000 rows × 4,096 numbers ≈ 200 million weights, and that's just for the lookup, before any attention or neurons. Ours is a toy 7 × 4.
The row for it (ID 4) is [0.8, 0.1, 0.2, 0.3]. The widget doesn't store that string anywhere — it fetches E[TOKENS[4]] from the data array by index every time you click. Edit a number in the table's source and the readout changes with it: the value is addressed, not memorized.
One line ahead — right after this lookup, the model literally adds a second "where it sits in the sentence" vector onto this row, element by element: vector = meaning + position. Not a metaphor — real addition. (That step is covered in the classic walkthrough.)
An embedding is one row of a learned lookup table, one row per token ID. Getting it is indexing, not computing.