Attention begins by making three new vectors out of each token's embedding — same input row, three different weight matrices. This page builds those pieces. It does not score them yet.
Every token already has an embedding — its row of numbers from the lookup table. Attention's first move is to turn that one row into three new vectors by multiplying it through three different learned weight matrices: Wq, Wk, Wv.
The plain gloss: the Query is what this token is looking for; the Key is what it advertises it offers; the Value is the content it will hand over if it gets attended to. Each one is just a matrix multiply — q = Wq·e — using the same row·vector rule from the neuron page. Nothing is compared or decided yet; we are only preparing the pieces.
TRY → Click any token. Its embedding lights up, then three violet vectors — Q, K, V — are computed from it live, each showing one worked row of the multiply. The three matrices are fixed; the token is what changes.
The three weight matrices (head 1), each 4×4. Non-zero weights are highlighted. Wq boosts the animacy and function slots into the Query; Wk passes the raw dimensions through but damps the function slot (×0.3); Wv is the embedding with dim 3 (function) halved (×0.5), so the Value resembles the token but is not identical to it.
The full multiply for it's Query, q = Wq · e_it with e_it = [0.8, 0.1, 0.2, 0.3] — all four output slots, each one row of Wq dotted with the embedding:
Same embedding, three matrices, three different vectors. That is the whole idea of this page — the comparison between a Query and a Key is the next page.
q = Wq·e, k = Wk·e, v = Wv·e.One embedding in, three vectors out — Q, K and V — each the product of a different weight matrix. This page prepared the pieces; scoring them comes next.
q = Wq·e, k = Wk·e, v = Wv·e — same input row, three learned matrices, three different vectors.