Query, key and value

A Transformer first learns three projections of the token embeddings X: Q = X @ W_Q, K = X @ W_K and V = X @ W_V, each of shape (B, S, D) — batch, token, features.

• Query — asks what to retrieve. Spanish: consulta.
• Key — determines its match. Spanish: clave.
• Value — carries what gets retrieved. Spanish: valor.

The notebook uses small random projections to show the mechanics, not a trained language model, so its attention patterns have no learned meaning.