Query, Key and Value from the input representation
Model
Query head
Query token
Key / Value token
Measurement stage
Immediately after WQ / WK / WV
Attention inputs (after normalization / RoPE)
Input
Q — Query
K — Key
V — Value
Component
Attention from this Query to every Key
Layer 0, one head, every component. Each plot has its own vertical scale. Attention uses transformed Q/K; V has no RoPE.