Faster query-key learning relative to output-value learning sharpens attention onto task-relevant tokens at comparable prediction performance, derived from gradient-flow dynamics and shown on synthetic and real tasks.
On the optimization and generalization of multi-head attention
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Faster Query-Key Learning Sharpens Attention in Self-Attention Models
Faster query-key learning relative to output-value learning sharpens attention onto task-relevant tokens at comparable prediction performance, derived from gradient-flow dynamics and shown on synthetic and real tasks.