Derives closed-form optimal attention temperature minimizing ICL generalization error under distribution shift, linked to pre-softmax score moments, with LLM validation.
(2024) and introduce noisy labels—incorrect but semantically related—to the in-context demonstrations (Appendix K.4)
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
stat.ML 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Optimal Attention Temperature Improves the Robustness of In-Context Learning under Distribution Shift in High Dimensions
Derives closed-form optimal attention temperature minimizing ICL generalization error under distribution shift, linked to pre-softmax score moments, with LLM validation.