Importance-weighted on-policy distillation counters position bias by scaling token weights according to cumulative student-teacher distribution discrepancy, improving efficiency and final performance over uniform averaging.
Disentangling reasoning tokens and boilerplate tokens for language model fine-tuning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
On the Position Bias of On-Policy Distillation
Importance-weighted on-policy distillation counters position bias by scaling token weights according to cumulative student-teacher distribution discrepancy, improving efficiency and final performance over uniform averaging.