REVIEW 4 cited by
Following Length Constraints in Instructions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Aligned instruction following models can better fulfill user requests than their unaligned counterparts. However, it has been shown that there is a length bias in evaluation of such models, and that training algorithms tend to exploit this bias by learning longer responses. In this work we show how to train models that can be controlled at inference time with instructions containing desired length constraints. Such models are superior in length instructed evaluations, outperforming standard instruction following models such as GPT4, Llama 3 and Mixtral.
Forward citations
Cited by 4 Pith papers
-
Test-Time Scaling via Error Localization
TTEL uses feedback-induced token probability drops to localize the first error in a failed reasoning trace and branch a new generation from that prefix, improving pass@k per token on coding and math benchmarks.
-
Revisiting the Reliability of Language Models in Instruction-Following
LLMs exhibit up to 61.8% performance drops on nuanced rephrasings of instruction-following tasks, revealing insufficient nuance-oriented reliability across 46 tested models.
-
Med-U1: Incentivizing Unified Medical Reasoning in LLMs via Large-scale Reinforcement Learning
A pure reinforcement learning recipe with mixed rule-based rewards and length control improves Qwen2.5-based models across diverse medical QA formats.
-
LIFEBench: Evaluating Length Instruction Following in Large Language Models
LIFEBench's evaluation of 26 LLMs shows most follow short length instructions but degrade sharply beyond a few hundred words, and none reliably hit vendor-claimed maximum output lengths.
Discussion (0). Continue with ORCID to comment.