{"id":"14420ff7-3fdb-4751-a106-1d78c20d85ac","arxiv_id":"2507.20398","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Output length in LLMs is encoded in early attention-layer units, and scaling those units changes generated summary length while largely preserving content.","lead":"Large language models store information about how long their output should be in specific internal units, mostly in the second attention layer. Scaling those units makes models write longer or shorter summaries without retraining, a finding that could give developers a new internal control for length.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The probe target is the generation time step, which is confounded with token position and token identity; the identified 'length units' may be position units, so a position-controlled probe is needed before the disentanglement claim can be accepted.","rationale":"I agree with the reader's weakest_assumption. The central claim rests on identifying hidden units whose activity correlates with generation time step; if that correlation is driven by positional encoding or token identity, the causal scaling result does not by itself prove a length representation. This concern is more load-bearing than the grouped-query attention limitation: the latter narrows the scope but does not invalidate the finding for multi-head attention models, whereas the position confound affects the interpretation of every R2 table and every unit-selected scaling result. The paper includes a smallest-k control and human evaluation, which are genuine strengths, but neither rules out position/token confounds. The scaling effect could arise from perturbing a position-correlated direction and thereby changing end-of-sequence decisions. I therefore keep the reader's conditional verdict: the paper is valuable, but the identification step needs a position-controlled probe. If the proposed test shows that R2 survives when position is held fixed, the central claim is substantially supported; if not, the claim should be restated as position-based length modulation rather than a length representation partially disentangled from semantic information.","tokens_in":21390,"tokens_out":7938,"duration_ms":97069,"concrete_test":"Retrain the per-unit probes of Table 4 with position controlled. For a fixed generation position n (e.g., n = 1, 2, 3), train a probe to predict the final summary length L, or remaining length L - n, from the hidden state at that position, using the same Llama-2-13B-Chat generations and the same 90/10 split. If the top units from Table 4 do not retain above-chance R2 on this length-given-position task, then the original time-step R2 was largely position/order information. Then rerun the Figure 2 scaling experiment using only units that survive this position-controlled selection; if length control disappears, the disentanglement claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the identification of length units in Sections 3-5. The regression target is the generation time step n (Section 3.1), and hidden states are collected from generated tokens that carry RoPE-based positional information and token-specific embeddings. Because n is exactly the absolute position during generation, a probe can achieve high R2 by decoding position, or token order/identity correlated with position, rather than any representation of planned output length. Table 4 then selects units by per-unit R2 on this same time-step target, and Figure 2 scales those units. If the selected units are position/order units, the scaling experiment shows that perturbing position-correlated directions changes when the model stops. That is a real but different phenomenon: it does not establish that the model encodes output length as an attribute partially disentangled from semantic information. The paper's own discussion of 'positional context' in the final layer (Section 4) shows that the confound is present. Per-unit R2 values are low (0.11-0.42), so the risk that selection is driven by incidental position/token correlations is non-trivial. For the central claim to hold, the probe must predict length beyond position; this is not currently tested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether large language models encode output sequence length as an internal, partially disentangled feature. Using Google sentence summarization data with three prompt formats (No-constraint, Length, Priming), the authors train a two-layer neural network to predict the generation time step from hidden states extracted at different transformer components and layers, reporting R2 scores across Llama-2, Llama-3, Phi-3, and Qwen-2.5 models under different precision settings. They find that the layer-2 attention output yields the highest R2, identify individual hidden units with high per-unit R2, and show that scaling the top-k units with positive or negative factors changes summary length while leaving the smallest-k units ineffective. They also report human evaluation results, cross-task transfer to machine translation and story generation, and shared units between zero-shot and fine-tuned settings. The paper concludes that length information is partially disentangled from semantic information and that LLMs possess internal, unit-level length control mechanisms.","tokens_in":21567,"tokens_out":4985,"duration_ms":54602,"significance":"If the central claim holds, the paper makes a useful contribution to mechanistic interpretability of LLMs by identifying unit-level length-controlling directions that are partly independent of content, with potential applications to controllable generation. The strengths of the paper are its breadth of models, sizes, precision settings, and prompting conditions; the inclusion of an actual intervention (scaling units) with a smallest-k control; human evaluation; cross-task transfer experiments; and released code. However, the interpretation of the probe as measuring length rather than position is the load-bearing step. Because the probe target is the generation time step, which coincides with absolute position during generation and is confounded with positional encoding and token order, the current evidence does not yet separate length representations from position representations. The modest per-unit R2 values and the fact that unit selection and scaling evaluation share the same corpus further weaken the identification step. These issues are fixable, so the result is potentially valuable but not yet established.","major_comments":[{"comment":"The probe target is the generation time step n, which is identical to the absolute position of the token being generated. Because Llama, Phi, and Qwen hidden states carry positional information through RoPE and through token order, the high R2 of the layer-2 attention output may simply reflect how well position is decodable, not how much planned output length is represented. The per-unit R2 values in Table 4 are modest (0.11–0.42), so the top-k units selected by this target could be position/order units rather than length units; in that case, the scaling intervention in Figure 2 would show that perturbing position-correlated directions changes stopping behavior, which is a real but different phenomenon. This does not establish the paper's central disentanglement claim. I ask for a position-controlled probe, for example, train the regression on hidden states collected at a fixed generation step across outputs of different total lengths, or predict the remaining number of steps while conditioning on n, or remove/randomize the positional contribution, and show that the same units remain predictive of length beyond position. The authors' own remark in Section 4 that the final-layer increase may 'reinforce positional context' indicates the confound is present and needs to be addressed explicitly.","section":"Section 3.1 and Table 4"},{"comment":"The length units are selected by per-unit R2 on the same Google summarization corpus that is later used for the scaling evaluation and human evaluation; no cross-validation or held-out unit selection is reported. If the selection step overfits to corpus-specific generation patterns, the scaling result in Figure 2 is partly a confirmation of the selection criterion. I request that units be selected on a training/validation split and evaluated on a held-out test split, and that the cross-task results in Appendix D be reported both with units selected independently per task and with units transferred from summarization. The current design does not separate discovery from evaluation.","section":"Section 5.2, Figure 2, Table 4"},{"comment":"The claim that LLMs have robust internal mechanisms for length control is directly qualified by the authors' own finding that scaling top-k units in grouped-query attention models (Qwen-2.5, Llama-3, Phi-3) does not control length in zero-shot settings, with success only after fine-tuning. Since the abstract and conclusion claim a general LLM mechanism, this limitation should be promoted into the main claims, or the claim should be explicitly restricted to standard multi-head attention models. In its current form, the paper demonstrates a mechanism in Llama-2 family models; the broader conclusion exceeds the evidence.","section":"Section 5.2, Figure 3, Limitations"}],"minor_comments":[{"comment":"There are several typos and reference formatting errors, including 'Googlesentence' in Section 3, 'Juseon-Do Juseon-Do' in the bibliography, a doubled comma in the Goh et al. entry, and 'Guangyi, Zhang' in the Llama 3 reference; these should be corrected.","section":"Throughout"},{"comment":"The column 'Avg 30' is not defined; please state whether it is the mean R2 over the top-30 units, and report standard errors or a statistical test for the differences between top-unit R2 values across prompting conditions.","section":"Table 4"},{"comment":"The sentence describing the dagger symbol, 'the improvement for scales between 10 and -10 is significant', is unclear; specify the exact paired comparison and report confidence intervals for the human evaluation scores in Table 5.","section":"Section 5.1"},{"comment":"The horizontal axis is labeled only as 'scale'; please label the axis as the multiplier applied to the selected units, state the range, and clarify whether error bars are standard errors across the five runs or across the test instances.","section":"Figures 2 and 3"},{"comment":"Please specify which encoder/decoder model was used for the machine translation experiment, whether the transferred length units were selected on the same summarization split as in the main experiments, and why BLEU-1/2 rather than a length-sensitive metric was selected for the translation evaluation.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"I agree with the stress-test concern that the probe target is confounded with absolute position, and this is the main obstacle to the paper's central claim. The intervention experiment is valuable and should be retained; the paper needs a position-controlled probe and independent unit selection before the disentanglement claim can be accepted. The GQA zero-shot failure also needs to be reconciled with the general claim in the abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid probe-and-scale study with a new object—length control in early attention units—but the confound between generation step and absolute position cuts into the central claim, and the in-sample unit selection needs fixing.\n\nWhat's genuinely new: first systematic localization of output-length information to second-layer attention outputs across Llama, Phi, and Qwen families, with unit-level scaling that changes completion length in a direction-consistent way. The smallest-k control is the right experimental move, and the human evaluation is a nice addition. Releasing code helps. The finding that length-related units transfer across summarization, MT, and story generation is genuinely interesting.\n\nSoft spots, in order of severity:\n\nThe probe target is the generation time step, which is the same as absolute position during generation. Hidden states carry RoPE and token identity, so a probe can hit high R2 by decoding position rather than any planned length. That makes the identification step in Sections 3–5 ambiguous. The authors even note final-layer 'positional context,' which is the same confound. They need a position-controlled probe—for instance, subtracting the positional component, or predicting the remaining length from a fixed early token across prompts that yield different total lengths.\n\nRelatedly, top-k units are selected on per-unit R2 from the same test corpus used for the scaling evaluation. That is in-sample selection, and given per-unit R2 values are modest (0.11–0.42), the selected units could be idiosyncratic to that data. Held-out selection is a must.\n\nThe GQA failure in zero-shot is a major scope limit. Llama-2 is the only MHA family, and many current models use GQA, so the 'robust internal mechanism' claim is not as broad as stated.\n\nFinally, the abstract says scaling works 'without losing informativeness,' but the human eval shows a small dip at positive scale (3.63 vs 3.70). That's a minor overstatement, easily fixed.\n\nBottom line: the paper is honest and the intervention is genuinely causal in the MHA models. It deserves a serious referee, but the position confound and in-sample selection need to be addressed before the disentanglement claim can stand. I'd send it to review with major-revision expectations.","headline":"Useful empirical mapping of length control to early attention units, but the position confound and in-sample selection make the disentanglement claim weaker than the abstract suggests.","tokens_in":22137,"tokens_out":2776,"would_cite":false,"duration_ms":28455,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The second attention layer of an LLM carries a length dial: probing finds individual hidden units, and scaling them lengthens or shortens output without losing meaning.","keywords":["length control","mechanistic interpretability","multi-head attention","probing classifiers","hidden unit scaling","sentence summarization","generation time step","in-context learning"],"falsifier":"Train the per-unit probes only on hidden states taken at the first generated token, a fixed position, across summaries that end at different lengths; if the top-k 'length units' no longer predict final output length with R2 far above zero, the probes were reading absolute position rather than length, and the scaling result would not prove length control.","tokens_in":21163,"feed_emoji":"📏","tokens_out":8011,"duration_ms":72721,"temperature":0.7,"pith_summary":"This paper tries to show that output length is not just a prompt-level behavior in large language models but a real, locatable internal feature. By training a simple regression probe to predict the generation time step from hidden states, the authors find that the second layer's multi-head attention output carries the most length information across several open LLM families and sizes, even under 4- and 8-bit quantization. They then identify individual hidden units with high per-unit probe accuracy and show that multiplying the most active of them by positive or negative numbers produces shorter or longer summaries while preserving informativeness, as measured by ROUGE-L and human evaluation. The result is evidence, the authors argue, that length-specific information is partially disentangled from semantic information inside LLMs and can be controlled without external scaffolding.","feed_headline":"Scaling a few hidden units lengthens or shortens LLM output","feed_subtitle":"Length probes locate the control in the second attention layer, separate from meaning.","key_machinery":"The load-bearing instruments are (1) a two-layer neural-network regressor that predicts the generation time step from hidden states and per-unit R2 values naming the length units, and (2) a causal scaling intervention that multiplies those top-k units in the second layer's attention output by positive or negative constants during decoding. The second-layer attention output is the object: it is where length predictions concentrate, and the scaling experiment is what turns correlational probes into evidence of control. The Priming prompt, which states source length and target kept-token count, is the condition that shifts activation onto a distinct set of units and makes single-unit scaling effective.","core_discovery":"Stated on the paper's own terms: large language models encode output sequence length as a locally concentrated, unit-level feature in the attention output of the second transformer layer, and this feature is partially separable from the semantic content being generated. A two-layer regression probe trained to predict the generation time step from hidden states gives the highest R2 for that layer's attention output across model families and precisions; per-unit probes then single out a handful of hidden units whose activity tracks length, and the top-1 unit alone can shift compression ratio when scaled. Multiplying these units by negative constants makes the model generate longer summaries, positive constants shorter ones, and human evaluation under the length-priming prompt shows informativeness is preserved. The same units recur after fine-tuning, and units found on summarization also steer translation and story generation, which the authors read as evidence of a learned length mechanism rather than a prompt artifact.","pith_inferences":["If length is a genuinely separable early-layer feature, then production systems could steer length by a direct unit-level gain instead of prompt phrasing, and the same identification-plus-scaling recipe could be tried for other discrete generation attributes such as formality, stance, or domain.","The paper's own negative result for grouped-query attention in zero-shot settings suggests that shared key/value projections move or scatter the length code; a testable extension is to probe the shared projections and per-head output rather than the concatenated hidden units.","Because the probe target is the current generation step, the identified units may encode how far the sequence has gone rather than when it should stop; an experiment that conditions on fixed total lengths or predicts remaining steps would separate these two readings.","The transfer of summarization-identified units to translation and story generation implies a task-general length axis; if confirmed, length steering should work by the same unit gains across decoding strategies with only calibration of the gain value."],"forward_implications":["Prompt-free length control is possible for standard multi-head-attention models by rescaling the top-k units in the second layer's attention output.","Length information survives 4-bit and 8-bit quantization, so compressed models retain the internal length code.","The same top-3 length units are active under Priming prompts in both zero-shot and fine-tuned models, supporting the view that in-context learning recruits the same circuitry that fine-tuning strengthens.","Length units identified on sentence compression transfer to machine translation and story generation, indicating a shared length axis across tasks.","Scaling the least-active units does not change the generated text, which is the control showing the top-k effect is not a generic perturbation."],"supporting_citations":[{"why":"Supplies the InstructCMP instruction templates and the length-priming prompting method whose zero-shot and fine-tuned settings structure the experiments.","marker":"(Juseon-Do et al., 2024)"},{"why":"Supplies the Google sentence summarization dataset that all generation and probing experiments run on.","marker":"(Filippova and Altun, 2013)"},{"why":"Establishes the regression-on-time-step probe for length information that this paper adapts to LLM hidden states.","marker":"(Shi et al., 2016)"},{"why":"Provides the precedent that time-related linear representations in LLMs can be changed by disentangling hidden units, motivating the scaling intervention.","marker":"(Gurnee and Tegmark, 2024)"},{"why":"Shows numeric attributes are encoded in interpretable directions, supporting the view that one attribute can be unit-specifically disentangled.","marker":"(Heinzerling and Inui, 2024)"},{"why":"Supports the interpretation that in-context learning mirrors fine-tuning, which the paper invokes when the same top-3 units appear in both settings.","marker":"(Dai et al., 2023)"},{"why":"Documents why probe accuracy alone does not prove functional involvement, forcing the paper to add the scaling experiment as validation.","marker":"(Sajjad et al., 2022)"},{"why":"Documents the same probing-classifier limitation, used to justify validating identified units by causal intervention.","marker":"(Belinkov, 2022)"},{"why":"Provides the paired bootstrap resampling used to test the significance of the human-evaluation differences.","marker":"(Koehn, 2004)"}],"fun_headline_variants":["LLM length control traced to second attention layer","Scaling hidden units shifts LLM output length","Length found as separate units in LLM attention","Probe pinpoints length units in LLM second layer","Hidden units in attention layer adjust LLM length"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The identification of 'length units' assumes that high probe accuracy for the generation time step means a hidden unit encodes output length, rather than the position of the token being generated or the identity of that token.","fun_headline_variants_meta":{"raw":{"variants":["LLM length control traced to second attention layer","Scaling hidden units shifts LLM output length","Length found as separate units in LLM attention","Probe pinpoints length units in LLM second layer","Hidden units in attention layer adjust LLM length"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1380,"prompt_tokens":873,"completion_tokens":507,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":434}},"tokens_in":489,"tokens_out":507,"duration_ms":4997,"temperature":1.0,"reasoning_tokens":434,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:43:31.320095+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the per-unit probes only on hidden states taken at the first generated token, a fixed position, across summaries that end at different lengths; if the top-k 'length units' no longer predict final output length with R2 far above zero, the probes were reading absolute position rather than length, and the scaling result would not prove length control.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the InstructCMP instruction templates and the length-priming prompting method whose zero-shot and fine-tuned settings structure the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the regression-on-time-step probe for length information that this paper adapts to LLM hidden states."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the precedent that time-related linear representations in LLMs can be changed by disentangling hidden units, motivating the scaling intervention."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the paired bootstrap resampling used to test the significance of the human-evaluation differences."}],"review_version":2}