{"id":"9e106799-e3b6-4ef7-a1fe-d6dc32174411","arxiv_id":"2505.08463","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Adding a single learned, input-independent offset to encoder outputs before decoding produces small but consistent downstream gains on many encoder-decoder PLMs.","lead":"RepCali inserts a tiny trainable block between a language model's encoder and decoder, adding a learned offset to the encoder's hidden states before decoding. Across 25 model variants and 8 tasks, the plug-in nudges most baselines upward while adding no more than 0.8 percent parameters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (2) makes RepCali's offset input-independent: a single vector is added to every encoder output, so with a frozen decoder it cannot close a content-dependent encoder-decoder gap; a constant-shift control is needed.","rationale":"Reader's weakest_assumption is input-independence of the calibration signal, and the math supports it: Eq. (2) is a function only of the constant ShapeSeed, so d_i is content-independent. The stronger point is that for a frozen decoder this degenerates even further: a position-independent additive shift leaves softmax attention over encoder keys unchanged, so the operation behaves as a constant bias rather than a representation-level calibration. This makes the central 'minimizing discrepancies' claim vulnerable. The empirical sweep is broad and consistently positive, which is real evidence of some benefit, but the paper does not provide a control distinguishing the proposed mechanism from (a) a constant offset, (b) extra trainable parameters, or (c) full-model fine-tuning; Appendix A's 'full-model fine-tuned' statement conflicts with §4.2's frozen-decoder description and must be resolved per table. I am not rejecting the empirical results, and the conditional verdict already captures the need for these controls, so my stress-test leaves the verdict unchanged.","tokens_in":15690,"tokens_out":8913,"duration_ms":93562,"concrete_test":"On MultiWOZ with MinTL(T5-base), run the exact RepCali recipe but replace Eq. (2)-(3) with p_i = h_i + c, where c is a single learned vector (same seed, same λ, same optimizer, same frozen/full fine-tuning protocol as the reported setting). If this constant-shift control matches RepCali's Inform/Success/BLEU-4 gains within seed-level noise, the improvements cannot be attributed to the proposed calibration block, since a content-independent offset is the only mechanism the paper specifies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (2) defines d_i = LearnEmb(ShapeSeed) with ShapeSeed an all-ones matrix, so after training d_i is the same vector for every token and every example. Eq. (3), p_i = h_i + λ d_i, therefore adds a fixed offset to all encoder outputs. For an encoder-decoder model with a frozen decoder, adding the same vector D to every encoder hidden state shifts each cross-attention key by W_K^T D; the additive term is identical for all keys, so the softmax attention weights are unchanged, and each attention head output merely gains a constant vector. The block is thus equivalent to a learned constant bias on decoder hidden states, not a content-dependent calibration. No evidence is provided that the encoder-decoder discrepancy takes this constant-offset form, and the paper's own Appendix A says the models were 'full-model fine-tuned' after adding the block, which conflicts with §4.2's claim that the decoder was frozen on NLU tasks. Until a control shows that a constant shift (or an equal-parameter bias) does not reproduce the reported gains, the central mechanistic claim that minimizing encoder-decoder discrepancies via calibration drives the improvements is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RepCali, a parameter-efficient fine-tuning method for encoder-decoder pre-trained language models. A calibration block comprising a learnable embedding applied to an all-ones ShapeSeed produces an offset d_i that is added with strength lambda to the encoder hidden states before they are passed to the decoder (Eqs. (1)-(4)). The authors claim this minimizes the discrepancy between the encoder's representation and the decoder's optimal input, and they report improvements over baselines across 25 PLM-based models and 8 tasks, including English and Chinese data, at a parameter overhead of 0-0.8%. The core empirical claim is that adding this block yields consistent downstream gains; the core mechanistic claim is that those gains arise from representation calibration.","tokens_in":15945,"tokens_out":7312,"duration_ms":70470,"significance":"If the empirical results are correct, RepCali would be an attractive plug-and-play component: it is simple, adds very few parameters, and the authors provide a broad sweep over BART, T5, PEGASUS, and derived models on multiple generation and understanding tasks, including a Chinese dialogue summarization dataset. The t-SNE visual analysis is a useful qualitative addition. However, the mechanistic interpretation is not currently established: as defined, the calibration offset is input-independent, and the paper does not provide the controlled experiments needed to distinguish calibration from a constant bias or from the effects of added parameters and changed optimization dynamics. The empirical claims are further weakened by missing variance information and by a training-protocol contradiction. The contribution is potentially publishable, but the load-bearing evidence must be strengthened.","major_comments":[{"comment":"Eq. (2) defines d_i = LearnEmb(ShapeSeed) with ShapeSeed an all-ones matrix, so d_i is the same vector for every token and every input. Eq. (3) then adds this fixed vector to every encoder hidden state. Under the §4.2 protocol with a frozen decoder, shifting every encoder output by a constant vector changes each cross-attention key by the same vector, so the softmax attention weights are unchanged and the calibration block reduces to a learned constant bias on decoder hidden states. Because the encoder-decoder discrepancy described in Section 1 is presumably content-dependent, this input-independent offset cannot by itself perform the claimed calibration. The paper needs a control experiment with an equal-parameter constant bias or learned decoder bias to show that the reported gains are not due merely to added parameters or changed optimization. Without that control, the mechanistic claim in Section 5.2 that minimizing encoder-decoder discrepancies drives the improvements is unsupported.","section":"§3, Eqs. (2)-(3) and §4.2"},{"comment":"Section 4.2 states that 'we froze the entire PLM decoder in NLU tasks' to reduce the fine-tuning parameter size and to validate RepCali's calibration, but Appendix A states that the baseline models were full-model fine-tuned and that 'we also full-model fine-tuned after integrating our representation calibration block into the baseline models.' These protocols are incompatible, and the manuscript does not state which protocol produced Table 1 and Tables 3-11. If the full-model protocol was used, the input-independence argument is moot because the entire model, including the decoder, is updated; if the frozen-decoder protocol was used, the parameter counts and training settings for those runs need to be described. This contradiction must be resolved before the results can be interpreted.","section":"§4.2 vs Appendix A"},{"comment":"The claim that RepCali 'significantly improves' performance is not supported by the evidence as reported. The text says results are averaged over three seeds, but no standard deviations, confidence intervals, or significance tests are provided. Many reported improvements are very small (e.g., Table 6 GAP on BLEU-4 is 0.00; Table 7 BRIO-Mul ROUGE-1 is +0.11) and could be within seed noise. In Table 1, baseline numbers are taken from Ding et al. (2022) rather than re-run under the authors' own training protocol (only BitFit is marked as reproduced), so the comparison to prompt, prefix, adapter, and LoRA is not matched. Please re-run all baselines under identical conditions or clearly restrict the comparison to comparable setups, and report seed-level variance or significance tests.","section":"§4.1-4.2 and Tables 1, 3-11"}],"minor_comments":[{"comment":"Section 4.1 says 'We compare the proposed method with three fine-tuning methods,' but Table 1 lists five baselines (Prompt tuning, Prefix-tuning, Adapter, LoRA, and BitFit); please correct the count or the table.","section":"§4.1"},{"comment":"The statement that ShapeSeed has size batchsize x n with 'n equals the length of token embedding' conflicts with the indexing {d_i}_{i=1}^n, which suggests sequence positions; please clarify the dimensions of ShapeSeed and the operation of LearnEmb.","section":"§3, Eq. (2)"},{"comment":"The parameter count for RepCali appears to omit the LayerNorm scale and bias parameters, which contribute 2 x d_h parameters; please update the formula or clarify what is counted.","section":"Table 2"},{"comment":"The abstract and Section 5 say the experiments include both English and Chinese datasets, but the only Chinese dataset in the manuscript appears in Appendix C (CSDS); please state this clearly in the main text or move the CSDS results into the main experimental section.","section":"Abstract and Section 5"},{"comment":"Several entries in Table 10 show no parameter increase after adding RepCali (e.g., MinTL(T5-small) 102M + RepCali 102M), while Section 5.3 reports 0-0.8% additional parameters; please report exact values or explain the rounding.","section":"Table 10"},{"comment":"The claim that RepCali produces a 'smoother' and 'more compact' latent space is based on visual inspection only; please add quantitative latent-space metrics or at least state the t-SNE settings and the number of points plotted.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript does not include a reproducibility statement or code release. Given the breadth of the empirical claims, I would ask the authors to provide code or detailed configuration files as part of the revision. The paper is in scope for cs.CL, but its contribution is mainly empirical; the editor may want to prioritize the controlled experiments and the frozen-versus-full-fine-tuning clarification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the short version: RepCali adds one learned vector to all encoder outputs before the decoder, and the paper reports consistent small gains across 25 encoder-decoder models. That sweep is real work and worth acknowledging. But the mechanism, as written, does not do what the paper says it does. Eq. (2) has d_i = LearnEmb(ShapeSeed) with ShapeSeed an all-ones matrix, so after training d_i is the same vector for every token and every example. Eq. (3) adds that constant to every encoder hidden state. With a frozen decoder, this shift cancels in the softmax attention weights and just adds a per-head constant to the decoder's cross-attention output; it is a learned bias, not a content-dependent calibration. The paper's own Appendix A says models were full-model fine-tuned after adding the block, which conflicts with §4.2's claim that the decoder was frozen on NLU tasks. Without a control (constant shift, or equal-parameter bias), the central mechanistic claim is unsupported.\n\nWhat is actually new: inserting a shift at the encoder-decoder interface and testing it on 25 models across 8 tasks. That breadth is the paper's main asset. The module is simple, cheap, and the tables are consistently positive, even if most gains are under a point. The comparison against Ding et al.'s numbers in Table 1 is not apples-to-apples, and no error bars or significance tests back the word 'significant.' SSF, the closest prior method, is cited but never compared. No code or key hyperparameters (λ) are released.\n\nThe stress-test note is right on the math. I'd add that the input-independence is not a fatal flaw for the empirical claim if the paper is reframed as 'a constant bias helps fine-tuning,' but that claim is much weaker than 'calibrates the representation gap.' The paper needs a control, a corrected training-protocol description, and a discussion of SSF. Then the empirical sweep could support a modest, useful conclusion.\n\nWho this is for: people working on parameter-efficient fine-tuning for seq2seq models. It deserves referee time, but it should not be accepted as is. I'd send it back for major revision with the control and SSF comparison as required.\n\nBest,\n[Your name]","headline":"RepCali is a simple additive-shift tuning module with a broad empirical sweep, but the mechanism as written reduces to a constant bias and the central calibration claim is not yet supported.","tokens_in":16463,"tokens_out":2612,"would_cite":false,"duration_ms":26949,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single learned latent-space offset added to encoder outputs can improve virtually any encoder-decoder PLM, the paper claims.","keywords":["representation calibration","latent space","parameter-efficient fine-tuning","encoder-decoder language models","plug-and-play adaptation","fine-tuning for LLMs","sequence-to-sequence learning","downstream task adaptation"],"falsifier":"Probe the input-independence assumption directly: on a trained RepCali model, split the test set by topic or input type, then for each group solve for the group-specific offset $d_g$ that minimizes decoder loss with all other weights frozen. If the optimal $d_g$ vectors differ materially across groups, one shared offset cannot be closing the true encoder-decoder gap, so the calibration mechanism is not what the results are showing.","tokens_in":15498,"feed_emoji":"📐","tokens_out":9626,"duration_ms":96082,"temperature":0.7,"pith_summary":"RepCali starts from a diagnosis: after fine-tuning, encoder-decoder PLMs still leave a gap between the distribution of the encoder's hidden states and the distribution the decoder was optimized to consume. The paper proposes closing that gap in latent space with a tiny plug-in block, made of an all-ones matrix passed through a learnable embedding and layer normalization, whose output is scaled and added to every encoder state before the decoder. The paper reports that this block works across 25 PLM-based models, 8 tasks, English and Chinese data, with only 0–0.8% additional parameters, and beats prompt tuning, prefix tuning, adapters, LoRA, and BitFit on a four-task head-to-head comparison. The payoff, if correct, is a nearly free universal upgrade for any encoder-decoder PLM that requires no change to the base model's loss or architecture.","feed_headline":"One learned vector lifts 25 language models across 8 tasks","feed_subtitle":"RepCali adds 0–0.8% parameters, sits between encoder and decoder, and beats prompt tuning, adapters, and LoRA.","key_machinery":"The load-bearing object is the calibration block, which sits between the encoder and decoder. It has three pieces: ShapeSeed, an input-shaped matrix initialized to all ones; LearnEmb, a learnable embedding layer that maps ShapeSeed to the calibration vector $d_i$; and layer normalization. The block's output is added to the encoder outputs through $p_i = h_i + \\lambda\\, d_i$, where $\\lambda$ controls the strength of the calibration, and the calibrated sequence $p_i$ is what the decoder consumes. This is the entire new machinery: it is position-wise, input-independent, introduces only one extra embedding matrix, and is trained by whatever downstream loss the original model uses.","core_discovery":"The paper's central claim is that a large share of the remaining gap in encoder-decoder PLMs can be removed by a single learned correction in latent space, inserted between the encoder and decoder. The correction is parameterized by a calibration block: ShapeSeed, an input-shaped matrix initialized to all ones, is mapped by a learnable embedding and layer normalization to a calibration vector $d_i$, and the decoder receives $p_i = h_i + \\lambda\\, d_i$ instead of the raw encoder output $h_i$. Because the correction is input-independent, once trained the block is effectively a task-specific offset added to every encoder state. The paper reports that 25 PLM-based models improve on 8 downstream tasks, including generation, understanding, summarization, and dialogue tasks, with parameter growth of at most 0.8%, and that the block outperforms prompt tuning, prefix tuning, adapters, LoRA, and BitFit on a four-task T5-base comparison.","pith_inferences":["Because the trained calibration is input-independent, RepCali is equivalent to adding one global bias in the encoder's hidden space; comparing it against per-layer or per-head bias vectors at matched parameter counts would show whether a single surface offset is the right granularity.","A content-dependent variant, where $d_i$ is conditioned on a pooled encoder state, would be the natural next test: the paper's diagnosis predicts larger gains exactly where the optimal correction varies across topics.","Another testable extension is cross-task transfer: freeze a calibration block trained on one task and reuse it on another with the same encoder-decoder; positive transfer would support the claim that the gap being corrected is a general property of the model rather than a task-specific artifact."],"forward_implications":["Any encoder-decoder PLM, from BART-base to a 4.5-billion-parameter T5-3B, can carry the calibration block with at most 0.8% added parameters and no change to the task loss.","On the four-task T5-base comparison, RepCali outperforms prompt tuning, prefix tuning, adapters, LoRA, and BitFit, with a 77.55 average score.","Inserting the block improves every one of the 25 baseline models across the eight English and Chinese tasks reported, including state-of-the-art systems such as MoKGE, JointGT, GAP, BRIO, and RE-BART.","Because the correction happens at the encoder-decoder interface, RepCali is agnostic to the inner structure of the PLM and can be layered on top of existing task-specific models such as MinTL, KB_BART, and keyword-control systems."],"supporting_citations":[{"why":"Supplies the baseline results for prompt tuning, prefix tuning, adapter, LoRA, and BitFit in the four-task comparison.","marker":"Ding et al. (2022)"},{"why":"Provides the MinTL framework and the MultiWOZ baselines that RepCali is inserted into.","marker":"Lin et al. (2020)"},{"why":"Defines LoRA, the strongest parameter-efficient baseline in the head-to-head comparison.","marker":"Hu et al. (2022)"},{"why":"Defines the adapter module used as one of the compared fine-tuning methods.","marker":"Houlsby et al. (2019)"},{"why":"Defines prompt tuning, one of the compared fine-tuning methods.","marker":"Lester et al. (2021)"},{"why":"Defines the T5 model family used as the backbone in the head-to-head comparison and multiple downstream tasks.","marker":"Raffel et al. (2020)"},{"why":"Supplies the ART dataset and the abductive-commonsense-reasoning task used in the alphaNLG experiments.","marker":"Bhagavatula et al. (2020)"},{"why":"Provides MoKGE, the strongest baseline on alphaNLG that RepCali improves on.","marker":"Yu et al. (2022)"}],"fun_headline_variants":["One learned offset boosts 25 language models on 8 tasks","Single latent-space fix lifts 25 PLMs, adds under 1% params","RepCali: a tiny latent-space correction that outperforms LoRA","Adds 0.8% params, one vector lifts 25 models","Plug-and-play calibration block improves 25 PLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one fixed calibration vector can close the encoder-decoder gap for every input, position, and domain in a task; if the discrepancy is content-dependent, the block is not actually calibrating, and the reported gains would need to be explained by added parameters, optimization dynamics, or regularization.","fun_headline_variants_meta":{"raw":{"variants":["One learned offset boosts 25 language models on 8 tasks","Single latent-space fix lifts 25 PLMs, adds under 1% params","RepCali: a tiny latent-space correction that outperforms LoRA","Adds 0.8% params, one vector lifts 25 models","Plug-and-play calibration block improves 25 PLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000506,"raw_usage":{"total_tokens":2465,"prompt_tokens":942,"completion_tokens":1523,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":1430}},"tokens_in":558,"tokens_out":1523,"duration_ms":11670,"temperature":1.0,"reasoning_tokens":1430,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:54:24.698981+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Probe the input-independence assumption directly: on a trained RepCali model, split the test set by topic or input type, then for each group solve for the group-specific offset $d_g$ that minimizes decoder loss with all other weights frozen. If the optimal $d_g$ vectors differ materially across groups, one shared offset cannot be closing the true encoder-decoder gap, so the calibration mechanism is not what the results are showing.","supporting_citations":[],"review_version":1}