{"id":"a89593ca-2ffc-4e6c-99b1-42ffc07a349f","arxiv_id":"2411.15661","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A decoder-only model trained to predict the second-to-last token can slightly improve next-token predictions when used to re-rank a GPT's top-k candidates.","lead":"This paper trains a second model to predict the second-to-last token and uses it to re-rank the top candidates a GPT proposes for the next token. The second-to-last predictions are far more accurate, but the re-ranking improves next-token accuracy by only about 0.02 to 0.35 percentage points.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified","rationale":"The reader's verdict is CONDITIONAL, and I agree that the paper should not be accepted as-is without additional validation. However, I identify a different load-bearing concern than the reader's weakest assumption. The reader selected the theoretical locality assumption (Section 4, Eq. 5-6) as the load-bearing premise; I see the within-run-only statistical testing as more load-bearing. The paper's central empirical claim is that gains are 'consistent and statistically significant' across datasets. Because each model was trained once, the reported p-values are not evidence about training-seed variability. The gains on two of three datasets are 0.02 to 0.06 percentage points, which is the same order as typical seed-to-seed variance for GPT-2-scale training. The locality assumption, by contrast, is a plausibility argument for why the refiner might be more accurate; the empirical accuracy of fs is directly measured and high, so the method's mechanism is at least partially supported even without a proof of the inequality. Therefore the decisive check is a multi-seed replication. My recommendation remains CONDITIONAL: accept only after the multi-seed test confirms the gains are not a single-seed artifact, and after the unresolved top-k coverage inconsistency in Section 3.2 is corrected. This does not move the reader's verdict but sharpens the condition.","tokens_in":58,"tokens_out":5570,"duration_ms":171509,"concrete_test":"For the openwebtext and simplewiki configurations with the vanilla setting (k=15, w=0.05), train fs and fn from scratch with at least 5 different random seeds, holding data splits and hyperparameters fixed. For each seed, compute AGR minus fn accuracy on the same 70,000-token evaluation sample, and report the per-seed distribution. Test whether the mean gain across seeds is greater than zero with a one-sided paired t-test or sign test. If any seed yields a negative gain or the cross-seed 95% confidence interval includes zero, the claim of consistent significant gains fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 3.1 states 'We trained each model in Table 2 once.' All reported means and standard deviations come from ten evaluation runs that re-sample 70,000 token predictions from the same fixed model. The t-test in Section 3.2 (p<0.05 for each dataset) therefore only tests whether AGR beats fn for those particular trained instances, not whether the improvement is reproducible across training runs. The largest claimed vanilla gains outside simplewiki are 0.04 to 0.06 percentage points (Table 3), with evaluation-run SDs of 0.02 to 0.04; run-to-run training variance at GPT-2 scale is typically at least this large. Thus the central phrase 'consistent and statistically significant gains' is not supported by the experiments as reported. The locality assumption in Section 4 (Eq. 5-6) is also unproved, but the empirical claim would stand if fs happened to be more accurate; the statistical foundation is more load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a generate-then-refine method (Algorithm 1, AGR) for improving next-token predictions of a standard autoregressive GPT model. A second, decoder-only model fs is trained to predict the second-to-last token given the preceding context and a candidate last token; fs is then used to re-rank the top-k next-token candidates produced by the base model fn by upweighting candidates for which fs correctly identifies the actual second-to-last token. Training of fs uses a deterministic permutation of input subsequences that avoids a mask token, yielding an efficiency gain relative to BERT-style masking. Experiments on three GPT-2-scale models and three datasets report that fs has substantially higher accuracy than fn, and that AGR yields small but claimed statistically significant gains over fn on held-out validation data, with the largest gains on SimpleWiki (about 0.2-0.35 percentage points) and smaller gains on Stories and OpenWebText (about 0.02-0.06 points).","tokens_in":9314,"tokens_out":4302,"duration_ms":39424,"significance":"If the central claim were fully supported, this would be a modest but useful contribution: it demonstrates a non-circular, inference-time refinement mechanism that uses a second-to-last-token predictor to improve next-token accuracy, and it introduces a mask-free deterministic masking scheme for decoder-only bidirectional training. The paper ships a public code repository, evaluates on standard datasets and model sizes, and reports results on held-out data, so the main result is not circular. However, the statistical foundation is currently too weak to establish the headline claim of 'consistent and statistically significant gains' across datasets, because each model is trained only once and the reported t-tests reflect evaluation-sampling noise rather than training-seed variability. Given that the gains outside SimpleWiki are on the order of a few hundredths of a percentage point, run-to-run training variance at this scale could easily account for the observed differences. The theoretical justification in Section 4 is also presented imprecisely.","major_comments":[{"comment":"The paper states 'We trained each model in Table 2 once.' All reported means and standard deviations come from ten evaluation runs that re-sample 70,000 token predictions from the same fixed model. The t-test reported in Section 3.2 therefore only tests whether AGR beats fn for these particular trained instances, not whether the improvement is reproducible across training runs. This is load-bearing for the abstract's claim of 'consistent and significant gains.' For the two larger datasets, the vanilla configuration (k=15, w=0.05) yields gains of only 0.04±0.02 (Stories) and 0.06±0.04 (OpenWebText) percentage points, with standard deviations that are already large relative to the gains. Run-to-run training variance at GPT-2 scale is typically at least this large, so the reported t-test results cannot rule out the possibility that the gains are due to the particular trained instances. The authors should train multiple seeds (at least 3-5) per dataset and report the distribution of AGR minus fn across seeds, or otherwise provide evidence that training variance is negligible compared to the reported gains.","section":"Section 3.1 and 3.2, Table 3"},{"comment":"The formal statement of the locality assumption is not a well-defined inequality. Both sides of the inequality in Eq. (5) and Eq. (6) begin with the same term ||p*(yt|yi≠t) - p*(yt|y0, y1, ..., yt-1)||, so the inequality reduces to a comparison between distances from p*(yt|yi≠t) to two different conditional distributions, but the second distribution in Eq. (6) is written with a lowercase p(yt|y1, y2, ..., yt-1, yt+1) rather than p*(·), and no distance metric is specified. The prose then asserts that 'This holds true if knowing the next token yt+1 is more useful than knowing a token y0 that occurred much earlier,' but this assertion does not follow from the displayed equations and the cited references on distributed symmetry breaking and correlated initialization do not directly test the language-model setting. The authors should either rigorously reformulate this assumption (e.g., in terms of conditional entropy or Bayes error with a specified distance) or clearly label it as a heuristic conjecture; the empirical results should not be presented as a consequence of this assumption.","section":"Section 4, Eqs. (5)-(6)"},{"comment":"The paper explores several (k, w) configurations and highlights the best one (e.g., w=0.1 for SimpleWiki with p<0.001) without any correction for multiple comparisons. The text mixes the pre-specified vanilla configuration (k=15, w=0.05) with post-hoc selected configurations when discussing the strength of the results. This inflates the apparent evidence for the method. The primary analysis should be the pre-specified vanilla configuration (or an explicit multi-seed comparison), and the exploratory results should be clearly labeled as such, with appropriate caveats about selection-on-validation.","section":"Section 3.2 and Table 3"}],"minor_comments":[{"comment":"The abstract contains a typo: 'taskssuchaspredicting' should be 'tasks such as predicting'.","section":"Abstract"},{"comment":"The formula for the gain is garbled: it reads 'ΔAGR−fn := Accva AGR − Accva s' but the second term should be Accva n (the validation accuracy of the next-token model), not Accva s.","section":"Section 3.2"},{"comment":"The captions for Tables 1-3 show 'T able' with a space; this should be 'Table'.","section":"Table captions"},{"comment":"The symbol y is reused both for a candidate next token and for the actual second-to-last token y_{t-1} in the indicator function on line 6. Please use distinct notation (e.g., y for the candidate and y*_{t-1} for the ground-truth second-to-last token) to avoid ambiguity.","section":"Algorithm 1 and Section 2"},{"comment":"The claimed training-efficiency gain ('from 15% up to 25%') is never measured or compared against a concrete baseline (e.g., wall-clock time against a standard BERT-style masking implementation). A brief quantitative comparison would make the claim more precise.","section":"Section 2.1 and Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a single-author manuscript that cites several of the author's own prior works (Refs. 2, 20, 21, 22) in support of the locality assumption and the general transformer discussion. While self-citation is not inherently problematic, the reliance on these references rather than on directly relevant language-modeling literature for the central assumption of Section 4 may deserve editorial attention. The main technical concern is the single-seed training protocol, which is fixable in revision but is load-bearing for the paper's main claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Johannes,\n\nThe core idea here is worth a look: train a decoder-only LM on a deterministic permutation that predicts the second-to-last token, then use it to re-rank the top-k next tokens from a standard GPT. Dropping the mask token and using fixed positions is a small but real efficiency trick, and the code is public. The observation that second-to-last prediction is 15-20 points more accurate than next-token prediction across all three datasets is credible and matches the intuition that bidirectional context helps.\n\nThe problem is the central claim. The abstract says gains are 'consistent and significant,' and Section 3.2 reports t-tests with p<0.05. But Section 3.1 states each model was trained once. The t-tests are over ten evaluation subsamples of the same fixed model, so they show the gain isn't noise in the evaluation sample, not that it reproduces across training runs. For simplewiki the gain is large enough (0.21-0.35 points) that I'd bet on it, but for openwebtext and Stories the gains are 0.02-0.06 points with standard deviations of 0.02-0.04. That's within typical run-to-run variance of GPT-2-scale training. Worse, the paper highlights the best (k,w) configuration per dataset, which is selection on validation. The multiple-comparison issue isn't addressed.\n\nThere are also a few concrete slips. Section 3.2 says the correct token is absent from the top-15 for openwebtext in 75% of predictions, but that gives a top-15 coverage of 25%, below the reported top-1 accuracy of 42.4%. That can't be right as stated. And the theoretical discussion in Section 4 dresses the locality assumption as a formal inequality, but it's a plausible conjecture, not a proof; the citation to distributed symmetry breaking doesn't test language.\n\nThe paper does some things well: it's honest about the small size of the gains, discusses when the refiner fails (common tokens, missing top-k), and releases code. But as it stands, the contribution is a promising trick with weak statistical support for two of its three datasets.\n\nI'd send it to peer review, not desk reject, because the idea is new and easily testable. A referee should ask for multi-seed training, corrected multiple-comparison statistics, a fix for the coverage contradiction, and a baseline comparison with BERT-style masking and maybe ELECTRA or speculative decoding. If the simplewiki-scale gains replicate, it's a publishable contribution; the other datasets need more evidence.","headline":"A neat decoding trick with a too-strong abstract; the simplewiki result is solid, but the 'consistent significant gains' claim rests on single training runs.","tokens_in":9794,"tokens_out":3552,"would_cite":false,"duration_ms":31113,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a decoder-only model trained to predict the second-to-last token can serve as a refiner that improves the next-token predictions of standard GPT models.","keywords":["large language models","generate-then-refine","second-to-last token prediction","bidirectional decoder-only model","self-correction","deterministic permutation","next-token prediction"],"falsifier":"On a constructed corpus where every target token is followed by a fixed or random token unrelated to the target, train f_s and f_n with the paper's setup and compare their accuracies; if f_s is not more accurate than f_n, or if AGR fails to improve next-token accuracy, the locality assumption is violated for that data.","tokens_in":8930,"feed_emoji":"🔄","tokens_out":7377,"duration_ms":60350,"temperature":0.7,"pith_summary":"The paper tries to show that an autoregressive language model's next-token predictions can be improved by a second, independently trained model that predicts the second-to-last token given the candidate next token. The authors train this refiner as a decoder-only transformer using a deterministic permutation of short token subsequences, which avoids the mask token and random masking of BERT-style training. At inference, the refiner re-ranks the top-k next-token candidates of GPT-2 by boosting candidates whose predicted second-to-last token matches the actual one. They report that second-to-last-token accuracy exceeds next-token accuracy by more than 15%, and that the generate-then-refine combination yields small (0.02 to 0.35 points) but consistent and statistically significant gains across GPT-2 variants and datasets. If true, this gives a low-cost path to bidirectional self-correction for unidirectional models.","feed_headline":"Generate-then-refine boosts GPT next-token accuracy","feed_subtitle":"A second-to-last-token scorer re-ranks GPT's top picks, giving small but consistent accuracy gains.","key_machinery":"The mechanism is Algorithm AGR (Auto-encoding Generate-then-Refine), which couples a next-token model f_n with a second-to-last-token model f_s. For each of the top k candidates, f_s receives the candidate token and the preceding context, and if its argmax is the correct second-to-last token, the candidate's score is multiplied by (1+w). The refiner is trained by a deterministic permutation: each input sequence is split into subsequences of length l=4, and the fourth token is moved to the end (so (y0,y1,y2,y3) becomes (y0,y1,y2,y4)) with only the predictions at the moved positions contributing to the loss. This gives a decoder-only transformer bidirectional information without a mask token or random masking, and it keeps training close to the speed of ordinary causal next-token training.","core_discovery":"The central claim is that predicting the token just before a target token, using both earlier context and the target itself, is easier than predicting the target from preceding tokens only, and that a model trained solely for that auxiliary task can act as a verifier to improve the main model's choices. Concretely, for each of the top k next-token candidates from an autoregressive model, the auxiliary bidirectional model is prompted with that candidate as the final token; if the auxiliary model's most likely second-to-last token equals the true one, the candidate's probability is multiplied by (1+w). The paper shows that this re-ranking improves next-token accuracy on every tested GPT-2-scale configuration, and that the improvements are statistically significant for the default parameter setting even though they are small in absolute size. The paper also shows that the same decoder architecture, trained with a deterministic permutation of l-token subsequences, learns second-to-last prediction much faster and more accurately than it learns next-token prediction, indicating that the bidirectional task is genuinely easier for the model family.","pith_inferences":["The same re-ranking should apply to any autoregressive decoder, not just GPT-2, and to auxiliary targets other than the immediately preceding token (e.g., two tokens back), because the locality assumption is not specific to the GPT family.","Combining AGR with beam search or contrastive decoding seems natural: those methods change the candidate set, and the refiner would simply re-score the new candidates.","A testable extension would be to use the refiner during training, e.g., as an auxiliary loss head, rather than only at inference, which could make the next-token model itself learn to produce candidates that the refiner confirms."],"forward_implications":["The refiner f_s achieves 15-20 percentage points higher accuracy than f_n on the same data, so bidirectional context is substantially easier to learn than next-token prediction for decoder-only transformers.","Algorithm AGR improves next-token accuracy on all three datasets with the default k=15, w=0.05; a t-test gives p<0.05 for each, so the gains are statistically significant.","Increasing the boost weight w helps only when f_s is reliable; at w=0.1 the method loses accuracy on openwebtext and Stories, where f_s scores only about 60%, so the refiner's trustworthiness should be measured before weighting it heavily.","The deterministic permutation scheme removes the mask token and raises training efficiency relative to BERT-style random masking, offering a faster route to bidirectional pretraining."],"supporting_citations":[{"why":"Supplies the transformer architecture on which both f_n and f_s are built and which the entire method modifies.","marker":"[23]"},{"why":"Defines the BERT-style masked-token prediction that the paper contrasts its deterministic permutation against, and provides the efficiency baseline.","marker":"[6]"},{"why":"Provides the GPT-2 variants used as the autoregressive next-token model and as the starting architecture for the refiner.","marker":"[17]"},{"why":"Offers the permutation-based bidirectional alternative that the paper's deterministic permutation simplifies by avoiding a mask token and separate content/query representations.","marker":"[27]"},{"why":"Motivates the generate-then-refine framing by questioning when LLMs can self-correct, making the paper a response to that critique.","marker":"[9]"}],"fun_headline_variants":["Second-to-last token prediction refines GPT's next choices","Penultimate-token verifier improves next-token accuracy","Generate then refine: penultimate predictions improve GPT","Second-to-last predictions as verifier offer small but consistent gains","Penultimate prediction: an easy task that verifies GPT's next tokens"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the token immediately following a target word is more useful than an earlier token when predicting that target, so a bidirectional predictor can beat a unidirectional one; if this locality assumption fails on a dataset, the refiner cannot improve next-token predictions.","fun_headline_variants_meta":{"raw":{"variants":["Second-to-last token prediction refines GPT's next choices","Penultimate-token verifier improves next-token accuracy","Generate then refine: penultimate predictions improve GPT","Second-to-last predictions as verifier offer small but consistent gains","Penultimate prediction: an easy task that verifies GPT's next tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001048,"raw_usage":{"total_tokens":4381,"prompt_tokens":903,"completion_tokens":3478,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":3393}},"tokens_in":519,"tokens_out":3478,"duration_ms":25363,"temperature":1.0,"reasoning_tokens":3393,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:03:31.266989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a constructed corpus where every target token is followed by a fixed or random token unrelated to the target, train f_s and f_n with the paper's setup and compare their accuracies; if f_s is not more accurate than f_n, or if AGR fails to improve next-token accuracy, the locality assumption is violated for that data.","supporting_citations":[{"cited_title":"Advances in neural information pro- cessing systems (2017)","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer architecture on which both f_n and f_s are built and which the entire method modifies."},{"cited_title":"OpenAI blog (2019)","cited_arxiv_id":null,"evidence_quote":"Provides the GPT-2 variants used as the autoregressive next-token model and as the starting architecture for the refiner."},{"cited_title":"Advances in neural information processing systems (2019)","cited_arxiv_id":null,"evidence_quote":"Offers the permutation-based bidirectional alternative that the paper's deterministic permutation simplifies by avoiding a mask token and separate content/query representations."},{"cited_title":"Transactions of the Association for Computational Linguistics12, 1417–1440 (2024)","cited_arxiv_id":null,"evidence_quote":"Motivates the generate-then-refine framing by questioning when LLMs can self-correct, making the paper a response to that critique."}],"review_version":1}