{"id":"d60abb2e-88d4-4441-8e0b-071380109e28","arxiv_id":"2510.02329","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SelfJudge automatically labels token-acceptance data via target-model semantic preservation scores and trains a logistic-regression judge verifier, improving speculative decoding speed while limiting task accuracy loss on five benchmarks.","lead":"SelfJudge trains a lightweight judge verifier for speculative decoding using the target LLM's own likelihood comparisons as training labels, replacing human annotations and answer-ground-truth signals. The method reports better accuracy-efficiency trade-offs than prior judge decoding baselines across math, code, summarization, and general QA benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'self-supervised' claim is undercut by an AutoJudge-calibrated threshold: τ in §4.1 is set from GSM8K answer-based labels, so the 'no ground truth' and cross-task generality claims are not yet demonstrated.","rationale":"The paper's strongest claim is that judge verifiers can be trained via self-supervision, with no human annotation or verifiable ground truth, and that this enables diverse NLP tasks. The threshold calibration in §4.1 is a direct, admitted use of AutoJudge—which itself relies on final-answer ground truth—to set the labeling threshold. This is not a side detail: it determines which of the 69,432 training tokens are labeled acceptable, and it is the only place the paper explains how τ is set. Without this step, the method as described cannot be applied to a truly open-ended task. A reviewer could accept the method as a label-efficient or semi-supervised variant, but the abstract's 'self-supervision' and 'without ground truths' wording is too strong. The reader's designated weakest assumption is the train/inference distribution shift; that concern is real but untested, whereas the τ calibration is explicitly present in the text and directly contradicts the central claim. I therefore treat the threshold dependency as the most load-bearing issue. If the proposed ablation shows stability, the concern is resolved; if not, the paper should either provide a self-contained τ selection procedure or qualify the self-supervision claim. The empirical trade-offs and Appendix B data-generation cost analysis are creditable; the issue does not warrant rejection, but it should be resolved before the strong claim is accepted as stated. I retain the reader's CONDITIONAL recommendation, hence no change to the verdict.","tokens_in":20249,"tokens_out":14008,"duration_ms":128923,"concrete_test":"Re-run the label-generation pipeline without any AutoJudge/ground-truth calibration. On the same 100 GSM8K queries, set τ to the 0.1 quantile of SelfJudge's own semantic-preservation score distribution (or another fixed quantile chosen a priori), generate labels for the full training set, retrain the logistic verifier, and recompute Table 1 (accepted length and task accuracy) on MMLU and CNN/DailyMail. If the trade-off is materially unchanged (e.g., Δm within 0.3 and Δtask within 0.5 points of the published AutoJudge-calibrated run), the calibration is not load-bearing; if performance degrades or becomes sensitive to the chosen quantile, the 'self-supervised across diverse tasks' claim fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SelfJudge trains judge verifiers without human annotations or verifiable ground truths and generalizes across diverse NLP tasks. Section 4.1 (Implementation Details) contradicts this: the label threshold τ is set using 100 GSM8K queries on which AutoJudge has identified 'ground-truth important tokens' (tokens whose replacement changes the final answer), and τ = quantile(semantic preservation scores of AutoJudge-rejected tokens, 0.1). This makes the binary accept/reject labels for the 69,432 training tokens (Table 2) depend on the very answer-based supervision the paper claims to avoid. It also entangles the published comparison against AutoJudge: SelfJudge's labeling is calibrated to reject everything AutoJudge rejects, so the evaluation does not test a fully self-supervised pipeline. Finally, because calibration is performed only on GSM8K, the claim of automatic verifier training across diverse NLP tasks (e.g., open-ended summarization/QA without answer labels) is not supported; no self-contained procedure for choosing τ on such tasks is provided. The empirical trade-offs in Table 1 may survive a different τ, but the paper as written does not show this.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SelfJudge, a method for training a judge verifier for speculative decoding without human annotations or verifiable ground truth. The method labels mismatched draft tokens by computing a semantic preservation score, defined as the target-model log-likelihood difference between a token-substituted response and the original response, decomposed into prefix and suffix terms (Eqs. 6-8); tokens above a threshold τ are labeled acceptable. A logistic-regression verifier is trained on target-model hidden states with these labels, and at inference it accepts draft tokens when the verifier score exceeds θ, with a two-stage fallback to standard rejection sampling for rejected tokens. Experiments on Llama-3.1-8B/Llama-3.2-1B and Qwen-2.5-7B/0.5B over GSM8K, MATH-500, LiveCodeBench, CNN/DailyMail, and MMLU report average accepted lengths and task accuracies, claiming superior accuracy-efficiency trade-offs over SD, top-k verification, AutoJudge, and AutoJudge+.","tokens_in":20575,"tokens_out":6707,"duration_ms":56262,"significance":"The contribution is potentially significant. If the claims hold, SelfJudge would remove the task-specific supervision bottleneck of judge decoding: the target model itself generates token-level acceptability labels, the labeling procedure is substantially cheaper than AutoJudge's (8 hours versus 82 hours on the reported workloads), and the verifier is a simple logistic regressor that operates on frozen hidden states. The decomposition of the semantic preservation score into a prefix term and a suffix log Bayes factor is a useful formalization, and the empirical results show SelfJudge achieving higher or comparable accepted lengths than AutoJudge while retaining accuracy across math, code, summarization, and general QA, including an MMLU setting where AutoJudge degrades. The paper also provides a threshold-sweep analysis rather than only a single operating point. These strengths make the work worth pursuing.","major_comments":[{"comment":"The central claim that SelfJudge trains verifiers without human annotations or verifiable ground truth is undercut by the threshold calibration: τ is set on 100 GSM8K queries using AutoJudge's answer-preservation labels, specifically τ = quantile(semantic preservation scores of unacceptable tokens determined by AutoJudge, 0.1), with the stated goal of guaranteeing that all tokens rejected by AutoJudge are also rejected by SelfJudge. As a result, the 69,432 training labels inherit the answer-based supervision that the paper claims to avoid, and the evaluation is entangled with the AutoJudge baseline it compares against. Moreover, no self-contained procedure is provided for choosing τ on tasks without verifiable answers, such as CNN/DailyMail or MMLU, so the claimed cross-task generality of 'automatic verifier training' is not demonstrated. The authors should add a sensitivity analysis that selects τ without AutoJudge labels, for example a fixed quantile of the score distribution or a validation set from Dolly15k, and report whether the Table 1 conclusions hold.","section":"§4.1, Implementation Details"},{"comment":"There is a train/inference distribution shift that the paper does not directly test. Training labels are computed with access to N future tokens y_{>i} of the target-complete response, with N=20 in the experiments, while at inference the verifier must judge acceptability from causal hidden states of a draft-token context (Eq. 4), where previously accepted draft tokens may already deviate from the target distribution. The two-stage rejection sampling can mask verifier errors, so end-to-end accuracy alone does not show that the verifier's ranking is reliable under this shift. I ask for a direct diagnostic, such as verifier precision/recall against oracle labels on actual draft contexts, the score distribution for tokens that are later rejected by the fallback, and an evaluation with stronger draft models or longer generation horizons to test whether the trade-off degrades.","section":"§3.3.2, Eq. (6); §3.4 and §4.1"},{"comment":"The headline aggregate Δm/Δtask comparison is not a like-for-like summary. SelfJudge uses 69,432 training labels versus 14,896 for AutoJudge and 39,069 for AutoJudge+ (Table 2), and the averaged row hides AutoJudge-R's collapse to 0.7 Pass@1 on LiveCodeBench, which inflates its average accepted length; SelfJudge's aggregate, by contrast, includes a LiveCodeBench row with 10.0 Pass@1. The paper should report per-task trade-offs, compare against an AutoJudge+ variant trained with the same Dolly15k data, and provide variance or repeated-run statistics for the small accuracy differences before claiming a consistent superior trade-off.","section":"Table 1 and Table 2; Section 4.2"},{"comment":"The paper reports average accepted length m as the sole efficiency metric and never reports wall-clock latency or the overhead of the logistic verifier. Since the title and abstract claim 'faster speculative decoding,' the method should be evaluated in end-to-end tokens/sec or latency; if accepted length is meant to be the only efficiency metric, that should be explicitly stated and justified.","section":"Throughout, especially §4.2 and Figure 3"}],"minor_comments":[{"comment":"Model names are inconsistent: Qwen-2.5-8B appears in §4.1 and Table 1, while Qwen-2.5-7B appears in Table 2 and §4.3; Table 3 uses Llama3.2-0.5B where the text reports Llama3.2-1B. Please harmonize all model names.","section":"§4.1, Table 2, Table 3"},{"comment":"Equation (7) contains notation typos, including the terms 'y_i>i' and the aligned cancellation symbols, which are hard to parse; please rewrite with clean y_{<i} and y_{>i} notation.","section":"Equation (7)"},{"comment":"The header 'Freuquently' in Tables 4-6 is misspelled and should be 'Frequently'; the column header 'OriginAlter' should be spaced; and 'Observartion1' in §4.2 should be 'Observation1.'","section":"Tables 4-6 and §4.2"},{"comment":"The AutoJudge+ labeling procedure using LLMs-as-judges is not described with enough detail to be reproducible; please provide the exact judge prompt, the [SAME]/[DIFFERENT] criterion, and the model used for labeling.","section":"Appendix B.1, AutoJudge+ description"}],"recommendation":"major_revision","confidential_remarks":"The τ-calibration issue in §4.1 is the deciding factor. If the authors cannot provide a self-contained procedure for selecting τ on tasks without verifiable answers, the paper's scope should be narrowed and the comparison with AutoJudge reframed as calibration-dependent. I see no evidence of misconduct or citation problems; the request for additional experiments is within the scope of a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on speculative decoding. The core idea is real: instead of human labels or answer-preservation, they define token acceptability by asking whether replacing a token with the draft's alternative changes the target model's likelihood of the rest of the response. The bidirectional likelihood-difference score (Eqs. 6-8) is a clean, parameter-free way to generate verifier training data, and it is much cheaper than AutoJudge's approach—8 hours for 69k labels versus 82 hours for 15k. That is a genuine practical win. The empirical section is also mostly honest: they report accepted-token length and task accuracy across five benchmarks, and SelfJudge consistently beats AutoJudge on the accuracy-efficiency frontier, especially on open-ended tasks where AutoJudge's answer-based labels are useless. The two-stage verification (judge first, rejection sampling for rejects) is sensible.\n\nNow the soft spots, in order. The abstract says 'without human supervision or verifiable ground truths,' but Section 4.1 sets tau using 100 GSM8K queries where AutoJudge's ground-truth-important tokens define unacceptable tokens. So the labeling pipeline does depend on answer-based supervision for its one global threshold. That does not kill the method—tau is a scalar and the labels themselves come from the new semantic-preservation score—but the 'fully self-supervised across diverse tasks' claim is not demonstrated. They would need to show a principled way to pick tau without AutoJudge, or at least report sensitivity to tau.\n\nSecond, the speedup is reported as average accepted length, not wall-clock latency. That is common in the SD literature, but the title says 'Faster,' and accepted length is not latency.\n\nThird, there is a train/inference shift they do not directly test: training labels use future tokens, while the verifier at inference sees only causal hidden states. Their end-to-end accuracy counters this concern somewhat, but a direct probe would be easy and would make the paper stronger.\n\nFourth, Appendix A is a tautology dressed as a theorem. It just restates the definition of conditional mutual information. Not a problem for the method, but it should be cut or reframed.\n\nNet: the central idea is sound and the results are favorable, but the 'no ground truth' framing overreaches. The paper deserves a serious referee; I would send it out and ask for the tau sensitivity analysis, a wall-clock measurement, and a more measured abstract. Conditional accept.","headline":"SelfJudge has a genuinely new labeling idea and mostly good results, but the 'no ground truth' claim does not survive contact with Section 4.1, where the threshold tau is calibrated using AutoJudge answer labels.","tokens_in":21049,"tokens_out":2627,"would_cite":true,"duration_ms":22399,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that a judge verifier for speculative decoding can be trained from the target model's own likelihoods, removing the need for human labels or verifiable ground truth.","keywords":["speculative decoding","judge verification","self-supervised labeling","semantic preservation","likelihood ratio","LLM inference acceleration","logistic regression verifier"],"falsifier":"Run SelfJudge on a held-out set of prompts, record every draft token the verifier accepts, and then ask independent human raters or a strong judge model whether replacing that token with the target model's own token changes the meaning of the final response; if a large fraction of accepted substitutions are meaning-changing, the self-supervised labels are not transferring to online contexts.","tokens_in":20011,"feed_emoji":"⚡","tokens_out":7455,"duration_ms":59781,"temperature":0.7,"pith_summary":"SelfJudge proposes that the verifier used in speculative decoding can be trained automatically from the target model's own judgments rather than from human annotations or task-specific answers. The paper defines a semantic preservation score that compares the target model's likelihood of its original response with the likelihood after replacing one token with the draft token, and labels a replacement as acceptable when the difference is small. These labels, computed offline, train a lightweight logistic-regression verifier on the target model's hidden states; at inference the verifier accepts draft tokens that preserve meaning, and any rejected tokens still go through standard rejection sampling. On math, code, general QA, and summarization tasks, the paper reports more accepted tokens per verification cycle than exact-alignment baselines while losing less task accuracy. If this holds, judge-based speculative decoding stops being limited to domains with checkable answers and becomes a general speed-up technique.","feed_headline":"Self-trained judge speeds up LLM decoding with less accuracy loss","feed_subtitle":"Token acceptance based on meaning preservation beats exact-match baselines on math, code, QA, and summarization.","key_machinery":"The central object is the semantic preservation score $s(y, z_i)$, a bidirectional likelihood comparison defined as the log-probability of the draft's alternative token minus the log-probability of the target's own token, both conditioned on the full original response around the position. Because autoregressive models cannot condition on the suffix directly, the score is rewritten as a prefix score plus a suffix likelihood ratio (a log Bayes factor), and computed by feeding the token-substituted response through the target model. This score generates the automatic training labels; the learned logistic regression on hidden states is what makes the score available online, when future tokens are unknown. A two-stage verification scheme, apply the judge first and then apply alignment-based rejection sampling to rejected tokens, keeps the method from discarding valid tokens.","core_discovery":"The central claim is that token-level acceptance in speculative decoding can be grounded in semantic preservation measured by the target model itself. For each position where the draft model's top token differs from the target token, SelfJudge computes $s(y,z_i) = \\log P(z_i \\mid y_{<i}, y_{>i}) - \\log P(y_i \\mid y_{<i}, y_{>i})$, decomposes it into a prefix log-probability difference plus a suffix log Bayes factor, and labels the replacement acceptable when $s$ exceeds a threshold. The labels become training data for a logistic regression verifier that reads the target model's hidden states during inference. The reported result is that this self-supervised verifier accepts semantically equivalent paraphrases and formatting variants, producing higher accepted length with smaller accuracy degradation than answer-preservation judges across diverse NLP tasks.","pith_inferences":["A testable extension would measure how well the verifier's online accept or reject decisions match the offline semantic preservation labels when the context contains previously accepted draft tokens; large disagreement would indicate that future-token supervision does not transfer to causal decoding contexts.","The same labeling procedure could be used to fine-tune or distill the draft model itself toward tokens the target model judges as semantically acceptable, potentially compounding the speedup by making drafts more acceptable before verification.","Because the score's suffix term is a Bayes factor, one could weight it by the mutual information between the token and future context, or tune the suffix length per position, rather than using a fixed $N=20$ for all tokens."],"forward_implications":["Judge verification becomes applicable to open-ended tasks like creative writing and instruction following, where no ground-truth answer exists to label accept or reject decisions.","Training verifier data is cheaper than answer-preservation labeling because SelfJudge prefills a short suffix instead of generating an entire alternative completion for every token.","A fixed lightweight verifier trained on one mixture of tasks transfers to held-out math, code, QA, and summarization benchmarks within the same target and draft model pair.","Because rejected tokens are still checked by rejection sampling, the method bounds quality loss: the only relaxed decisions are the ones the judge accepts, so accuracy degradation is controlled by the judge's precision.","End-to-end inference latency improves whenever the draft model produces semantically equivalent but lexically different tokens, which is common for sentence starters, punctuation, synonyms, and formatting."],"supporting_citations":[{"why":"Introduces judge decoding with a learned verifier on hidden states, the setup SelfJudge replaces the data labeling for.","marker":"Bachmann et al., 2025"},{"why":"Provides the answer-preservation automated labeling baseline whose domain limitation SelfJudge aims to overcome.","marker":"Garipov et al., 2025"},{"why":"Establishes draft-then-verify speculative decoding and the rejection-sampling acceptance criterion.","marker":"Leviathan et al., 2023"},{"why":"Formalizes modified rejection sampling that SelfJudge retains as the backstop verification stage.","marker":"Chen et al., 2023"},{"why":"Supplies GSM8K, used both for calibrating the labeling threshold and for math reasoning evaluation.","marker":"Cobbe et al., 2021"},{"why":"Supplies MMLU, the general knowledge benchmark used to show the verifier generalizes beyond math and code.","marker":"Hendrycks et al., 2021"}],"fun_headline_variants":["Self-supervised judge speeds LLM decoding via meaning preservation","SelfJudge: Self-trained verifier cuts decoding accuracy loss","Meaning-aware token acceptance accelerates LLM speculative decoding","SelfJudge: Auto-verifier improves decoding speed without accuracy hit","Self-supervised semantic check speeds up LLM token acceptance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a verifier trained on labels derived from the target model's complete responses will judge draft tokens correctly during inference, when it sees only a partial causal context and no future tokens; the paper does not directly measure this train and inference gap.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised judge speeds LLM decoding via meaning preservation","SelfJudge: Self-trained verifier cuts decoding accuracy loss","Meaning-aware token acceptance accelerates LLM speculative decoding","SelfJudge: Auto-verifier improves decoding speed without accuracy hit","Self-supervised semantic check speeds up LLM token acceptance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001129,"raw_usage":{"total_tokens":4640,"prompt_tokens":839,"completion_tokens":3801,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":3720}},"tokens_in":455,"tokens_out":3801,"duration_ms":21885,"temperature":1.0,"reasoning_tokens":3720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:45:37.479961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SelfJudge on a held-out set of prompts, record every draft token the verifier accepts, and then ask independent human raters or a strong judge model whether replacing that token with the target model's own token changes the meaning of the final response; if a large fraction of accepted substitutions are meaning-changing, the self-supervised labels are not transferring to online contexts.","supporting_citations":[{"cited_title":"Judge decoding: Faster speculative sampling requires going beyond model alignment","cited_arxiv_id":null,"evidence_quote":"Introduces judge decoding with a learned verifier on hidden states, the setup SelfJudge replaces the data labeling for."}],"review_version":2}