{"id":"2298f7c0-c91f-4ffe-8be2-fdad12dea9e3","arxiv_id":"2501.01668","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A small CoT-based synthesizer model trained on candidate-response analysis improves LLM reasoning accuracy, including cases where all sampled candidate answers are incorrect.","lead":"This paper trains a small language model to read several candidate answers to a reasoning question and write a new, better answer, rather than just picking the most common or best-scored one. Even when every candidate answer is wrong, the synthesizer sometimes produces the correct answer, improving accuracy on math and table reasoning benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training labels y may be produced from x alone, so Synthesizer-8B's gains could reflect a second-model ensemble rather than candidate synthesis; the R-dependence of both labels and outputs needs a perturbation test.","rationale":"After reading the paper carefully, I think the most load-bearing assumption is exactly the one flagged by the reader: the training labels y must encode information from R for the trained Synthesizer to learn candidate synthesis. The reported empirical gains are consistent with a simpler ensemble explanation, and the paper's own ablations do not rule it out. I do not see a more serious issue. Missing variance reporting and the uncited Mixture-of-Agents comparison are real but secondary; they would affect confidence, not the central mechanism. Lack of formal verification is not a concern for an empirical paper; reproducibility of code and data helps. The proposed perturbation test is cheap and decisive: if Synthesizer-8B's accuracy is unchanged when R is replaced by unrelated responses, then the model is not using candidate information, and the paper's title and abstract claim should be revised to describe an answer-generator ensemble rather than synthesis. If the test shows sensitivity to R, the conditional concern is resolved and the reader's verdict can stand. I therefore leave the verdict unchanged; it is conditional pending this check.","tokens_in":19386,"tokens_out":5164,"duration_ms":52653,"concrete_test":"Run a perturbation test on MATH500 with Llama3-8B as policy. For each of, say, 300 queries, sample R once. Then run Synthesizer-8B four ways: (i) true R; (ii) R randomly shuffled; (iii) R replaced by candidates sampled for a different query; (iv) x-only prompt with no R. Compare exact-match accuracy. If accuracy in (iii) and (iv) is statistically indistinguishable from (i) (e.g., within one standard error over 3 seeds), the model is solving x independently of R, so the synthesis claim is not supported. If (i) significantly exceeds (iii)/(iv), the concern is resolved. As a supplementary check on the data labels, take a random 100 (x,R,y) triples and ask Llama3.1-70B for y given x only; high reproduction rate would confirm labels are x-only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—'synthesize superior answers by analyzing complementary information from multiple candidate responses, even when all candidate responses are flawed'—requires that the trained model p_phi(y|x,R) actually use R. In the Section 4.2 pipeline, labels y are generated by Llama3.1-70B prompted with (x,R) and then filtered for correctness against the gold answer y_g. Nothing in this pipeline verifies that y is a function of R. Because the prompt contains the full problem x and the response LLM is a strong reasoner, y can be correct even when R is ignored; filtering on y_g does not remove such labels. The LLM Repair stage has the same property: it asks the model to reflect on R, but R is not necessary for producing a correct y. Training with Eq. (1) then fits p(y|x,R) under labels that may be independent of R, so Synthesizer-8B can learn a direct answer generator p(y|x). Under that alternative, the Table 1 gains and the '0 correct candidates -> 9 correct' result in Table 3 are explained by ensembling an 8B distilled-70B model with the policy model, not by candidate synthesis. Table 2's untrained baseline already gives +8.0 on Llama3-8B MATH500, showing that even without training the prompt can elicit useful x-only answers, which is exactly the confound. The 'w/o CoT training' ablation is not a control because it removes CoT labels but does not test whether R is used at inference. Thus the mechanism underlying the paper's headline claim is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoT-based Synthesizer, an inference-scaling method in which a trained small LLM takes a question and a set of candidate responses produced by a policy model, reasons about the candidates via chain-of-thought, and synthesizes a final answer. The authors introduce an automated pipeline that uses a sampling LLM to generate candidate sets and a response LLM to produce synthesis labels, filters those labels against gold answers, and then fine-tunes Llama3-8B-Instruct to imitate the synthesis process. Experiments on GSM8k, MATH500, WikiTQ, and FeTaQA across seven policy models report consistent gains over Self-consistency, Best-of-N, and LMCOR, including 11.8% absolute improvement for Llama3-8B and 10.3% for GPT-4o on MATH500. The paper also includes ablations of the data pipeline, scaling experiments over data size and candidate count, and an analysis table showing that the Synthesizer produces correct final answers on some problems where all candidates are incorrect.","tokens_in":19675,"tokens_out":5389,"duration_ms":54357,"significance":"If the central claim is correct, the method is a practical and cost-effective inference-scaling strategy: it trains a single 8B synthesizer once and then improves a variety of policy models, including closed API models, without requiring retraining or a separate verifier. The paper is commendable for releasing code and data, for evaluating across four benchmarks and seven policy models, and for including scaling analyses and ablations of the data-generation pipeline. The reproducibility-oriented reporting (three runs, public repository) is a strength. However, the load-bearing claim that the Synthesizer actually exploits the candidate set R is not established: the training-label construction does not guarantee that labels depend on R, and the ablation evidence is consistent with a simpler explanation in which the trained model is a direct answer generator distilled from the response LLM. Because the paper's novelty and headline results depend on the mechanism of candidate synthesis, this gap is central rather than cosmetic.","major_comments":[{"comment":"The headline claim of \"significant\" improvement is not supported by any variance or significance reporting. Each result is described as the average of three runs, but no standard deviation, confidence interval, or test (e.g., paired bootstrap or McNemar) is given. Many decisive cells, such as MATH500 for Qwen2.5-14B (78.2 vs. 78.2 for USC) and GSM8k for Llama3.1-70B (94.0 vs. 93.9 for SC), differ by well under the noise floor of a 500- or 1319-sample test set. The paper should report per-cell variance and a significance test for at least the headline comparisons, and should temper the claim of consistent superiority accordingly.","section":"5.1 / Table 1"},{"comment":"The training pipeline does not verify that the synthesized label y is actually a function of the candidate set R. In \"Synthetic Answer Generation\", the response LLM is prompted with x and R and multiple outputs are filtered by equality with the gold answer y_g; this filter removes incorrect answers but does not remove labels that were produced by solving x directly while ignoring R. The same issue applies to \"LLM Repair\". Because the response LLM is a strong reasoner, a large fraction of filtered labels may be independent of R. Training with Eq. (1) can then fit p_phi(y|x) rather than p_phi(y|x,R), and the gains in Tables 1 and 3 would be explained by ensembling or distillation, not by candidate synthesis. The paper needs a perturbation test that makes the R-dependence observable: for example, shuffle candidate sets across questions at inference (or mask R entirely) and measure the drop in accuracy. Without such a test, the central mechanism behind the paper's title and abstract is unverified.","section":"4.2 / Equation (1)"},{"comment":"The ablations do not control for the direct-distillation confound. Row \"w/o training\" shows that an untrained Llama3-8B, given the synthesis prompt, already improves MATH500 accuracy over CoT-prompting by +8.0 for Llama3-8B and +3.6 for Llama3.1-8B; this demonstrates that the prompt alone elicits useful x-only answers and therefore cannot distinguish true synthesis from direct solving. Row \"w/o CoT training\" even exceeds the full method on MATH500 for Llama3-8B (37.0 vs. 36.0), directly contradicting the claim that the CoT analysis of candidates is beneficial for this setting. The most informative control is missing: train the same 8B model on (x, y) pairs with no R in the input, using the same filtered response-LLM labels, and compare it with the full Synthesizer. If the two perform equally, the paper's stated contribution reduces to a distillation/ensemble effect.","section":"5.3 / Table 2"},{"comment":"The \"Correct Count 0\" row reports that Synthesizer-8B produces 9 correct final answers when zero of the five candidates are correct, while SC, ArmoRM, and Scalar RM produce 0. Without the denominator, i.e., the number of test instances having zero correct candidates, this figure cannot be interpreted. If, for example, the Synthesizer's own single-pass accuracy on MATH500 is roughly 20-25%, then on a few dozen such instances one would expect about this many correct answers from a model that ignores R entirely. The paper should report the denominator for this row and compare against the x-only control on the same subset, so that the \"synthesis from all-wrong candidates\" claim can be evaluated.","section":"5.5 / Table 3"}],"minor_comments":[{"comment":"The abstract first says \"gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset\" but the body reports these on MATH500; align the terminology in the abstract and introduction.","section":"Abstract / Section 1"},{"comment":"In the FeTaQA block, the entry for USC with GLM-4-Plus appears as \"87.187.0\", which is likely a missing space between two numbers; please correct.","section":"Table 1"},{"comment":"The table labels the MATH500 results as \"MATH\", while the text and other tables use MATH500; use one consistent name.","section":"Table 2"},{"comment":"The URL for ArmoRM-Llama3-8B-v0.1 points to a Mixtral-8x7B checkpoint, which appears to be a copy-paste error; the correct model link should be provided.","section":"Appendix C.2 / Table 6"},{"comment":"The caption says the scaling plot is \"across the MATH500 and GSM8k datasets,\" but the figure appears to show Llama3-8B and Qwen2-7B test sets; clarify which dataset/model combination is displayed.","section":"5.4 / Figure 4"},{"comment":"For the \"w/o training\" row, the paper should specify exactly which prompt is given to the untrained Llama3-8B-Instruct (the synthesis prompt from Appendix E or a variant), so the reader can assess whether the +8.0 gain reflects the prompt itself or the candidate content.","section":"5.3 / Table 2"},{"comment":"The example is titled \"synthesizing a correct answer from all the wrong candidate responses,\" but several candidate responses contain the correct element -4 (e.g., R1, R3, R4, R5). If \"wrong\" only means no candidate is fully correct, the caption should say so explicitly; otherwise the example does not demonstrate the stronger claim that no candidate contains the correct answer.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically interesting and the released artifacts are valuable, but the core scientific claim—that a trained synthesizer exploits complementary information in R—is currently indistinguishable from a direct-answer-generation effect. The single most important addition is an R-dependence test (shuffled or masked candidate sets) together with an (x,y)-only distillation control. I would also require per-cell variance or significance tests before accepting the headline superiority claims. If these experiments confirm that the model uses R, the paper could be a solid contribution; if they do not, the contribution shifts to a data-distillation/ensemble method that is still useful but should be framed differently."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the trained synthesizer is a plausible inference-scaling trick and the data pipeline is real work, but the paper's central claim—that the model synthesizes from complementary candidate information—is not established. The results are consistent with a simpler explanation: the 8B synthesizer is a second answer generator that sometimes fixes errors, and the candidate set is largely decorative.\n\nWhat's new and worth taking seriously: the automated data generation pipeline with the LLM Repair step is a genuine engineering contribution. The model is trained on 295k MATH and 87k WikiTQ examples, and the method improves over SC, USC, and reward-model baselines across four benchmarks and seven policy models. Table 3's demonstration that Synthesizer-8B produces correct answers even from all-wrong candidate sets is the most interesting result in the paper. The authors also release code and data (modulo an inconsistency: the abstract says data is publicly available, while Appendix A says it will be released upon acceptance).\n\nThe soft spots are proportional. The main one is the stress-test concern: the training labels y are generated by a strong Response LLM prompted with (x, R), then filtered on correctness against gold answers. Nothing ensures y actually depends on R. The LLM could be solving x from the question alone, and filtering on y_g doesn't remove such labels. If so, SFT fits p(y|x), not p(y|x,R), and the synthesizer's gains are a second-model ensemble, not synthesis. There is direct evidence for this concern: Table 2's 'w/o training' row, which uses the untrained Llama3-8B with the same prompt, already gives +8.0 on MATH500 for Llama3-8B and +4.4 for GLM-4-Plus. That means the prompt alone can elicit useful x-only answers. The 'w/o CoT training' ablation is also not a control, because it keeps R in the input. What's missing is a perturbation test: shuffle R, replace R with candidates from a different model, or train on (x) only. Until that's done, the causal role of R is unproven.\n\nAlso, the main results are averages of three runs with no variance or significance tests. On a 500-question test set, a 2-point gap can easily be noise. The authors should report standard deviations or at least a paired test.\n\nWho is this for? Researchers working on inference scaling, LLM post-processing, or cheap ensembling. The recipe could be useful even if the mechanism is just ensembling, but the paper overclaims the mechanism.\n\nRecommendation: I'd send it to peer review, with a clear request for the perturbation experiment and variance reporting. The engineering is solid enough that the paper has value, but the headline claim needs the control.","headline":"Useful empirical recipe, but the central claim about cross-candidate synthesis is not proven; the results are consistent with a second-model ensemble.","tokens_in":20302,"tokens_out":3134,"would_cite":false,"duration_ms":29351,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a lightweight synthesizer model, trained on examples where a large model analyzes and combines candidate responses, can produce correct answers even when every candidate is wrong, improving larger policy models on…","keywords":["inference scaling","answer synthesis","chain-of-thought","self-consistency","Best-of-N","mathematical reasoning","table question answering","LLM post-processing"],"falsifier":"Replace the candidate set at inference time with a deliberately misleading set -- for example, a set of wrong answers that share no correct sub-steps -- and measure whether Synthesizer-8B's output changes or degrades toward the direct answer; if the output is statistically the same as when the candidates are removed or randomized, the model is solving independently rather than synthesizing, which would falsify the central claim. A simpler version is to compare accuracy on identical queries with the genuine candidates versus with candidates drawn from a different question.","tokens_in":19118,"feed_emoji":"🧠","tokens_out":7375,"duration_ms":61945,"temperature":0.7,"pith_summary":"The paper proposes CoT-based Synthesizer, an inference-scaling method that takes a question together with several candidate answers, reasons through the candidates step by step, and writes a new synthesized answer instead of selecting one existing answer. The central claim is that this synthesis process can produce a correct answer even when every candidate is wrong, by extracting and recombining the correct fragments scattered across flawed responses. To make this practical, the paper builds an automated pipeline that samples candidate sets, has a large model produce and filter synthesis examples, and uses them to train a smaller 8-billion-parameter synthesizer. The result, if true, is that a small open model can serve as a post-processing layer that improves much larger and even API-only models, with reported gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on MATH500.","feed_headline":"Synthesis beats selection: 11.8% math gain with a small model","feed_subtitle":"A trained 8B synthesizer fuses fragments from flawed answers, lifting Llama3-8B by 11.8% and GPT-4o by 10.3% on MATH500.","key_machinery":"The central mechanism is the CoT-based Synthesizer: a small language model fine-tuned to map a query with its candidate set $(x, R)$ to a synthesized answer $y$, using chain-of-thought analysis of each candidate and of the relations among candidates. It is trained with a two-stage data pipeline -- synthetic answer generation, in which a large response model analyzes $(x, R)$ and correct outputs are kept by comparing to gold answers, and LLM repair, which prompts the response model to re-synthesize when all candidates are wrong. The same inference prompt is then applied to any policy model's candidates, which is what allows a single 8B model to improve larger and API-based models.","core_discovery":"The paper's discovery is that answer selection is not the only viable form of inference scaling: a model can analyze the relationship between a query and a set of candidate responses and produce a better answer than any candidate contains. Concretely, CoT-based Synthesizer treats the candidate responses as raw material, identifies which reasoning steps are valid, and synthesizes a final answer, so the method remains useful precisely when self-consistency and Best-of-N fail because the correct answer is absent from the candidate set. The experiments claim the trained Synthesizer-8B outperforms these selection baselines across four benchmarks and seven policy models, including generalization to models structurally different from those used to generate the training data.","pith_inferences":["Not tested in the paper, the synthesis mechanism plausibly transfers to other post-processing settings where candidates contain partial correctness, such as code repair or document-grounded QA.","If the label-generation assumption fails on some queries, the practical gains may partly reflect a two-model ensemble rather than candidate synthesis; an ablation that corrupts or removes candidates would separate those effects.","Because the paper groups candidates when the context is long, a long-context synthesizer that reads all candidates at once could improve both accuracy and cost, a direction the paper notes but does not pursue."],"forward_implications":["Selection-based inference scaling is not the ceiling: post-processing can produce correct answers that do not appear in any candidate, so the candidate set's quality no longer strictly bounds the final accuracy.","A small trained model can act as a portable synthesis layer that improves API models it never trained on, transferring across policy models with different architectures and behaviors.","Generative synthesis scales with training data differently than scalar verifiers: accuracy keeps rising log-linearly past the point where reward-model training saturates or declines on duplicated instructions.","Synthesis removes the exact-match voting requirement, extending inference scaling to open-ended tasks such as free-form table question answering."],"supporting_citations":[{"why":"Defines self-consistency, the main selection-based baseline that synthesis is designed to improve upon.","marker":"Wang et al., 2022"},{"why":"Introduces verifier-based Best-of-N and the scalar reward-model training used as a baseline.","marker":"Cobbe et al., 2021"},{"why":"Defines Universal Self-consistency, the generative-voting baseline closest to this synthesis approach.","marker":"Chen et al., 2023"},{"why":"LMCOR, the prior synthesis baseline that trains a model to rewrite outputs without CoT candidate analysis.","marker":"Vernikos et al., 2023"},{"why":"Chain-of-thought prompting, the base inference method and the reasoning style the synthesizer uses.","marker":"Wei et al., 2022"},{"why":"Provides the MATH dataset from which the MATH500 test set and training queries are drawn.","marker":"Hendrycks et al., 2021"},{"why":"Provides the MATH500 test set and the step-by-step verification lineage for reward-model baselines.","marker":"Lightman et al., 2024"},{"why":"Supplies ArmoRM, the reward model used as a Best-of-N baseline.","marker":"Wang et al., 2024"}],"fun_headline_variants":["When all candidates fail, synthesize the answer","Beyond selection: fuse flawed answers into correct ones","CoT synthesizer lifts Llama and GPT-4o on MATH","8B model turns bad answers into right ones, 11.8% gain","Synthesize, don't select: new inference scaling method"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the training labels are genuine syntheses of the candidate set: the large response model produces its answer by analyzing and combining $(x, R)$, so the filtered correct outputs teach the smaller model how to synthesize from candidates rather than how to solve the question directly.","fun_headline_variants_meta":{"raw":{"variants":["When all candidates fail, synthesize the answer","Beyond selection: fuse flawed answers into correct ones","CoT synthesizer lifts Llama and GPT-4o on MATH","8B model turns bad answers into right ones, 11.8% gain","Synthesize, don't select: new inference scaling method"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1220,"prompt_tokens":901,"completion_tokens":319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":234}},"tokens_in":517,"tokens_out":319,"duration_ms":3906,"temperature":1.0,"reasoning_tokens":234,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:22:41.760045+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the candidate set at inference time with a deliberately misleading set -- for example, a set of wrong answers that share no correct sub-steps -- and measure whether Synthesizer-8B's output changes or degrades toward the direct answer; if the output is statistically the same as when the candidates are removed or randomized, the model is solving independently rather than synthesizing, which would falsify the central claim. A simpler version is to compare accuracy on identical queries with the genuine candidates versus with candidates drawn from a different question.","supporting_citations":[{"cited_title":"Small Language Models Improve Giants by Rewriting Their Outputs","cited_arxiv_id":"2305.13514","evidence_quote":"LMCOR, the prior synthesis baseline that trains a model to rewrite outputs without CoT candidate analysis."}],"review_version":1}