{"id":"a6cf3ce8-de4b-458f-ade7-fd98166e27d3","arxiv_id":"2506.18330","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A 14B RL-trained model reports 96.2% on an internal Chinese K-12 benchmark, about 15x lower serving cost than DeepSeek-R1, with three new training tricks.","lead":"Confucius3-Math is a 14-billion-parameter math model trained with reinforcement learning for Chinese K-12 math. It reports top scores on several Chinese benchmarks while running on a consumer GPU, at a training cost of about $26,000.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CK12-MATH is sampled from the same business data as the proprietary training pool, with no overlap check or released items; the headline K-12 advantage could be a training-set artifact rather than a real capability gain.","rationale":"The strongest claim in the paper is not that a 14B model can do competition math, but that Confucius3-Math is specifically better for Chinese K-12 learning. The only benchmark that directly tests this is CK12-MATH, and it is the least externally verifiable piece of evidence: 500 proprietary items, not released, sampled from the same user-query stream as the 330k proprietary training examples. The paper's own data-cleaning section shows the authors know that exact, fuzzy, and semantic overlap matter for training data, yet no equivalent check is reported between CK12 and the training pool. Without that check, the headline 96.24 and the +3.5 over DeepSeek-R1 could simply reflect memorization of recurring homework and quiz items. The PSHW constant and throughput measurements are weaker concerns: PSHW is an ablatable implementation detail with a hand-set coefficient, and the throughput comparison is at least described with a concrete setup. The internal benchmark concern is load-bearing because it is the one piece of evidence the community cannot check without releasing data. The reader's CONDITIONAL verdict is appropriate: with CK12 released and a clean overlap analysis, the claim could be validated; without it, the central K-12 contribution remains unverified. I would not move the verdict to REJECT because the model weights and code are released, public GAOKAO and CMATH scores are still strong, and the training-cost figures are plausible, but the CK12 result should be treated as unconfirmed until the proposed test is run.","tokens_in":15968,"tokens_out":5473,"duration_ms":56570,"concrete_test":"Ask the authors to (1) release CK12-MATH, or an independently drawn 500-question sample from the same source with timestamps after the training-data cutoff, and (2) run exact-match, Jaccard, and embedding-based dedup of each CK12 item against all 540k training samples, reporting per-item nearest-neighbor similarities and the resulting overlap rate. In addition, have two independent annotators, or a blinded external judge with no access to model identity, re-score a random 100-item subsample of Confucius3 responses for both final-answer and intermediate-step correctness, and compare those scores against the paper's LLM-as-a-judge scores. If any CK12 item overlaps training data, or if judge concordance is below roughly 95%, the reported 96.24 CK12 score and the 3.5-point lead over DeepSeek-R1 should be recomputed on a clean split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central K-12 claim (CK12-MATH 96.24 vs. 92.74 for DeepSeek-R1, and the abstract's 'excels at solving mainstream Chinese K-12 problems') depends entirely on an unreleased 500-question benchmark. Section 3.1 says proprietary training data are 'math questions accumulated during the operation of our business'; Section 7 says CK12-MATH is 'sampled from actual user queries'—the same operational stream. The paper reports exact/fuzzy/semantic deduplication for open-source data (Section 3.2), but no analogous overlap analysis between the 500 CK12 items and the ~330k proprietary training samples, nor any release of CK12 items. Because identical or near-identical questions can recur in a tutoring business, train/eval contamination is a concrete risk rather than a purely hypothetical one. The LLM-as-a-judge step check for Confucius3 (Section 7) adds a second uncontrolled variable: neither the judge's identity nor its agreement with human graders is reported, and the protocol applied to baselines is unspecified. If CK12 is contaminated, the headline 'leads DeepSeek-R1 by 3.5 points' evaporates, and the paper's unique K-12 positioning is unsupported; the public benchmarks alone (MathBench K12: 95.10 vs. 96.51 for Qwen3-14B; AIME25: 69.95 vs. 70.40 for Qwen3-14B) do not establish SOTA.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces Confucius3-Math, a 14B-parameter reasoning LLM for Chinese K-12 mathematics, built by pure RL post-training from DeepSeek-R1-Distill-Qwen-14B. Three training innovations are proposed: Targeted Entropy Regularization (TER), Recent Sample Recovery (RSR), and Policy-Specific Hardness Weighting (PSHW). The paper reports large gains over the base model on public benchmarks, SOTA or near-SOTA scores on several competition-level benchmarks, a 3.5-point advantage over DeepSeek-R1 on an internal CK12-MATH benchmark, a training cost of $26K, and roughly 15x the serving throughput of DeepSeek-R1. The central K-12 claim rests on CK12-MATH, an unreleased internal benchmark sampled from the authors' business data, and the evaluation protocol applies an LLM-as-a-judge step check to Confucius3-Math only.","tokens_in":16266,"tokens_out":4149,"duration_ms":43878,"significance":"If the claims hold, the paper is a useful existence proof that domain-specific RL post-training of a 14B model can produce strong mathematical reasoning at low training and inference cost. The ablations in Figures 3 and 4 give credible internal support for each of the three proposed training components, the cost figures are concrete, and open-sourcing the model and code is a plus for reproducibility. The high scores on public benchmarks (MATH500 98.44, AIME24 81.15) are impressive and potentially meaningful. However, the unique K-12 SOTA claim is not independently verifiable: CK12-MATH is an internal, unreleased benchmark drawn from the same business stream that supplied the proprietary RL training data, and no overlap analysis between the 500 evaluation items and the 330k training samples is reported. The LLM-as-a-judge step verification used only for Confucius3-Math is an additional uncontrolled variable. These audit gaps are load-bearing for the paper's positioning and must be addressed before the central claim can be accepted.","major_comments":[{"comment":"The headline K-12 result, Confucius3-Math leading DeepSeek-R1 by 3.5 points on CK12-MATH, depends entirely on an internal 500-question benchmark sampled from actual user queries, while the proprietary training data (Section 3.1) are math questions accumulated during the same business's operations. The open-source data pipeline includes exact, fuzzy, and semantic deduplication (Section 3.2), but no comparable overlap, contamination, or deduplication analysis is reported between the CK12-MATH items and the 330k proprietary training samples. Because identical or near-identical questions plausibly recur in a tutoring business, the observed 3.5-point gap could be a training-set artifact. To support the central claim, the authors should release the CK12-MATH items, or at minimum provide a detailed overlap analysis (exact and fuzzy matching against the training pool), along with a clear statement of how the set was held out from training.","section":"Section 7 (CK12-MATH) and Section 3.1"},{"comment":"The evaluation section states that CK12-MATH includes intermediate solution steps and that for Confucius3-Math a problem is solved only when both the final answer and the intermediate steps are judged correct by LLM-as-a-judge, whereas the baselines are presumably evaluated on final answers only. This asymmetric protocol makes the comparison against DeepSeek-R1 and other baselines difficult to interpret. The paper does not report which judge model was used, how the judge was prompted or calibrated, what its agreement with human graders is, or whether the same step-checking protocol was applied to all baselines. The authors should report these details and, ideally, run a version of the evaluation with an identical protocol for all models, including a human-annotated subset.","section":"Section 7 (LLM-as-a-judge protocol)"},{"comment":"The abstract and introduction claim SOTA performance across a range of benchmarks, but Table 3 shows that Confucius3-Math is below Qwen3-14B on MathBench(K12) (95.10 vs. 96.51) and on AIME25 (69.95 vs. 70.40). The statement that the model outperforms all models on all but two benchmarks is technically accurate, but those two exceptions are central to the K-12 and current-year competition claims. The authors should temper the SOTA wording to 'comparable or leading on specific benchmarks' and should provide confidence intervals or error bars for the pass@1 estimates, especially for CK12-MATH where k=4 per question and the reported advantage is only 3.5 points.","section":"Table 3 and Abstract/Introduction SOTA claim"},{"comment":"The manuscript uses DeepSeek-R1-Distill-Qwen-14B, a model that already acquired reasoning capabilities through distillation from DeepSeek-R1, as the base. The phrase 'pure RL' in the introduction and the framing in Section 6 should be clarified to mean pure RL post-training of an already distilled reasoning model, not RL from a base pretrained model. This does not invalidate the approach, but it is material to the claim that RL alone elicits the observed capabilities.","section":"Section 4 and Section 6 (base model and 'pure RL' claim)"}],"minor_comments":[{"comment":"The three hyperparameters entropy_target (0.55), entropy_coeff (0.001), and the PSHW coefficient alpha (-0.256) are all set empirically, but the paper reports no sensitivity analysis or ablations for them. Since these parameters are integral to the claimed innovations, reporting at least a small sweep or noting the range of stable values would strengthen the contribution.","section":"Section 6.2 and Table 2"},{"comment":"Several baseline numbers are marked with an asterisk as taken from the respective publications, while others were produced with different sampling parameters and k values. The paper should state for each benchmark which settings were used for each model so that the cross-model comparisons are directly interpretable.","section":"Table 3 and Section 7"},{"comment":"The throughput comparison scales Confucius3-Math's single-GPU throughput by a factor of 8 to compare with DeepSeek-R1 on 8 GPUs. This assumes perfectly linear scaling across GPUs and should be stated explicitly as an idealized comparability assumption; reporting raw per-configuration numbers would be more informative.","section":"Section 8"},{"comment":"There are several typos and formatting issues, including 'summerarized' in Section 3.1, 'reword model' in Section 5, 'rigorious' in Section 6.3, and the mislabeled 'summarized' in the Table 1 caption. These should be corrected in a final revision.","section":"Throughout"},{"comment":"The claim that the model runs 'efficiently on a single consumer-grade GPU' is supported by the 4090D experiment, but the paper should clarify whether the model is open-sourced with a specific license and whether the released GitHub repository contains the model weights or only code and training recipes.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The primary evidence for the unique K-12 claim is an internal benchmark derived from the authors' own business data, with no overlap analysis and an asymmetric judge-based evaluation. If this paper is considered for publication, the editor should require that the CK12-MATH items and judge configuration be made available to reviewers, or that the central claim be reformulated around the public benchmarks. This is an audit and reproducibility concern, not an allegation of misconduct."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe real news here is not any single technique — entropy targeting resembles Skywork-OR1, sample reuse is a natural fix, and the advantage weighting is a hand-set coefficient — but that the combination works and is cheap: $26K in H800 rental, 15x throughput vs DeepSeek-R1, and large gains over its own base across public benchmarks. That part looks credible.\n\nThe soft spot is the headline K-12 benchmark. CK12-MATH is 500 questions sampled from the same tutoring business that supplied the proprietary training pool. The paper reports dedup for open-source data but no overlap check between CK12 and the ~330k proprietary training samples. At a tutoring company, real user queries repeat; identical or near-identical items are a concrete risk, not a hypothetical one. If CK12 is contaminated, the 96.24 vs 92.74 lead over DeepSeek-R1 shrinks from a real capability claim to a possible training-set artifact. The LLM-as-a-judge step verification adds a second unmeasured variable: no judge identity, no agreement numbers, and the protocol for baselines is not specified. The public benchmarks do not fully rescue the SOTA claim — on MathBench K12 and AIME25, Qwen3-14B scores slightly higher. So the paper is internally consistent (the ablations support the recipe) but the flagship external-positioning claim is not demonstrated.\n\nOne more minor point: the PSHW coefficient alpha is set empirically with no sensitivity analysis, and the paper admits the entropy target is empirical. That is fine for a recipe but not a derivation. Also, no error bars on any single-run numbers.\n\nWho is this for? Practitioners building cheap domain-specific reasoning models. They get a detailed, reproducible-looking recipe with open code and weights. Researchers benchmarking SOTA should treat the CK12 numbers with suspicion until the benchmark is released. I would send it to a competent referee — it is an honest engineering report with testable claims — but the referee should demand an overlap analysis or a release of CK12 items.\n\nRecommendation: engage, but with eyes open.","headline":"A credible low-cost RL recipe that is undermined by an internal, likely-contaminated benchmark; the public numbers and ablations are worth a look, the headline K-12 claim is not.","tokens_in":16806,"tokens_out":2081,"would_cite":true,"duration_ms":20873,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 14B model post-trained with pure reinforcement learning for about $26K reaches state-of-the-art accuracy on Chinese K-12 math benchmarks, matching or beating much larger models and serving about 15x the throughput of a 671B model.","keywords":["reinforcement learning","mathematical reasoning","Chinese K-12 education","large language models","entropy regularization","group relative policy optimization","data efficiency","LLM post-training"],"falsifier":"A reader could settle the main claim by checking whether the 500 CK12-MATH questions overlap the 330,000 proprietary training questions, and then by re-grading Confucius3-Math and DeepSeek-R1 on a freshly sampled, independently verified set of Chinese K-12 problems. If the 96% accuracy and the 3.5-point lead over DeepSeek-R1 persist under model-blind human grading on the fresh sample, the central claim stands; if they vanish, the benchmark is the load-bearing artifact.","tokens_in":15768,"feed_emoji":"🧮","tokens_out":9785,"duration_ms":85725,"temperature":0.7,"pith_summary":"This paper tries to establish that a 14-billion-parameter model, post-trained with pure reinforcement learning and no supervised fine-tuning, can match or beat much larger models on Chinese K-12 mathematics and competition benchmarks, at a training cost of roughly $26,000 and with about 15 times the serving throughput of a 671-billion-parameter model. The authors report that their model reaches 96.24 on an internal Chinese K-12 benchmark, 98.44 on MATH500, and 81.15 on AIME24, outperforming its own base model by up to 26.98 points. Three proposed mechanisms—targeted entropy regularization, recent sample recovery, and policy-specific hardness weighting—are what stabilize the RL run, cut wasted data, and raise accuracy. If the claims hold, they demonstrate that domain-focused, low-cost RL post-training is a practical route to deployable AI tutoring models, with consequences for educational access and affordability.","feed_headline":"14B math model tops 671B R1 on K-12 tests after $26K RL run","feed_subtitle":"Pure reinforcement learning plus three stability tricks reaches 96% on Chinese K-12 problems and runs 15x faster.","key_machinery":"The load-bearing machinery is a modified RL objective built on Group Relative Policy Optimization (GRPO) and its dynamic-sampling variant DAPO. Targeted Entropy Regularization adds $|L_{\\text{entropy}} - 0.55| \\times 0.001$ to the loss, constraining policy entropy to a narrow band; the paper finds this suppresses the mixed-language and repetition failure modes that otherwise destabilize the run. Policy-Specific Hardness Weighting changes the group-relative advantage estimator to $D(q) = \\alpha \\mu + 1.256$ with $\\alpha = -0.256$, where $\\mu$ is the mean reward on a question; this deliberately gives stronger advantage to questions the current policy finds hard, making difficulty relative to the evolving policy rather than a fixed curriculum. Recent Sample Recovery keeps the overflow samples that DAPO's dynamic sampling would discard and feeds them into the next batch, without importance sampling; the paper reports this not only reduces raw data consumption but also improves final accuracy at equal effective data, possibly through a smoothing effect. The paper also removes length normalization from the objective, following a critique that normalization biases the policy toward short correct answers and long incorrect ones.","core_discovery":"The paper's central claim is that pure RL post-training of a strong 14B base model is sufficient to elicit state-of-the-art mathematical reasoning in Chinese K-12 and competition settings, without supervised fine-tuning or distillation from a stronger teacher. Starting from DeepSeek-R1-Distill-Qwen-14B, the authors train Confucius3-Math in three stages with gradually expanding context windows, from 4K to 8K to 16K, using GRPO in the first stage and a modified DAPO in the later stages. They report that the final model beats DeepSeek-R1 on CK12-MATH (96.24 vs 92.74), GAOKAO-Bench Math (98.46 vs 93.27), CMATH (96.13 vs 95.81), MATH500 (98.44 vs 97.30), and AIME24 (81.15 vs 79.80), while remaining comparable on MathBench K12 and AIME25. The largest lift over its base model is 26.98 points on AIME25, and the entire training run costs $26K in rented H800 GPU time.","pith_inferences":["An implication the paper does not draw: the entropy-target mechanism is a generic treatment for mixed-language outputs in multilingual RL, so it could be tested directly on code generation or multilingual reasoning tasks where language mixing also appears.","A testable extension: vary the RSR buffer size and the policy distance between reused samples in controlled runs; if quality improves with distance, the effect is a form of implicit off-policy smoothing, a mechanism the paper only speculates about.","The reliance on an LLM-as-a-judge for proprietary K-12 data means the method's reward signal and its main benchmark share a judgment style; a natural extension is to train on judge-verified answers and then re-evaluate with a different judge or human graders to confirm the gains are not judge-specific.","The paper computes training cost at a fixed $2 per GPU-hour rental; at different rates the dollar figure scales, but the structural advantage over on-policy distillation, which requires a separate teacher-serving server, should survive because the RL pipeline uses only one server."],"forward_implications":["A 14B domain model can beat a 671B general reasoning model on the domain's own benchmarks, so model scale is not the only path to top-tier reasoning accuracy.","The training recipe is reusable on other verifiable reasoning domains at roughly $26K per run, making state-of-the-art RL post-training accessible to teams with a single 8-GPU server.","At $0.15 per million tokens and over 15x the throughput of DeepSeek-R1, the model can serve realistic tutoring traffic on consumer-grade hardware, which is the deployment condition the paper argues matters for educational equity.","The largest gains over the base model appear on competition-level sets, up to +26.98 on AIME25, suggesting pure RL extracts latent reasoning already present in a distilled base rather than only memorizing school problems.","Because the techniques modify the advantage estimator and data flow rather than the architecture, the paper argues they can be combined with other GRPO/DAPO-family algorithms and other base models."],"supporting_citations":[{"why":"Supplies the pure-RL training template, the R1 baseline, and the distilled 14B base model the paper starts from.","marker":"DeepSeek-AI et al., 2025"},{"why":"Supplies DAPO, the RL framework with dynamic sampling that stages two and three modify.","marker":"Yu et al., 2025"},{"why":"Supplies the GRPO algorithm and the group-relative advantage estimator that the paper's new weighting extends.","marker":"Shao et al., 2024"},{"why":"Identifies the length-normalization bias that the paper removes from the objective.","marker":"Liu et al., 2025"},{"why":"Provides Math-Verify, the rule-based extractor used to score open-source answers.","marker":"HuggingFace"},{"why":"Defines the Qwen3-14B baseline and the strong-to-weak distillation alternative that the RL approach is compared against.","marker":"Yang et al., 2025"},{"why":"Contributes the GAOKAO-Bench math subset used as an external K-12 evaluation benchmark.","marker":"Zhang et al., 2023"},{"why":"Contributes CMATH, the Chinese elementary-school word-problem benchmark used in evaluation.","marker":"Wei et al., 2023"}],"fun_headline_variants":["14B RL-only model beats 671B R1 on key math benchmarks","$26K RL run lets 14B model surpass 671B R1 on K-12 tests","Three RL innovations: 14B math model outperforms 671B R1","14B Confucius3-Math: SOTA math reasoning for $26K, one GPU","Pure RL, no SFT: 14B model beats larger rivals on math"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline K-12 result depends on CK12-MATH, an internal 500-question benchmark drawn from the same tutoring business that supplied the proprietary training data, and the paper does not show that those questions are cleanly held out or that the language-model judge used to verify answers and steps is unbiased toward the trained model.","fun_headline_variants_meta":{"raw":{"variants":["14B RL-only model beats 671B R1 on key math benchmarks","$26K RL run lets 14B model surpass 671B R1 on K-12 tests","Three RL innovations: 14B math model outperforms 671B R1","14B Confucius3-Math: SOTA math reasoning for $26K, one GPU","Pure RL, no SFT: 14B model beats larger rivals on math"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000516,"raw_usage":{"total_tokens":2550,"prompt_tokens":1037,"completion_tokens":1513,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":1401}},"tokens_in":653,"tokens_out":1513,"duration_ms":13947,"temperature":1.0,"reasoning_tokens":1401,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:51:19.393954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle the main claim by checking whether the 500 CK12-MATH questions overlap the 330,000 proprietary training questions, and then by re-grading Confucius3-Math and DeepSeek-R1 on a freshly sampled, independently verified set of Chinese K-12 problems. If the 96% accuracy and the 3.5-point lead over DeepSeek-R1 persist under model-blind human grading on the fresh sample, the central claim stands; if they vanish, the benchmark is the load-bearing artifact.","supporting_citations":[],"review_version":2}