{"id":"1000fa4e-157f-4d04-bbda-530776744a28","arxiv_id":"2606.13473","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"MaxProof uses a generative-verifier RL trained model with population-level test-time scaling to reach 35/42 on IMO 2025 and 36/42 on USAMO 2026.","lead":"The paper presents MaxProof, a framework that scales mathematical proof generation at test time by using one model for generation, verification, repair, and ranking via population search and tournament selection. Smart generalists might read it because it shows how AI can be pushed to solve very difficult math competition problems that test the limits of current systems.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Verifier's low false-positive rate is asserted but not independently validated on the reported IMO/USAMO solutions","rationale":"The reader's weakest_assumption directly identifies the same dependency; the full text does not add an external validation step that would remove the risk, so the concern remains load-bearing and the verdict should stay conditional pending that check.","tokens_in":1614,"tokens_out":292,"duration_ms":11566,"concrete_test":"Randomly sample 15 of the final selected proofs (stratified across the 42 problems) and submit them to two independent IMO/USAMO gold medalists for blind correctness grading; if the fraction judged incorrect exceeds 5/15, the 35/42 and 36/42 claims cannot be sustained.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline performance (35/42 IMO 2025, 36/42 USAMO 2026) is obtained by tournament selection over a population of proofs, where the final proof is chosen by the same defense-in-depth generative verifier used in RL training. The paper states this verifier was engineered for low false-positive rate, yet provides no external human audit or cross-check of the 71 selected proofs against the official problem statements. If the verifier systematically misses subtle logical gaps that human graders would catch, the reported scores are inflated; the entire test-time scaling pipeline collapses without this component functioning as claimed.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces MaxProof, a population-level test-time scaling framework for competition-level mathematical proof. It first trains the M3 model on three capabilities—proof generation, proof verification, and critique-conditioned proof repair—using a defense-in-depth generative verifier engineered for low false-positive rate. These are merged into a single model. At test time, MaxProof uses the model as generator, verifier, refiner, and ranker, searches over populations of candidate proofs, and selects the final proof via tournament selection, reporting 35/42 on IMO 2025 and 36/42 on USAMO 2026.","tokens_in":1731,"tokens_out":482,"duration_ms":18576,"significance":"If the central claims hold after proper validation, the work would represent a notable advance in scaling AI for formal mathematical reasoning, demonstrating that integrated generative verification plus population-level selection can exceed human gold-medal thresholds on recent contest problems. The RL training of verifier and repair capabilities alongside test-time tournament selection is a coherent technical direction with potential broader impact.","major_comments":[{"comment":"Abstract and experimental results: The reported scores (35/42 IMO 2025, 36/42 USAMO 2026) are obtained by tournament selection over populations using the same generative verifier trained in the RL stage; however, the manuscript provides no independent human audit, cross-check against official solutions, or false-positive rate measurement on the 71 selected proofs, leaving the low-FP claim unverified and the performance numbers potentially inflated.","section":"Abstract"},{"comment":"Abstract: No experimental setup, baselines, ablation studies, or error analysis are described, so it is impossible to assess whether the method (rather than unstated factors) supports the headline claims of surpassing human performance.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract could more explicitly state the total number of problems attempted and any filtering criteria applied before reporting the 35/42 and 36/42 figures.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The absence of any experimental details or verifier validation in the provided manuscript raises serious reproducibility concerns; this may affect suitability for the target journal until a full methods section with audits is added."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive report. We address each major comment below.","responses":[{"response":"We agree that the manuscript does not contain an independent human audit, cross-check against official solutions, or explicit false-positive rate measurement performed on the 71 final selected proofs. The low false-positive engineering is described for the verifier training stage and the tournament selection is presented as an additional safeguard, but these do not substitute for post-hoc verification of the contest outputs. We will add a limitations paragraph acknowledging this gap and will include a small-scale human verification study of a random subset of the proofs in the revised manuscript.","revision_made":"partial","referee_comment":"[Abstract] Abstract and experimental results: The reported scores (35/42 IMO 2025, 36/42 USAMO 2026) are obtained by tournament selection over populations using the same generative verifier trained in the RL stage; however, the manuscript provides no independent human audit, cross-check against official solutions, or false-positive rate measurement on the 71 selected proofs, leaving the low-FP claim unverified and the performance numbers potentially inflated."},{"response":"The abstract is intentionally concise. The full manuscript describes the RL training of the three capabilities (generation, verification, critique-conditioned repair), the defense-in-depth verifier, the merging into a single model, and the population-level tournament selection procedure at test time. We nevertheless accept that the current version lacks explicit baselines, ablations, and error analysis. We will expand the experimental section in the revision to include these elements.","revision_made":"yes","referee_comment":"[Abstract] Abstract: No experimental setup, baselines, ablation studies, or error analysis are described, so it is impossible to assess whether the method (rather than unstated factors) supports the headline claims of surpassing human performance."}],"tokens_in":1309,"tokens_out":446,"duration_ms":23701,"standing_objections":["Independent human audit and false-positive measurement on the complete set of 71 selected proofs, as this verification step was not performed during the original experiments and cannot be supplied without new work."]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that MaxProof combines proof generation, verification, and critique-based repair into one model, then runs a population of attempts at test time and picks the winner with tournament selection using the same verifier. That setup is the concrete piece worth noting if the numbers hold.\n\nThe paper does show a clear pipeline: train the three capabilities with a defense-in-depth verifier tuned for low false positives, merge them into M3, and apply the model to itself for search and ranking. The idea of using the model as its own ranker and refiner in a closed loop is a straightforward extension of existing test-time methods, and the population approach is a natural next step once you have a verifier you trust.\n\nThe soft spot is exactly where the stress test flags it. The headline scores come from selecting among many candidates with that verifier, yet the abstract gives no human audit or cross-check of the 71 final proofs against the official statements. If the verifier misses subtle gaps that graders would catch, the reported performance is overstated and the whole scaling story rests on an untested assumption. Without those details or baselines in the provided text, it is difficult to judge how much the method actually moves the needle beyond prior work.\n\nThis is aimed at labs working on AI for competition math and automated theorem proving. Readers who track test-time scaling or verifier training will get something usable from the framework description, even if they end up questioning the numbers. The paper deserves a serious referee because the core engineering choices are specific enough to evaluate once the full methods and validation data are on the table. Any review should start with whether the verifier was tested independently on the selected solutions.","headline":"The abstract claims 35/42 on IMO 2025 and 36/42 on USAMO 2026 via population test-time scaling, but the verifier's accuracy on those exact proofs is asserted without shown external checks.","tokens_in":2258,"tokens_out":427,"would_cite":false,"duration_ms":15724,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"MaxProof lets one model exceed human gold-medal scores on IMO 2025 and USAMO 2026 by running tournament selection over populations of candidate proofs at test time.","keywords":["mathematical proof","test-time scaling","generative verifier","tournament selection","IMO","USAMO","proof repair","population search"],"falsifier":"Running the same population search on a fresh set of competition problems where the verifier's accuracy is measured independently and the final selected proofs are graded by human experts; if the selected proofs are mostly incorrect despite high reported scores, the claim fails.","tokens_in":2539,"feed_emoji":"📐","tokens_out":674,"duration_ms":17448,"temperature":0.7,"pith_summary":"The paper presents MaxProof as a test-time framework that merges proof generation, verification, and repair into a single model, then searches a population of proofs and selects the best one via tournament. The central claim is that this population-level scaling, powered by a low false-positive generative verifier, pushes performance past the human gold-medal threshold without further training. A sympathetic reader would care because the approach shows how test-time compute can be applied to formal reasoning tasks that normally require deep search or external solvers. The results are reported on recent competition problems, suggesting the method scales with the number of candidates considered.","feed_headline":"Model tops gold-medal scores on IMO 2025 and USAMO 2026","feed_subtitle":"Population search and tournament selection over generated proofs lets the M3 model reach 35/42 and 36/42 without extra training.","key_machinery":"Population-level test-time scaling that treats the model as a combined generator-verifier-refiner-ranker and applies tournament selection over candidate proofs.","core_discovery":"MaxProof trains the M3 model on three capabilities using a defense-in-depth generative verifier, then at test time deploys the model simultaneously as generator, verifier, refiner, and ranker; it samples populations of candidate proofs and returns a final proof through tournament selection, reaching 35 out of 42 on IMO 2025 and 36 out of 42 on USAMO 2026.","pith_inferences":["The same population-search pattern could be tested on other verifiable domains such as code synthesis or formal theorem proving in different libraries.","If verifier accuracy improves further, the same framework might push scores higher without changing the base model.","The approach highlights that verification quality, not just generation quality, is the bottleneck for test-time scaling in reasoning."],"forward_implications":["A single trained model can reach competition-level performance on proof tasks by allocating more test-time compute to population search rather than larger training runs.","Verification and repair capabilities become directly usable at inference time to refine and rank outputs.","Tournament selection among many candidates becomes an effective way to leverage the verifier for final answer choice.","Performance scales with the size and diversity of the proof population generated at test time."],"fun_headline_variants":["M3 scores 35/42 on IMO 2025 with MaxProof scaling","Tournament selection scores 36/42 on USAMO 2026","Population-level scaling scores 35/42 on IMO 2025","MaxProof uses verifier RL for 35/42 IMO 2025 proofs"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The generative verifier can reliably separate correct proofs from incorrect ones with a low false-positive rate so that tournament selection actually picks the right answer.","fun_headline_variants_meta":{"raw":{"variants":["M3 scores 35/42 on IMO 2025 with MaxProof scaling","Tournament selection scores 36/42 on USAMO 2026","Population-level scaling scores 35/42 on IMO 2025","MaxProof uses verifier RL for 35/42 IMO 2025 proofs"]},"model":"grok-4.3","cost_usd":0.009139,"raw_usage":{"total_tokens":4062,"prompt_tokens":598,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":91387000,"prompt_tokens_details":{"text_tokens":598,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3386,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":598,"tokens_out":78,"duration_ms":19139,"temperature":1.0,"reasoning_tokens":3386,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T07:35:20.177609+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the same population search on a fresh set of competition problems where the verifier's accuracy is measured independently and the final selected proofs are graded by human experts; if the selected proofs are mostly incorrect despite high reported scores, the claim fails.","supporting_citations":[],"review_version":1}