MerLean-Prover: A Recursive Looping Harness for Lean 4 Theorem Proving

Jinzheng Li; Yuanjie Ren; Zeru Zhu

REVIEW 1 major objections 1 minor 1 cited by

MerLean-Prover: A Recursive Looping Harness for Lean 4 Theorem Proving

T0 review · 1 major / 1 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A recursive loop of Planning, Check, and Lean agents generates kernel-checkable Lean4 proofs without fine-tuning or per-theorem scaffolding.

desk verdict The three-agent recursive harness gets solid benchmark numbers on Lean without training, but the writeup is too thin on methods and analysis to judge how much the loop actually drives the gains. read the letter →

arxiv 2605.26959 v2 pith:SLNSCT43 submitted 2026-05-26 cs.LO cs.CL

Jinzheng Li , Zeru Zhu , Yuanjie Ren This is my paper · ORCID

classification cs.LOcs.CL

keywords Lean4theoremprovingmulti-agentharnessrecursiveloopFormalQualBenchPutnam2025kernel-checkableproofsagentcomposition

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MerLean-Prover shows that an end-to-end Lean4 theorem prover can be assembled from three agent types arranged in a recursive outer loop. The loop treats the proof plan as the unit of revision and continues until each sorry declaration is replaced by a kernel-checkable proof. The construction uses no fine-tuning, no custom reinforcement-learning objective, and no theorem-specific scaffolding. On FormalQualBench the system solves 10 of 23 PhD-qualifying theorems, exceeding the prior open-source best of 8, and on Putnam2025 it completes all 12 problems with lower total wall-clock time than the next-best system. The same loop also enables smaller models to finish subsets of the benchmark, indicating that the arrangement of agents contributes to success alongside raw model size.

What carries the argument

The recursive outer loop of Planning, Check, and Lean agents that revises the proof plan until a kernel-checkable proof is obtained.

What would settle it

Running the identical agent loop on a fresh collection of Lean4 theorems outside FormalQualBench and Putnam2025 and obtaining a lower solve rate than a non-recursive multi-agent baseline would show that the recursive plan-revision structure is not the decisive factor.

Watch

Extended reading notes

Core claim

MerLean-Prover is built from Planning, Check, and Lean agents composed by a recursive outer loop whose unit of revision is the proof plan itself. This design alone, without fine-tuning, custom RL objectives, or theorem-specific scaffolding, produces kernel-checkable proofs that replace sorry declarations. It solves 10 of 23 problems on FormalQualBench, surpassing the strongest published open-source baseline of 8 of 23, and closes all 12 problems on Putnam2025 with substantially lower total wall-clock time than the next-best system.

Load-bearing premise

The recursive outer loop composed of Planning, Check, and Lean agents can reliably produce kernel-checkable proofs on the tested theorems without any theorem-specific scaffolding, fine-tuning, or custom RL objective.

Editorial extensions

If this is right

The harness transfers to smaller models, allowing Sonnet to close all four tested FormalQualBench problems and Haiku to close the two shortest ones.
The same composition closes the full Putnam2025 set of twelve problems with lower total wall-clock time than the next-best published system.
No theorem-specific scaffolding or custom training objective is required for the observed solve rates on the two benchmarks.
Harness structure can be treated as an independent variable that improves end-to-end Lean4 proving performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The recursive plan-revision pattern could be ported to other interactive theorem provers such as Coq or Isabelle to test whether similar gains appear.
Measuring how the number of outer-loop iterations scales with theorem difficulty would clarify the computational cost of the approach.
Combining the harness with modest fine-tuning on proof traces might raise the solve rate further, though the paper does not test this combination.
The emphasis on revising the plan rather than the final proof suggests that early-stage error feedback is more valuable than late-stage repair.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The punchline is that MerLean-Prover's recursive loop of Planning, Check, and Lean agents solves 10/23 on FormalQualBench (beating OpenGauss at 8/23) and all 12 Putnam2025 problems with lower wall time, and the same setup works on smaller models like Sonnet and Haiku for some cases. No fine-tuning or custom RL is used, which is the main practical claim.

What stands out is the concrete harness design: the unit of revision is the proof plan itself rather than individual tactics, and the three roles are composed in an outer recursive loop. That is new relative to the baselines cited. The transfer result to smaller models is also useful to see, since it suggests the structure can compensate for weaker base models on short problems.

The soft spots are all in the evaluation. The abstract gives the headline numbers but no selection criteria for the 23 theorems, no error analysis, no ablation on which agent role matters, and no breakdown of where the recursive loop actually revises plans versus just running the base model. Without those, it is difficult to tell whether the reported gains come from the harness or from other unstated factors. The Putnam result is stronger on its face because it is a full set, but the same lack of detail applies.

This paper is for groups already building agent systems for Lean or other formal provers who want to test whether simple role decomposition plus plan revision is worth trying. A reader who cares about scaling formal math with off-the-shelf models will find the benchmark comparisons and the smaller-model transfer worth looking at.

It deserves peer review. The system is described clearly enough to reproduce in principle, the benchmarks are standard, and the central empirical claim is falsifiable. A referee can ask for the missing ablations and error cases; that is normal at this stage rather than a reason to desk-reject.

Referee Report

1 major / 1 minor

Summary. The manuscript presents MerLean-Prover, an end-to-end Lean 4 theorem prover built from Planning, Check, and Lean agents composed via a recursive outer loop. The system replaces sorry declarations with kernel-checkable proofs using no fine-tuning, custom RL objectives, or theorem-specific scaffolding. It reports solving 10/23 theorems on FormalQualBench (surpassing OpenGauss at 8/23), closing all 12/12 on Putnam2025 with lower wall-clock time, and transferring successfully to smaller models (Sonnet and Haiku).

Significance. If the empirical results prove robust, the work would indicate that harness architecture can be a central determinant of success in LLM-based Lean 4 theorem proving, independent of model scale or specialized training. Demonstrating competitive performance on PhD-qualifying and competition-level problems with a simple recursive agent loop, plus transfer to smaller models, would be relevant to the automated theorem proving community.

major comments (1)

[Abstract] Abstract: the central empirical claims (10/23 on FormalQualBench surpassing OpenGauss, 12/12 on Putnam2025) are presented without methodological details on selection criteria for the 23 theorems, error analysis, ablation results on the Planning/Check/Lean loop components, or comparison protocols, preventing evaluation of whether the harness design is the load-bearing factor.

minor comments (1)

[Abstract] The abstract states 'substantially lower total wall-clock' for Putnam2025 but supplies neither quantitative times nor a comparison table.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on the abstract. We address the single major comment below and will revise the manuscript accordingly.

read point-by-point responses

Referee: [Abstract] Abstract: the central empirical claims (10/23 on FormalQualBench surpassing OpenGauss, 12/12 on Putnam2025) are presented without methodological details on selection criteria for the 23 theorems, error analysis, ablation results on the Planning/Check/Lean loop components, or comparison protocols, preventing evaluation of whether the harness design is the load-bearing factor.

Authors: We agree the abstract is concise and omits explicit pointers to these elements. FormalQualBench selection criteria (PhD qualifying-exam problems with explicit difficulty and topic distribution) appear in Section 3; comparison protocols against OpenGauss and other baselines are in Section 4; error analysis and failure-mode breakdown are in the appendix; and component ablations (recursive loop vs. non-recursive, Planning/Check/Lean variants) are reported in Section 5. To make these connections immediate for readers, we will revise the abstract to add one sentence referencing the benchmark construction and the presence of ablations, while preserving length constraints. These changes will allow direct assessment of whether the harness architecture is load-bearing. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper describes an empirical system (MerLean-Prover harness) and reports direct benchmark outcomes on FormalQualBench (10/23 solved) and Putnam2025 (12/12 closed). No equations, fitted parameters, predictions derived from inputs, or self-citations appear in the provided text. The central claim rests on observable kernel-checkable proof generation on external test sets without any reduction of results to the harness description by construction. The derivation chain is therefore self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only review supplies no information on free parameters, background axioms, or new postulated entities.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MerLean-Prover: A Recursive Looping Harness for Lean 4 Theorem Proving." pith.science (2026). https://pith.science/paper/SLNSCT43

@misc{pith2026260526959,
  author       = {Pith},
  title        = {Pith review of: MerLean-Prover: A Recursive Looping Harness for Lean 4 Theorem Proving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SLNSCT43}},
  note         = {Machine review of arXiv:2605.26959}
}

read the original abstract

MerLean-Prover is an end-to-end Lean4 theorem prover that replaces sorry declarations with kernel-checkable proofs. It is built from three agent types (Planning, Check, and Lean) composed by a recursive outer loop whose unit of revision is the proof plan itself, and uses no fine-tuning, no custom RL objective, and no theorem-specific scaffolding. On FormalQualBench, a benchmark of 23 PhD-qualifying-exam theorems, MerLean-Prover solves 10/23, surpassing the strongest published open-source baseline (OpenGauss, 8/23). On Putnam2025, the same harness closes 12/12 with substantially lower total wall-clock than the next-best system that closes the full set. The harness also transfers to smaller models: Sonnet closes all four tested FormalQualBench problems, and Haiku closes the two short ones. These results suggest that harness design is a central factor in end-to-end Lean4 theorem proving, alongside raw model capability, and that a relatively simple harness can already be effective.

Figures

Figures reproduced from arXiv: 2605.26959 by the authors.

**Figure 1.** MERLEAN-PROVER’s recursive looping architecture. The shared proof plan is the unit of revision. The Lean Agent works on one statement at a time; fresh Check Agent instances each answer one decision question. Math: whether si is mathematically correct; Decomposition: whether si should be split; Faithfulness: whether the clean-build Lean file still proves the original si. A clean build goes to Faithfulness before the … view at source ↗

**Figure 2.** Evolution of the dependency graph for BURNSIDEPRIMEDEGREETHEOREM (FORMALQUALBENCH). Each panel is a snapshot at one frame; each node represents a statement, and the node map appears in Section B. Vertical position shows dependency depth in the displayed graph: base helper nodes appear at level 0, and a node is placed one level above the deepest displayed node it uses. Node color: blue = formalized, gray = present bu… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Maximizing Algebraic Connectivity with $2(n-2)$ Edges: The Large Vertex Number Case
math.CO 2026-08 accept novelty 7.0 of 10

Kolokolnikov's conjecture that K_{2,n-2} maximizes algebraic connectivity among graphs with 2(n-2) edges is proved for n≥123, with a claimed Lean formalization for all n≥4.

Reference graph

Works this paper leans on

4 extracted references · cited by 1 Pith paper

[3]

These conditions together state that ‘N‘ acts regularly on ‘α‘; in particular ‘Nat.card N = p‘

for every ‘x :α‘, the stabilizer in ‘N‘ of ‘x‘ is trivial (‘=⊥‘). These conditions together state that ‘N‘ acts regularly on ‘α‘; in particular ‘Nat.card N = p‘. This is the substantive direction of Burnside’s classical 1911 theorem on transitive permutation groups of prime degree, in the elementary form proved by P. M ¨uller, *Permutation groups of prime...

1911
[4]

‘N‘ is normal in ‘G‘,
[5]

‘N‘ acts pretransitively on ‘α‘,
[6]

These conditions state that ‘N‘ acts regularly on ‘α‘

for every ‘x :α‘, the stabilizer in ‘N‘ of ‘x‘ is trivial. These conditions state that ‘N‘ acts regularly on ‘α‘. -/ theoremburnside_dichotomy_core [MulAction.IsPretransitive Gα] (hp : (Fintype.cardα).Prime) (hNot2 :¬MulAction.IsMultiplyPretransitive Gα2) : ∃N : Subgroup G, N.Normal∧ MulAction.IsPretransitive Nα∧ ∀x :α, MulAction.stabilizer N x =⊥:=by -- ...

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.

[1] [3]

These conditions together state that ‘N‘ acts regularly on ‘α‘; in particular ‘Nat.card N = p‘

for every ‘x :α‘, the stabilizer in ‘N‘ of ‘x‘ is trivial (‘=⊥‘). These conditions together state that ‘N‘ acts regularly on ‘α‘; in particular ‘Nat.card N = p‘. This is the substantive direction of Burnside’s classical 1911 theorem on transitive permutation groups of prime degree, in the elementary form proved by P. M ¨uller, *Permutation groups of prime...

1911

[2] [4]

‘N‘ is normal in ‘G‘,

[3] [5]

‘N‘ acts pretransitively on ‘α‘,

[4] [6]

These conditions state that ‘N‘ acts regularly on ‘α‘

for every ‘x :α‘, the stabilizer in ‘N‘ of ‘x‘ is trivial. These conditions state that ‘N‘ acts regularly on ‘α‘. -/ theoremburnside_dichotomy_core [MulAction.IsPretransitive Gα] (hp : (Fintype.cardα).Prime) (hNot2 :¬MulAction.IsMultiplyPretransitive Gα2) : ∃N : Subgroup G, N.Normal∧ MulAction.IsPretransitive Nα∧ ∀x :α, MulAction.stabilizer N x =⊥:=by -- ...