Pith. sign in

REVIEW 4 major objections 5 minor 33 references

Can LLMs Prove Robotic Path Planning Optimality? A Benchmark for Research-Level Algorithm Verification

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces a 34-task benchmark of approximation-optimality proofs for robotic path planning and finds that, without injected domain lemmas, even the strongest LLMs produce fully valid proofs only about a quarter of the time; supp

desk verdict The benchmark is a genuine contribution, but the headline lemma-vs-posterior comparison sits on an unvalidated LLM-as-judge and should be met with caution until human-agreement numbers appear. read the letter →

arxiv 2603.19464 v3 pith:LO3E3EU2 submitted 2026-03-19 cs.RO

Zhengbang Yang , Md. Tasin Tazwar , Minghan Wei , Zhuangdi Zhu This is my paper · ORCID
classification cs.RO
keywords LLMbenchmarkapproximationratiopathplanningalgorithmverificationin-contextlemmaschain-of-thoughtpromptingproofgenerationLLM-as-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether LLMs can do research-grade mathematical verification in robotics: specifically, prove that a given path-planning algorithm's cost is at most α times optimal. It builds a 34-task benchmark from published planning papers and tests leading models under four information settings. The central finding is that unaided models succeed on at most 26.47% of tasks, while injecting task-specific lemmas extracted from the ground-truth proofs improves performance more than generic chain-of-thought prompting or than being told the correct approximation ratio in advance. The authors argue this shows domain knowledge—not prompting strategy—is the binding constraint on LLM proof generation, and that the oracle ratio mainly helps once the model is anchored by lemmas. A reader should care because the result delineates where LLM-assisted algorithm verification is feasible today and what infrastructure would be needed to move it forward.

What carries the argument

The operating machinery is a controlled benchmark design: 34 anonymized proof tasks, each combining a path-planning problem T, an algorithm A, optional in-context lemmas C, and an optional ground-truth ratio α, with four settings toggling C and α to isolate their effect. Anonymization of problem and algorithm names prevents retrieval by memorized content. Success is measured by an LLM judge scoring final answer, reasoning correctness, and proof relevance, plus a fine-grained error taxonomy and a first-error-position metric that tracks how far into a proof the model stays sound. The key contrast driving the headline result is between the lemma-augmented setting and the no-context, oracle-rati

What would settle it

Re-score a random subset of the 34 tasks' model outputs with human experts who are blinded to which setting produced each proof, and compare their validity ratings with the LLM judge's; if human experts find the lemma-augmented proofs no more valid than the oracle-ratio proofs once lemma citations are ignored, the paper's central comparison collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is empirical: given only a problem description and an algorithm, current LLMs cannot reliably reconstruct the multi-step geometric and combinatorial reasoning needed to certify approximation ratios for constrained path planning. Across 34 tasks from 11 published papers, the best unaided success rate is 26.47%. When prompts include the key lemmas from the original proof, reasoning-correctness and relevance scores rise consistently, and one open-source model surpasses all proprietary counterparts; telling the model the true approximation ratio helps less and mainly acts as a consistency check once lemmas anchor the reasoning. The paper also catalogs failure modes—unsu

Load-bearing premise

The load-bearing premise is that the LLM judge's scores measure actual mathematical proof validity; since the rubric explicitly rewards precise use of the supplied in-context lemmas, the measured advantage of the lemma-augmented setting could partly reflect the judge rewarding lemma citation rather than sound reasoning, and no quantitative human-judge agreement is reported.

Editorial extensions

If this is right

  • Even the strongest current LLMs cannot be trusted as standalone verifiers of approximation-optimality proofs for robotic path-planning algorithms.
  • Providing task-specific lemmas is a more effective intervention than generic chain-of-thought prompting or revealing the correct answer, so improving domain-knowledge injection is the practical lever for LLM proof assistance.
  • The ground-truth approximation ratio is not a substitute for lemmas; it is most useful as a consistency check after the model is anchored by domain knowledge.
  • Generic chain-of-thought prompting alone does not meaningfully reduce task difficulty and can even increase hallucination, though it complements posterior information.
  • First errors tend to appear earlier in weaker proofs, suggesting that catching and correcting early unsupported inferences could substantially improve long-horizon deductive chains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would build a reusable, compositional library of lemmas indexed by planning-problem structure and retrieve them for new algorithms; the paper's results predict this would raise proof success without retraining models.
  • If the judge-reward confound is controlled, formal proof assistants would be a natural next evaluator: checking generated steps against certified lemmas would separate 'lemma citation' from 'valid reasoning'.
  • The findings suggest LLM proof assistance in robotics will likely take the form of interactive, lemma-guided theorem proving rather than one-shot generation.
  • A cheap falsification check would be to supply irrelevant but plausible lemmas in place of the paper-specific ones; if proof quality rises just as much, the effect is prompt length or generic scaffolding, not domain knowledge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a benchmark of 34 approximation-ratio proof tasks derived from 11 published robotics and path-planning papers, and uses it to evaluate four LLMs (GPT-5.2, Gemini 3 Pro, Grok 4.1, Qwen3.5) under four information settings: no context, with in-context lemmas, with the oracle approximation ratio, and with both. The central empirical claims are that without external domain knowledge only up to 26.47% of tasks are successfully addressed, that task-specific in-context lemmas improve reasoning quality more than generic chain-of-thought prompting or providing the ground-truth ratio, and that a fine-grained error taxonomy can characterize failures and show how context augmentation mitigates them. The evaluation relies on GPT-5.2-as-judge scoring generated proofs along final-answer, reasoning-correctness, and proof-relevance dimensions.

Significance. If the evaluation pipeline were validated, this would be a useful and timely benchmark: it targets research-level proofs with operational constraints, going beyond Olympiad-style math benchmarks, and the anonymized task design and multi-setting comparisons are sensible. The paper also provides a fine-grained error taxonomy and an automated extraction pipeline, both potentially valuable to the community. However, the quantitative findings are not yet established because the LLM-as-judge is unvalidated, the rubric contains a near-mechanical advantage for the lemma-augmented setting, and no statistical reliability measures are reported. The benchmark and its qualitative observations are promising, but the headline comparisons currently rest on fragile evaluation foundations.

major comments (4)
  1. [§IV-B, §III-C] The 'proof relevance' dimension explicitly rewards precise use of supplied lemmas, and Setting-2 supplies exactly the ground-truth lemma set, so relevance scores are almost mechanically higher in that setting. Since the same GPT-5.2 judge also scores reasoning correctness, this can inflate the headline Setting-2-over-Setting-3 comparison. The paper states that 4 human annotators verified the evaluation quality of the LLM-as-judge, but reports no agreement statistics, no kappa, no confusion matrix, and no blinded human rescoring of generated proofs. This validation gap is load-bearing for the central claim; please add quantitative judge–human agreement per dimension and on a representative sample of proofs.
  2. [§IV-C, Table I] Success is defined differently across settings: Setting-1/2 require both final-answer and reasoning scores ≥7, while Setting-3/4 require only a reasoning score ≥7. The paper directly compares success rates across these settings to conclude that in-context lemmas outperform posterior information. This is confounded because Setting-3/4 have a strictly easier success criterion. Report comparable metrics across settings (e.g., reasoning-only success everywhere, and final-answer success where it applies) and re-evaluate the comparative claims.
  3. [§IV-A, Table I note] The table note states that consistent timeout runs are assigned a score of 1 for every metric, and Claude 4.6 was excluded because it frequently timed out. Assigning the lowest score conflates failure-to-finish with an incorrect proof and can bias model rankings, especially if timeout frequency differs across models. Report per-model and per-setting timeout counts, and either exclude timeouts from the aggregated scores or handle them with a separate category.
  4. [§IV-C, Table I] All results appear to be single generations scored once by a stochastic LLM judge; no standard deviations, confidence intervals, or repeated runs are reported. Differences such as GPT-5.2 at 26.47% versus Gemini 3 Pro at 11.76% in Setting-1, or the Setting-2 success rates of 32.35% versus 35.29%, may be within run-to-run noise. Please run multiple independent generations and judge calls and report variance or significance tests.
minor comments (5)
  1. [§III-A] Typo: 'to sovle T' should be 'to solve T'. Also clarify in the task definition that 'optimality' means proving the approximation-ratio bound, not exact optimality.
  2. [Table I] The header row repeats 'Final Answer↑' and lists relevance scores even though the table note omits final-answer scores in Settings 3 and 4; make the column structure clearer (e.g., separate sub-headers for w/o CoT and w/ CoT, and mark '–' consistently).
  3. [§IV-D, Table IV] The claim of a negative correlation between reasoning score and first-error position is based on only four aggregated data points (one per setting). This is not enough to support a correlation claim; either report per-task data with a proper correlation test or soften the statement.
  4. [Figure 2] The illustrative example uses the named Christofides algorithm, but the benchmark anonymizes algorithm names to avoid contamination. It would help to state explicitly that this example is illustrative and not a benchmark task, so readers do not confuse the two.
  5. [References] In §IV-E, 'a proposition [33]' refers to a dataset source paper; consider citing the specific lemma or theorem from that paper for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline comparisons rest on separate final-answer and reasoning scores, not on the rubric dimension that explicitly rewards supplied lemmas.

full rationale

The paper's central claims are empirical: LLMs struggle on 34 proof tasks (Setting-1 success capped at 26.47%), and in-context lemmas improve reasoning more than oracle ratios or generic CoT. There is no formal derivation chain in which an output is defined in terms of its own input. The only potentially self-referential element is the LLM-as-judge rubric: the proof-relevance dimension 'rewards precise use of supplied lemmas and definitions.' But the headline Setting-2 vs Setting-3 comparison is gated by Reasoning score (and, in Setting-2, also Final Answer), not by the Relevance score; the success-rate thresholds make Setting-2 stricter, so the lemma advantage is not an artifact of the relevance rubric alone. The benchmark's ground-truth proofs and lemmas come from peer-reviewed papers, including some by co-authors ([9],[33]), but those citations serve as external data sources rather than as load-bearing justifications of the paper's conclusions. Concerns about GPT-5.2-as-judge reliability, same-family judging, and the absence of quantitative human-agreement statistics are validity/correctness risks, not instances of circular reduction. No fitted parameter is renamed as a prediction, and no author-imported uniqueness theorem or ansatz is invoked. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

This is an empirical benchmark paper, not a mathematical derivation, so the knowledge budget is dominated by assumptions about data faithfulness, judge validity, contamination controls, and task representativeness. No new physical or mathematical entities are postulated.

free parameters (2)
  • success threshold = 7 (on a 1–10 scale)
    Success is defined as final answer and reasoning scores ≥7 in Settings 1–2 and reasoning ≥7 in Settings 3–4; changing this threshold would change all reported success rates and model rankings.
  • timeout substitution score = 1
    Timeouts in LLM reasoning are assigned the lowest score 1, conflating API/refusal failure with genuine proof failure in all averages.
assumptions (4)
  • domain assumption Ground-truth proofs from the 11 source papers are correct and the GPT-5.2 extraction pipeline reproduced them faithfully (Section III-C).
    The benchmark's correctness depends on the source proofs and the automated extraction; human verification is asserted, but no agreement metric or audit trail is reported.
  • domain assumption LLM-as-judge (GPT-5.2) scores generated proofs consistently with expert human judgment (Section IV-B).
    No human-judge agreement statistics, inter-annotator reliability, or judge validation set is provided; this is the main measurement assumption.
  • domain assumption Anonymized problem and algorithm names prevent data contamination (Section III-C).
    Anonymization is plausible, but no contamination test (e.g., paraphrasing probes or memorization checks) is reported.
  • domain assumption The expert-selected 11 papers and 34 tasks are representative of research-level approximation-optimality proofs in robotic path planning (Section III-B).
    Representativeness is asserted by expert choice; no sampling frame, coverage analysis, or comparison to a broader corpus is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can LLMs Prove Robotic Path Planning Optimality? A Benchmark for Research-Level Algorithm Verification." pith.science (2026). https://pith.science/paper/LO3E3EU2

@misc{pith2026260319464,
  author       = {Pith},
  title        = {Pith review of: Can LLMs Prove Robotic Path Planning Optimality? A Benchmark for Research-Level Algorithm Verification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LO3E3EU2}},
  note         = {Machine review of arXiv:2603.19464}
}
read the original abstract

Robotic path planning problems are often NP-hard, and practical solutions typically rely on approximation algorithms with provable performance guarantees for general cases. While designing such algorithms is challenging, formally proving their approximation optimality is even more demanding, which requires domain-specific geometric insights and multi-step mathematical reasoning over complex operational constraints. Recent Large Language Models (LLMs) have demonstrated strong performance on mathematical reasoning benchmarks, yet their ability to assist with research-level optimality proofs in robotic path planning remains under-explored. In this work, we introduce the first benchmark for evaluating LLMs on approximation-ratio proofs of robotic path planning algorithms. The benchmark consists of 34 research-grade proof tasks spanning diverse planning problem types and complexity levels, each requiring structured reasoning over algorithm descriptions, problem constraints, and theoretical guarantees. Our evaluation of state-of-the-art proprietary and open-source LLMs reveals that even the strongest models struggle to produce fully valid proofs without external domain knowledge. However, providing LLMs with task-specific in-context lemmas substantially improves reasoning quality, a factor that is more effective than generic chain-of-thought prompting or supplying the ground-truth approximation ratio as posterior knowledge. We further provide fine-grained error analysis to characterize common logical failures and hallucinations, and demonstrate how each error type can be mitigated through targeted context augmentation.

Figures

Figures reproduced from arXiv: 2603.19464 by the authors.

Figure 1
Figure 1. LLM proof generation pipeline. The LLM receives a path-planning [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An Example Comparison of a Path Planning Optimality Proof [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Distribution of the reasoning errors occurring in Setting 1. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 5 linked inside Pith

  1. [1]

    Solving the multi-objective path planning problem for mobile robot using an improved nsga-ii algorithm,

    P. Duan, Z. Yu, K. Gao, L. Meng, Y . Han, and F. Ye, “Solving the multi-objective path planning problem for mobile robot using an improved nsga-ii algorithm,”Swarm and Evolutionary Computation, vol. 87, p. 101576, 2024. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S2210650224001147

    2024

  2. [2]

    Approximation algorithms for some vehicle routing problems,

    C. Bazgan, R. Hassin, and J. Monnot, “Approximation algorithms for some vehicle routing problems,”Discrete Applied Mathematics, vol. 146, no. 1, pp. 27–42, 2005. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0166218X04002860

    2005

  3. [3]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . K. Li, Y . Wu, and D. Guo, “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,” 2024. [Online]. Available: https://arxiv.org/abs/2402.03300

    arXiv 2024

  4. [4]

    Omni-MATH: A universal olympiad level mathematic benchmark for large language models,

    B. Gao, F. Song, Z. Yang, Z. Cai, Y . Miao, Q. Dong, L. Li, C. Ma, L. Chen, R. Xu, Z. Tang, B. Wang, D. Zan, S. Quan, G. Zhang, L. Sha, Y . Zhang, X. Ren, T. Liu, and B. Chang, “Omni-MATH: A universal olympiad level mathematic benchmark for large language models,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Avail...

    2025

  5. [5]

    OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems,

    C. He, R. Luo, Y . Bai, S. Hu, Z. Thai, J. Shen, J. Hu, X. Han, Y . Huang, Y . Zhang, J. Liu, L. Qi, Z. Liu, and M. Sun, “OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),...

    2024

  6. [6]

    Matharena: Evaluating LLMs on uncontaminated math competitions,

    M. Balunovic, J. Dekoninck, I. Petrov, N. Jovanovi ´c, and M. Vechev, “Matharena: Evaluating LLMs on uncontaminated math competitions,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. [Online]. Available: https://openreview.net/forum?id=y0zL9IZxZ7

    2025

  7. [7]

    Training verifiers to solve math word problems,

    K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, “Training verifiers to solve math word problems,” 2021. [Online]. Available: https://arxiv.org/abs/2110.14168

    arXiv 2021

  8. [8]

    Measuring mathematical problem solving with the MATH dataset,

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt, “Measuring mathematical problem solving with the MATH dataset,” inThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. [Online]. Available: https://openreview.net/forum? id=7Bywt2mQsCe

    2021

Show all 33 references
  1. [9]

    A log-approximation for coverage path planning with the energy constraint,

    M. Wei and V . Isler, “A log-approximation for coverage path planning with the energy constraint,”Proceedings of the International Conference on Automated Planning and Scheduling, vol. 28, no. 1, pp. 532–539, Jun. 2018. [Online]. Available: https: //ojs.aaai.org/index.php/ICAP...

    2018

  2. [10]

    An approximation algorithm for the pickup and delivery vehicle routing problem on trees,

    N. Katoh and T. Yano, “An approximation algorithm for the pickup and delivery vehicle routing problem on trees,”Discrete Applied Mathematics, vol. 154, no. 16, pp. 2335–2349, 2006, discrete Algorithms and Optimization, in Honor of Professor Toshihide Ibaraki at His Retirement ...

    2006

  3. [11]

    Chain of thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds.,...

    2022

  4. [12]

    Can language models solve graph problems in natural language?

    H. Wang, S. Feng, T. He, Z. Tan, X. Han, and Y . Tsvetkov, “Can language models solve graph problems in natural language?” inThirty- seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=UDqHhbqYJV

    2023

  5. [13]

    Llm+p: Empowering large language models with optimal planning proficiency,

    B. Liu, Y . Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone, “Llm+p: Empowering large language models with optimal planning proficiency,” 2023. [Online]. Available: https: //arxiv.org/abs/2304.11477

    2023 arXiv

  6. [14]

    LLM-a*: Large language model enhanced incremental heuristic search on path planning,

    S. Meng, Y . Wang, C.-F. Yang, N. Peng, and K.-W. Chang, “LLM-a*: Large language model enhanced incremental heuristic search on path planning,” inFindings of the Association for Computational Linguistics: EMNLP 2024, Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, Eds. Miami, Flor...

    2024

  7. [15]

    Grapharena: Evaluating and exploring large language models on graph computation,

    J. Tang, Q. Zhang, Y . Li, N. Chen, and J. Li, “Grapharena: Evaluating and exploring large language models on graph computation,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https: //openreview.net/forum?id=Y1r9yCMzeA

    2025

  8. [16]

    Travelplanner: a benchmark for real-world planning with language agents,

    J. Xie, K. Zhang, J. Chen, T. Zhu, R. Lou, Y . Tian, Y . Xiao, and Y . Su, “Travelplanner: a benchmark for real-world planning with language agents,” inProceedings of the 41st International Conference on Machine Learning, ser. ICML’24. JMLR.org, 2024

    2024

  9. [17]

    Personal travel solver: A preference-driven LLM-solver system for travel planning,

    Z. Shao, J. Wu, W. Chen, and X. Wang, “Personal travel solver: A preference-driven LLM-solver system for travel planning,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M....

    2025

  10. [18]

    Let’s verify step by step,

    H. Lightman, V . Kosaraju, Y . Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe, “Let’s verify step by step,” inThe twelfth international conference on learning representations, 2023

    2023

  11. [19]

    ARB: Advanced reasoning benchmark for large language models,

    T. Sawada, D. Paleka, A. Havrilla, P. Tadepalli, P. Vidas, A. P. Kranias, J. J. Nay, K. Gupta, and A. Komatsuzaki, “ARB: Advanced reasoning benchmark for large language models,” 2024. [Online]. Available: https://openreview.net/forum?id=gsZAtAdzkY

    2024

  12. [20]

    Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai,

    E. Glazer, E. Erdil, T. Besiroglu, D. Chicharro, E. Chen, A. Gunning, C. F. Olsson, J.-S. Denain, A. Ho, E. de Oliveira Santos, O. J ¨arviniemi, M. Barnett, R. Sandler, M. Vrzala, J. Sevilla, Q. Ren, E. Pratt, L. Levine, G. Barkley, N. Stewart, B. Grechuk, T. Grechuk, S. V . E...

    2025 arXiv

  13. [21]

    Gold-medalist performance in solving olympiad geometry with alphageometry2,

    Y . Chervonyi, T. H. Trinh, M. Ol ˇs´ak, X. Yang, H. Nguyen, M. Menegali, J. Jung, J. Kim, V . Verma, Q. V . Le, and T. Luong, “Gold-medalist performance in solving olympiad geometry with alphageometry2,” 2025. [Online]. Available: https://arxiv.org/abs/ 2502.03544

    2025

  14. [22]

    Solving inequality proofs with large language models,

    J. Sheng, L. Lyu, J. Jin, T. Xia, A. Gu, J. Zou, and P. Lu, “Solving inequality proofs with large language models,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. [Online]. Available: https://openreview.net/fo...

    2025

  15. [23]

    The open proof corpus: A large-scale study of LLM-generated mathematical proofs,

    J. Dekoninck, I. Petrov, K. Minchev, M. Marinov, M. Drencheva, L. Konova, M. M. Shumanov, K. Tsvetkov, N. Drenchev, L. D. Todorov, K. Nikolova, N. Georgiev, V . Kalinkova, M. Ismoldayev, M. Balunovic, and M. Vechev, “The open proof corpus: A large-scale study of LLM-generated ...

    2026

  16. [24]

    Judging llm-as-a-judge with mt-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging llm-as-a-judge with mt-bench and chatbot arena,” inPro- ceedings of the 37th International Conference on Neural Information Proc...

    2023

  17. [25]

    A survey on llm-as-a-judge,

    J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y . Shen, S. Ma, H. Liuet al., “A survey on llm-as-a-judge,”The Innovation, 2024

    2024

  18. [26]

    Language models as science tutors,

    A. Chevalier, J. Geng, A. Wettig, H. Chen, S. Mizera, T. Annala, M. J. Aragon, A. R. Fanlo, S. Frieder, S. Machadoet al., “Language models as science tutors,”arXiv preprint arXiv:2402.11111, 2024

    2024 arXiv

  19. [27]

    Autograding mathematical induc- tion proofs with natural language processing,

    C. Zhao, M. Silva, and S. Poulsen, “Autograding mathematical induc- tion proofs with natural language processing,”International Journal of Artificial Intelligence in Education, pp. 1–31, 2025

    2025

  20. [28]

    Language models are few-shot graders,

    ——, “Language models are few-shot graders,” inInternational Con- ference on Artificial Intelligence in Education. Springer, 2025, pp. 3–16

    2025

  21. [29]

    Approximation algorithms for distance constrained vehicle routing problems,

    V . Nagarajan and R. Ravi, “Approximation algorithms for distance constrained vehicle routing problems,”Networks, vol. 59, no. 2, pp. 209–214, 2012. [Online]. Available: https://onlinelibrary.wiley.com/ doi/abs/10.1002/net.20435

    2012 doi

  22. [30]

    A new approximation algorithm for the capacitated vehicle routing problem on a tree,

    T. Asano, N. Katoh, and K. Kawashima, “A new approximation algorithm for the capacitated vehicle routing problem on a tree,”Journal of Combinatorial Optimization, vol. 5, no. 2, p. 213–231, Jun. 2001. [Online]. Available: http://dx.doi.org/10.1023/A: 1011461300596

    2001 doi

  23. [31]

    Approximation algorithms for some routing problems,

    G. N. Frederickson, M. S. Hecht, and C. E. Kim, “Approximation algorithms for some routing problems,” in17th Annual Symposium on Foundations of Computer Science (sfcs 1976), 1976, pp. 216–227

    1976

  24. [32]

    Approximation algorithms for tours of height-varying view cones,

    P. A. Plonski and V . Isler, “Approximation algorithms for tours of height-varying view cones,”The International Journal of Robotics Research, vol. 38, no. 2-3, pp. 224–235, 2019. [Online]. Available: https://doi.org/10.1177/0278364918784353

    2019 doi

  25. [33]

    Coverage path planning under the energy constraint,

    M. Wei and V . Isler, “Coverage path planning under the energy constraint,” in2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, pp. 368–373

    2018

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.