CLAPP: The CLASS LLM Agent for Pair Programming

Boris Bolliet; Christian Fidler; Francisco Villaescusa-Navarro; Julien Lesgourgues; Santiago Casas

REVIEW 3 major objections 3 minor 2 cited by

CLAPP: The CLASS LLM Agent for Pair Programming

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CLAPP aims to give the CLASS cosmology solver a conversational AI pair programmer

desk verdict The submission is an abstract about a CLASS coding assistant attached to the body of an unrelated galaxy-evolution paper; as presented, there is no paper to referee. read the letter →

arxiv 2508.05728 v1 pith:L6ATNPTC submitted 2025-08-07 astro-ph.IM astro-ph.COcs.AIcs.MA

Santiago Casas , Christian Fidler , Boris Bolliet , Francisco Villaescusa-Navarro , Julien Lesgourgues This is my paper · ORCID

classification astro-ph.IMastro-ph.COcs.AIcs.MA

keywords CLAPPCLASSsolverLLMagentssemanticsearchcodegenerationpairprogrammingcosmologicalcomputationscientificsoftware

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper's abstract introduces CLAPP, a web-based AI assistant for the Einstein-Boltzmann solver CLASS, and claims it offers conversational coding support: answering questions, generating CLASS code, debugging errors, and producing plots. The proposed mechanism combines multi-agent LLM orchestration, semantic search across CLASS documentation, and a live Python execution environment. If the claim holds, researchers who are not CLASS experts—and not yet comfortable with AI tools—could write, run, and debug CLASS computations through natural language. The supplied manuscript body is a different article, so the abstract is the only source for this account, and no evaluation is reported.

What carries the argument

The load-bearing architecture is CLAPP's three-part integration: multi-agent LLM orchestration (decomposing a user request into specialized sub-tasks handled by different LLM agents), semantic search over CLASS documentation (retrieving the relevant parameter and function details so answers stay CLASS-specific), and a live Python execution environment (running generated code and returning outputs and plots). The combination is what distinguishes CLAPP from an ordinary LLM chat: retrieval is supposed to keep it grounded, and execution is supposed to make it verifiable in the loop.

What would settle it

Take a set of CLASS tasks (for example, computing CMB power spectra for a specified cosmology, producing a matter power spectrum, or repairing a deliberately broken parameter file), run them through CLAPP, and compare the generated code's execution and outputs with CLASS's own results. If a substantial fraction of responses fail to run or disagree with CLASS, the central claim collapses.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a retrieval-augmented, execution-backed LLM agent can serve as a pair programmer for CLASS. The central discovery asserted is that combining three components—documentation search, a multi-agent LLM workflow, and a Python sandbox—in one conversational web app is sufficient to answer questions, generate code, debug errors, and produce plots for CLASS users. If true, the practical consequence is that CLASS-specific computational tasks become accessible through natural language, lowering the entry barrier for newcomers and making human-AI collaboration more productive in numerical cosmology.

Load-bearing premise

The value of CLAPP depends on the LLM-and-retrieval pipeline producing CLASS answers and code that are correct often enough to trust; if it produces fluent but wrong code, using it would cost more time and trust than it saves.

Editorial extensions

If this is right

Researchers can pose CLASS questions in natural language and receive documentation-grounded answers rather than generic LLM text.
Generated CLASS code can be executed immediately, so syntax and physics errors surface in the conversation and can be debugged interactively.
Plot requests are turned into actual figures by the execution environment, removing a separate scripting and visualization step.
The assistant lowers the barrier for scientists with little CLASS experience or little exposure to AI-assisted coding.
If the tool is adopted, a larger fraction of CLASS usage could shift from hand-written scripts to conversational specification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

This submission alone does not establish that CLAPP produces correct CLASS code; a companion technical description, user study, or benchmark would be needed to verify the abstract's claims.
The same three-component pattern—documentation retrieval, LLM agent orchestration, and a sandboxed interpreter—could be adapted to other cosmological and scientific software, turning any well-documented code into a conversational assistant.
A straightforward test follows: run a set of standard CLASS tasks through the app, compare the generated code and computed outputs against CLASS itself, and measure the failure rate.
Because the retrieval step is grounded in CLASS documentation, the tool's ceiling is set by that documentation; undocumented or rarely used CLASS features are where confident but wrong answers would most likely appear.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's the quick take: this submission is a header-plus-abstract for CLAPP, an LLM-assisted pair-programming agent for the CLASS Einstein-Boltzmann solver, followed by the full text of a completely different paper (Barrufet et al., on red galaxies and the cosmic star-formation rate density). The body never mentions CLAPP, CLASS, or any of the claimed architecture. So as a manuscript, it is internally incoherent: the claims in the abstract have zero support in the supplied text.

What is actually new: on the abstract alone, the CLAPP idea is a sensible integration of established pieces—multi-agent LLM orchestration, semantic search over CLASS documentation, a live Python sandbox, and a Streamlit deployment. That combination, applied specifically to CLASS, is a reasonable engineering contribution. If the tool works, it could genuinely lower the entry barrier for cosmology researchers who use CLASS but aren't familiar with LLM tooling. Those benefits are plausible.

The soft spots are proportionate to the evidence, which is to say they are load-bearing and complete. There is no architecture description, no retrieval corpus, no code repository, no commit hash, no benchmark questions, no success rate, no user study, and no comparison against a generic coding assistant. The central premise—that the LLM returns CLASS-specific answers and code that are correct often enough to be useful—is asserted, not demonstrated. And because the body is an unrelated paper, even the modest value that might come from reading the full text is absent. There is no math or data to audit, and no citation pattern beyond the unrelated body's references.

For whom is this paper? For CLASS users and tool developers, a properly documented CLAPP with evaluation would be worth reading. This submission gives them nothing beyond an abstract. It should not be sent to peer review in its current form; it should be returned to the authors with a request to resubmit the correct manuscript, including at least a basic evaluation (task set, success metrics, error analysis, maybe a comparison against the underlying LLM alone). If the real paper ships that, it deserves referee time. As it stands, no.

Best, [You]

Referee Report

3 major / 3 minor

Summary. The manuscript, arXiv:2508.05728, presents CLAPP, an LLM-based conversational assistant for the CLASS Einstein-Boltzmann solver, claiming that its combination of multi-agent LLM orchestration, semantic search over CLASS documentation, and a live Python execution environment lowers entry barriers and enables more productive human-AI collaboration in cosmology. The abstract provides an architecture sketch and a public web-app URL. However, the supplied full text is not the CLAPP paper: it is a galaxy-evolution manuscript (arXiv:2508.05740v1, 'Strength in Numbers: Red Galaxies Bolster the Cosmic Star Formation Rate Density at z>3') that never mentions CLAPP, its architecture, its implementation, or any evaluation. As submitted, the only evidence for the central claim is the abstract itself.

Significance. If CLAPP works as claimed, it could be a useful community tool for lowering the barrier to CLASS usage in cosmology, particularly for students and researchers new to Einstein-Boltzmann solvers. The core idea of combining retrieval-augmented generation with an execution sandbox for domain-specific scientific software is sensible and timely, and the paper names a concrete deployment target. However, the manuscript does not provide architecture details, prompt/agent design, retrieval corpus specifications, code repository, benchmark suite, user study, correctness metrics, or error analysis. The central effectiveness claim is therefore completely unevidenced in the submitted document. The paper also contains no machine-checked proofs, reproducible code artifacts, or falsifiable predictions that could be assessed independently.

major comments (3)

[Full text (all sections)] The supplied full text is arXiv:2508.05740v1, a paper on red galaxies and the cosmic star formation rate density, not a paper about CLAPP. None of Sections 1-6 describes the assistant's architecture, multi-agent orchestration, retrieval index, execution environment, or user interface. The manuscript's central subject is therefore entirely absent from the body. This is not a local gap but a wholesale mismatch between the abstract's claims and the manuscript content, and it prevents any substantive review of the tool.
[Abstract] The abstract claims that CLAPP 'lowers the entry barrier for scientists unfamiliar with AI tools' and 'enables more productive human-AI collaboration', but no evaluation is reported anywhere: no benchmark questions, no success rate, no user study, no comparison against a baseline coding assistant, and no error analysis. For a software/tools paper, such evidence is load-bearing: the stated value proposition is empirical and cannot be verified from the architecture sentence alone.
[Abstract] The architecture claim rests on three components: multi-agent LLM orchestration, semantic search across CLASS documentation, and a live Python execution environment. The manuscript gives no details on any of these—no retrieval corpus description, no prompt design, no agent topology, no sandbox safety or error-handling procedures. Without these, the reader cannot assess whether the system is likely to produce correct CLASS parameter files and code rather than fluent but physically wrong output.

minor comments (3)

[Title and abstract] The title and abstract describe CLAPP, but the body is a different paper. Even in a corrected submission, the authors should ensure the manuscript text, references, and acknowledgments all correspond to the CLAPP work.
[Abstract] The web app URL is mentioned, but no code repository, version information, or documentation link is provided. A tools paper should include a repository with a versioned release and installation/usage instructions.
[Abstract] The phrase 'domain-specific retrieval' would benefit from a precise statement of what documents are indexed (e.g., CLASS documentation version, examples, forum threads) and how retrieval quality was validated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the document has no derivational content to audit; the abstract's claims are unsupported by the body, which is an unrelated paper, but unsupportedness is not circularity.

full rationale

The submitted material consists of an abstract describing the CLAPP tool and a full text that is an entirely different paper (Barrufet et al., 'Strength in Numbers: Red Galaxies Bolster the Cosmic Star Formation Rate Density at z>3'), which never mentions CLAPP. There is therefore no derivation chain, no equations, no fitted parameters, and no prediction compared to its inputs. The abstract's effectiveness claims — that CLAPP 'lowers the entry barrier for scientists unfamiliar with AI tools and enables more productive human-AI collaboration' — rest solely on the asserted architecture (multi-agent LLM orchestration, semantic search over CLASS documentation, live Python execution). This is a missing-support or correctness concern, not a circularity concern: the architecture is not defined in terms of the claimed benefit, and no result is shown to reduce to its inputs by construction. The only mild self-reference is that CLASS is authored in part by the same community (e.g., Lesgourgues), and the retrieval corpus is built from CLASS documentation; however, the abstract invokes no specific prior work, makes no uniqueness claim, and draws no derivation from those documents, so this is not load-bearing self-citation. Under the hard rule that circularity requires a quoted reduction, no circular step can be identified.

Assumptions & free parameters 0 free parameters · 2 assumptions · 1 invented entities

This is a software tool paper, so the ledger has no fitted parameters. The paper's contribution rests on two domain assumptions: that LLM-based agents actually deliver the claimed coding support for CLASS, and that the CLASS documentation is an adequate retrieval corpus. The only invented entity is CLAPP itself, which is externally testable through the deployed app, so independent_evidence is True. The supplied full text (arXiv:2508.05740v1) provides no evidence on any of these points.

assumptions (2)

domain assumption LLM agents with domain-specific retrieval and a live Python execution environment can provide reliable, useful coding support for CLASS.
This is the core value proposition stated in the abstract ('answering questions, generating code, debugging errors, and producing plots'); it is asserted without any evaluation, benchmark, or user study in the supplied material.
domain assumption The CLASS documentation and codebase form a sufficient and well-indexed knowledge source for the retrieval system.
The architecture in the abstract ('semantic search across CLASS documentation') presupposes that the documentation corpus contains what users need and that retrieval over it yields correct answers; this is not demonstrated.

invented entities (1)

CLAPP (the assistant itself) independent evidence
purpose: Interactive AI pair-programming assistant for the CLASS Einstein-Boltzmann solver: answers questions, generates code, debugs, and produces plots.
CLAPP is a new artifact introduced by the paper. Its falsifiable handle is the deployed Streamlit app URL given in the abstract (https://classclapp.streamlit.app), so an interested user can test the claimed behavior directly; however, no performance characterization is supplied, and the full text is an unrelated paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CLAPP: The CLASS LLM Agent for Pair Programming." pith.science (2026). https://pith.science/paper/L6ATNPTC

@misc{pith2026250805728,
  author       = {Pith},
  title        = {Pith review of: CLAPP: The CLASS LLM Agent for Pair Programming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6ATNPTC}},
  note         = {Machine review of arXiv:2508.05728}
}

read the original abstract

We introduce CLAPP (CLASS LLM Agent for Pair Programming), an interactive AI assistant designed to support researchers working with the Einstein-Boltzmann solver CLASS. CLAPP leverages large language models (LLMs) and domain-specific retrieval to provide conversational coding support for CLASS-answering questions, generating code, debugging errors, and producing plots. Its architecture combines multi-agent LLM orchestration, semantic search across CLASS documentation, and a live Python execution environment. Deployed as a user-friendly web application, CLAPP lowers the entry barrier for scientists unfamiliar with AI tools and enables more productive human-AI collaboration in computational and numerical cosmology. The app is available at https://classclapp.streamlit.app

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Agentic Exploration of Physics Models
cs.AI 2025-09 conditional novelty 7.0 of 10

A general-purpose LLM agent can discover physics models, including ODEs and spin Hamiltonians, by autonomously choosing experiments and fitting hypotheses to numeric data.
DeepInflation: an AI agent for research and model discovery of inflation
astro-ph.CO 2026-01 conditional novelty 6.0 of 10

An LLM agent with symbolic regression finds simple inflation potentials that match target CMB observables, but the outputs are fitted to the targets rather than independently predicted.

Reference graph

Works this paper leans on

6 extracted references · 5 canonical work pages · cited by 2 Pith papers

[1]

Barrufet1⋆, J.S

MNRAS000, 1–11 (2025) Preprint 13 September 2025 Compiled using MNRAS L ATEX style file v3.0 Strength in Numbers: Red Galaxies Bolster the Cosmic Star Formation Rate Density atz≳3 L. Barrufet1⋆, J.S. Dunlop1, R. Begley1, S. Flury1, D.J. McLeod1, K. Arellano-Cordova1, A. Carnall1, F. Cullen1, C. T. Donnan2, F. Liu1, R. McLure1, D. Scholte1, T. M. Stanton1,...

arXiv 2025
[6]

cosmic noon

Throughout the paper, we assume a flat cold dark matter cos- mology withH0 =67.4 km s−1 Mpc−1,Ω m =0.315andΩ Λ =0.685 (Planck Collaboration et al. 2020). All quoted magnitudes are in the AB system (Oke & Gunn 1983), and all derived star-formation rates (SFR) and stellar masses (M∗) assume a Kroupa (2001) IMF. MNRAS000, 1–11 (2025) Strength in Numbers: Red...

work page 2020
[2017]

cosmic noon

by an order of magnitude highlighting the significant contribution of red galaxies to the total SFRD. Despite lower individual SFRs, their high number density makes red galaxies a dominant contributor to the obscured SFRD across cosmic time. log(n/Mpc−3)=−3.42±0.03atz∼1.8and−4.9±0.3atz∼8.5. Fig. 5 shows that the total galaxy population follows similar red...

work page 2025
[2022]

2017; Zavala et al

is incom- plete and is explicitly biased against the observation of the reddest dust-obscuredsources.Oneconsequenceofthisisthat,asmentioned above,whiletheinventoryofdust-enshroudedstar-formationactivity aroundcosmicnoonisnowreasonablywellestablished(e.g.,Dunlop et al. 2017; Zavala et al. 2021), the situation atz>3has remained much less clear. In this stud...

work page 2017
[2023]

Unveiling the hidden universe with JWST: The contribution of dust-obscured galaxies to the stellar mass function at $z\sim3-8$

At z∼5.4, excluding LRDs lowers the SFRD from7.7×10 −3 to 4.8×10−3 M⊙,yr−1,Mpc−3, a∼38%reduction, though still within the uncertainty range. This contrast is less pronounced than that re- ported by Williams et al. (2024), likely due to our nearly tenfold largerredgalaxysample,whichmitigatesstatisticalfluctuations.We also highlight that the majority of LRD...

work page Pith review arXiv 2024
[2024]

2023; Barrufet et al

and spectroscopically con- firming the existence of quiescent galaxies atz>4(Carnall et al. 2023; Barrufet et al. 2024). These findings demonstrate that both populationsaremoreabundantthananticipatedfrompre-JWSTsur- veys. However, the evolutionary connection between these massive MNRAS000, 1–11 (2025) Strength in Numbers: Red Galaxies Bolster the Cosmic S...

work page 2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Barrufet1⋆, J.S

MNRAS000, 1–11 (2025) Preprint 13 September 2025 Compiled using MNRAS L ATEX style file v3.0 Strength in Numbers: Red Galaxies Bolster the Cosmic Star Formation Rate Density atz≳3 L. Barrufet1⋆, J.S. Dunlop1, R. Begley1, S. Flury1, D.J. McLeod1, K. Arellano-Cordova1, A. Carnall1, F. Cullen1, C. T. Donnan2, F. Liu1, R. McLure1, D. Scholte1, T. M. Stanton1,...

arXiv 2025

[2] [6]

cosmic noon

Throughout the paper, we assume a flat cold dark matter cos- mology withH0 =67.4 km s−1 Mpc−1,Ω m =0.315andΩ Λ =0.685 (Planck Collaboration et al. 2020). All quoted magnitudes are in the AB system (Oke & Gunn 1983), and all derived star-formation rates (SFR) and stellar masses (M∗) assume a Kroupa (2001) IMF. MNRAS000, 1–11 (2025) Strength in Numbers: Red...

work page 2020

[3] [2017]

cosmic noon

by an order of magnitude highlighting the significant contribution of red galaxies to the total SFRD. Despite lower individual SFRs, their high number density makes red galaxies a dominant contributor to the obscured SFRD across cosmic time. log(n/Mpc−3)=−3.42±0.03atz∼1.8and−4.9±0.3atz∼8.5. Fig. 5 shows that the total galaxy population follows similar red...

work page 2025

[4] [2022]

2017; Zavala et al

is incom- plete and is explicitly biased against the observation of the reddest dust-obscuredsources.Oneconsequenceofthisisthat,asmentioned above,whiletheinventoryofdust-enshroudedstar-formationactivity aroundcosmicnoonisnowreasonablywellestablished(e.g.,Dunlop et al. 2017; Zavala et al. 2021), the situation atz>3has remained much less clear. In this stud...

work page 2017

[5] [2023]

Unveiling the hidden universe with JWST: The contribution of dust-obscured galaxies to the stellar mass function at $z\sim3-8$

At z∼5.4, excluding LRDs lowers the SFRD from7.7×10 −3 to 4.8×10−3 M⊙,yr−1,Mpc−3, a∼38%reduction, though still within the uncertainty range. This contrast is less pronounced than that re- ported by Williams et al. (2024), likely due to our nearly tenfold largerredgalaxysample,whichmitigatesstatisticalfluctuations.We also highlight that the majority of LRD...

work page Pith review arXiv 2024

[6] [2024]

2023; Barrufet et al

and spectroscopically con- firming the existence of quiescent galaxies atz>4(Carnall et al. 2023; Barrufet et al. 2024). These findings demonstrate that both populationsaremoreabundantthananticipatedfrompre-JWSTsur- veys. However, the evolutionary connection between these massive MNRAS000, 1–11 (2025) Strength in Numbers: Red Galaxies Bolster the Cosmic S...

work page 2023