Pith. sign in

REVIEW 4 major objections 7 minor 6 cited by

This concept paper argues that LLM-based multi-agent systems—teams of specialized, communicating agents—are becoming the central paradigm for software engineering, and that the main obstacles to adoption are orchestration, human-agent coord

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:27 UTC pith:4D25KSCB

load-bearing objection A competent, readable survey of multi-agent LLM systems for SE, but its central premise that multi-agent beats single-agent is asserted rather than tested—fine as a map, not as an argument. the 4 major comments →

arxiv 2601.09822 v2 pith:4D25KSCB submitted 2026-01-14 cs.SE cs.AI

LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities

Yongjian Tang , Thomas Runkler This is my paper
classification cs.SE cs.AI
keywords LLM-based agentsmulti-agent systemssoftware engineeringsoftware development lifecyclecode generationtesting and debuggingagent orchestrationSE benchmarks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to establish that LLM-based multi-agent systems are the next step beyond single-model assistants for software engineering, because modern development tasks are inherently collaborative. It reviews evidence across five stages of the software development lifecycle—requirements engineering, code generation, static checking, testing, and debugging—and argues that role-specialized agents that exchange feedback and use tools can cover the whole pipeline. The authors are not reporting new experiments; they are making a synthesis claim: the field is converging on collaborative agent teams, and the bottlenecks are now orchestration, human-agent coordination, computational cost, data collection, and the absence of benchmarks that measure collaboration. A sympathetic reader should care because if this framing is right, research priorities and evaluation standards in software engineering will shift from single-model capability to multi-agent coordination.

Core claim

On its own terms, the paper's central claim is that a multi-agent system composed of specialized LLM agents—each handling roles like planning, coding, reviewing, testing, or debugging—can provide comprehensive, end-to-end solutions to real-world software engineering problems, where single LLM calls fall short. The authors survey dozens of systems that embody this idea and group them by lifecycle stage, showing that collaboration patterns such as role specialization, iterative feedback from models/tools/humans, and parallel execution recur across stages. They then argue that the primary open problems are no longer basic model competence but how to orchestrate agents, how to coordinate them wi

What carries the argument

The central object is the LLM-based multi-agent system: a set of specialized agents, each built on a language model, that communicate through messages, call external tools, and refine their outputs through feedback loops. The paper identifies role specialization (orchestrator, programmer, reviewer, tester) and iterative feedback—from other models, from compilers and test runners, and from humans—as the mechanisms that let these teams outperform a single model. A secondary piece of machinery is the agent communication protocol, which standardizes inter-agent messaging and tool access, and the surrounding frameworks that put these components together. These mechanisms are doing the work of tur

Load-bearing premise

The paper assumes, without a controlled comparison of its own, that splitting work among specialized agents produces better software-engineering outcomes than a single capable LLM equipped with the same tools and feedback.

What would settle it

A controlled experiment that holds the underlying LLM fixed and compares three configurations—a single model with tools, a simple pipeline with one feedback loop, and a full multi-agent team—on the same set of repository-level software engineering tasks, recording correctness, cost, and latency. If the single model or simple pipeline matches or exceeds the multi-agent system on typical tasks, the paper's central premise fails. A complementary check would be a benchmark that measures collaborative outcomes and asks whether agent teams produce measurably better end products than a single agent g

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the paper's synthesis is right, next-generation software engineering tooling will look like a team of specialized agents rather than a single chat assistant, with orchestrator and reviewer roles becoming standard.
  • Software engineering benchmarks will need to test collaboration—joint requirements gathering, code integration, peer review—not just single-task correctness, because single-task scores miss what multi-agent systems add.
  • Human oversight will remain part of the loop, so the useful question becomes when and how humans should intervene, not whether to automate fully.
  • Deploying these systems in production will require cutting compute, for example by distilling large models into smaller task-specific agents and by minimizing iterations between agents.
  • Collecting data across the entire development lifecycle—design documents, developer discussions, historical changes, runtime logs—will be a prerequisite for agents that mimic real software engineering roles.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that evaluation metrics for agentic software engineering should include process cost—tokens consumed, number of agent turns, wall-clock time, and human corrections—alongside correctness; these will determine whether multi-agent overhead pays off.
  • A testable extension of the survey's premise: multi-agent gains should be largest on loosely specified, multi-file tasks and smallest on well-scoped, single-function tasks; re-analyzing existing benchmark results by task granularity could check this.
  • The paper's emphasis on SDLC-wide data suggests that organizations with rich development histories have a structural advantage in building specialized agents, which could widen the gap between large platforms and smaller teams.
  • The authors' stated future plan—comparing planner-executor pairs, self-refinement loops, and human-in-the-loop designs—implies that orchestration patterns, not underlying model choice, may be the decisive variable for multi-agent success.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This concept paper surveys LLM-based multi-agent systems for software engineering across the SDLC: requirements engineering, code generation, static code checking, testing, and debugging. It reviews selected systems, discusses language model selection, SE benchmarks, agentic frameworks, and communication protocols, and closes with future challenges and a research agenda. The paper argues that multi-agent systems are a promising paradigm for end-to-end SE tasks, but it reports no original experiments; the conclusion explicitly defers empirical comparisons of multi-agent design patterns to future work.

Significance. The paper occupies a useful niche: it organizes a fast-moving body of work into SDLC stages, connects it to benchmarks and frameworks, and names the important open problems—orchestration, human-agent coordination, cost, and data collection. Its breadth and the explicit enumeration of research directions make it a reasonable entry point for practitioners. However, the central premise, that multi-agent systems outperform single-LLM pipelines or simpler tooling, is not established by the evidence presented; the paper itself defers the needed comparisons. Several specific summarization and framing issues also need correction. These problems are fixable through reframing and more careful reporting, after which the manuscript would be acceptable as a position/concept paper.

major comments (4)
  1. [§1, §2.2, §6] The core claim that LLM-based multi-agent systems 'provide comprehensive, end-to-end solutions for real-world SE challenges' is asserted rather than demonstrated. Section 2.2 lists five advantages (specialization, modularity, collaboration, tool use, parallelism) without citing a single comparative study showing these advantages over single-LLM pipelines. Section 3 reviews heterogeneous works on different benchmarks with no synthesis of effect sizes or baselines. Section 6 then states that comparing multi-agent design patterns is future work. This is load-bearing for the survey's framing. Please either explicitly reframe the advantages as hypotheses and the central claim as potential, or add an evidence table reporting, for each reviewed system, the baseline, metric, and quantitative outcome.
  2. [§3.1] The MARE claim—'surpassing SOTA baselines using text parser and logical reasoning by 15.4%'—is under-specified: no baseline names, metric, or evaluation setup are given. In a review, an isolated numerical comparison without context is not verifiable and can mislead readers. Either remove the number or reproduce the source's precise comparative result with the relevant metric and baselines.
  3. [§4.2] The sentence 'By evaluating LLM-based solutions across these established benchmarks, we provide empirical evidence for the effectiveness of our approaches' reads as a statement of completed work, but no such evaluation appears in the manuscript. This conflates the research agenda with achieved results. Additionally, the description of GSM8K as a benchmark that 'assesses mathematical reasoning in code' is inaccurate; GSM8K is a math word-problem benchmark, and its role in a code-generation section should be clarified or removed.
  4. [§3.3] In the static code checking subsection, the text says 'Preliminary studies show that fine-tuning LLMs on code base or simply prompting them has demonstrated superior effectiveness in identifying bugs in code snippets [FC24; Yu23a].' Reference [FC24] is about automated Android bug replay from issue reports, not static bug detection, and 'superior effectiveness' is not quantified or compared. This is a summary accuracy problem that should be corrected.
minor comments (7)
  1. [Abstract, §3] The paper calls itself a 'systematic' review, but Section 3 is a narrative selection of works without a search protocol or inclusion criteria. If systematicity is intended, add a methodology subsection; otherwise, soften the wording.
  2. [§3.1] 'use stories' should read 'user stories'.
  3. [§4.3] The agent frameworks (CrewAI, LangGraph, Google ADK, AutoGen, n8n) are listed without citations. Add pointers to primary sources or documentation.
  4. [§4.1] The claim that 'Proprietary frontier models represent performance upper bounds' should be hedged (e.g., 'generally considered') or supported with specific benchmark references.
  5. [§4.2] Capitalize 'HumanEval'; also, the GSM8K entry should be corrected as noted in the major comments.
  6. [References] Several references are incomplete or lack identifiers (e.g., [Ga23], [Li14], [Zh23], [Zh24b]). Add arXiv IDs or venue information.
  7. [§6] Minor grammar issue: 'As for future work, We plan...' should be 'As for future work, we plan...'.

Circularity Check

0 steps flagged

No significant circularity: survey claims summarize external work; self-citations are not load-bearing.

full rationale

This is a concept/survey paper, not a derivation. The central claim—that LLM-based multi-agent systems are a promising paradigm for SE—is supported by citations to a heterogeneous body of external work, and no equation, fitted parameter, or benchmark result is renamed as a prediction. The three self-citations ([ELT25], [Ta25], [THR24]) appear only as examples of techniques (notebook transformation, few-shot dilemma, prompt optimization for NER) and are not load-bearing; the survey's framing does not reduce to them. The skeptical objection—that Section 2.2's advantages are asserted without a controlled comparison, and Section 6 defers empirical evaluation with 'we plan to compare multi-agent design patterns, such as planner–executor pairs, self-refinement loops, and human-in-the-loop, and perform empirical evaluations'—is a critique of evidential support, not circularity. The paper makes no formal derivation from an input to an output, and no step exhibits the required equivalence of a result to its own inputs by construction. Accordingly, the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper introduces no free parameters, mathematical axioms, or invented entities. Its load-bearing assumptions are domain-level: that multi-agent orchestration is beneficial and that current benchmarks capture relevant SE ability. These are reasonable for a survey but remain unvalidated.

axioms (2)
  • domain assumption Specialization, modularity, collaboration, tool use, and parallelism make multi-agent systems superior to single-LLM approaches for SE tasks.
    Section 2.2 lists these as 'main advantages' without any empirical comparison or controlled evidence; the entire review and research agenda depend on this premise.
  • domain assumption Success on existing individual-task benchmarks (HumanEval, GSM8K, BugBench, Tests4Py) is a meaningful proxy for real-world SE capability.
    Section 4.2 proposes evaluating approaches with these benchmarks, implicitly assuming transfer to production software engineering; the paper itself later notes benchmarks overlook collaborative aspects.

pith-pipeline@v1.3.0-alltime-deepseek · 7843 in / 5299 out tokens · 59008 ms · 2026-08-03T10:27:10.873815+00:00 · methodology

0 comments
read the original abstract

Despite recent advancements in Large Language Models (LLMs), complex Software Engineering (SE) tasks require more collaborative and specialized approaches. This concept paper systematically reviews the emerging paradigm of LLM-based multi-agent systems, examining their applications across the Software Development Life Cycle (SDLC), from requirements engineering and code generation to static code checking, testing, and debugging. We delve into a wide range of topics such as language model selection, SE evaluation benchmarks, state-of-the-art agentic frameworks and communication protocols. Furthermore, we identify key challenges and outline future research opportunities, with a focus on multi-agent orchestration, human-agent coordination, computational cost optimization, and effective data collection. This work aims to provide researchers and practitioners with valuable insights into the current forefront landscape of agentic systems within the software engineering domain.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems

    cs.AI 2026-05 unverdicted novelty 7.0

    A survey that unifies prior work on multi-agent LLM systems via the LIFE framework, mapping dependencies across collaboration, failure attribution, and autonomous self-evolution while identifying cross-stage challenges.

  2. PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization

    cs.SE 2026-05 unverdicted novelty 7.0

    PerfCodeBench reveals that state-of-the-art LLMs produce functionally correct but significantly slower code than expert-optimized versions on system-level tasks, especially those involving parallelism and GPUs.

  3. Specifying the Delegated-Autonomy Boundary: Requirements Engineering for Agentic AI

    cs.SE 2026-07 conditional novelty 6.0

    A requirements-engineering framework that justifies when agentic AI is warranted (AJR) and specifies graduated autonomy, memory, coordination, assurance, and evolution policies (ADP).

  4. Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systems

    cs.AI 2026-05 conditional novelty 5.0

    The survey proposes the LIFE framework to unify fragmented research on collaboration, failure attribution, and self-evolution in LLM multi-agent systems into a progression toward self-organizing intelligence.

  5. PromptMN: Pseudo Prompting Language

    cs.CL 2026-06 unverdicted novelty 4.0

    PromptMN is a pseudo-prompting DSL that adds compact typed directives to natural language to improve clarity, reusability, and reverse engineering of AI instructions.

  6. Code Broker: A Multi-Agent System for Automated Code Quality Assessment

    cs.SE 2026-04 unverdicted novelty 3.0

    Code Broker deploys a five-agent hierarchy that combines LLM semantic analysis with static linting to generate actionable Python code quality reports.

Reference graph

Works this paper leans on

35 extracted references · 26 linked inside Pith · cited by 5 Pith papers

  1. [1]

    [Ba24] Bairi, R

    [An25] Anthropic: Introducing Claude Sonnet 4.5d, Accessed September 29, 2025, https: //www.anthropic.com/news/claude-sonnet-4-5. [Ba24] Bairi, R. et al.: Codeplan: Repository-level coding using llms and planning. Proceedings of the ACM on Software Engineering 1(FSE), pp. 675–698,

    2025

  2. [5]

    et al.: A survey on in-context learning

    [Do24] Dong, Q. et al.: A survey on in-context learning. In: Proceedings of the 2024 conference on empirical methods in natural language processing. Pp. 1107–1128,

    2024

  3. [6]

    arXiv preprint arXiv:2511.07257,

    [ELT25] Elhashemy, H.; Lotfy, Y.; Tang, Y.: Bridging the Prototype-Production Gap: A Multi- Agent System for Notebooks Transformation. arXiv preprint arXiv:2511.07257,

    arXiv

  4. [8]

    In: 2017 IEEE 25th international requirements engineering conference (RE)

    [FSG17] Ferrari,A.;Spagnolo,G.O.;Gnesi,S.:Pure:Adatasetofpublicrequirementsdocuments. In: 2017 IEEE 25th international requirements engineering conference (RE). IEEE, pp. 502–505,

    2017

  5. [10]

    et al.: CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing

    [He24] He, X. et al.: CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing. arXiv preprint arXiv:2403.13583,

    Pith/arXiv arXiv

  6. [11]

    et al.: Cigar: Cost-efficient program repair with llms

    [Hi24] Hidvégi, D. et al.: Cigar: Cost-efficient program repair with llms. arXiv preprint arXiv:2402.06598,

    Pith/arXiv arXiv

  7. [12]

    et al.: Large language model-powered smart contract vulnerability detection: New perspectives

    [Hu23a] Hu, S. et al.: Large language model-powered smart contract vulnerability detection: New perspectives. In: 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). IEEE, pp. 297–306,

    2023

  8. [13]

    et al.: MARE: Multi-Agents Collaboration Framework for Requirements Engi- neering

    [Ji24] Jin, D. et al.: MARE: Multi-Agents Collaboration Framework for Requirements Engi- neering. arXiv preprint arXiv:2405.03256,

    Pith/arXiv arXiv

  9. [14]

    arXiv preprint arXiv:2306.02907,

    [JWW23] Jiang,S.;Wang,Y.;Wang,Y.:Selfevolve:Acodeevolutionframeworkvialargelanguage models. arXiv preprint arXiv:2306.02907,

    Pith/arXiv arXiv

  10. [15]

    [KAY23] Kang,S.;An,G.;Yoo,S.:Apreliminaryevaluationofllm-basedfaultlocalization.arXiv preprint arXiv:2308.05487,

    Pith/arXiv arXiv

  11. [16]

    arXiv preprint arXiv:2310.08992,

    [Le23] Le,H.etal.:Codechain:Towardsmodularcodegenerationthroughchainofself-revisions with representative sub-modules. arXiv preprint arXiv:2310.08992,

    Pith/arXiv arXiv

  12. [17]

    et al.: Non-functional requirements as qualities, with a spice of ontology

    [Li14] Li, F.-L. et al.: Non-functional requirements as qualities, with a spice of ontology. In: 2014 IEEE 22nd International Requirements Engineering Conference (RE). IEEE, pp. 293–302,

    2014

  13. [19]

    et al.: Deepseek-v3 technical report

    [Li24b] Liu, A. et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437,

    Pith/arXiv arXiv

  14. [21]

    et al.: Multi-role consensus through llms discussions for vulnerability detection

    [Ma24b] Mao, Z. et al.: Multi-role consensus through llms discussions for vulnerability detection. In: 2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C). IEEE, pp. 1318–1319,

    2024

  15. [22]

    [Mu23] Mu, F

    [Me25] Meta:LLaMA4:Leadingmultimodalintelligence,2025,https://ai.meta.com/blog/llama- 4-multimodal-intelligence/. [Mu23] Mu, F. et al.: ClarifyGPT: Empowering LLM-based Code Generation with Intention Clarification. arXiv preprint arXiv:2310.10996,

    Pith/arXiv arXiv 2025

  16. [23]

    [PB24] Pizzorno, J.A.; Berger, E.D.: CoverUp: Coverage-Guided LLM-Based Test Generation

    [Op25] OpenAI: GPT-5 System Card, Accessed August 7, 2025, https://openai.com/index/gpt-5- system-card/. [PB24] Pizzorno, J.A.; Berger, E.D.: CoverUp: Coverage-Guided LLM-Based Test Generation. arXiv preprint arXiv:2403.16218,

    Pith/arXiv arXiv 2025

  17. [24]

    et al.: AgentFL: Scaling LLM-based Fault Localization to Project-Level Context

    [Qi24] Qin, Y. et al.: AgentFL: Scaling LLM-based Fault Localization to Project-Level Context. arXiv preprint arXiv:2403.16362,

    Pith/arXiv arXiv

  18. [25]

    et al.: A systematic survey of prompt engineering in large language models: Techniques and applications

    [Sa24a] Sahoo, P. et al.: A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927,

    Pith/arXiv arXiv

  19. [26]

    et al.: CodeAgent: Collaborative Agents for Software Engineering

    [Ta24] Tang, D. et al.: CodeAgent: Collaborative Agents for Software Engineering. ArXiv abs/2402.02172, 2024, https://api.semanticscholar.org/CorpusID:270865202. [Ta25] Tang, Y. et al.: The few-shot dilemma: Over-prompting large language models. arXiv preprint arXiv:2509.13196,

    Pith/arXiv arXiv 2024

  20. [27]

    et al.: Mint: Evaluating llms in multi-turn interaction with tools and language feedback

    [Wa23] Wang, X. et al.: Mint: Evaluating llms in multi-turn interaction with tools and language feedback. arXiv preprint arXiv:2309.10691,

    Pith/arXiv arXiv

  21. [28]

    et al.: Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation

    [Wa24a] Wang, C. et al.: Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation. arXiv preprint arXiv:2401.06391,

    Pith/arXiv arXiv

  22. [29]

    et al.: Emergent Abilities of Large Language Models, 2022, arXiv: 2206.07682 [cs.CL]

    [We22] Wei, J. et al.: Emergent Abilities of Large Language Models, 2022, arXiv: 2206.07682 [cs.CL]. [Wu24] Wu,Q.etal.:Autogen:Enablingnext-genLLMapplicationsviamulti-agentconversations. In: First Conference on Language Modeling

    Pith/arXiv arXiv 2022

  23. [30]

    et al.: A survey of ai agent protocols

    [Ya25] Yang, Y. et al.: A survey of ai agent protocols. arXiv preprint arXiv:2504.16736,

    Pith/arXiv arXiv

  24. [31]

    arXiv preprint arXiv:2308.01240,

    [Yu23a] Yuan,Z.etal.:Evaluatinginstruction-tunedlargelanguagemodelsoncodecomprehension and generation. arXiv preprint arXiv:2308.01240,

    Pith/arXiv arXiv

  25. [32]

    et al.: No more manual tests? evaluating and improving chatgpt for unit test generation

    [Yu23b] Yuan, Z. et al.: No more manual tests? evaluating and improving chatgpt for unit test generation. arXiv preprint arXiv:2305.04207,

    Pith/arXiv arXiv

  26. [33]

    et al.: Toolcoder: Teach code generation models to use api search tools

    [Zh23] Zhang, K. et al.: Toolcoder: Teach code generation models to use api search tools. arXiv preprint arXiv:2305.04032,

    Pith/arXiv arXiv

  27. [34]

    et al.: Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges

    [Zh24b] Zhang, K. et al.: Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges. arXiv preprint arXiv:2401.07339,

    Pith/arXiv arXiv

  28. [35]

    et al.: A Survey of Large Language Models, 2024, arXiv: 2303.18223 [cs.CL], https://arxiv.org/abs/2303.18223

    [Zh24c] Zhao, W.X. et al.: A Survey of Large Language Models, 2024, arXiv: 2303.18223 [cs.CL], https://arxiv.org/abs/2303.18223

    Pith/arXiv arXiv 2024

  29. [2005]

    et al.: SpecGen: Automated Generation of Formal Program Specifications via Large Language Models

    [Ma24a] Ma, L. et al.: SpecGen: Automated Generation of Formal Program Specifications via Large Language Models. arXiv preprint arXiv:2401.08807,

    Pith/arXiv arXiv

  30. [2014]

    et al.: Camel: Communicative agents for"mindëxploration of large language model society.AdvancesinNeuralInformationProcessingSystems36,pp.51991–52008,2023

    [Li23] Li, G. et al.: Camel: Communicative agents for"mindëxploration of large language model society.AdvancesinNeuralInformationProcessingSystems36,pp.51991–52008,2023. [Li24a] Li, H. et al.: Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach.ProceedingsoftheACMonProgrammingLanguages8(OOPSLA1),pp.474– 499,

    2023

  31. [2017]

    [Go25] Google: Gemini 3 Pro - Best for complex tasks and bringing creative concepts to life, 2025, https://deepmind.google/models/gemini/pro/

    [Ga23] Gao,Y.etal.:Retrieval-augmentedgenerationforlargelanguagemodels:Asurvey.2023. [Go25] Google: Gemini 3 Pro - Best for complex tasks and bringing creative concepts to life, 2025, https://deepmind.google/models/gemini/pro/. [Ha24] Han, Z. et al.: Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey. Transactions on Machine Learning...

    2023

  32. [2021]

    [DBM25] Derouiche,H.;Brahmi,Z.;Mazeni,H.:AgenticAIFrameworks:Architectures,Protocols, and Design Challenges, 2025, arXiv: 2508.10146[cs.AI], https://arxiv.org/abs/2508. 10146. [De19] Devlin, J. et al.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: North American Chapter of the Association for Computational Linguisti...

    Pith/arXiv arXiv 2025

  33. [2023]

    et al.: Training Verifiers to Solve Math Word Problems

    [Co21] Cobbe, K. et al.: Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168,

    Pith/arXiv arXiv

  34. [2024]

    et al.: Evaluating Large Language Models Trained on Code, 2021, arXiv: 2107.03374[cs.LG]

    [Ch21] Chen, M. et al.: Evaluating Large Language Models Trained on Code, 2021, arXiv: 2107.03374[cs.LG]. [Ch23] Chen, X. et al.: Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128,

    Pith/arXiv arXiv 2021

  35. [2025]

    et al.: Static Code Analysis in the AI Era: An In-depth Exploration of the Concept, Function, and Potential of Intelligent Code Analysis Agents

    [Fa23] Fan, G. et al.: Static Code Analysis in the AI Era: An In-depth Exploration of the Concept, Function, and Potential of Intelligent Code Analysis Agents. arXiv preprint arXiv:2310.08837,

    Pith/arXiv arXiv