An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding

Dan Li; Li Chen; Xizheng Wang; Yanyu Ren; Yunfeng Bai

REVIEW 4 major objections 6 minor 14 references

For identical tasks and models, changing only how agents coordinate shifts functional scores by over 30 points and doubles wall-clock time, placing organizational topology on par with model capability.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A new multi-agent coding benchmark (MSEval) shows that collaboration topology—not just model ability—strongly shifts the speed, cost, and quality of LLM-built software.

T0 review reviewed 2026-07-31 challenge →

load-bearing objection MSEval is a genuinely useful evaluation instrument for multi-agent coding, but the headline causal claims about topology outrun the evidence; send it to review, don't take the 30-point claim at face value. the 4 major comments →

arxiv 2607.27877 v1 pith:FJWILHWG submitted 2026-07-30 cs.AI

An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding

Yanyu Ren , Yunfeng Bai , Xizheng Wang , Li Chen , Dan Li This is my paper

classification cs.AI

keywords multi-agent codingcollaboration topologycoordination modeLLM agentssoftware engineering benchmarkfrom-scratch codingfunctional completion scoretoken cost

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MSEval puts coordination policy on the same footing as model capability by measuring how teams of LLM agents build ten full-stack projects from scratch across ten collaboration topologies. For identical tasks and models, swapping the topology shifted functional scores by more than 30 points and roughly doubled wall-clock time, making organizational design a first-order determinant of software delivery. Structured pipeline handoffs converged fastest at the highest quality, while heavy managerial oversight and open-source-style review degraded performance. The authors note the evaluation scale is bounded by cost—three parallel trials on the reference project, spot reruns elsewhere—and scope is limited to web applications. The benchmark's ceiling appears to be integration completeness and cross-agent contracts rather than raw code generation.

Core claim

The paper's central claim is that the collaboration mode—defined by an ownership map, activation schedule, decision rule, and required handoff artifact—is a first-class experimental variable in multi-agent software development, not a prompt flourish. Using MSEval, the authors show that with the same task, model family, requirement document, deployment path, and rubric, varying only the topology moves the best-round functional score by over 30 points (e.g., one model ranged from 89.9 under pipeline to 43.0 under open-source review on the instant-messaging project) and doubles wall-clock time. Mode rankings hold in the aggregate—QA-first and rotation tie at 83.3 across ten projects while open-

What carries the argument

The mode template: each of the ten collaboration topologies is an instance of four concrete objects—ownership map (who owns which part of the repository), activation schedule (who is awake in which round), decision rule (how conflicts resolve), and required handoff artifact (what must exist before a round is accepted). LegoGent executes this template with a periodic sync loop that broadcasts a progress snapshot roughly every four minutes, an active peer mailbox for targeted questions, and an all-idle-and-artifacts completion gate; TAgent then probes the deployed artifact through UI, API, and code checks and returns weighted item-level feedback that routes the next round.

Load-bearing premise

The paper asserts that the collaboration mode is the only independent variable across runs, which requires that the ten mode templates differ only in the four defined template fields (ownership map, activation schedule, decision rule, handoff artifact) and not in prompt wording, timing noise, or hidden runtime details; no manipulation check is reported to confirm that agents actually followed the intended mode, so the 30-point spread could in principle be an artifact of accid

What would settle it

Run the same project and model under two modes, then swap only the mode-template text while keeping the rest of the agent prompts, sync schedule, and runtime identical; if the score difference persists, it is caused by the template content rather than the coordination policy. Alternatively, instrument agent logs to verify mode-specific behaviors—e.g., in pipeline mode confirm the backend stage never writes code before the architect stage completes, and in rotation mode confirm roles actually swap each round; a mismatch between commanded and observed behavior would undermine the causal claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Coordination policy should be treated as a controlled experimental variable in future multi-agent coding benchmarks, not a prompt detail.
Fit matters more than a global best: serialized-handoff modes win projects with long requirement chains or clean stage boundaries, while parallel-contention modes pay a reconciliation tax unless redundancy is cheap to score.
Feedback-driven refinement works: 82% of adjacent rounds improve and 94.7% of runs finish above their first round, so three rounds capture most of the available gain.
The score frontier is not the cost frontier: high raw token counts do not necessarily mean high budget, since cache reads are far cheaper than fresh input, and the highest-scoring model can be several times slower and more expensive than a near-tied alternative.
The benchmark's ceiling is integration completeness and cross-agent contracts, not raw code generation: one broken prerequisite can cascade-skip up to 14 downstream checks, and security/transport items recur in nearly 80% of runs due to deployment-layer integration.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the mode-fidelity assumption holds, organization design becomes a tunable hyperparameter of an agentic coding stack; engineering teams could choose a mode per project by predicting the dependency structure (e.g., pipeline for transactional CRUD, feature squad for realtime composition).
A testable extension would be an ablation that varies only one template field at a time (e.g., fix the ownership map, swap the activation schedule) to isolate which of the four fields drives most of the 30-point spread.
The mode template suggests a taxonomy for future benchmarks: any multi-agent coding system could be described by an ownership map, activation schedule, decision rule, and handoff artifact, enabling cross-benchmark comparison without adopting this paper's runtime.
Because security/transport deductions come from the deployment layer in nearly 80% of runs, re-scoring with a more permissive deployment ceiling could change relative mode rankings on integration-heavy projects.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

MSEval is a genuinely useful evaluation instrument for multi-agent coding, but the headline causal claims about topology outrun the evidence; send it to review, don't take the 30-point claim at face value.

read the letter

Candid take: MSEval is the first benchmark I know that combines from-scratch full-stack projects, explicit coordination topologies, CI/CD deployment, and token/latency/cost accounting in one instrument. That combination is real and useful; the failure taxonomy (1,440 checks) and the worked refinement trace are the strongest parts. The paper is worth a serious referee. The headline causal claim—topology shifts scores by over 30 points—should not be accepted as stated.

Strengths: Table 7's four-template mode definitions (ownership map, activation schedule, decision rule, handoff artifact) are concrete enough to reproduce. Cost accounting with provider price tables and per-round token breakdown is careful. TAgent's dependency-aware scheduling and item-level feedback are thoughtful. The 0.87 Spearman correlation with TA scores is useful validity evidence, even if limited.

Soft spots, in order: (1) No manipulation check. The paper says mode is the only independent variable, but no audit shows pipeline stages were serial, rotation changed roles, PM gates were enforced, etc. The single feature-squad trace doesn't validate the other nine. Without that, the 30-point spread could be prompt wording, template bugs, or agents ignoring constraints. This is a load-bearing gap for the causal claim, not fatal for the benchmark's value. (2) Statistics. Most cells are single runs, no confidence intervals or tests. Triplicates only for P00 token/time. The abstract's 'structured pipelines converge fastest with highest quality' is not what Table 8 shows: QA-first and rotation tie for best average, pipeline is mid-pack, and PM oversight is near the top despite 'managerial oversight degrades'. (3) TAgent is both grader and feedback signal, so the 82% improvement is partly a property of the loop, not independent evidence about topologies. (4) Abstract says released, appendix says will release; that needs to be resolved before reproducibility claims. (5) Dropping Qwen after zero across the board is defensible, but it means 'five models' is really three for the full grid.

Bottom line: for benchmark designers and anyone studying multi-agent coordination, this is a valuable scaffold. If the code appears with a commit hash and the manipulation check closes, it could become a standard. I'd send it to peer review with a request for code, a manipulation check, and conservative rewording of the causal claims. Desk rejection would be a loss.

Referee Report

4 major / 6 minor

Summary. The paper introduces MSEval, a benchmark for from-scratch multi-agent software construction, together with the LegoGent execution runtime and the TAgent automated grader. Ten real-world web projects, ten collaboration topologies, and five LLMs are combined into a 100-configuration grid (with three full-grid models run on all ten projects). The central claim is that, for identical tasks and models, the collaboration topology changes functional scores by over 30 points and doubles wall-clock time, making organizational policy a first-order determinant of the speed–cost–quality trade-off. The paper also reports per-round refinement under TAgent feedback, a failure taxonomy, and a detailed worked trace. The benchmark and infrastructure are released.

Significance. If the central claim is substantiated, this is a genuinely useful contribution: it moves multi-agent coding evaluation from synthetic, single-attempt settings to deployment-grounded, cost-aware, multi-round measurement. The design has real strengths: deterministic rubric-based scoring, native CI/CD deployment, cache-aware token/cost accounting, three-round refinement with item-level feedback, a clear template-based mode taxonomy, and a released codebase. The paper also ships a concrete worked example with execution logs. However, the quantitative support for the headline claim is currently fragile: most project-by-mode cells are single runs, no statistical inference is reported, and the causal attribution to 'topology' rests on an untested assumption that the ten modes were behaviorally instantiated as specified. These issues are fixable and do not undermine the value of the infrastructure, but they must be addressed before the empirical conclusions can be accepted.

major comments (4)

[Methodology, 'The collaboration mode is the only independent variable'; Table 7; Figure 7] The causal claim—'varying the topology shifts scores by over 30 points and doubles wall-clock time'—requires that the ten LegoGent modes actually implemented the intended coordination policies and that nothing else varied systematically. The paper states that the collaboration mode is the only independent variable, but it reports no manipulation check. Table 7 defines each mode by ownership map, activation schedule, decision rule, and handoff artifact, yet there is no evidence from execution logs that, for example, pipeline stages were truly serial, rotation actually permuted roles each round, PM oversight gated commits, or feature-squad agents respected module boundaries. Figure 7 shows a single worked trace from the feature-squad mode and cannot validate the other nine. Without an audit of the remaining 90+ configurations, the observed score and latency differences could be driven by p
[Evaluation, Tables 2/3/8 and Limitations ('Randomness and scale')] The quantitative evidence for the headline 30-point spread is thin. Most project-by-mode cells in Tables 2 and 3, and all cells in Table 8, appear to derive from a single run (the limitation only says three parallel trials were run for the instant-messaging token/time evaluation). No confidence intervals, standard errors, or significance tests are provided. This matters because the cross-mode average range in Table 8 is narrow (73.5 to 83.3), and the 'over 30 points' figure comes from one model/project comparison (DeepSeek v4 Pro: pipeline 89.9 vs open-source 43.0 on P00). Without variance estimates, it is impossible to tell whether the observed mode differences are robust or partly noise. Please report per-condition repetition (at least for the headline claims), or provide a clear statistical model and effect sizes with uncertainty.
[Evaluation, 'Qwen3.6-Flash... we omit it from the results'] Qwen3.6-Flash is run on all ten modes of the reference project, scores 0 everywhere, and is then dropped from the results. This changes the claimed model comparison and the '100 runs' count. The decision to omit is reasonable only if the zero is diagnostic of the model rather than a bug in the experimental setup (e.g., a systematic failure to interact with the LegoGent runtime). Please report the Qwen runs in a table, state the exact failure mode (deployment? rubric mismatch? agent noncompliance?), and include this exclusion in the limitations. As written, the reader cannot tell whether Qwen is incapable or whether the harness was incompatible.
['TAgent Feedback Drives Refinement' and 'TAgent Verification Methodology'] The claim that '82.0% of adjacent transitions improve' is partly a property of the loop under study: TAgent defines the Functional Completion Score and its feedback is the refinement signal, so improvement toward the same rubric is expected. This is not a fitted-parameter circularity, but it means the refinement result is not an independent validation of TAgent or of the agents' learning ability. The reported Spearman correlation of 0.87 with TA scores is on student projects, not on agent-generated artifacts. Please add a validation of TAgent scoring on a sample of agent-generated runs (e.g., human expert re-scoring), and discuss the degree to which the refinement numbers are a self-consistency measure rather than external evidence of quality.

minor comments (6)

[Abstract and Figure 1] The abstract says 'Across 100 runs' and Figure 1 says '100 Team Configurations', but the full-grid evaluation (10 projects × 10 modes × 3 models) implies 300 runs. Please clarify whether '100' refers to configurations, project–mode pairs, or something else, and align the wording.
[Table 2 and Table 3] The column headers 'T', 'C', 'Tok' are used without definition in the caption or text. Define them explicitly (wall-clock minutes, USD cost, million tokens) in each table caption.
[Figure 4] The axis titles 'USD to best round' and 'Minutes to best round' are ambiguous: are these cumulative to the best round, or the value at the best round? Clarify the exact definition of the plotted quantities.
[Figure 5] The radar figure uses ten normalized axes, with seven from logs and three from protocols, but the mapping from axis labels to raw metrics is not given in the caption or text. Add a table or legend specifying the construction of each axis, especially 'Ownership', 'Inspect', and 'Parallel'.
[Table 9] The pricing table lists 'Cache-wr' for Claude and Qwen but '—' for others. This is plausible for provider billing, but the phrase 'cache writes fold into new input' is stated only in the caption. State explicitly which models bill cache-write separately and how the conversion from CNY to USD is applied (the fixed 6.6 rate is given, but the date of the rate is not).
[Minor editorial] There are several typographical and formatting issues: the abstract is a single run-on paragraph, the reference 'Qwen Team 2026' is incomplete (no arXiv or URL), and some table cells in Table 2 contain ambiguous values (e.g., '1,481' vs '11481' in the Feat./Claude row seems to be a line-break artifact). Please proofread the final version.

Circularity Check

0 steps flagged

No significant circularity: the paper is an empirical benchmark study whose central claims are measurements, not derivations that reduce to their inputs.

full rationale

MSEval does not contain a derivation chain whose output is equivalent to its input. Its central claims—that topology shifts scores by over 30 points and doubles wall-clock time, that QA-first and rotation tie for the best cross-project mean, that regression is a distinct failure mode—are empirical observations over 100 runs, not consequences of an assumed equation. The only formal equation in the paper is the cost identity USD = Σ(n·π), which is a definitional accounting of logged token fields at published prices, not a prediction. The TAgent feedback loop uses the same weighted rubric to score and to generate the repair agenda; this means the 82% improvement figure should be interpreted as measuring whether agents can act on the grader's own item-level evidence, but it is not a fitted parameter renamed as a prediction and it does not make the observed improvement equal to the input by construction. The reported Spearman 0.87 correlation of TAgent scores with human TA scores is external validation, not circular. There are no load-bearing self-citations and no imported uniqueness theorems; cited works are prior benchmarks, pricing pages, and general background. The absence of a manipulation check for whether each of the ten modes was actually instantiated as intended is a construct-validity and confound concern about causal attribution, but it is not a circularity step under the definitions used here: the mode effect is not defined in terms of the outcome score. The paper's claims are therefore self-contained as an empirical study, with no significant circularity found.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The paper contains no mathematical derivation, so the free quantities are benchmark design choices: rubric weights, sync interval, round budget, and grader thresholds. These choices directly determine every reported score and cost. The load-bearing assumptions are that TAgent scores track real quality, that the mode templates cleanly instantiate the named collaboration policies, that contamination is negligible, and that single runs are representative. Provider price tables and the CNY/USD conversion are external inputs, not fitted parameters.

free parameters (4)

Per-project rubric weights = P00: 40 system / 50 functional / 10 docs; other projects described qualitatively
Hand-authored weighting of requirements into a 100-point score. Every comparison of modes and models is a function of these weights; they are chosen by the authors, not derived from an external standard.
Sync interval = ~4 minutes (244s observed)
The periodic sync broadcast interval is a design choice that directly affects communication overhead, token usage, and wall-clock time.
Round budget and per-round cap = 3 rounds; 90-minute cap
Convergence and regression results depend on this budget; a different budget would change R1/R2/R3 trajectories and cost figures.
TAgent pass/partial/fail thresholds = Not reported per check
The grader's verdict boundaries are hand-set and influence every score; no sensitivity analysis is given.

axioms (4)

domain assumption TAgent's Functional Completion Score is a valid proxy for real software quality.
Central scores are produced by TAgent; validation is one Spearman correlation of 0.87 on student projects (Limitations). No randomized human-grader comparison is reported on the ten benchmark projects.
ad hoc to paper The ten collaboration modes are faithfully implemented in LegoGent and behaviorally distinct.
The paper asserts 'The collaboration mode is the only independent variable' (Methodology) but provides no manipulation check that the mode templates produce the intended team dynamics rather than accidental prompt differences.
domain assumption LLM training-data contamination does not meaningfully inflate scores.
The paper argues that 2000+ generated lines make MSEval 'not sensitive to data contamination' without a contamination audit; modern LLMs may have seen similar capstone-style web apps.
domain assumption Non-P00 runs are representative despite single trials.
Only P00 is run three times; all other project × mode × model cells are single runs, so stochasticity could change rankings (Limitations).

invented entities (3)

MSEval independent evidence
purpose: Benchmark measuring speed, cost, and quality of multi-agent LLM coding across 10 projects and 10 topologies.
Public GitHub URL claimed; third parties can run it, though no commit hash is given in the paper.
LegoGent independent evidence
purpose: Runtime that instantiates 10 collaboration topologies with periodic sync, mailbox, and CI/CD deployment.
Released as part of MSEval; external execution is possible if the repository is complete.
TAgent independent evidence
purpose: Automated grader that discovers UI/API/code surfaces and returns weighted scores and item-level feedback.
Comparable against human TA grading (paper reports Spearman 0.87 on student projects), giving a falsifiable handle outside the paper.

reviewed 2026-07-31 · how reviews work

0 comments

Cite this review

Pith. "Pith review of An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding." pith.science (2026). https://pith.science/paper/FJWILHWG

@misc{pith2026260727877,
  author       = {Pith},
  title        = {Pith review of: An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FJWILHWG}},
  note         = {Machine review of arXiv:2607.27877}
}

Share X Bluesky LinkedIn Reddit HN

read the original abstract

Multi-agent vibe coding promises to accelerate software development, yet existing benchmarks rely on synthetic environments that ignore practical time and monetary costs, conflate reasoning with communication, and reward only superficial completion. We introduce multi-agent from-scratch evaluation benchmark, MSEval, evaluating multi-agent coding on real-world tasks. Grounded in 10 authentic, full-stack projects across 10 domains, MSEval scores performance using hierarchical requirements and deterministic rubrics. Its execution engine, LegoGent, tests 10 collaboration topologies where agents coordinate via periodic sync intervals and deploy through native CI/CD pipelines. Concurrently, the automated grader TAgent dynamically probes implementations to jointly measure functional success, latency, and prefix-cached token cost. Across 100 runs, MSEval reveals that organizational topology rivals model capability in shaping the speed--cost--quality trade-off. For identical tasks and models, varying the topology shifts scores by over 30 points and doubles wall-clock time. Structured pipelines converge fastest with the highest quality, whereas heavy managerial oversight degrades performance. Ultimately, MSEval establishes a rigorous, reproducible standard for measuring how multi-agent teams actually build software. The benchmark is released at https://github.com/robinren03/MSEval.

Figures

Figures reproduced from arXiv: 2607.27877 by Dan Li, Li Chen, Xizheng Wang, Yanyu Ren, Yunfeng Bai.

**Figure 1.** Figure 1: MSEval overview. MSEval builds and evaluates a web app from requirement document without detailed API specifications and evaluate the projects comprehensively. project and model family, every run receives the same requirement document, round budget, deployment path, and TAgent rubric. What changes is the organizational policy: who owns each artifact, which agents may run concurrently, where handoffs occur… view at source ↗

**Figure 2.** Figure 2: An illustration of LegoGent execution under the rotation collaboration mode, where the agents switch their roles in each round. BE and FE mean backend and frontend. work, test evidence, blockers, and next steps, and the runtime broadcasts a consolidated state snapshot to the team. An active mailbox supports targeted peer questions and handoffs between sync rounds. This design gives parallel workers a share… view at source ↗

**Figure 3.** Figure 3: TAgent parses a requirement document into weighted checks, discovers the implementation, and merges parallel UI, API, and code evidence into a score. LegoGent enables validated artifacts as unit of collaboration. The useful unit of collaboration is validated artifacts, such as a schema, route contract, integration point, test report, or deployable repository state. LegoGent crops the overall failure repor… view at source ↗

**Figure 4.** Figure 4: Pareto-frontier collaboration modes on instant messaging, scores against USD cost and wall-clock time. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

**Figure 5.** Figure 5: Collaboration-mode radar on ten normalized axes. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

**Figure 6.** Figure 6: Per-round trajectories of instant messaging scores over R1–R3. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

**Figure 7.** Figure 7: One MSEval round end to end, from the feature-squad / Claude Opus 4.8 / instant-messaging run. Top (collaboration): a periodic LegoGent sync broadcast—the runtime-state block and the “idle-and-stable + required-artifacts” gate are the actual basis for accepting a round, and each agent reports the same four fields every cycle; here dev-4 crashed on a provider API error and dev-1 absorbed its frontend work, … view at source ↗

discussion (0)

Reference graph

Works this paper leans on

14 extracted references · 5 linked inside Pith

[3]

arXiv:2107.03374

Evaluating Large Language Models Trained on Code. arXiv:2107.03374. DeepSeek

Pith/arXiv arXiv
[4]

https://api-docs

Models and Pricing. https://api-docs. deepseek.com/quick_start/pricing/. Accessed: 2026-07-18. DeepSeek-AI; Xu, A.; Lin, B.; Xue, B.; Wang, B.; Xu, B.; Wu,B.;Zhang,B.;Lin,C.;Dong,C.;Ling,C.;Lu,C.;Zhao, C.; Deng, C.; Hou, C.; Xu, C.; Shao, C.; Ruan, C.; Sun, C.; Dai, D.; Guo, D.; Yang, D.; Chen, D.; Li, D.; Ji, D.; Li, E.; Wei,F.;Lin,F.;Yuan,F.;Xia,F.;Dai,...

2026
[5]

arXiv:2606.19348

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence. arXiv:2606.19348. Du,M.;Xu,B.;Zhu,C.;Wang,X.;andMao,Z.2025. Deep- ResearchBench:AComprehensiveBenchmarkforDeepRe- search Agents. arXiv:2506.11763. Grötschla, F.; Müller, L.; Tönshoff, J.; Galkin, M.; and Per- ozzi, B

arXiv 2025
[7]

InProceedings of the 2025 6th International ConferenceonComputerScienceandManagementTechnol- ogy, ICCSMT ’25, 1285–1290

A Multi-Agent Coding Assistant for Cloud- NativeDevelopment:FromRequirementstoDeployableMi- croservices. InProceedings of the 2025 6th International ConferenceonComputerScienceandManagementTechnol- ogy, ICCSMT ’25, 1285–1290. New York, NY, USA: Asso- ciation for Computing Machinery. ISBN 9798400719981. Huang, W.; Lee, C.; Tng, L.; and Ge, S

2025
[8]

arXiv:2607.07946

Deep- SWE:MeasuringFrontierCodingAgentsonOriginal,Long- Horizon Engineering Tasks. arXiv:2607.07946. Jimenez, C. E.; Yang, J.; Wettig, A.; Yao, S.; Pei, K.; Press, O.; and Narasimhan, K

Pith/arXiv arXiv
[10]

In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.;andSun,Y.,eds.,InternationalConferenceonLearning Representations, volume 2024, 47832–47850

RepoBench: Bench- marking Repository-Level Code Auto-Completion Systems. In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.;andSun,Y.,eds.,InternationalConferenceonLearning Representations, volume 2024, 47832–47850. Lu, E.; Pan, B.; Birlikci, D.; Lee, S.; Wang, R.; Choudhury, R.;Ma,F.;Qin,T.;Baronio,C.;Alberti,S.;etal.2026. Intro- ducing Fronti...

2024
[11]

Wang, X.; Li, B.; Song, Y.; Xu, F

Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All. Wang, X.; Li, B.; Song, Y.; Xu, F. F.; Tang, X.; Zhuge, M.; Pan,J.;Song,Y.;Li,B.;Singh,J.;etal.2025.Openhands:An openplatformforaisoftwaredevelopersasgeneralistagents. InInternational Conference on Learning Representations, volume 2025, 65882–65919. Xia, C. S.; Deng, Y.; Dunn, S.; and Zhang, L

2025
[12]

arXiv:2604.25256

AutoResearchBench:BenchmarkingAIAgentsonComplex Scientific Literature Discovery. arXiv:2604.25256. Xu,F.F.;Song,Y.;Li,B.;Tang,Y.;Jain,K.;Bao,M.;Wang, Z.; Zhou, X.; Guo, Z.; Cao, M.; et al

Pith/arXiv arXiv
[13]

InTheThirty-eighthAnnualConferenceonNeuralInforma- tion Processing Systems

Swe-agent: Agent- computerinterfacesenableautomatedsoftwareengineering. InTheThirty-eighthAnnualConferenceonNeuralInforma- tion Processing Systems. Z.AI.2026a. GLM-5.2:BuiltforLong-HorizonTasks. https: //z.ai/blog/glm-5.2. Accessed: 2026-07-15. Z.AI. 2026b. Z.AI Pricing. https://docs.z.ai/guides/ overview/pricing. Accessed: 2026-07-15. Zhang, Y.; Ruan, H....

2026
[14]

theteamstopped talking

Autocoderover:Autonomousprogramimprovement. InPro- ceedings of the 33rd ACM SIGSOFT International Sympo- sium on Software Testing and Analysis, 1592–1604. Zhu, K.; Du, H.; Hong, Z.; Yang, X.; Guo, S.; Wang, D. Z.; Wang,Z.;Qian,C.;Tang,R.;Ji,H.;etal.2025. Multiagent- bench: Evaluating the collaboration and competition of llm agents. InProceedings of the 63...

2025
[2021]

arXiv:2108.07732

Program Synthesis with Large Language Models. arXiv:2108.07732. Campbell, G. A.; and Papapetrou, P. P. 2013.SonarQube in action. Manning Publications Co. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; de Oliveira Pinto, H.P.;Kaplan,J.;Edwards,H.;Burda,Y.;Joseph,N.;Brock- man, G.; Ray, A.; Puri, R.; Krueger, G.; Petrov, M.; Khlaaf, H.; Sastry, G.; Mishkin, P.; C...

Pith/arXiv arXiv 2013
[2024]

Liu, T.; Xu, C.; and McAuley, J

Swe-bench: Can lan- guage models resolve real-world github issues? InInter- national Conference on Learning Representations, volume 2024, 54107–54157. Liu, T.; Xu, C.; and McAuley, J

2024
[2025]

arXiv:2507.08616

AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs. arXiv:2507.08616. Guan, T

Pith/arXiv arXiv
[2026]

https://www.alibabacloud.com/help/en/ model-studio/model-pricing

Alibaba Cloud Model Studio: Model Inference Pricing. https://www.alibabacloud.com/help/en/ model-studio/model-pricing. Accessed: 2026-07-17. Anthropic. 2026a. Claude API Pricing. https://platform. claude.com/docs/en/about-claude/pricing. Accessed: 2026- 07-20. Anthropic. 2026b. Introducing Claude Opus 4.8. https: //www.anthropic.com/news/claude-opus-4-8. ...

2026

This paper was first reviewed by deepseek-v4-flash on July 31, 2026.

[1] [3]

arXiv:2107.03374

Evaluating Large Language Models Trained on Code. arXiv:2107.03374. DeepSeek

Pith/arXiv arXiv

[2] [4]

https://api-docs

Models and Pricing. https://api-docs. deepseek.com/quick_start/pricing/. Accessed: 2026-07-18. DeepSeek-AI; Xu, A.; Lin, B.; Xue, B.; Wang, B.; Xu, B.; Wu,B.;Zhang,B.;Lin,C.;Dong,C.;Ling,C.;Lu,C.;Zhao, C.; Deng, C.; Hou, C.; Xu, C.; Shao, C.; Ruan, C.; Sun, C.; Dai, D.; Guo, D.; Yang, D.; Chen, D.; Li, D.; Ji, D.; Li, E.; Wei,F.;Lin,F.;Yuan,F.;Xia,F.;Dai,...

2026

[3] [5]

arXiv:2606.19348

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence. arXiv:2606.19348. Du,M.;Xu,B.;Zhu,C.;Wang,X.;andMao,Z.2025. Deep- ResearchBench:AComprehensiveBenchmarkforDeepRe- search Agents. arXiv:2506.11763. Grötschla, F.; Müller, L.; Tönshoff, J.; Galkin, M.; and Per- ozzi, B

arXiv 2025

[4] [7]

InProceedings of the 2025 6th International ConferenceonComputerScienceandManagementTechnol- ogy, ICCSMT ’25, 1285–1290

A Multi-Agent Coding Assistant for Cloud- NativeDevelopment:FromRequirementstoDeployableMi- croservices. InProceedings of the 2025 6th International ConferenceonComputerScienceandManagementTechnol- ogy, ICCSMT ’25, 1285–1290. New York, NY, USA: Asso- ciation for Computing Machinery. ISBN 9798400719981. Huang, W.; Lee, C.; Tng, L.; and Ge, S

2025

[5] [8]

arXiv:2607.07946

Deep- SWE:MeasuringFrontierCodingAgentsonOriginal,Long- Horizon Engineering Tasks. arXiv:2607.07946. Jimenez, C. E.; Yang, J.; Wettig, A.; Yao, S.; Pei, K.; Press, O.; and Narasimhan, K

Pith/arXiv arXiv

[6] [10]

In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.;andSun,Y.,eds.,InternationalConferenceonLearning Representations, volume 2024, 47832–47850

RepoBench: Bench- marking Repository-Level Code Auto-Completion Systems. In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.;andSun,Y.,eds.,InternationalConferenceonLearning Representations, volume 2024, 47832–47850. Lu, E.; Pan, B.; Birlikci, D.; Lee, S.; Wang, R.; Choudhury, R.;Ma,F.;Qin,T.;Baronio,C.;Alberti,S.;etal.2026. Intro- ducing Fronti...

2024

[7] [11]

Wang, X.; Li, B.; Song, Y.; Xu, F

Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All. Wang, X.; Li, B.; Song, Y.; Xu, F. F.; Tang, X.; Zhuge, M.; Pan,J.;Song,Y.;Li,B.;Singh,J.;etal.2025.Openhands:An openplatformforaisoftwaredevelopersasgeneralistagents. InInternational Conference on Learning Representations, volume 2025, 65882–65919. Xia, C. S.; Deng, Y.; Dunn, S.; and Zhang, L

2025

[8] [12]

arXiv:2604.25256

AutoResearchBench:BenchmarkingAIAgentsonComplex Scientific Literature Discovery. arXiv:2604.25256. Xu,F.F.;Song,Y.;Li,B.;Tang,Y.;Jain,K.;Bao,M.;Wang, Z.; Zhou, X.; Guo, Z.; Cao, M.; et al

Pith/arXiv arXiv

[9] [13]

InTheThirty-eighthAnnualConferenceonNeuralInforma- tion Processing Systems

Swe-agent: Agent- computerinterfacesenableautomatedsoftwareengineering. InTheThirty-eighthAnnualConferenceonNeuralInforma- tion Processing Systems. Z.AI.2026a. GLM-5.2:BuiltforLong-HorizonTasks. https: //z.ai/blog/glm-5.2. Accessed: 2026-07-15. Z.AI. 2026b. Z.AI Pricing. https://docs.z.ai/guides/ overview/pricing. Accessed: 2026-07-15. Zhang, Y.; Ruan, H....

2026

[10] [14]

theteamstopped talking

Autocoderover:Autonomousprogramimprovement. InPro- ceedings of the 33rd ACM SIGSOFT International Sympo- sium on Software Testing and Analysis, 1592–1604. Zhu, K.; Du, H.; Hong, Z.; Yang, X.; Guo, S.; Wang, D. Z.; Wang,Z.;Qian,C.;Tang,R.;Ji,H.;etal.2025. Multiagent- bench: Evaluating the collaboration and competition of llm agents. InProceedings of the 63...

2025

[11] [2021]

arXiv:2108.07732

Program Synthesis with Large Language Models. arXiv:2108.07732. Campbell, G. A.; and Papapetrou, P. P. 2013.SonarQube in action. Manning Publications Co. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; de Oliveira Pinto, H.P.;Kaplan,J.;Edwards,H.;Burda,Y.;Joseph,N.;Brock- man, G.; Ray, A.; Puri, R.; Krueger, G.; Petrov, M.; Khlaaf, H.; Sastry, G.; Mishkin, P.; C...

Pith/arXiv arXiv 2013

[12] [2024]

Liu, T.; Xu, C.; and McAuley, J

Swe-bench: Can lan- guage models resolve real-world github issues? InInter- national Conference on Learning Representations, volume 2024, 54107–54157. Liu, T.; Xu, C.; and McAuley, J

2024

[13] [2025]

arXiv:2507.08616

AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs. arXiv:2507.08616. Guan, T

Pith/arXiv arXiv

[14] [2026]

https://www.alibabacloud.com/help/en/ model-studio/model-pricing

Alibaba Cloud Model Studio: Model Inference Pricing. https://www.alibabacloud.com/help/en/ model-studio/model-pricing. Accessed: 2026-07-17. Anthropic. 2026a. Claude API Pricing. https://platform. claude.com/docs/en/about-claude/pricing. Accessed: 2026- 07-20. Anthropic. 2026b. Introducing Claude Opus 4.8. https: //www.anthropic.com/news/claude-opus-4-8. ...

2026

An Empirical Study of Coordination Mode as the First-Class Citizen in From-Scratch Multi-Agent Coding

The reading

Core claim §

What carries the argument

Load-bearing premise §

What would settle it §

If this is right

Where Pith is reading between the lines

discussion (0)

Core claim

Load-bearing premise

What would settle it