Pith. sign in

REVIEW 3 major objections 5 minor 23 references

How much correctness costs in a weak multi-agent swarm is decided by where oracles sit and by the curvature of the cost-quality law.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 16:09 UTC pith:Q4TDN2RJ

load-bearing objection Clean theory package on budgeted corrector placement with a real curvature dichotomy; the deployment reading is only partially bridged to actual LLM swarms. the 3 major comments →

arxiv 2607.09765 v1 pith:Q4TDN2RJ submitted 2026-07-07 cs.AI cs.CLcs.MAcs.SYeess.SYmath.OC

How Much Does Correctness Cost? Budgeted Placement of Strong Correctors in a Weak Multi-Agent Swarm

Igor Itkin This is my paper
classification cs.AI cs.CLcs.MAcs.SYeess.SYmath.OC
keywords multi-agent systemsoracle placementsubmodular maximizationgrounded Laplacianbudget-correctness frontiercost-quality lawconsensus dynamicsLLM swarms
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

A swarm of cheap unreliable agents can still reach a correct consensus if a few expensive oracles pin selected nodes toward the truth. The paper models that loop as a grounded consensus on a graph, measures tracking quality by the coherence H(R) = tr M(R)^{-1}, and couples each oracle's pin strength to its cost through a quality law w(c). It proves that the error reduction stays monotone submodular even when the oracles have different strengths, so a simple cost-benefit greedy placement is within a 1-1/e factor of optimal at any budget and yields the least spend B*(eps) that certifies an eps-correct consensus. On the complete graph that frontier is closed-form, and a single curvature test on w decides whether the same budget is better spent on many medium oracles or on a few strong ones. Measurements on model ladders show the law is concave for factual and math verification (spread wins) but convex for emergent code tracing (concentrate wins), so the practical verdict is task-dependent.

Core claim

The coherence objective remains monotone submodular under heterogeneous positive pin strengths, so cost-benefit greedy is a (1-1/e) approximation for budgeted oracle placement; on the complete graph the same budget optimally spreads under a concave cost-quality law and concentrates only under a sharply convex law whose curvature exceeds a precise threshold, and real LLM ladders realize both regimes depending on the task.

What carries the argument

The grounded operator M(R) = L + kappa I + sum of diagonal pin strengths, whose coherence H(R) = tr M(R)^{-1} is the quality measure; its M-matrix structure keeps the resolvent entrywise nonnegative and thereby preserves submodularity of the error reduction even for unequal wi.

Load-bearing premise

That a model's measured accuracy on a two-choice verification task is a valid stand-in for the abstract pin strength that appears in the linear consensus equations, and that those linear dynamics adequately describe how multi-agent LLM swarms actually correct each other.

What would settle it

Measure cost-quality curves and run real multi-agent swarms on the same tasks: if a clearly concave law still produces better average correctness by concentrating on one large model, or if a convex law still prefers many medium correctors, the curvature dichotomy fails to transfer.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies budgeted placement of strong, cost-coupled oracle correctors in a weak multi-agent consensus swarm. Agents evolve under a grounded Laplacian; each oracle at node i adds a diagonal pin of strength wi = w(ci). Quality is the coherence H(R) = tr M(R)^{-1}. Theorem 1 shows that the error reduction ρ(R) = H(∅) − H(R) remains monotone submodular for heterogeneous wi > 0 via an M-matrix / entrywise-nonnegative-inverse argument, so cost-benefit greedy attains a (1−1/e) guarantee (Theorem 2) and yields a budget–correctness frontier B⋆(ε). On the complete graph, Proposition 1 gives a closed form for H, and Theorem 3 shows that the fixed-budget optimum is decided by the sign of (N+κ+w)w'' − 3(w')²: concave w spreads, sufficiently convex w concentrates. Empirically, Qwen3 (and Gemma-4) cost–quality laws are concave for factual and math 2-AFC verification but convex for emergent code tracing. Stage B treats a nonlinear majority cascade, giving influence-balance thresholds (exact on KN, rigorous on dense ER) and small real-LLM swarm checks.

Significance. If the results hold, the paper supplies a missing dynamical foundation for budgeted multi-agent LLM assembly: a consensus-correctness object with a cost-coupled quality law, a submodular knapsack guarantee under heterogeneous pins, a closed-form frontier on KN, and a single scalar (curvature of w) that decides few-strong versus many-medium oracles. Strengths include carefully scoped proofs (M-matrix argument; pairwise-balancing for equal spread), recovery of the homogeneous companion as Corollary 1, released code and per-item scores, bootstrap CIs, a second-family replication, and an honest Limitations section. The Stage-B influence balance and fallible-corrector extension (Proposition 2, Corollary 2) usefully connect average-error and high-probability views. The task-dependent empirical curvature finding is practically relevant if the accuracy-to-pin proxy transfers.

major comments (3)
  1. [Section 7, Theorem 3, Abstract] Section 7 explicitly states that 2-AFC log-probability accuracy is only a proxy for abstract pin strength wi and is “not otherwise validated,” yet the abstract and Remark 3 present the task-dependent spread/concentrate verdict as a deployment conclusion of Theorem 3. Limitations claim the concave sign survives identity, logit, and tested monotone reparametrizations because saturation is per-item, but this is not elevated to a formal statement with conditions under which sign((N+κ+w)w''−3(w')²) is preserved. Either (i) add a short proposition quantifying curvature-sign preservation under monotone maps of measured accuracy to wi, with the adversarial maps tested, or (ii) soften the abstract/conclusion so the empirical half is clearly a measurement of a proxy scalar, not a validated transfer into M(R). Without one of these, the headline “verdict is genuinely task-dependent” overreaches the
  2. [Section 3, Eq. (2); Theorem 3; Appendix D] The optimization object throughout Theorems 1–3 is H = tr M^{-1}, while the motivating tracking error under isotropic fault covariance is σ² tr M^{-2} (Section 3). Submodularity and the Sherman–Morrison gain (3) are proved for H; the complete-graph curvature dichotomy (Theorem 3, Appendix D) uses the closed form (6) specific to tr M^{-1}. Remark 6 notes both functionals are minimized by leverage placement near consensus, but that does not automatically transfer the equal-spread / single-oracle dichotomy or the factor 3 in (7) to tr M^{-2}. Please either (a) state that the paper’s object is deliberately the standard coherence criterion and that the MSE dichotomy is left open, or (b) sketch whether the pairwise-balancing argument extends to tr M^{-2} on KN (even a numerical check on the equal-split vs single-oracle comparison under concave/convex w would help).
  3. [Section 8, Proposition 2, Corollary 2, Figs. 6–7] The real-swarm confirmations that partially close the proxy gap (Section 8, Figs. 6–7) are small and narrow: N=25, one BA graph, one hardness band, 30–40 trials, and for real reasoning correctors the leverage curve reaches parity rather than certainty. Proposition 2 / Corollary 2 give a clean composition condition (reliability-weighted degree balance plus no single fallible hub dominating), but the experiments do not check the dominance condition d_free_i ≤ μ_{-i} under the measured medium-reliability correctors on hubs. Given that Corollary 2 is what makes “many medium on high-degree nodes” composition-safe, a brief check or explicit caveat that the N=25 runs do not verify the safety margin would keep Stage A/B composition claims proportionate.
minor comments (5)
  1. [Abstract] The abstract has several garbled phrases (“manye curvature”, “sharply increasion”, “math verificatio convex foremergent”, truncated GitHub URL). These should be cleaned to match the body abstract.
  2. [Assumption 1; Theorem 3] Assumption 1 forces w concave, while Theorem 3 and the code-tracing experiment treat sharply convex w. A one-sentence note that Assumption 1 is the diminishing-returns case and that Theorem 3 also covers the opposite curvature regime would avoid a framing inconsistency.
  3. [Figures 1–2, 6–8] Figure captions are dense; labeling panels (a)/(b) more explicitly in the text when first cited (Figs. 1–2, 6–8) would help navigation.
  4. [Section 2] Related work on model cascades and FrugalGPT [18] is cited; a brief contrast with cascade escalation (confidence-triggered single strong model) versus simultaneous multi-oracle placement would clarify the operational difference.
  5. [Appendix H; Section 4] Appendix H correctly notes that submodularity does not transfer to directed M; flagging this earlier (e.g., end of Section 4 or Limitations) would help readers who jump to directed swarms.

Circularity Check

0 steps flagged

No significant circularity: submodularity, budgeted greedy, and curvature dichotomy are parameter-free mathematical derivations; empirical w(c) is an independent measurement, not a fitted input renamed as prediction.

full rationale

The load-bearing claims are Theorems 1–3 and the complete-graph frontier. Theorem 1 proves monotone submodularity of ρ(R)=H(∅)−H(R) for any wi>0 via the M-matrix / entrywise-nonnegative inverse argument and Sherman–Morrison marginal gains (Appendix A); it does not define H in terms of the conclusion. Theorem 2 is the standard single-knapsack (1−1/e) guarantee applied to that submodular ρ (Appendix B). Proposition 1 and Theorem 3 invert the closed-form H on KN and decide spread vs concentrate from the sign of (N+κ+w)w''−3(w')² under Assumption 1 (Appendices C–D); the curvature criterion is derived, not fitted. Corollary 1 recovers the homogeneous unit-cost placement of companion [1] as the special case ci≡1, wi≡w—a legitimate limit, not a self-definitional premise. The Qwen3/Gemma cost–quality measurements (Section 7) are independent empirical inputs to which Theorem 3 is applied; they are not parameters fitted to force the placement theorems. Stage-B thresholds (Propositions 2–5, Theorem 4) are separate combinatorial/cavity arguments. Self-citation of [1] is present but not load-bearing for any central derivation. No self-definitional loop, fitted-input-as-prediction, uniqueness import, or ansatz smuggling is present.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The central theory rests on standard grounded-Laplacian consensus and M-matrix facts, plus one modeling law (cost-coupled pin strength) and the choice of coherence as the correctness metric. Empirical claims add fitted saturation scales and the unvalidated-then-partially-checked accuracy-to-pin proxy. No new physical entities; the 'oracle pin' is a modeling abstraction with partial real-swarm support.

free parameters (3)
  • anchor strength κ
    Positive scalar that keeps the uncorrected operator M0 positive definite; calibrated in simulations and the budget-correctness sweep but not derived from first principles.
  • saturation scale c0 in w(c)=w̄(1-e^{-c/c0})
    Fitted per difficulty band (0.20→0.25→0.31B) and used to describe the measured cost-quality law; location of the plateau is task-dependent.
  • reliability pr and seed/oracle fractions in Stage B
    Simulation and threshold statements depend on chosen reliability and planted falsehood counts; real-swarm runs fix specific N, f, and model sizes.
axioms (5)
  • domain assumption Agents run linear consensus on a connected graph with grounded Laplacian M(R)=L+κI+diag(wi 1[i∈R]); correctness is H(R)=tr M(R)^{-1} ≤ ε.
    Section 3 model; Stage A is defined as this linear system. Remark 6 notes it is the near-consensus linearization of the majority cascade.
  • domain assumption Assumption 1: pin strength wi=w(ci) for a strictly increasing concave w (diminishing-returns cost-quality law).
    Section 3; needed for the spread-vs-concentrate dichotomy (Theorem 3). Submodularity (Theorem 1) needs only wi>0.
  • standard math Symmetric M-matrices have entrywise nonnegative inverses, which forces nonnegative mixed second derivatives of H and thus submodularity.
    Appendix A proof of Theorem 1; standard M-matrix fact.
  • standard math Monotone submodular maximization under a single knapsack admits a (1-1/e) cost-benefit greedy guarantee (Sviridenko).
    Theorem 2 cites [5]; used off-the-shelf once Theorem 1 is established.
  • ad hoc to paper Measured 2-AFC log-probability of the gold choice is a usable proxy for abstract pin strength wi.
    Section 7 explicitly calls this a proxy 'we do not otherwise validate'; later partially closed by fallible-corrector swarm checks (Section 8, Fig. 7).
invented entities (2)
  • Cost-coupled oracle pin wi=w(ci) on a consensus node independent evidence
    purpose: Turns classical identical-leader selection into a budgeted knapsack with a few-strong vs many-medium tradeoff.
    Modeling abstraction introduced in Section 3; not a new physical particle, but the paper's central modeling invention. Independent evidence is partial via real LLM corrector swarms.
  • Budget-correctness frontier B*(ε) no independent evidence
    purpose: Least spend guaranteeing ε-correct consensus; headline object of Problem 2.
    Defined from the model; closed-form on KN, greedy elsewhere. Not an external entity but a derived object.

pith-pipeline@v1.1.0-grok45 · 29270 in / 3813 out tokens · 46710 ms · 2026-07-14T16:09:51.879586+00:00 · methodology

0 comments
read the original abstract

A cheap swarm of unreliable agents can be steered to a correct consensus by a few strong, expensive "oracle" correctors. We ask how much one must spend, and where to place the oracles. We model the swarm as a consensus on a graph in which each oracle pins one node toward the truth at a cost-coupled, concave strength, and measure quality by the coherence H(R)=tr M(R)^{-1}. Our first result is that H stays submodular (each added oracle helps less than the last) even when the oracles differ in strength, so a cost-benefit greedy comes within 1-1/e of the best placement at any budget. Inverting the budget gives the budget-correctness frontier B*(eps), the least spend that guarantees an eps-correct consensus: closed-form on the complete graph, and a minimal oracle count k* when oracles cost the same. Whether a budget then buys a few strong oracles or many medium onese curvature of the cost-quality law: diminishing returns favour spreadsharply increasion. Measured onthe Qwen3 ladder (0.6-32B), the law is concave for math verificatio convex foremergent code tracing, so the verdict is genuinely task-dependent.https://github.com/YehudaItkin/budgeted-oracle-placemen

Figures

Figures reproduced from arXiv: 2607.09765 by Igor Itkin.

Figure 1
Figure 1. Figure 1: Stage A on a random graph. (a) at a fixed budget the truth-tracking error H/H0 is minimized by spreading over many medium correctors when w is concave (the regime we measure for Qwen3 in Section 7) and by concentrating on a few strong ones when w is sharply convex (Eq. (7); Theorem 3). (b) cost-benefit greedy placement reaches a lower error per unit budget than degree-based or random placement (Theorem 2).… view at source ↗
Figure 2
Figure 2. Figure 2: The budget–correctness frontier. (a) greedy placement reaches a target error ε at a smaller budget than random, so B⋆ greedy < B⋆ rand. (b) for the coherence objective the minimal budget k ⋆ is a constant fraction of N (here ≈0.3 at mean degree d=6), well above the worst-case λmin scaling ∼1/d. language-inference (NLI) contradiction between the gold answer and its hallucinated twin and rerun the 0.6–14B la… view at source ↗
Figure 3
Figure 3. Figure 3: The saturation scale grows as the task gets harder. The same Qwen3 ladder (0.6–14B, logprob 2-AFC, n=200) on three disjoint PsiloQA difficulty bands, graded by the gold– hallucination NLI contradiction. Easy and hard distractors are flat between 1.7 and 4B; the subtlest band is still rising there (paired-bootstrap slope CI excludes zero; marginal under an independent bootstrap). The fitted scale c0 in w = … view at source ↗
Figure 4
Figure 4. Figure 4: The measured cost–quality law is concave (Qwen3, 0.6–32B, n=200). Each model chooses, from the evidence, between the gold answer and a plausible hallucination (PsiloQA); w is the model’s probability of the gold choice (scored from token log-probabilities), position￾averaged over both answer orders (solid: continuous, 95% bootstrap CI; dashed: robust pick in both orders). Strength rises steeply to ∼1.7B the… view at source ↗
Figure 5
Figure 5. Figure 5: The concave law replicates across families. The Qwen3 (0.6–32B) and Gemma-4 (E2B/E4B/12/31B) ladders on the same PsiloQA hard band, same 2-AFC logprob protocol (n=200). Both rise then plateau near ∼0.95; the newer Gemma-4 saturates earlier. Error bars are 95% bootstrap CIs; x-axis is nominal size (log). a content-free opinion-consensus swarm of real LLM agents (Qwen3-1.7B, N=25 on a heavy-tailed Barab´asi–… view at source ↗
Figure 6
Figure 6. Figure 6: Leverage placement in a real LLM swarm. P(truth wins) versus corrector count k for high-degree (leverage) and random placement in a content-free opinion-consensus swarm (Qwen3-1.7B, N=25, f=6 false seeds, 40 trials; Wilson 95% intervals). Leverage crosses P=1/2 at k ⋆≈1.7 versus ≈7 for random, matching the predicted ¯d/dmax. proxy under leverage, so we claim only that real reasoning correctors reproduce th… view at source ↗
Figure 7
Figure 7. Figure 7: Two mechanism checks in a real LLM swarm. (a) With fallible correctors, P(truth wins) is linear in the effective strength 2q −1 (q=reliability, r=0.98), so measured reliability is the effective pin weight of (8). (b) The leverage−random placement gap vanishes as agents become more grounded (content-free → partial → full evidence), confirming that placement matters only in the majority-copy regime. Swarms o… view at source ↗
Figure 8
Figure 8. Figure 8: The budget–correctness phase transition (Stage B). Majority-cascade model: f false-pinned seeds versus k truth-pinned oracles on an Erd˝os–R´enyi swarm. (a) P(truth wins) jumps sharply with the oracle count, and leverage placement reaches certainty at roughly half the oracles random placement needs (k ⋆ ≈ f ¯d/dmax versus f). (b) against the oracle fraction k/N the curves steepen with N at an N-independent… view at source ↗
Figure 9
Figure 9. Figure 9: Coherence of a directed swarm as edges turn one-way (reciprocity [PITH_FULL_IMAGE:figures/full_fig_p029_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 10 linked inside Pith

  1. [1]

    I. Itkin. Delayed verification destabilizes multi-agent LLM belief: instability thresholds and optimal corrector placement. arXiv:2606.27409 (2026)

    Pith/arXiv arXiv 2026

  2. [2]

    I. Neri, F. L. Metz. Spectra of sparse non-Hermitian random matrices: an analytical solution. Phys. Rev. Lett.109, 030602 (2012). arXiv:1205.0702

    Pith/arXiv arXiv 2012

  3. [3]

    E. Kim, A. Garg, K. Peng, N. Garg. Correlated errors in large language models.ICML(2025). arXiv:2506.07962

    Pith/arXiv arXiv 2025

  4. [4]

    M. Yuan, K. Pahwa, S. Chang, et al. Automated composition of agents: a knapsack approach for agentic component selection.NeurIPS(2025). arXiv:2510.16499

    arXiv 2025

  5. [5]

    Sviridenko

    M. Sviridenko. A note on maximizing a submodular set function subject to a knapsack constraint. Oper. Res. Lett.32(1):41–43 (2004)

    2004

  6. [6]

    G. L. Nemhauser, L. A. Wolsey, M. L. Fisher. An analysis of approximations for maximizing submodular set functions—I.Math. Program.14:265–294 (1978)

    1978

  7. [7]

    Leskovec, A

    J. Leskovec, A. Krause, C. Guestrin, et al. Cost-effective outbreak detection in networks.KDD (2007). 18

    2007

  8. [8]

    Clark, B

    A. Clark, B. Alomair, L. Bushnell, R. Poovendran.Submodularity in Dynamics and Control of Networked Systems.Springer (2016)

    2016

  9. [9]

    Mackin, S

    E. Mackin, S. Patterson. Submodular optimization for consensus networks with noise-corrupted leaders. arXiv:1712.08212 (2017)

    Pith/arXiv arXiv 2017

  10. [10]

    Bamieh, M

    B. Bamieh, M. R. Jovanovi´ c, P. Mitra, S. Patterson. Coherence in large-scale networks: dimension-dependent limitations of local feedback.IEEE Trans. Autom. Control57(9):2235–2249 (2012)

    2012

  11. [11]

    Pirani, S

    M. Pirani, S. Sundaram. On the smallest eigenvalue of grounded Laplacian matrices.IEEE TAC61(2):509–514 (2016). arXiv:1406.2271

    Pith/arXiv arXiv 2016

  12. [12]

    X. Zhou, R. Wang, W. Li, Z. Zhang. Maximizing the smallest eigenvalue of the grounded Laplacian matrix.J. Glob. Optim.(2025). arXiv:2110.12576

    Pith/arXiv arXiv 2025

  13. [13]

    B. Liu, L. Kong, J. Pei. Phase transition for budgeted multi-agent synergy. arXiv:2601.17311 (2026)

    arXiv 2026

  14. [14]

    F. Lin, M. Fardad, M. R. Jovanovi´ c. Algorithms for leader selection in stochastically forced consensus networks.IEEE TAC59(7):1789–1802 (2014). arXiv:1302.0450

    Pith/arXiv arXiv 2014

  15. [15]

    Krause, A

    A. Krause, A. Singh, C. Guestrin. Near-optimal sensor placements in Gaussian processes.JMLR 9:235–284 (2008)

    2008

  16. [16]

    Abbas, A

    W. Abbas, A. Laszka, X. Koutsoukos. Improving network connectivity and robustness using trusted nodes with application to resilient consensus.IEEE TCNS5(4):2036–2048 (2018)

    2036

  17. [17]

    H. J. LeBlanc, H. Zhang, X. Koutsoukos, S. Sundaram. Resilient asymptotic consensus in robust networks.IEEE JSAC31(4):766–781 (2013)

    2013

  18. [18]

    L. Chen, M. Zaharia, J. Zou. FrugalGPT: how to use large language models while reducing cost and improving performance. arXiv:2305.05176 (2023)

    Pith/arXiv arXiv 2023

  19. [19]

    Rykov, et al

    E. Rykov, et al. When models lie, we learn: multilingual span-level hallucination detection with PsiloQA.Findings of EMNLP(2025). arXiv:2510.04849. Dataset: https://huggingface. co/datasets/s-nlp/PsiloQA

    arXiv 2025

  20. [20]

    Yang, et al

    A. Yang, et al. (Qwen Team). Qwen3 technical report. arXiv:2505.09388 (2025)

    Pith/arXiv arXiv 2025

  21. [21]

    Gemma Team, Google DeepMind. Gemma 4. https://huggingface.co/google/gemma-4-1 2B-it(2026)

    2026

  22. [22]

    M. A. Di Muro, S. V. Buldyrev, L. A. Braunstein. Reversible bootstrap percolation: fake news and fact checking.Phys. Rev. E101, 042307 (2020). arXiv:1910.09516

    Pith/arXiv arXiv 2020

  23. [23]

    Balogh, B

    J. Balogh, B. G. Pittel. Bootstrap percolation on the random regular graph.Random Struct. Alg.30(1–2):257–286 (2007). 19 Appendix A Proof of Theorem 1 (submodularity under heteroge- neous pins) Throughout, fixκ >0 and write M(R) =L+κI+ X i∈R wi eie⊤ i , H(R) = trM(R) −1, where ei is the i-th standard basis vector, so that eie⊤ i has a single nonzero entry...

    2007