Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives

Elliot Meyerson; Xin Qiu

Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives

T0 review · 0 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Counting LLM forward passes as the atomic cost unit exposes asymptotic separations in multi-agent systems that role-based design misses.

desk verdict A clear, honest position paper on LLM-agent complexity analysis, but the 'required' claim is stronger than the evidence and the headline gains depend on free, exact decompositions that the paper itself leaves open. read the letter →

arxiv 2502.04358 v2 pith:MIMG7R6Q submitted 2025-02-04 cs.CL cs.AIcs.CCcs.LGcs.NE

Elliot Meyerson , Xin Qiu This is my paper · ORCID

classification cs.CLcs.AIcs.CCcs.LGcs.NE

keywords LLMagentsasymptoticanalysisprimitivesoracledecompositionmulti-agentorchestrationproblemevolutionaryoptimizationagentscaling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's position is that scaling LLM-based agentic systems to large, many-task workloads will require asymptotic analysis in which the LLM forward pass is the atomic unit of cost. It introduces a minimal framework built on language-based algorithms, capabilities sets, and oracle decompositions, then uses three worked examples to argue that this analysis exposes efficiency separations that intuitive, human-team-inspired designs miss. If the position holds, multi-agent system builders should invest in complexity analysis of orchestration rather than treating role assignment as the main design decision, and the largest gains should come from decomposing problems into independently solvable, precisely specified pieces.

What carries the argument

The central object is the LLM primitive, a single forward pass of a model of size $m$ over an input of length $n$, counted as cost $mn$; each model also carries a capabilities set describing what it can reliably do. The frameworks around it are the language-based algorithm, an algorithm whose steps include LLM calls, and the oracle decomposition, an assumed zero-cost, non-LLM operation that splits a problem in a specified way. The argument's engine is the asymptotic comparison of orchestration schemes that differ only in how they route, focus, and decompose LLM calls; the oracle decomposition is what lets a proposed decomposition be turned into a calculable separation.

What would settle it

Run the evolutionary example with real LLMs on a family of block-fitness functions where each of $k$ blocks is either correct or incorrect, count forward passes for whole-string mutation versus per-block mutation with known boundaries, and check whether the measured ratio of total calls grows roughly as $2^k k$ as $k$ increases; if it does not, the paper's central separation fails to appear in practice.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that asymptotic analysis with LLM primitives (AALPs) is critical to scaling LLM agents, and the evidence is a set of derived cost separations. An LLM primitive costs $mn$ for a model of size $m$ applied to a string of length $n$, and counting these primitives isolates orchestration efficiency from any particular model's internals. In the three examples, the optimized scheme beats the intuitive one by $\Theta(k)$ for $k$-task routing, by $\Theta(bk^2)$ for debugging a $k$-function codebase with $b$ bugs, and by $\Theta(2^k k)$ for evolutionary optimization on a block-structured fitness landscape. The largest separation is presented as evidence that intuitive designs can be extraordinarily far from optimal and as motivation to develop AALPs into a practical engineering tool.

Load-bearing premise

The load-bearing premise is that a problem can be split, at no cost, into independent, precisely specified subproblems—as when debugging assumes a completely correct constant-size specification for every function and evolution assumes programmatic block boundaries—so if that decomposition is unreliable or itself needs LLM calls, the promised speed-ups collapse.

Editorial extensions

If this is right

Routing increases the value of specialization linearly in the number of tasks: a delegator plus $k$ specialists costs $\Theta(m_s n)$ versus $\Theta(k m_s n)$ for a generalist of the same per-task size, so large many-task systems should be built from many small specialists rather than one large model.
For iterative debugging, the analysis says focusing each QA and fixing call on a single function, rather than on the whole accumulating code, yields a speed-up of $\Theta(bk^2)$ in the dominant term, making extreme decomposition the target design for scaling coding agents.
For evolutionary optimization, local block-wise mutation has expected LLM-primitive cost $\Theta(m_u k l^2)$ versus $\Theta(m_u 2^k k^2 l^2)$ for whole-string mutation, a separation of $\Theta(2^k k)$ that makes block-structured solution representations a primary design choice.
Because the delegator and QA costs are asymptotically small relative to specialists and debuggers, research and development effort on model-size reduction should be concentrated on the agents that are executed most and process the largest inputs.
AALPs lets designers compare orchestration schemes before the required capabilities exist, so insights about decomposition can guide development of future LLMs rather than only analyzing current ones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One consequence the paper leaves implicit: the oracle-decomposition assumption in Sections 3.2 and 3.3 means all separations are upper bounds on what a free decomposition can buy; counting the LLM calls spent to find and verify decompositions would shrink the gap, so a natural next step is to derive closed-form costs for LLM-based decomposers.
The same accounting extends by analogy to non-text modalities: a compact vision-only model is the asymptotically right engine for high-volume visual subtasks, with multi-modal foundation models reserved for cases that genuinely need them, a direction the paper touches on but does not quantify.
A quantitative falsification target follows from the evolutionary example: if per-block mutation's success probability decays with $k$, the $\Theta(2^k k)$ separation in expected calls may not hold for real LLMs; an empirical measurement of per-block success as a function of $k$ would test the position's most extreme claim.
The debugging example's assumption of perfect constant-size specifications could be relaxed by letting the specification itself be generated by an LLM; the position would be more robust if the separation persisted after subtracting the cost of generating and checking specifications.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: This is a clear, honest position paper, but its central claim—that AALPs is required—is stronger than the evidence supports. The framework itself is not new: the authors cite Chen et al. (2024) as the first formal treatment. What this paper adds is a plain-language argument, three simple worked examples, and a research agenda.

The examples are simple arithmetic, but they serve a real purpose. They show how counting LLM forward passes as the cost unit separates orchestration efficiency from model internals, and the Θ(2^k k) separation in the evolutionary example is genuinely striking. The paper is also unusually candid. It flags its own assumptions—the oracle decomposition, linear cost, no stochastic errors—and lists them as open directions in Section 5. That honesty is valuable and makes the paper a good discussion piece.

The soft spot is exactly where the stress-test note lands. The headline separations in Sections 3.2 and 3.3 depend on free, exact decompositions: perfect function specifications and programmatic block boundaries. Section 5.4 acknowledges that automatic decomposition will likely require LLM calls whose cost must be counted, but the paper never re-runs its examples with that cost included. If decomposition is imperfect or itself costs Θ(mn) per scan, the Θ(bk²) and Θ(2^k k) improvements shrink or vanish. For a position paper, flagging these assumptions is acceptable, but it means 'required' is not supported. The argument would be on much firmer ground if framed as 'promising and worth developing.'

The capability-scaling rule in Section 3.1 (mg = Θ(k·ms)) is also just assumed, not argued from data. That is fine as an illustration, but it should be labeled as an assumption. The linear input-cost model is similarly approximate, and the authors do note this.

Who is this for? Researchers designing multi-agent LLM systems and anyone thinking about a complexity theory for LLM-based algorithms. It deserves a serious referee—I would send it out. The main revision I would ask for is to soften the central claim and add a short discussion of how the examples change when decomposition costs are included. With that change, the paper is a solid call to action.

Referee Report

0 major / 7 minor

Summary. This position paper argues that scaling LLM-based agentic systems requires asymptotic analysis that treats a single LLM forward pass as the atomic unit of cost, an abstraction the authors call the LLM primitive. The paper defines a minimal framework (LLM primitives, capabilities sets, oracle decomposition), then presents three worked examples—k-task routing, iterative code debugging, and evolutionary optimization—in which a carefully decomposed agent design is compared with a more intuitive, role-based design. The claimed gains are Θ(k), Θ(bk²), and Θ(2^k·k) respectively. The paper also discusses alternative views (e.g., that people would do this anyway, or that direct cost optimization suffices) and lists open research directions, including automatic decomposition, stochasticity, error correction, and parallelization.

Significance. If the position is accepted, the paper supplies a useful and much-needed notation for reasoning about the orchestration-level efficiency of LLM-agent systems, independent of any particular model implementation. The worked examples are transparent, the assumptions are mostly stated explicitly, and the arithmetic is straightforward to verify; this makes the paper a good starting point for a formal theory of LLM-agent algorithms. The paper is appropriately modest in presenting itself as a call to action rather than a complete framework. Its main limitation—that the headline separations depend on oracle decompositions and on reliability assumptions—is real, but it is explicitly acknowledged in Sections 5.1, 5.2, and 5.4, and for a position paper this is an acceptable research-direction rather than a fatal flaw.

minor comments (7)

[Definition 2.2] The definition says a single application of M to a string of length n costs mn, but the examples in Section 3 count cost as m·n² when the model produces n output tokens (e.g., Equations (5), (7), and (12)). Please clarify that the primitive is a single forward pass per output token, with cost m times the current context length, or define a separate cost rule for generation, so that the definitions match the example derivations.
[Table 1 / Section 3.3.3] The improvement values in Table 1, especially Θ(2^k·k), are conditional on the oracle-decomposition assumption and on zero cost for decomposition, but Table 1 presents them without qualification. Please add an explicit footnote or phrase in the table caption and in the conclusion noting that these gains assume the oracle-decomposition setting described in Sections 2 and 3, and that LLM-based decomposition costs are not yet incorporated into the examples.
[Section 3.1.3, Eq. (3)] The assumption that 'the required size of the LLM scales linearly with the number of required capabilities' is presented in the text as following from neural scaling laws, but this is a modeling choice rather than a standard scaling-law statement. Please mark it explicitly as an assumption at the point of use, in addition to the later mention in Section 5.1.
[Section 3.2.1, Eq. (5)] The notation m_d(in + (in + 1) + ... + (in + n)) is ambiguous; it should be clear that the cost of generating the n output tokens accumulates the growing context length at each output-token step, and the line 'm_d(in + (in + 1) + ... + (in + n)) = Θ(m_d·i·n²)' would benefit from an explicit intermediate expression.
[Section 3.3.1] The statement that the expected number of applications of the global mutator is 2^k assumes that each of the k blocks is sampled independently with probability 1/2 of being correct on every application, and that the initial state does not affect the expected hitting time. Please state this independence assumption explicitly, since the later local-mutation comparison depends on it.
[Section 5.7] The discussion of machine sentience and suffering is speculative and tangential to the technical position of the paper. Consider shortening it to a brief note or moving it to the impact statement, so that the main argument about asymptotic analysis remains focused.
[Section 4, Alternative View 2] There is a typo: 'aymptotic' should be 'asymptotic'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: asymptotic separations are conditional arithmetic from explicit oracle-decomposition assumptions, not fitted predictions or self-citation-dependent results.

full rationale

No significant circularity found. The paper's three worked examples are self-contained asymptotic arithmetic from explicitly stated cost and capability assumptions (Definition 2.2 and Definition 2.4). The Theta(k) routing improvement follows from the assumed linear scaling of model size with capability count; the Theta(bk^2) debugging improvement follows from the explicitly assumed completely correct, precise and constant size specification for every function; and the Theta(2^k k) evolution improvement is the ratio of the assumed global and local mutation costs when block boundaries are available programmatically. These are conditional derivations, not predictions fitted to data, and they do not redefine the target result as its own input. The paper repeatedly flags these assumptions in Sections 3.2.3, 3.3.3, 5.1, and 5.4, and it explicitly lists automatic decomposition with its associated LLM cost as an open research direction, so the idealization is transparent rather than hidden. Self-citations (e.g., Meyerson et al. 2024) appear only as contextual examples of LLM-based optimization and related work; they do not supply a uniqueness theorem, a fitted parameter, or any load-bearing premise for the derivations. The central position is an argument for a style of analysis, not an empirical claim whose support reduces to its own assumptions.

Assumptions & free parameters 0 free parameters · 6 assumptions · 2 invented entities

The central arguments rest on three modeling conventions: linear LLM cost, linear capability scaling, and free perfect decomposition. No parameters are fitted to data; the examples are analytic.

assumptions (6)

domain assumption LLM primitive cost is mn for a forward pass over an input of length n with model size m (Definition 2.2).
Underlies all three example cost derivations; justified by API pricing, but the paper notes superlinear costs would sharpen separations.
domain assumption Required model size grows linearly with the number of capabilities, so m_g = Θ(k m_s) (Section 3.1.3).
Based on scaling laws; converts the routing example into a Θ(k) speed-up; if the relationship is sublinear, the advantage shrinks.
ad hoc to paper Oracle decomposition is available: subproblems are independent and precisely specified, with zero decomposition cost (Definition 2.4, Sections 3.2.2 and 3.3.2).
The main idealized assumption; the predicted separations depend on it, and the paper flags it as a strong assumption.
domain assumption LLMs execute in-scope capabilities reliably (Sections 2 and 3; relaxed in Section 5.2).
The analysis ignores stochastic errors, which could change costs if failures require retries.
domain assumption The delegator can detect the task from a constant number of metadata tokens (Section 3.1.2).
Makes the delegator cost Θ(m_d) independent of input length; if not true, the routing conclusion changes.
ad hoc to paper The mutator has a 50% chance per block of producing a correct block (Section 3.3.1).
Chosen for convenience; gives expected 2^k global attempts; other probabilities shift constants but not the exponential separation.

invented entities (2)

LLM primitive
purpose: Atomic cost unit for asymptotic analysis
A modeling abstraction, not an empirical object; its cost rule mn is assumed.
Oracle decomposition
purpose: Assumed cost-free problem decomposition that enables the asymptotic separations
No evidence such decompositions exist generically; the paper itself calls this a strong assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives." pith.science (2026). https://pith.science/paper/MIMG7R6Q

@misc{pith2026250204358,
  author       = {Pith},
  title        = {Pith review of: Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MIMG7R6Q}},
  note         = {Machine review of arXiv:2502.04358}
}

read the original abstract

Decomposing hard problems into subproblems often makes them easier and more efficient to solve. With large language models (LLMs) crossing critical reliability thresholds for a growing slate of capabilities, there is an increasing effort to decompose systems into sets of LLM-based agents, each of whom can be delegated sub-tasks. However, this decomposition (even when automated) is often intuitive, e.g., based on how a human might assign roles to members of a human team. How close are these role decompositions to optimal? This position paper argues that asymptotic analysis with LLM primitives is needed to reason about the efficiency of such decomposed systems, and that insights from such analysis will unlock opportunities for scaling them. By treating the LLM forward pass as the atomic unit of computational cost, one can separate out the (often opaque) inner workings of a particular LLM from the inherent efficiency of how a set of LLMs are orchestrated to solve hard problems. In other words, if we want to scale the deployment of LLMs to the limit, instead of anthropomorphizing LLMs, asymptotic analysis with LLM primitives should be used to reason about and develop more powerful decompositions of large problems into LLM agents.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Imprompt: A Language Framework for Prompt Programming
cs.AI 2026-07 conditional novelty 7.0 of 10

Prompt programs can be typed like programs and compiled in different ways; Imprompt formalizes two such compilers and tests them on tax-law reasoning and data-labeling tasks.

Reference graph

Works this paper leans on

96 extracted references · 40 canonical work pages · cited by 1 Pith paper

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Aho, A. V. and Hopcroft, J. E. The design and analysis of computer algorithms. Pearson Education India, 1974

1974
[3]

Akl, S. G. The design and analysis of parallel algorithms. Prentice-Hall, Inc., 1989

1989
[4]

The intelligence age

Altman, S. The intelligence age. https://ia.samaltman.com/, September 2024. Accessed: 2025-05-28

2024
[5]

Anthropic API Pricing , 2025

Anthropic . Anthropic API Pricing , 2025. URL https://www.anthropic.com/pricing#anthropic-api. Accessed: 2025-01-30

2025
[6]

B., and Michalewicz, Z

B \"a ck, T., Fogel, D. B., and Michalewicz, Z. Handbook of evolutionary computation. Release, 97 0 (1): 0 B1, 1997

1997
[7]

Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds

Basmov, V., Goldberg, Y., and Tsarfaty, R. Simple linguistic inferences of large language models (llms): Blind spots and blinds. arXiv preprint arXiv:2305.14785, 2023

work page Pith review arXiv 2023
[8]

Baudet, G. M. The design and analysis of algorithms for asynchronous multiprocessors. Carnegie Mellon University, 1978

1978

Show all 96 references

[9]

M., Gebru, T., McMillan-Major, A., and Shmitchell, S

Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pp.\ 610--623, 2021

2021
[10]

Has ai progress really slowed down? Time, November 2024

Booth, H. Has ai progress really slowed down? Time, November 2024

2024
[11]

B., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J

Bradley, H., Dai, A., Teufel, H. B., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J. Quality-diversity through ai feedback. In The Twelfth International Conference on Learning Representations, 2024

2024
[12]

T., Li, Y., Lundberg, S., et al

Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023

2023 arXiv
[13]

Frugalgpt: How to use large language models while reducing cost and improving performance

Chen, L., Zaharia, M., and Zou, J. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176, 2023

2023 arXiv
[14]

On the design and analysis of llm-based algorithms

Chen, Y., Li, Y., Ding, B., and Zhou, J. On the design and analysis of llm-based algorithms. arXiv preprint arXiv:2407.14788, 2024

2024
[15]

Why computers won't make themselves smarter

Chiang, T. Why computers won't make themselves smarter. The New Yorker, 2021

2021
[16]

H., Leiserson, C

Cormen, T. H., Leiserson, C. E., Rivest, R. L., and Stein, C. Introduction to algorithms. MIT press, 2022

2022
[17]

Ai models' slowdown spells end of gold rush era

Cyran, R. Ai models' slowdown spells end of gold rush era. Reuters, December 2024

2024
[18]

Parameter-efficient fine-tuning of large-scale pre-trained language models

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5 0 (3): 0 220--235, 2023

2023
[19]

Probabilistic tools for the analysis of randomized optimization heuristics

Doerr, B. Probabilistic tools for the analysis of randomized optimization heuristics. Theory of evolutionary computation: Recent developments in discrete optimization, pp.\ 1--87, 2020

2020
[20]

and Neumann, F

Doerr, B. and Neumann, F. A survey on recent progress in the theory of evolutionary algorithms for discrete optimization. ACM Transactions on Evolutionary Learning and Optimization, 1 0 (4): 0 1--43, 2021

2021
[21]

On the analysis of the (1+ 1) evolutionary algorithm

Droste, S., Jansen, T., and Wegener, I. On the analysis of the (1+ 1) evolutionary algorithm. Theoretical Computer Science, 276 0 (1-2): 0 51--81, 2002

2002
[22]

B., and Mordatch, I

Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning, 2024

2024
[23]

L., Jiang, L., Lin, B

Dziri, N., Lu, X., Sclar, M., Li, X. L., Jiang, L., Lin, B. Y., Welleck, S., West, P., Bhagavatula, C., Le Bras, R., et al. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems, 36, 2024

2024
[24]

C., Sharma, P., Chen, F., and Jiang, L

Faiz, A., Kaneda, S., Wang, R., Osi, R. C., Sharma, P., Chen, F., and Jiang, L. Llmcarbon: Modeling the end-to-end carbon footprint of large language models. In The Twelfth International Conference on Learning Representations, 2024

2024
[25]

Graphrouter: A graph-based router for LLM selections

Feng, T., Shen, Y., and You, J. Graphrouter: A graph-based router for LLM selections. In The Thirteenth International Conference on Learning Representations, 2025

2025
[26]

and Weiss, G

Ferber, J. and Weiss, G. Multi-agent systems: an introduction to distributed artificial intelligence, volume 1. Addison-wesley Reading, 1999

1999
[27]

Anthropic chief: 'by next year, ai could be smarter than all humans'

Fortson, D. Anthropic chief: 'by next year, ai could be smarter than all humans'. The Times, March 2025. Accessed: 2025-05-28

2025
[28]

Multi-modal agent tuning: Building a vlm-driven agent for efficient tool usage

Gao, Z., Zhang, B., Li, P., Ma, X., Yuan, T., Fan, Y., Wu, Y., Jia, Y., Zhu, S.-C., and Li, Q. Multi-modal agent tuning: Building a vlm-driven agent for efficient tool usage. arXiv preprint arXiv:2412.15606, 2024

2024 arXiv
[29]

F., Thomas, S., Weinstein-Raun, B., and Brauner, J

Grace, K., Stewart, H., Sandk \"u hler, J. F., Thomas, S., Weinstein-Raun, B., and Brauner, J. Thousands of ai authors on the future of ai. arXiv preprint arXiv:2401.02843, 2024

2024
[30]

V., Wiest, O., and Zhang, X

Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N. V., Wiest, O., and Zhang, X. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680, 2024

2024 arXiv
[31]

Hamming, R. W. Error detecting and error correcting codes. The Bell system technical journal, 29 0 (2): 0 147--160, 1950

1950
[32]

Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608, 2024

2024 arXiv
[33]

Superintelligence strategy: Expert version

Hendrycks, D., Schmidt, E., and Wang, A. Superintelligence strategy: Expert version. arXiv preprint arXiv:2503.05628, 2025

2025 arXiv
[34]

AI and agents

Hodjat, B. AI and agents. AI Magazine , 45 0 (1): 0 1--3, 2024. doi:10.1002/aaai.12170. URL https://onlinelibrary.wiley.com/doi/10.1002/aaai.12170

2024 doi
[35]

J., and Amamiya, M

Hodjat, B., Savoie, C. J., and Amamiya, M. An adaptive agent oriented software architecture. In Proceedings of the 5th Pacific Rim International Conference on Artificial Intelligence (PRICAI '98), pp.\ 33--46, 1998

1998
[36]

D., and Finn, C

Hu, N., Mitchell, E., Manning, C. D., and Finn, C. Meta-learning online adaptation of language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 4418--4432, 2023

2023
[37]

J., Bieker, J., Li, X., Jiang, N., Keigwin, B., Ranganath, G., Keutzer, K., and Upadhyay, S

Hu, Q. J., Bieker, J., Li, X., Jiang, N., Keigwin, B., Ranganath, G., Keutzer, K., and Upadhyay, S. K. Routerbench: A benchmark for multi-llm routing system. arXiv preprint arXiv:2403.12031, 2024

2024 arXiv
[38]

Benchmarking large language models as ai research agents

Huang, Q., Vora, J., Liang, P., and Leskovec, J. Benchmarking large language models as ai research agents. In NeurIPS 2023 Foundation Models for Decision Making Workshop

2023
[39]

Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024 a

2024 arXiv
[40]

J., Taylor, C

Jiang, B., Xie, Y., Wang, X., Su, W. J., Taylor, C. J., and Mallick, T. Multi-modal and multi-agent systems meet rationality: A survey. In ICML 2024 Workshop on LLMs and Cognition, 2024 b

2024
[41]

B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

2001 arXiv
[42]

Text modular networks: Learning to decompose tasks in the language of existing models

Khot, T., Khashabi, D., Richardson, K., Clark, P., and Sabharwal, A. Text modular networks: Learning to decompose tasks in the language of existing models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hum...

2021
[43]

Knuth, D. E. The Art of Computer Programming: Fundamental Algorithms, Volume 1. Addison-Wesley Professional, 1968

1968
[44]

Handbuch der Lehre von der Verteilung der Primzahlen, volume 1

Landau, E. Handbuch der Lehre von der Verteilung der Primzahlen, volume 1. BG Teubner, 1909

1909
[45]

Evolving deeper llm thinking

Lee, K.-H., Fischer, I., Wu, Y.-H., Marwood, D., Baluja, S., Schuurmans, D., and Chen, X. Evolving deeper llm thinking. arXiv preprint arXiv:2501.09891, 2025

2025 arXiv
[46]

Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., and Stanley, K. O. Evolution through large models. In Handbook of Evolutionary Machine Learning, pp.\ 331--366. Springer, 2023

2023
[47]

O., and Ziyaee, T

Lehman, J., Meyerson, E., El-Gaaly, T., Stanley, K. O., and Ziyaee, T. Evolution and the knightian blindspot of machine learning. arXiv preprint arXiv:2501.13075, 2025

2025 arXiv
[48]

A survey of multimodel large language models

Liang, Z., Xu, Y., Hong, Y., Shang, P., Wang, Q., Fu, Q., and Liu, K. A survey of multimodel large language models. In Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering, pp.\ 405--409, 2024

2024
[49]

Lidar, D. A. and Brun, T. A. Quantum error correction. Cambridge university press, 2013

2013
[50]

Deepseek-v3 technical report

Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

2024 arXiv
[51]

T., Foerster, J., Clune, J., and Ha, D

Lu, C., Lu, C., Lange, R. T., Foerster, J., Clune, J., and Ha, D. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024

2024 arXiv
[52]

Keep the cost down: A review on methods to optimize llm’s kv-cache consumption

Luohe, S., Zhang, H., Yao, Y., Li, Z., et al. Keep the cost down: A review on methods to optimize llm’s kv-cache consumption. In First Conference on Language Modeling, 2024

2024
[53]

Artificial suffering: An argument for a global moratorium on synthetic phenomenology

Metzinger, T. Artificial suffering: An argument for a global moratorium on synthetic phenomenology. Journal of Artificial Intelligence and Consciousness, 8 0 (01): 0 43--66, 2021

2021
[54]

and Miikkulainen, R

Meyerson, E. and Miikkulainen, R. Modular universal reparameterization: Deep multi-task learning across diverse domains. In Advances in Neural Information Processing Systems, volume 32, 2019

2019
[55]

Simple genetic operators are universal approximators of probability distributions (and other advantages of expressive encodings)

Meyerson, E., Qiu, X., and Miikkulainen, R. Simple genetic operators are universal approximators of probability distributions (and other advantages of expressive encodings). In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '22, pp.\ 739–748, 2022

2022
[56]

J., Bradley, H., Gaier, A., Moradi, A., Hoover, A

Meyerson, E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., and Lehman, J. Language model crossover: Variation through few-shot prompting. ACM Transactions on Evolutionary Learning, 4 0 (4): 0 1--40, 2024

2024
[57]

and Upfal, E

Mitzenmacher, M. and Upfal, E. Probability and computing: Randomization and probabilistic techniques in algorithms and data analysis. Cambridge university press, 2017

2017
[58]

Quantum algorithms: an overview

Montanaro, A. Quantum algorithms: an overview. npj Quantum Information, 2 0 (1): 0 1--8, 2016

2016
[59]

and Raghavan, P

Motwani, R. and Raghavan, P. Randomized algorithms. ACM Computing Surveys (CSUR), 28 0 (1): 0 33--37, 1996

1996
[60]

Minions: Cost-efficient collaboration between on-device and cloud language models

Narayan, A., Biderman, D., Eyuboglu, S., May, A., Linderman, S., Zou, J., and Re, C. Minions: Cost-efficient collaboration between on-device and cloud language models. arXiv preprint arXiv:2502.15964, 2025

2025 arXiv
[61]

W., Teodorescu, L., Hayes, C

Nisioti, E., Glanois, C., Najarro, E., Dai, A., Meyerson, E., Pedersen, J. W., Teodorescu, L., Hayes, C. F., Sudhakaran, S., and Risi, S. From text to life: On the reciprocal relationship between artificial life and large language models. In Artificial Life Conference Proceedi...

2024
[62]

OpenAI API Pricing , 2025

OpenAI . OpenAI API Pricing , 2025. URL https://openai.com/api/pricing/. Accessed: 2025-01-30

2025
[63]

S., O'Brien, J., Cai, C

Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp.\ 1--22, 2023

2023
[64]

Introduction to the theory of error-correcting codes

Pless, V. Introduction to the theory of error-correcting codes. John Wiley & Sons, 2011

2011
[65]

Chatdev: Communicative agents for software development

Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\...

2024
[66]

W., and Titsias, M

Rannen-Triki, A., Bornschein, J., Pascanu, R., Hutter, M., Gy \"o rgy, A., Galashov, A., Teh, Y. W., and Titsias, M. K. Revisiting dynamic evaluation: Online adaptation for large language models. arXiv preprint arXiv:2403.01518, 2024

2024 arXiv
[67]

Quantum error correction: an introductory guide

Roffe, J. Quantum error correction: an introductory guide. Contemporary Physics, 60 0 (3): 0 226--245, 2019

2019
[68]

P., Dupont, E., Ruiz, F

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, 625 0 (7995): 0 468--475, 2024

2024
[69]

and Bradley, A

Saad, B. and Bradley, A. Digital suffering: Why it’sa problem and how to prevent it. Inquiry, pp.\ 1--36, 2022

2022
[70]

Design and analysis of distributed algorithms

Santoro, N. Design and analysis of distributed algorithms. John Wiley & Sons, 2006

2006
[71]

H., Jang, L., Tarr, M

Sarch, G. H., Jang, L., Tarr, M. J., Cohen, W. W., Marino, K., and Fragkiadaki, K. Vlm agents generate their own memories: Distilling experience into embodied programs of thought. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

2024
[72]

Beyond chinchilla-optimal: Accounting for inference in language model scaling laws

Sardana, N., Portes, J., Doubov, S., and Frankle, J. Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. In Forty-first International Conference on Machine Learning, 2024

2024
[73]

An introduction to the analysis of algorithms

Sedgewick, R. An introduction to the analysis of algorithms. Pearson Education India, 1996

1996
[74]

J., Casper, S., Tegmark, M., Saunders, W., Bau, D., Todd, E., Geiger, A., Geva, M., Hoogland, J., Murfet, D., and McGrath, T

Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., Biderman, S., Garriga-Alonso, A., Conmy, A., Nanda, N., Rumbelow, J., Wattenberg, M., Schoots, N., Miller, J., Michaud, E. J., Casper, S., Tegma...

2025 arXiv
[75]

Towards optimizing the costs of llm usage

Shekhar, S., Dubey, T., Mukherjee, K., Saxena, A., Tyagi, A., and Kotla, N. Towards optimizing the costs of llm usage. arXiv preprint arXiv:2402.01742, 2024

2024 arXiv
[76]

Adaptive in-conversation team building for language model agents

Song, L., Liu, J., Zhang, J., Zhang, S., Luo, A., Wang, S., Wu, Q., and Wang, C. Adaptive in-conversation team building for language model agents. arXiv preprint arXiv:2405.19425, 2024

2024 arXiv
[77]

G., Chen, L., Chiang, W.-L., and Davis, J

Stoica, I., Zaharia, M., Gonzalez, J., Goldberg, K., Zhang, H., Angelopoulos, A., Patil, S. G., Chen, L., Chiang, W.-L., and Davis, J. Q. Specifications: The missing link to making the development of llm systems an engineering discipline. arXiv preprint arXiv:2412.05299, 2024

2024 arXiv
[78]

and Belle, V

Sypherd, C. and Belle, V. Practical considerations for agentic llm systems. arXiv preprint arXiv:2412.04093, 2024

2024 arXiv
[79]

and Le, Q

Tan, M. and Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp.\ 6105--6114. PMLR, 2019

2019
[80]

Sparse sinkhorn attention

Tay, Y., Bahri, D., Yang, L., Metzler, D., and Juan, D.-C. Sparse sinkhorn attention. In International Conference on Machine Learning, pp.\ 9438--9447. PMLR, 2020

2020
[81]

M., Hauth, A., Millican, K., et al

Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

2023 arXiv
[82]

Position: Enforced amnesia as a way to mitigate the potential risk of silent suffering in the conscious ai

Tkachenko, Y. Position: Enforced amnesia as a way to mitigate the potential risk of silent suffering in the conscious ai. In Forty-first International Conference on Machine Learning, 2024

2024
[83]

On computable numbers, with an application to the entscheidungs problem

Turing, A. On computable numbers, with an application to the entscheidungs problem. Proceedings of the London Mathematical Society Series/2 (42), pp.\ 230--42, 1936

1936
[84]

V., Singh, A

Urlana, A., Kumar, C. V., Singh, A. K., Garlapati, B. M., Chalamala, S. R., and Mishra, R. Llms with industrial lens: Deciphering the challenges and prospects--a survey. arXiv preprint arXiv:2402.14558, 2024

2024 arXiv
[85]

and Wooldridge, M

Van der Hoek, W. and Wooldridge, M. Multi-agent systems. Foundations of Artificial Intelligence, 3: 0 887--928, 2008

2008
[86]

Attention is all you need

Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

2017
[87]

Voyager: An open-ended embodied agent with large language models

Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024 a . ISSN 2835-8856. URL https://openreview.net/forum?id=ehfRiF0R3a

2024
[88]

A survey on large language model based autonomous agents

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18 0 (6): 0 186345, 2024 b

2024
[89]

and Huckle, J

Williams, S. and Huckle, J. Easy problems that llms get wrong. arXiv preprint arXiv:2405.19616, 2024

2024 arXiv
[90]

Tight bounds on the optimization time of a randomized search heuristic on linear functions

Witt, C. Tight bounds on the optimization time of a randomized search heuristic on linear functions. Combinatorics, Probability and Computing, 22 0 (2): 0 294--318, 2013

2013
[91]

Autogen: Enabling next-gen llm applications via multi-agent conversation

Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. Autogen: Enabling next-gen llm applications via multi-agent conversation. In ICLR 2024 Workshop on Large Language Model (LLM) Agents

2024
[92]

Wu, X., Wu, S.-h., Wu, J., Feng, L., and Tan, K. C. Evolutionary computation in the era of large language model: Survey and roadmap. arXiv preprint arXiv:2401.10034, 2024

2024 arXiv
[93]

The rise and potential of large language model based agents: A survey

Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68 0 (2): 0 121101, 2025

2025
[94]

Opencity: A scalable platform to simulate urban activities with massive llm agents

Yan, Y., Zeng, Q., Zheng, Z., Yuan, J., Feng, J., Zhang, J., Xu, F., and Li, Y. Opencity: A scalable platform to simulate urban activities with massive llm agents. arXiv preprint arXiv:2410.21286, 2024

2024 arXiv
[95]

V., Zhou, D., and Chen, X

Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Bb4VGOWELI

2024
[96]

Researchtown: Simulator of human research community

Yu, H., Hong, Z., Cheng, Z., Zhu, K., Xuan, K., Yao, J., Feng, T., and You, J. Researchtown: Simulator of human research community. arXiv preprint arXiv:2412.17767, 2024

2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Aho, A. V. and Hopcroft, J. E. The design and analysis of computer algorithms. Pearson Education India, 1974

1974

[3] [3]

Akl, S. G. The design and analysis of parallel algorithms. Prentice-Hall, Inc., 1989

1989

[4] [4]

The intelligence age

Altman, S. The intelligence age. https://ia.samaltman.com/, September 2024. Accessed: 2025-05-28

2024

[5] [5]

Anthropic API Pricing , 2025

Anthropic . Anthropic API Pricing , 2025. URL https://www.anthropic.com/pricing#anthropic-api. Accessed: 2025-01-30

2025

[6] [6]

B., and Michalewicz, Z

B \"a ck, T., Fogel, D. B., and Michalewicz, Z. Handbook of evolutionary computation. Release, 97 0 (1): 0 B1, 1997

1997

[7] [7]

Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds

Basmov, V., Goldberg, Y., and Tsarfaty, R. Simple linguistic inferences of large language models (llms): Blind spots and blinds. arXiv preprint arXiv:2305.14785, 2023

work page Pith review arXiv 2023

[8] [8]

Baudet, G. M. The design and analysis of algorithms for asynchronous multiprocessors. Carnegie Mellon University, 1978

1978