pith. sign in

arxiv: 2606.25960 · v1 · pith:Q4XYIWEYnew · submitted 2026-06-24 · 💻 cs.AI

Agentic System as Compressor: Quantifying System Intelligence in Bits

Zihan Qin , Hongrui Zhang This is my paper

Pith reviewed 2026-06-25 20:01 UTC · model grok-4.3

classification 💻 cs.AI
keywords agentic systemscompressioncodelengthintelligence measurearithmetic codingLLM agentsinformation theorytask reconstruction
0
0 comments X

The pith

Agentic systems act as compressors that reconstruct targets with fewer bits under fixed task distributions, interfaces, and compute budgets.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper treats agentic AI systems, which incorporate tools, retrieval, verifiers, and multi-turn interaction, as mechanisms that reduce uncertainty about a target object. It claims that when task distribution, interface, and budget stay constant, stronger agentic components produce shorter codelengths for reconstruction. This is demonstrated across five controlled settings using arithmetic coding, seed coding, and fallback methods, where agentic elements consistently lower the bits required. A reader would care because the approach supplies an information-theoretic yardstick for comparing agents that does not depend on task-specific performance scores.

Core claim

Under a fixed task distribution, interface, and compute budget, a stronger agentic system lets a target object be reconstructed with fewer bits; this is shown by evaluating reversed text, chess moves, protein sequences, retrieval-augmented question answering, and semantic story compression, where agentic components reduce codelength in each case.

What carries the argument

The compression-is-intelligence viewpoint operationalized through arithmetic coding, seed coding, and fallback to measure how agentic components reduce residual uncertainty.

If this is right

  • Codelength becomes a usable metric for analyzing how specific agent components, observers, and budgets alter residual uncertainty.
  • Agent design can be guided by identifying which components most effectively shorten the description of targets.
  • The same measure applies uniformly across domains including language, games, and sequences.
  • Evaluation of real agent systems can shift toward bit-based comparisons rather than isolated accuracy scores.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The bit-reduction view could be tested on non-LLM agents or human problem solvers performing analogous reconstruction tasks.
  • If the relation holds, minimum-description-length ideas from learning theory might connect directly to agent evaluation.
  • Compute-budget allocation in deployed agents could be optimized by tracking codelength changes rather than downstream metrics alone.

Load-bearing premise

The five controlled settings together with the chosen coding methods are sufficient to isolate the contribution of agentic components to reduced uncertainty.

What would settle it

A run of the same five settings in which adding the agentic components leaves codelength unchanged or increases it.

Figures

Figures reproduced from arXiv: 2606.25960 by Hongrui Zhang, Zihan Qin.

Figure 1
Figure 1. Figure 1: Overview of agentic compression. Tools, constraints, context, search, and verifiers [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Bit value of agentic components across the five experiments (A–E). The first five panels [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Retrieval value on HotpotQA, priced in bits. (a) Per-question effect of relevant vs. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Observer monotonicity. The stronger the observer, the higher the mean codelength [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Compute–codelength Pareto frontier: (a) schematic and (b) measured on TinyStories. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
read the original abstract

Large language models are turning from isolated predictors into agentic systems: they call tools, retrieve evidence, obey environment constraints, use verifiers, and complete tasks through search and multi-turn interaction. We adopts an analytical viewpoint based on "compression is intelligence": under a fixed task distribution, interface, and compute budget, a stronger agentic system lets a target object be reconstructed with fewer bits. We operationalize the measure with arithmetic coding, seed coding, and a fallback, and evaluate it in five settings: reversed text, chess moves, protein sequences, retrieval-augmented question answering, and semantic story compression; in all of them agentic components reduce codelength. These small, controlled experiments cover component types typical of real agentic systems, show that codelength can analyze how components, observers, and budgets change residual uncertainty, and offer guidance for evaluating real agent systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper claims that agentic systems (incorporating tools, verifiers, search, and multi-turn interaction) function as compressors: under fixed task distribution, interface, and compute budget, stronger agentic components reduce the codelength needed to reconstruct a target object. This is operationalized via arithmetic coding, seed coding, and fallback, and demonstrated across five controlled settings (reversed text, chess moves, protein sequences, RAG QA, semantic story compression), where agentic elements consistently lower codelength.

Significance. If the fixed-budget controls and attribution to agentic components hold, the work supplies a concrete, information-theoretic metric for comparing agentic designs that could complement accuracy-based benchmarks and inform scaling decisions. The small-scale, component-isolated experiments are a strength for reproducibility and falsifiability.

major comments (3)
  1. [Experimental settings (Sections 4–8)] The central claim requires that compute budget (model calls, search steps, total FLOPs) remains strictly fixed when adding agentic components; the manuscript must explicitly document how this is enforced in each of the five settings (e.g., via token or call limits) rather than allowing extra inference in the agentic case to drive the reported codelength reduction.
  2. [Coding methods and §3] It is unclear whether the probability models used for arithmetic/seed coding are held identical across baseline and agentic conditions or whether agentic retrieval/verification alters the effective predictive distribution; without an ablation that isolates the coding model from the agentic interface, the reduction cannot be attributed solely to agenticity.
  3. [Evaluation protocol] The fallback procedure and any exclusion criteria for failed reconstructions must be stated with quantitative thresholds; otherwise the reported codelength averages may be conditioned on easier instances in the agentic arm, undermining the cross-setting comparison.
minor comments (2)
  1. [Abstract] Abstract: 'We adopts' is a grammatical error.
  2. [§2–3] Notation for codelength (bits vs. nats) and the exact definition of 'fixed compute budget' should be introduced with an equation in §2 or §3 for clarity.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The comments highlight important aspects of experimental rigor that we address point-by-point below, with commitments to revisions that strengthen transparency without altering the core claims or results.

read point-by-point responses

  1. Referee: [Experimental settings (Sections 4–8)] The central claim requires that compute budget (model calls, search steps, total FLOPs) remains strictly fixed when adding agentic components; the manuscript must explicitly document how this is enforced in each of the five settings (e.g., via token or call limits) rather than allowing extra inference in the agentic case to drive the reported codelength reduction.

    Authors: We agree that explicit per-setting documentation is required for full reproducibility. In the reported experiments, compute was controlled by enforcing identical maximum model calls, search depth, and token budgets between baseline and agentic arms (e.g., single forward pass in reversed-text baseline vs. one tool-augmented pass in agentic; fixed iteration caps in chess and protein settings). We will add a dedicated subsection (new §3.4) that tabulates the exact call limits, token caps, and estimated FLOPs for each of the five settings to make the fixed-budget enforcement transparent. revision: yes

  2. Referee: [Coding methods and §3] It is unclear whether the probability models used for arithmetic/seed coding are held identical across baseline and agentic conditions or whether agentic retrieval/verification alters the effective predictive distribution; without an ablation that isolates the coding model from the agentic interface, the reduction cannot be attributed solely to agenticity.

    Authors: The underlying LLM probability model remains identical across conditions; agentic elements only supply additional context, retrieval results, or verification signals that improve the conditional distribution fed to the same coder. We acknowledge that an explicit ablation isolating the interface would further strengthen attribution and will add such an ablation (comparing agentic retrieval against equivalent non-agentic context padding) in the revised §3 and corresponding experimental sections. revision: yes

  3. Referee: [Evaluation protocol] The fallback procedure and any exclusion criteria for failed reconstructions must be stated with quantitative thresholds; otherwise the reported codelength averages may be conditioned on easier instances in the agentic arm, undermining the cross-setting comparison.

    Authors: The fallback (baseline codelength) is applied whenever the agentic procedure fails to produce a valid reconstruction within the fixed turn limit; no instances are excluded from the reported averages. We will revise §3 to state the precise quantitative failure thresholds (e.g., no further codelength reduction after maximum turns or when remaining entropy exceeds a fixed bit threshold) and confirm that all test instances contribute to the averages under the same protocol. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical evaluation under adopted viewpoint

full rationale

The paper adopts the 'compression is intelligence' viewpoint as an analytical lens rather than deriving it, then operationalizes a codelength measure via arithmetic/seed coding and evaluates it empirically across five settings. The reported reductions are outputs of those controlled comparisons, not reductions of a claimed first-principles result to the inputs by construction. No equations, self-citations, uniqueness theorems, or ansatzes are invoked that would make the central claim tautological. The work is self-contained as an empirical quantification exercise.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review performed from abstract only; no explicit free parameters, axioms, or invented entities are stated beyond the adopted viewpoint that compression equals intelligence.

axioms (1)
  • domain assumption Compression is intelligence under fixed task distribution, interface, and compute budget
    Stated as the analytical viewpoint adopted in the abstract.

pith-pipeline@v0.9.1-grok · 5673 in / 1133 out tokens · 38854 ms · 2026-06-25T20:01:55.788718+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references

  1. [1]

    2006 , publisher =

    Elements of Information Theory , author =. 2006 , publisher =

    2006

  2. [2]

    Communications of the ACM , volume =

    Arithmetic Coding for Data Compression , author =. Communications of the ACM , volume =. 1987 , doi =

    1987

  3. [3]

    International Conference on Learning Representations (ICLR) , year =

    Language Modeling Is Compression , author =. International Conference on Learning Representations (ICLR) , year =

  4. [4]

    Conference on Language Modeling (COLM) , year =

    Compression Represents Intelligence Linearly , author =. Conference on Language Modeling (COLM) , year =

  5. [5]

    2023 , note =

    An Observation on Generalization , author =. 2023 , note =

    2023

  6. [6]

    Automatica , volume =

    Modeling by Shortest Data Description , author =. Automatica , volume =. 1978 , doi =

    1978

  7. [7]

    IEEE Transactions on Information Theory , volume =

    The Minimum Description Length Principle in Coding and Modeling , author =. IEEE Transactions on Information Theory , volume =. 1998 , doi =

    1998

  8. [8]

    2007 , publisher =

    The Minimum Description Length Principle , author =. 2007 , publisher =

    2007

  9. [9]

    International Conference on Learning Representations (ICLR) , year =

    ReAct: Synergizing Reasoning and Acting in Language Models , author =. International Conference on Learning Representations (ICLR) , year =

  10. [10]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  11. [11]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Toolformer: Language Models Can Teach Themselves to Use Tools , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  12. [12]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  13. [13]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Reflexion: Language Agents with Verbal Reinforcement Learning , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  14. [14]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    SWE-agent: Agent--Computer Interfaces Enable Automated Software Engineering , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  15. [15]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  16. [16]

    International Conference on Learning Representations (ICLR) , year =

    Self-Consistency Improves Chain-of-Thought Reasoning in Language Models , author =. International Conference on Learning Representations (ICLR) , year =

  17. [17]

    2026 , howpublished =

    Harness Engineering: Leveraging Codex in an Agent-First World , author =. 2026 , howpublished =

    2026

  18. [18]

    2026 , howpublished =

    OpenClaw: An Open-Source Agent Harness , author =. 2026 , howpublished =

    2026

  19. [19]

    2026 , howpublished =

    Hermes Agent: An Open-Source Autonomous Agent Runtime , author =. 2026 , howpublished =

    2026

  20. [20]

    2022 , note =

    Holistic Evaluation of Language Models (HELM) , author =. 2022 , note =

    2022

  21. [21]

    2022 , note =

    Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models (BIG-bench) , author =. 2022 , note =

    2022

  22. [22]

    Communications of the ACM , volume =

    The JPEG Still Picture Compression Standard , author =. Communications of the ACM , volume =. 1991 , doi =

    1991

  23. [23]

    AES 17th International Conference: High-Quality Audio Coding , year =

    MP3 and AAC Explained , author =. AES 17th International Conference: High-Quality Audio Coding , year =

  24. [24]

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    The Perception--Distortion Tradeoff , author =. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2018 , doi =

    2018

  25. [25]

    International Conference on Very Large Data Bases (VLDB) , pages =

    Semantic Compression and Pattern Extraction with Fascicles , author =. International Conference on Very Large Data Bases (VLDB) , pages =

  26. [26]

    Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models , author =. Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2023 , doi =

    2023

  27. [27]

    Advances in Cryptology -- EUROCRYPT , series =

    Conditional Computational Entropy, or Toward Separating Pseudoentropy from Compressibility , author =. Advances in Cryptology -- EUROCRYPT , series =. 2007 , doi =

    2007

  28. [28]

    Proceedings of the 23rd Annual Symposium on Foundations of Computer Science (FOCS) , pages =

    Theory and Application of Trapdoor Functions , author =. Proceedings of the 23rd Annual Symposium on Foundations of Computer Science (FOCS) , pages =. 1982 , doi =

    1982

  29. [29]

    RANDOM-APPROX , series =

    Computational Analogues of Entropy , author =. RANDOM-APPROX , series =. 2003 , doi =

    2003