Pith Number

pith:A4YI2J37

pith:2026:A4YI2J375WJRRNHU6DCBBGSGXT

not attested not anchored not stored refs pending

AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents

Dongyu Ru, Jingwen Xv, Lin Qiu, Xiaohua Wang, Xiaoqing Zheng, Xiaoyu Li, Xuezhi Cao, Xunliang Cai, Yiyang Li, Zhengkang Guo

LLM agents handle short tool sequences but lose substantial accuracy when required to track deep chains of dependencies across novel procedures.

arxiv:2605.07926 v2 · 2026-05-08 · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{A4YI2J375WJRRNHU6DCBBGSGXT}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Experiments with sixteen LLM agents and human participants show that performance drops sharply as dependency depth increases: humans decline from 98.3% success at difficulty-5 to 80.0% at difficulty-25, while the best model drops from 90.0% to 60.0%.

C2weakest assumption

That the escape-room tasks with explicit DAG constraints and incremental state revelation accurately capture the core challenges of out-of-domain tool-grounded reasoning without introducing benchmark-specific artifacts or overly artificial constraints.

C3one line summary

AgentEscapeBench shows LLM agents' success rates drop from 90% to 60% as tool-dependency depth increases from 5 to 25 steps, while humans drop only from 98% to 80%.

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-05-21T01:05:20.639499Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

07308d277fed9318b4f4f0c4109a46bcdc086e6710acb73e1deee590fb307d83

Aliases

arxiv: 2605.07926 · arxiv_version: 2605.07926v2 · doi: 10.48550/arxiv.2605.07926 · pith_short_12: A4YI2J375WJR · pith_short_16: A4YI2J375WJRRNHU · pith_short_8: A4YI2J37

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/A4YI2J375WJRRNHU6DCBBGSGXT \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 07308d277fed9318b4f4f0c4109a46bcdc086e6710acb73e1deee590fb307d83

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "c7bd849a0bedf3ca53e795edddb5a69745ff9fd911a56ad5f4458166bcfdb295",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-05-08T15:59:27Z",
    "title_canon_sha256": "298835dc23693e4627548307586cb086366c3bd4f1c2abdfeaf337fd1cc579e6"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.07926",
    "kind": "arxiv",
    "version": 2
  }
}