Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis

Changhao Wang; Haoran Tan; Minlong Peng; Ruifeng Ren; Xinhao Yao; Yong Liu; Yuanzhuo Liu; Yunfei Yu; Yuyao Zhang

End-to-end answer accuracy alone cannot diagnose whether models know when and how to search.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 20:12 UTC pith:PFHQRP5U

load-bearing objection Solid disentangled eval recipe with real Setting I signal; the named Setting II “delegation profiles” are underpowered and the closed-pool search is a weak stand-in for open web.

arxiv 2607.23524 v1 pith:PFHQRP5U submitted 2026-07-26 cs.AI

Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis

Xinhao Yao , Yuanzhuo Liu , Changhao Wang , Yunfei Yu , Haoran Tan , Yuyao Zhang , Ruifeng Ren , Minlong Peng

show 1 more author

Yong Liu

This is my paper

classification cs.AI

keywords Delegation Intelligencedeep searchagent evaluationsearch decision-makinginformation synthesisevidence verificationcontrollable synthesisDelegSearchBench

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Deep-search agents are usually scored only on whether the final answer is right. That single number mixes retrieval luck, long-context reading, source judgment, and the decision to search at all, so failures stay hard to attribute. This paper names the missing meta-skill Delegation Intelligence: knowing when evidence is insufficient, whether and how to search, and how to verify and fuse what comes back under noise and adversarial distractors. It offers a controllable, document-grounded reverse-engineering recipe that builds tasks with known answers, labeled evidence, natural noise, and targeted distractors, then instantiates DelegSearchBench and a protocol that isolates synthesis versus search decisions by changing document composition and tool access. Across models, full-context results show mode-specific weaknesses and lost-in-the-middle sensitivity even when all evidence is present; partial-context results show premature answering on one side and aggressive but ineffective searching on the other. The sympathetic takeaway is that reliable deep search needs calibrated coordination of uncertainty awareness, acquisition, and faithful synthesis—not merely calling a search tool.

Core claim

Deep-search competence cannot be characterized by final-answer accuracy alone. Under controlled full-context and partial-context protocols on DelegSearchBench, models show mode-dependent synthesis and verification failures, position sensitivity to supporting evidence even when the full pool is available, unstable reasoning across repeated attempts (Pass@1 vs Pass@3 gaps), and two opposing delegation failures—answering too early despite missing evidence, or searching without retrieving and integrating what is needed—so reliable agents must coordinate uncertainty recognition, evidence acquisition, and verification, not only invoke search.

What carries the argument

Delegation Intelligence, decomposed into Search Decision-Making and Information Synthesis & Verification, measured via a controllable synthesis pipeline of document-grounded reverse engineering (evidence-first query/answer construction, evidence-anchored distractors, multi-view rejection filtering) and a disentangled protocol that varies document composition and tool access (Setting I full-context, search off; Setting II partial golden evidence, search on over a closed item pool).

Load-bearing premise

If tasks built by reverse-engineering real documents with model generators and filters, plus a small closed search pool in the partial-context setting, do not stand in for open-web search, then the measured “delegation” gaps may not transfer beyond the synthetic bench.

What would settle it

Re-run the same models on live open-web deep-search tasks with matched queries, or expand Setting II to a large open corpus: if mode rankings, position effects, and premature-vs-ineffective-search profiles collapse or reverse while end-to-end accuracy stays high, the claim that these controlled gaps diagnose real Delegation Intelligence fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Agent benchmarks should report decoupled scores for search decision-making and synthesis/verification, not only task success rate.
Full-context evaluation with fixed document order variants can expose lost-in-the-middle failures independent of retrieval quality.
Pass@1 versus Pass@3 (and related stability metrics) should be standard for agents, because stochastic success is not the same as reliable evidence use.
Training and product design should target calibrated under- and over-search, not only higher tool-call rates or longer contexts.
The reverse-engineering recipe can be reused to build new controlled deep-search suites as domains and tools change.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Position-robust evidence localization may need architectural or training fixes separate from better retrievers, since the paper’s full-context setting already removes retrieval noise.
Judge-model and construction-model separation reduces self-preference, but residual alignment between synthetic distractors and particular model families could still skew comparative rankings—worth an independent cross-generator audit.
A natural next stress test is multi-turn, budgeted search where cost of each call is explicit, to see whether “knowing when not to search” survives under real latency and quota pressure.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid disentangled eval recipe with real Setting I signal; the named Setting II “delegation profiles” are underpowered and the closed-pool search is a weak stand-in for open web.

read the letter

The useful core here is not another end-to-end agent leaderboard. It’s a clean separation of Search Decision-Making from Information Synthesis & Verification, plus a reverse-engineering pipeline that fixes answer and evidence structure before task construction. That control is what lets them run full-context position ablations and partial-context action metrics without the usual retrieval confounds.

Setting I is the part I’d trust. 429 items, six shaping modes, three evidence orders, Pass@1/Pass@3 across a wide model panel. Mode-dependent rankings and lost-in-the-middle effects with retrieval removed are real and useful. The Pass@1–Pass@3 gaps make the stability point concrete rather than hand-wavy. The construction recipe (Stages A–C, evidence-anchored distractors, multi-view rejection) is more interesting as a reusable method than as a one-shot dataset, and the 100-item human audit is at least honest about residual defect rates.

Setting II is thinner than the prose implies. Thirty-three curated items, split across two modes, N=3, no intervals. Naming doubao/deepseek as premature answerers, hunyuan as aggressive-but-ineffective, and claude as calibrated is the narrative the abstract leans on, but those fractions move on a handful of items, and Hit@FirstSearch is conditioned on searching at all. Worse, “search” is top-k over the item’s own ~9-doc closed pool, so query quality mostly means lexical hit on a withheld golden doc—not open-web information need expression. That doesn’t kill the premature-vs-ineffective contrast as a qualitative observation; it does mean the central “calibrated coordination” claim is oversold relative to the n and the retrieval setup.

Other soft spots in proportion: code/data still “after internal review,” heavy LLM generate/filter/judge stack, and transfer to live web unproven. Citations to multi-hop, BrowseComp-Plus, GAIA, τ-bench, lost-in-the-middle are appropriate; novelty is incremental disentanglement, not a new paradigm.

Who it’s for: people building or auditing search-agent benchmarks and training signals. Worth a serious referee. I’d bring the pipeline and Setting I tables to reading group; treat Table 2 model portraits as provisional. Engage the method; don’t overfit the named failure profiles until the set is larger and the search backend is less toy.

Circularity Check

0 steps flagged

No significant circularity: empirical evaluation framework whose behavioral findings are not forced by construction or self-citation.

full rationale

This is a methods-and-benchmark paper, not a first-principles derivation. Delegation Intelligence is defined operationally (Def. 3.1; §3), then measured under controlled full-context and partial-context protocols on a reverse-engineered document pool (pipeline §4; Settings I–II §5). Controllability steps—grounding queries in golden evidence, synthesizing query-constrained distractors, and rejection-filtering for answerability/non-memorability/label correctness—ensure valid diagnostic items; they do not algebraically or statistically force the reported outcomes (mode-dependent rankings, golden_mid degradation, Pass@1–Pass@3 gaps, or opposing Search vs Direct profiles). Those are independent behavioral readouts across models. LLM use in synthesis, filtering, and judging is a methodological risk, not a circular reduction of a claimed prediction to its fitted inputs. No uniqueness theorem, self-citation chain, or renamed known law carries the central claim. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 3 invented entities

The work is empirical/methodological rather than axiomatic math. Load-bearing premises are domain assumptions about what controlled synthetic documents measure, plus operational choices in generation, filtering, judging, and the closed search pool. No physical constants or fitted scientific parameters; free parameters are design knobs of the benchmark and protocol.

free parameters (5)

DelegSearchBench size and mode mix (429 items; per-mode counts) = 429 items; averages ~2.76 golden / 1.56 irrelevant / 4.54 distractor docs
Final pool composition (e.g., sufficiency 97, intra_doc 94, publish_time 86, multi_hop 79) is a construction choice that shapes all aggregate rankings.
Setting II curated subset size and selection = 33 items
Only 33 manually retained multi_hop/sufficiency items define the search-decision results; selection criteria are qualitative (“genuine information gap”).
Pass@k sample count N=3 = N=3
Stability conclusions (Pass@1 vs Pass@3 gaps) depend on drawing three samples per condition.
Document body truncation length = 4000 characters
Each document content truncated to at most 4,000 characters in evaluation prompts, affecting available evidence.
Complexity controls (golden doc count, hops, information points)
Stage A lightweight constraints on numbers of golden documents, reasoning hops, and independent information points are hand-set synthesis knobs.

axioms (6)

domain assumption Final-answer accuracy on coupled retrieval+reasoning systems is insufficient to attribute deep-search competence.
Stated in Introduction and Abstract as the motivating premise for disentangled evaluation; standard in process-vs-outcome agent eval arguments but not proved universally.
ad hoc to paper Document-grounded reverse engineering from high-authority real documents yields natural, uniquely answerable, non-memorable tasks whose labels are trustworthy after automated filters plus light human audit.
Core of §4 Stages A–C and Appendix B; the benchmark’s validity rests on this construction epistemology.
ad hoc to paper Isolating synthesis (full context, search off) versus decision-making (partial golden evidence, search on, closed pool) cleanly separates the two Delegation Intelligence dimensions.
§5.1 protocol design; assumes models cannot smuggle parametric knowledge or exploit pool artifacts enough to collapse the isolation.
domain assumption LLM-as-judge semantic equivalence (held-out deepseek-v4-flash) is an adequate correctness criterion for short answers.
§5.1 scoring protocol; common practice but an unproven reliability assumption for the reported Pass rates.
ad hoc to paper Closed item-level top-k retrieval is an acceptable proxy for testing search decision-making without open-web variance.
Explicit Setting II design choice in §5.2; trades realism for reproducibility.
domain assumption Lost-in-the-middle-style position effects and Pass@k gaps are meaningful indicators of fragile evidence localization rather than benign sampling noise.
Interpretation of Table 1 patterns in §5.2; depends on standard long-context literature (e.g., Liu et al.) plus the authors’ agent-reliability framing.

invented entities (3)

Delegation Intelligence (in deep search) no independent evidence
purpose: Name and organize the metacognitive capacity to decide when/how to search and how to verify/synthesize returned evidence.
Definition 3.1 operationalizes a new evaluation target; related “delegation” citations exist, but this task-specific construct is introduced here as the paper’s organizing entity.
DelegSearchBench no independent evidence
purpose: Concrete benchmark instantiation of the controllable synthesis recipe for disentangled measurement.
New dataset/protocol artifact; existence is by construction, not an external natural kind.
Controllable document-grounded reverse-engineering synthesis pipeline (Stages A–C with shaping modes) no independent evidence
purpose: Generate queries, golden evidence, irrelevant noise, and query-constrained distractors with auditable labels.
Methodological invention enabling the claimed experimental control; validity is internal to the pipeline plus human audit sample.

pith-pipeline@v1.2.0-grok45-kimik3 · 30135 in / 4124 out tokens · 78089 ms · 2026-07-30T20:12:16.384398+00:00 · methodology

0 comments

read the original abstract

Deep search is becoming a core capability of modern agent systems, yet it is typically evaluated solely based on end-to-end answer accuracy. This coupled evaluation paradigm entangles retrieval quality, long-context comprehension, evidence verification, and tool-use decisions, making it difficult to determine whether a model truly knows when and how to delegate information seeking to search. To this end: (1) We formalize this meta-capability as Delegation Intelligence in deep search and decompose it into complementary dimensions-Search Decision-Making (recognizing information insufficiency and deciding whether, when, and how to search) and Information Synthesis and Verification (aggregating evidence from multiple sources, judging source reliability, and synthesizing information under noisy, potentially adversarial conditions). (2) To enable disentangled and reproducible measurement, we develop a controllable synthesis pipeline built on document-grounded reverse engineering. This yields a general recipe for constructing controlled deep-search evaluations rather than a single fixed dataset. (3) As a concrete instantiation, we construct DelegSearchBench, together with a disentangled evaluation protocol that isolates each capability dimension by varying document composition and tool access. (4) Across representative models, we demonstrate that deep-search competence cannot be adequately characterized by final-answer accuracy alone...

discussion (0)

Reference graph

Works this paper leans on

44 extracted references · 13 linked inside Pith

[1]

Ctrlsynth: Controllable image text synthesis for data-efficient multimodal learning

Qingqing Cao, Mahyar Najibi, and Sachin Mehta. Ctrlsynth: Controllable image text synthesis for data-efficient multimodal learning. InForty-second International Conference on Machine Learning, 2025

2025
[2]

A benchmark of expert-level academic questions to assess AI capabilities.Nature, 649:1139–1146, 2026

Center for AI Safety, Scale AI, and HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities.Nature, 649:1139–1146, 2026. doi: 10.1038/s41586 -025-09962-4. URLhttps://arxiv.org/abs/2501.14249

Pith/arXiv arXiv 2026
[3]

xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations, 2025

Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, Chen Sun, Han Hou, Hui Yang, James Pan, Jianan Lou, Jiayi Mao, Jizheng Liu, Jinpeng Li, Kangyi Liu, Kenkun Liu, Rui Wang, Run Li, Tong Niu, Wenlong Zhang, Wenqi Yan, Xuanzheng Wang, Yuchen Zhang, Yi-Hsin Hung, Yuan Jiang, Zexuan Liu,...

Pith/arXiv arXiv 2025
[4]

Warner, Hugo Aerts, Thomas Hartvigsen, Jack Gallifant, and Danielle Bitterman

Shan Chen, Pedro José Ferreira Moreira, Yuxin Xiao, Samuel Schmidgall, Jeremy L. Warner, Hugo Aerts, Thomas Hartvigsen, Jack Gallifant, and Danielle Bitterman. Medbrowsecomp: Benchmarking medical deep research and computer use. InNeurIPS 2025 Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling, 2025

2025
[5]

BrowseComp-plus: A fair and disentangled evaluation benchmark for deep search agents

ZijianChen, XueguangMa, ShengyaoZhuang, PingNie, KaiZou, SahelSharifymoghaddam, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Hosna Oyarhoseini, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, and Jimmy Lin. BrowseComp-plus: A fair and disentangled evaluation benchmark for deep search agent...

2026
[6]

Beyond itinerary planning—a real-world benchmark for multi-turn and tool-using travel tasks

Xiang Cheng, Yulan Hu, Xiangwen Zhang, Lu Xu, Lide Tan, Zheng Pan, Xin Li, and Yong Liu. Beyond itinerary planning—a real-world benchmark for multi-turn and tool-using travel tasks. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29200–29251, 2026

2026
[7]

Evoclaw: Evaluating AI agents on continuous software evolution

Gangda Deng, Zhaoling Chen, Zhongming Yu, Haoyang Fan, Yuhong Liu, Yuxin Yang, Dhruv Parikh, Rajgopal Kannan, Le Cong, Mengdi Wang, Qian Zhang, Viktor Prasanna, Xiangru Tang, and Xingyao Wang. Evoclaw: Evaluating AI agents on continuous software evolution. InForty-third International Conference on Machine Learning, 2026

2026
[8]

Cl-bench life: Can language models learn from real-life context?, 2026

Shihan Dou, Yujiong Shen, Chenhao Huang, Junjie Ye, Jiayi Chen, Junzhe Wang, Qianyu He, Shichun Liu, Changze Lv, Jiahang Lin, Jiazheng Zhang, Ming Zhang, Shaofan Liu, Tao Ji, Zhangyue Yin, Cheng Zhang, Huaibing Xie, Jianglu Hu, Jingcheng Deng, Lincheng Li, Minda Hu, Shaolei Wang, Syrus Zhao, Weichao Wang, Yan Lei, Yang Liu, Yanling Xiao, Yiting Liu, Zenan...

Pith/arXiv arXiv 2026
[9]

Cl-bench: A benchmark for context learning, 2026

Shihan Dou, Ming Zhang, Zhangyue Yin, Chenhao Huang, Yujiong Shen, Junzhe Wang, Jiayi Chen, Yuchen Ni, Junjie Ye, Cheng Zhang, Huaibing Xie, Jianglu Hu, Shaolei Wang, Weichao Wang, Yanling Xiao, Yiting Liu, Zenan Xu, Zhen Guo, Pluto Zhou, Tao Gui, Zuxuan Wu, Xipeng Qiu, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Di Wang, and Shunyu Yao. Cl-bench: A benchmar...

arXiv 2026
[10]

Glm-5: from vibe coding to agentic engineering, 2026

GLM-5-Team, :, Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, Chenzheng Zhu, Congfeng Yin, Cunxiang Wang, Gengzheng Pan, Hao Zeng, Haoke Zhang, Haoran Wang, Huilong Chen, Jiajie Zhang, Jian Jiao, Jiaqi Guo, Jingsen Wang, Jingzhao Du, Jinzhu Wu, Kedong Wang, Lei Li, Lin Fan, Lucen Zho...

Pith/arXiv arXiv 2026
[11]

A survey on llm-as-a-judge, 2025

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on llm-as-a-judge, 2025. URLhttps://arxiv.org/abs/2411.15594

Pith/arXiv arXiv 2025
[12]

Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps

Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. InProceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, 2020

2020
[13]

Step-deepresearch technical report, 2025

Chen Hu, Haikuo Du, Heng Wang, Lin Lin, Mingrui Chen, Peng Liu, Ruihang Miao, Tianchi Yue, Wang You, Wei Ji, Wei Yuan, Wenjin Deng, Xiaojian Yuan, Xiaoyun Zhang, Xiangyu Liu, Xikai Liu, Yanming Xu, Yicheng Cao, Yifei Zhang, Yongyao Wang, Yubo Shu, Yurong Zhang, Yuxiang Zhang, Zheng Gong, Zhichao Chang, Binyan Li, Dan Ma, Furong Jia, Hongyuan Wang, Jiayu L...

arXiv 2025
[14]

Building a foundational guardrail for general agentic systems via synthetic data

Yue Huang, Hang Hua, Yujun Zhou, Pengcheng Jing, Manish Nagireddy, Inkit Padhi, Greta Dolcetti, Zhangchen Xu, Subhajit Chaudhury, Ambrish Rawat, Liubov Nedoshivina, Pin-Yu Chen, Prasanna Sattigeri, and Xiangliang Zhang. Building a foundational guardrail for general agentic systems via synthetic data. InThe Fourteenth International Conference on Learning R...

2026
[15]

Livecodebench: Holistic and contamination free evaluation of large language models for code

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=chfJJYC3iL

2025
[16]

DiningBench: A hierarchical multi-view benchmark for perception and reasoning in the dietary domain

Song Jin, Juntian Zhang, Xun Zhang, Zeying Tian, Fei Jiang, Guojun Yin, Wei Lin, Yong Liu, and Rui Yan. DiningBench: A hierarchical multi-view benchmark for perception and reasoning in the dietary domain. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 35289–35310, 2026

2026
[17]

Delegation to artificial intelligence can increase dishonest behaviour.Nature, 646(8083):126–134, 2025

Nils Köbis, Zoe Rahwan, Raluca Rilla, Bramantyo Ibrahim Supriyatno, Clara Bersch, Tamer Ajaj, Jean-François Bonnefon, and Iyad Rahwan. Delegation to artificial intelligence can increase dishonest behaviour.Nature, 646(8083):126–134, 2025

2025
[18]

Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation

Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies ...

2025
[19]

Claw-eval-live: A live agent benchmark for evolving real-world workflows, 2026

Chenxin Li, Zhengyang Tang, Mingxin Huang, Yunlong Lin, Shijue Huang, Shengyuan Liu, Bowen Ye, Rang Li, Lei Li, Benyou Wang, and Yixuan Yuan. Claw-eval-live: A live agent benchmark for evolving real-world workflows, 2026. URLhttps://arxiv.org/abs/2604.28139

Pith/arXiv arXiv 2026
[20]

From generation to judgment: Opportunities and challenges of LLM-as-a-judge

Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. From generation to judgment: Opportunities and challenges of LLM-as-a-judge. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2757–2...

2025
[21]

Reportbench: Evaluating deep research agents via academic survey tasks, 2025

Minghao Li, Ying Zeng, Zhihao Cheng, Cong Ma, and Kai Jia. Reportbench: Evaluating deep research agents via academic survey tasks, 2025. URLhttps://arxiv.org/abs/2508.15804

Pith/arXiv arXiv 2025
[22]

Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, pages 157–173, 2024. Work in Progress 14 Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capa...

2024
[23]

Search self-play: Pushing the frontier of agent capa- bility without supervision

Hongliang Lu, Yuhang Wen, Pengyu Cheng, Ruijin Ding, Jiaqi Guo, Haotian Xu, Chutian Wang, Haonan Chen, xiaoxi jiang, and guanjunjiang. Search self-play: Pushing the frontier of agent capa- bility without supervision. InThe Fourteenth International Conference on Learning Representations, 2026

2026
[24]

Natural emergent misalignment from reward hacking in production rl, 2025

Monte MacDiarmid, Benjamin Wright, Jonathan Uesato, Joe Benton, Jon Kutasov, Sara Price, Naia Bouscal, Sam Bowman, Trenton Bricken, Alex Cloud, Carson Denison, Johannes Gasteiger, Ryan Greenblatt, Jan Leike, Jack Lindsey, Vlad Mikulik, Ethan Perez, Alex Rodrigues, Drake Thomas, Albert Webson, Daniel Ziegler, and Evan Hubinger. Natural emergent misalignmen...

arXiv 2025
[25]

GAIA: a benchmark for general AI assistants

Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: a benchmark for general AI assistants. InThe Twelfth International Conference on Learning Represen- tations, 2024

2024
[26]

Searchswarm: Towards delegation intelligence in agentic llms for long-horizon deep research, 2026

Pu Ning, Quan Chen, Kun Tao, Xinyu Tang, Tianshu Wang, Qianggang Cao, Xinyu Kong, Zujie Wen, Zhiqiang Zhang, and Jun Zhou. Searchswarm: Towards delegation intelligence in agentic llms for long-horizon deep research, 2026. URLhttps://arxiv.org/abs/2606.09730

Pith/arXiv arXiv 2026
[27]

Measuring and narrowing the compositionality gap in language models

Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, 2023

2023
[28]

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

2024
[29]

Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, Z. F. Wu, Zhibin Gou, Shirong Ma, Hongxuan Tang, Yuxuan Liu, Wenjun Gao, Daya Guo, and Chong Ruan. Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition, 2025. URLhttps://ar...

Pith/arXiv arXiv 2025
[30]

Jinzhu Song and Hengyu Lin. Exploring the effect of artificial intelligence intellect on consumer decision delegation: The role of trust, task objectivity, and anthropomorphism.Journal of Consumer Behaviour, 23(2):727–747, 2024

2024
[31]

Recent frontier models are reward hacking.https: //metr.org/blog/2025-06-05-recent-reward-hacking/, 06 2025

Beth Barnes Sydney Von Arx, Lawrence Chan. Recent frontier models are reward hacking.https: //metr.org/blog/2025-06-05-recent-reward-hacking/, 06 2025

2025
[32]

Kimi Team, Yifan Bai, Yiping Bao, Y. Charles, Cheng Chen, Guanduo Chen, Haiting Chen, Huarong Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Chenxiao Gao, Hongcheng Gao, Peizhong Ga...
[33]

Intelligent ai delegation, 2026

Nenad Tomašev, Matija Franklin, and Simon Osindero. Intelligent ai delegation, 2026. URL https://arxiv.org/abs/2602.11865

arXiv 2026
[34]

Musique: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, pages 539–554, 2022

Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, pages 539–554, 2022

2022
[35]

Browsecomp: A simple yet challenging benchmark for browsing agents, 2025

JasonWei, ZhiqingSun, SpencerPapay, ScottMcKinney, JeffreyHan, IsaFulford, HyungWonChung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents, 2025. URLhttps://arxiv.org/abs/2504.12516

Pith/arXiv arXiv 2025
[36]

Webwalker: Benchmarking LLMs in web traversal

Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and Fei Huang. Webwalker: Benchmarking LLMs in web traversal. In Workshop on Reasoning and Planning for Large Language Models, 2025

2025
[37]

TRIDENT: Enhancing large language model safety with tri-dimensional diversified red-teaming data synthesis

Xiaorui Wu, Xiaofeng Mao, Fei Li, Xin Zhang, Xuanhong Li, Chong Teng, Donghong Ji, and Zhuang Li. TRIDENT: Enhancing large language model safety with tri-dimensional diversified red-teaming data synthesis. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15077–15099, 2025

2025
[38]

HotpotQA:Adatasetfordiverse, explainablemulti-hopquestionanswering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and ChristopherD.Manning. HotpotQA:Adatasetfordiverse, explainablemulti-hopquestionanswering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380, 2018

2018
[39]

{$\tau$}-bench: Abenchmark for \underline{T}ool-\underline{A}gent-\underline{U}ser interaction in real-world domains

ShunyuYao, NoahShinn, PedramRazavi, andKarthikRNarasimhan. {$\tau$}-bench: Abenchmark for \underline{T}ool-\underline{A}gent-\underline{U}ser interaction in real-world domains. In The Thirteenth International Conference on Learning Representations, 2025. Work in Progress 16 Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled ...

2025
[40]

Claw-eval: Towards trustworthy evaluation of autonomous agents, 2026

Bowen Ye, Rang Li, Qibin Yang, Yuanxin Liu, Linli Yao, Hanglong Lv, Zhihui Xie, Chenxin An, Lei Li, Lingpeng Kong, Qi Liu, Zhifang Sui, and Tong Yang. Claw-eval: Towards trustworthy evaluation of autonomous agents, 2026. URLhttps://arxiv.org/abs/2604.06132

Pith/arXiv arXiv 2026
[41]

Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese,

Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong, Jing Ren, Jian Chen, Chao Liu, and Yining Hua. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese,
[42]

synthetic document

Yutao Zhu, Xingshuo Zhang, Maosen Zhang, Jiajie Jin, Liancheng Zhang, Xiaoshuai Song, Kangzhi Zhao, Wencong Zeng, Ruiming Tang, Han Li, Ji-Rong Wen, and Zhicheng Dou. Gisa: A benchmark for general information-seeking assistant, 2026. URLhttps://arxiv.org/abs/2602.08543. Work in Progress 17 Delegation Intelligence in Deep Search: A Controllable Framework f...

arXiv 2026
[2025]

URLhttps://arxiv.org/abs/2504.19314

Pith/arXiv arXiv
[2026]

URLhttps://arxiv.org/abs/2507.20534

Pith/arXiv arXiv

[1] [1]

Ctrlsynth: Controllable image text synthesis for data-efficient multimodal learning

Qingqing Cao, Mahyar Najibi, and Sachin Mehta. Ctrlsynth: Controllable image text synthesis for data-efficient multimodal learning. InForty-second International Conference on Machine Learning, 2025

2025

[2] [2]

A benchmark of expert-level academic questions to assess AI capabilities.Nature, 649:1139–1146, 2026

Center for AI Safety, Scale AI, and HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities.Nature, 649:1139–1146, 2026. doi: 10.1038/s41586 -025-09962-4. URLhttps://arxiv.org/abs/2501.14249

Pith/arXiv arXiv 2026

[3] [3]

xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations, 2025

Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, Chen Sun, Han Hou, Hui Yang, James Pan, Jianan Lou, Jiayi Mao, Jizheng Liu, Jinpeng Li, Kangyi Liu, Kenkun Liu, Rui Wang, Run Li, Tong Niu, Wenlong Zhang, Wenqi Yan, Xuanzheng Wang, Yuchen Zhang, Yi-Hsin Hung, Yuan Jiang, Zexuan Liu,...

Pith/arXiv arXiv 2025

[4] [4]

Warner, Hugo Aerts, Thomas Hartvigsen, Jack Gallifant, and Danielle Bitterman

Shan Chen, Pedro José Ferreira Moreira, Yuxin Xiao, Samuel Schmidgall, Jeremy L. Warner, Hugo Aerts, Thomas Hartvigsen, Jack Gallifant, and Danielle Bitterman. Medbrowsecomp: Benchmarking medical deep research and computer use. InNeurIPS 2025 Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling, 2025

2025

[5] [5]

BrowseComp-plus: A fair and disentangled evaluation benchmark for deep search agents

ZijianChen, XueguangMa, ShengyaoZhuang, PingNie, KaiZou, SahelSharifymoghaddam, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Hosna Oyarhoseini, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, and Jimmy Lin. BrowseComp-plus: A fair and disentangled evaluation benchmark for deep search agent...

2026

[6] [6]

Beyond itinerary planning—a real-world benchmark for multi-turn and tool-using travel tasks

Xiang Cheng, Yulan Hu, Xiangwen Zhang, Lu Xu, Lide Tan, Zheng Pan, Xin Li, and Yong Liu. Beyond itinerary planning—a real-world benchmark for multi-turn and tool-using travel tasks. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29200–29251, 2026

2026

[7] [7]

Evoclaw: Evaluating AI agents on continuous software evolution

Gangda Deng, Zhaoling Chen, Zhongming Yu, Haoyang Fan, Yuhong Liu, Yuxin Yang, Dhruv Parikh, Rajgopal Kannan, Le Cong, Mengdi Wang, Qian Zhang, Viktor Prasanna, Xiangru Tang, and Xingyao Wang. Evoclaw: Evaluating AI agents on continuous software evolution. InForty-third International Conference on Machine Learning, 2026

2026

[8] [8]

Cl-bench life: Can language models learn from real-life context?, 2026

Shihan Dou, Yujiong Shen, Chenhao Huang, Junjie Ye, Jiayi Chen, Junzhe Wang, Qianyu He, Shichun Liu, Changze Lv, Jiahang Lin, Jiazheng Zhang, Ming Zhang, Shaofan Liu, Tao Ji, Zhangyue Yin, Cheng Zhang, Huaibing Xie, Jianglu Hu, Jingcheng Deng, Lincheng Li, Minda Hu, Shaolei Wang, Syrus Zhao, Weichao Wang, Yan Lei, Yang Liu, Yanling Xiao, Yiting Liu, Zenan...

Pith/arXiv arXiv 2026

[9] [9]

Cl-bench: A benchmark for context learning, 2026

Shihan Dou, Ming Zhang, Zhangyue Yin, Chenhao Huang, Yujiong Shen, Junzhe Wang, Jiayi Chen, Yuchen Ni, Junjie Ye, Cheng Zhang, Huaibing Xie, Jianglu Hu, Shaolei Wang, Weichao Wang, Yanling Xiao, Yiting Liu, Zenan Xu, Zhen Guo, Pluto Zhou, Tao Gui, Zuxuan Wu, Xipeng Qiu, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Di Wang, and Shunyu Yao. Cl-bench: A benchmar...

arXiv 2026

[10] [10]

Glm-5: from vibe coding to agentic engineering, 2026

GLM-5-Team, :, Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, Chenzheng Zhu, Congfeng Yin, Cunxiang Wang, Gengzheng Pan, Hao Zeng, Haoke Zhang, Haoran Wang, Huilong Chen, Jiajie Zhang, Jian Jiao, Jiaqi Guo, Jingsen Wang, Jingzhao Du, Jinzhu Wu, Kedong Wang, Lei Li, Lin Fan, Lucen Zho...

Pith/arXiv arXiv 2026

[11] [11]

A survey on llm-as-a-judge, 2025

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on llm-as-a-judge, 2025. URLhttps://arxiv.org/abs/2411.15594

Pith/arXiv arXiv 2025

[12] [12]

Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps

Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. InProceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, 2020

2020

[13] [13]

Step-deepresearch technical report, 2025

Chen Hu, Haikuo Du, Heng Wang, Lin Lin, Mingrui Chen, Peng Liu, Ruihang Miao, Tianchi Yue, Wang You, Wei Ji, Wei Yuan, Wenjin Deng, Xiaojian Yuan, Xiaoyun Zhang, Xiangyu Liu, Xikai Liu, Yanming Xu, Yicheng Cao, Yifei Zhang, Yongyao Wang, Yubo Shu, Yurong Zhang, Yuxiang Zhang, Zheng Gong, Zhichao Chang, Binyan Li, Dan Ma, Furong Jia, Hongyuan Wang, Jiayu L...

arXiv 2025

[14] [14]

Building a foundational guardrail for general agentic systems via synthetic data

Yue Huang, Hang Hua, Yujun Zhou, Pengcheng Jing, Manish Nagireddy, Inkit Padhi, Greta Dolcetti, Zhangchen Xu, Subhajit Chaudhury, Ambrish Rawat, Liubov Nedoshivina, Pin-Yu Chen, Prasanna Sattigeri, and Xiangliang Zhang. Building a foundational guardrail for general agentic systems via synthetic data. InThe Fourteenth International Conference on Learning R...

2026

[15] [15]

Livecodebench: Holistic and contamination free evaluation of large language models for code

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=chfJJYC3iL

2025

[16] [16]

DiningBench: A hierarchical multi-view benchmark for perception and reasoning in the dietary domain

Song Jin, Juntian Zhang, Xun Zhang, Zeying Tian, Fei Jiang, Guojun Yin, Wei Lin, Yong Liu, and Rui Yan. DiningBench: A hierarchical multi-view benchmark for perception and reasoning in the dietary domain. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 35289–35310, 2026

2026

[17] [17]

Delegation to artificial intelligence can increase dishonest behaviour.Nature, 646(8083):126–134, 2025

Nils Köbis, Zoe Rahwan, Raluca Rilla, Bramantyo Ibrahim Supriyatno, Clara Bersch, Tamer Ajaj, Jean-François Bonnefon, and Iyad Rahwan. Delegation to artificial intelligence can increase dishonest behaviour.Nature, 646(8083):126–134, 2025

2025

[18] [18]

Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation

Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies ...

2025

[19] [19]

Claw-eval-live: A live agent benchmark for evolving real-world workflows, 2026

Chenxin Li, Zhengyang Tang, Mingxin Huang, Yunlong Lin, Shijue Huang, Shengyuan Liu, Bowen Ye, Rang Li, Lei Li, Benyou Wang, and Yixuan Yuan. Claw-eval-live: A live agent benchmark for evolving real-world workflows, 2026. URLhttps://arxiv.org/abs/2604.28139

Pith/arXiv arXiv 2026

[20] [20]

From generation to judgment: Opportunities and challenges of LLM-as-a-judge

Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. From generation to judgment: Opportunities and challenges of LLM-as-a-judge. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2757–2...

2025

[21] [21]

Reportbench: Evaluating deep research agents via academic survey tasks, 2025

Minghao Li, Ying Zeng, Zhihao Cheng, Cong Ma, and Kai Jia. Reportbench: Evaluating deep research agents via academic survey tasks, 2025. URLhttps://arxiv.org/abs/2508.15804

Pith/arXiv arXiv 2025

[22] [22]

Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the Association for Computational Linguistics, pages 157–173, 2024. Work in Progress 14 Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capa...

2024

[23] [23]

Search self-play: Pushing the frontier of agent capa- bility without supervision

Hongliang Lu, Yuhang Wen, Pengyu Cheng, Ruijin Ding, Jiaqi Guo, Haotian Xu, Chutian Wang, Haonan Chen, xiaoxi jiang, and guanjunjiang. Search self-play: Pushing the frontier of agent capa- bility without supervision. InThe Fourteenth International Conference on Learning Representations, 2026

2026

[24] [24]

Natural emergent misalignment from reward hacking in production rl, 2025

Monte MacDiarmid, Benjamin Wright, Jonathan Uesato, Joe Benton, Jon Kutasov, Sara Price, Naia Bouscal, Sam Bowman, Trenton Bricken, Alex Cloud, Carson Denison, Johannes Gasteiger, Ryan Greenblatt, Jan Leike, Jack Lindsey, Vlad Mikulik, Ethan Perez, Alex Rodrigues, Drake Thomas, Albert Webson, Daniel Ziegler, and Evan Hubinger. Natural emergent misalignmen...

arXiv 2025

[25] [25]

GAIA: a benchmark for general AI assistants

Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. GAIA: a benchmark for general AI assistants. InThe Twelfth International Conference on Learning Represen- tations, 2024

2024

[26] [26]

Searchswarm: Towards delegation intelligence in agentic llms for long-horizon deep research, 2026

Pu Ning, Quan Chen, Kun Tao, Xinyu Tang, Tianshu Wang, Qianggang Cao, Xinyu Kong, Zujie Wen, Zhiqiang Zhang, and Jun Zhou. Searchswarm: Towards delegation intelligence in agentic llms for long-horizon deep research, 2026. URLhttps://arxiv.org/abs/2606.09730

Pith/arXiv arXiv 2026

[27] [27]

Measuring and narrowing the compositionality gap in language models

Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, 2023

2023

[28] [28]

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

2024

[29] [29]

Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, Z. F. Wu, Zhibin Gou, Shirong Ma, Hongxuan Tang, Yuxuan Liu, Wenjun Gao, Daya Guo, and Chong Ruan. Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition, 2025. URLhttps://ar...

Pith/arXiv arXiv 2025

[30] [30]

Jinzhu Song and Hengyu Lin. Exploring the effect of artificial intelligence intellect on consumer decision delegation: The role of trust, task objectivity, and anthropomorphism.Journal of Consumer Behaviour, 23(2):727–747, 2024

2024

[31] [31]

Recent frontier models are reward hacking.https: //metr.org/blog/2025-06-05-recent-reward-hacking/, 06 2025

Beth Barnes Sydney Von Arx, Lawrence Chan. Recent frontier models are reward hacking.https: //metr.org/blog/2025-06-05-recent-reward-hacking/, 06 2025

2025

[32] [32]

Kimi Team, Yifan Bai, Yiping Bao, Y. Charles, Cheng Chen, Guanduo Chen, Haiting Chen, Huarong Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Chenxiao Gao, Hongcheng Gao, Peizhong Ga...

[33] [33]

Intelligent ai delegation, 2026

Nenad Tomašev, Matija Franklin, and Simon Osindero. Intelligent ai delegation, 2026. URL https://arxiv.org/abs/2602.11865

arXiv 2026

[34] [34]

Musique: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, pages 539–554, 2022

Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition.Transactions of the Association for Computational Linguistics, pages 539–554, 2022

2022

[35] [35]

Browsecomp: A simple yet challenging benchmark for browsing agents, 2025

JasonWei, ZhiqingSun, SpencerPapay, ScottMcKinney, JeffreyHan, IsaFulford, HyungWonChung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents, 2025. URLhttps://arxiv.org/abs/2504.12516

Pith/arXiv arXiv 2025

[36] [36]

Webwalker: Benchmarking LLMs in web traversal

Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and Fei Huang. Webwalker: Benchmarking LLMs in web traversal. In Workshop on Reasoning and Planning for Large Language Models, 2025

2025

[37] [37]

TRIDENT: Enhancing large language model safety with tri-dimensional diversified red-teaming data synthesis

Xiaorui Wu, Xiaofeng Mao, Fei Li, Xin Zhang, Xuanhong Li, Chong Teng, Donghong Ji, and Zhuang Li. TRIDENT: Enhancing large language model safety with tri-dimensional diversified red-teaming data synthesis. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15077–15099, 2025

2025

[38] [38]

HotpotQA:Adatasetfordiverse, explainablemulti-hopquestionanswering

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and ChristopherD.Manning. HotpotQA:Adatasetfordiverse, explainablemulti-hopquestionanswering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380, 2018

2018

[39] [39]

{$\tau$}-bench: Abenchmark for \underline{T}ool-\underline{A}gent-\underline{U}ser interaction in real-world domains

ShunyuYao, NoahShinn, PedramRazavi, andKarthikRNarasimhan. {$\tau$}-bench: Abenchmark for \underline{T}ool-\underline{A}gent-\underline{U}ser interaction in real-world domains. In The Thirteenth International Conference on Learning Representations, 2025. Work in Progress 16 Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled ...

2025

[40] [40]

Claw-eval: Towards trustworthy evaluation of autonomous agents, 2026

Bowen Ye, Rang Li, Qibin Yang, Yuanxin Liu, Linli Yao, Hanglong Lv, Zhihui Xie, Chenxin An, Lei Li, Lingpeng Kong, Qi Liu, Zhifang Sui, and Tong Yang. Claw-eval: Towards trustworthy evaluation of autonomous agents, 2026. URLhttps://arxiv.org/abs/2604.06132

Pith/arXiv arXiv 2026

[41] [41]

Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese,

Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong, Jing Ren, Jian Chen, Chao Liu, and Yining Hua. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese,

[42] [42]

synthetic document

Yutao Zhu, Xingshuo Zhang, Maosen Zhang, Jiajie Jin, Liancheng Zhang, Xiaoshuai Song, Kangzhi Zhao, Wencong Zeng, Ruiming Tang, Han Li, Ji-Rong Wen, and Zhicheng Dou. Gisa: A benchmark for general information-seeking assistant, 2026. URLhttps://arxiv.org/abs/2602.08543. Work in Progress 17 Delegation Intelligence in Deep Search: A Controllable Framework f...

arXiv 2026

[43] [2025]

URLhttps://arxiv.org/abs/2504.19314

Pith/arXiv arXiv

[44] [2026]

URLhttps://arxiv.org/abs/2507.20534

Pith/arXiv arXiv