Pith. sign in

REVIEW 3 major objections 5 minor 5 cited by

Maestro: Joint Graph & Config Optimization for Reliable AI Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Jointly optimizing agent graphs and configurations beats prompt-only search on IFBench and HotpotQA.

desk verdict A plausible and well-formulated idea for joint graph+config agent optimization, but the headline numbers rest on cited baselines and withheld optimizer details. read the letter →

arxiv 2509.04642 v1 pith:67AFTUFA submitted 2025-09-04 cs.AI cs.CLcs.LGcs.SE

Wenxiao Wang , Priyatham Kattakinda , Soheil Feizi This is my paper
classification cs.AIcs.CLcs.LGcs.SE
keywords LLMagentsagentgraphoptimizationpromptconfigurationsearchjointreflectivefeedbackHotpotQAIFBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Maestro is a framework-agnostic optimizer for LLM agents that treats an agent as a typed computation graph and searches both the graph's structure (which modules exist and how they connect) and each node's configuration (prompts, model, tools, control knobs). The paper's central claim is that this joint search beats prompt-only optimizers that keep the architecture fixed: on IFBench and HotpotQA, Maestro surpasses GEPA, MIPROv2, and GEPA+Merge by averages of about 12%, 4.9%, and 4.86%, respectively, and does so with far fewer rollouts. The paper also argues that structural edits—adding an entity-extraction step, a constraints validator, an explicit state variable, or a numeric-computation tool—fix failure modes prompt tuning alone cannot, and that reflective textual feedback from traces steers the search toward those fixes. If right, the result would give practitioners a practical path to improving agent reliability without hand-rewiring graph skeletons.

What carries the argument

The central object is the typed stochastic computation graph G=(V,E): nodes carry capabilities (LLM invocation, retrieval, tool calls, memory modules, validators), edges carry adapters that transform a parent's output into a child's input, and each node has a merge operator combining incoming messages. Maestro's search alternates C-steps (fix the graph, optimize configurations) and G-steps (mutate the graph within an edit-distance trust region), accepting changes under a guarded improvement rule. Reflective non-numeric feedback from execution traces is distilled into targeted proposals, which is what lets structural edits—validators, entity extraction, persistent state, new tools—be discover

What would settle it

Re-run the HotpotQA and IFBench comparisons in one harness with identical evaluator prompts, data splits, LLM version (gpt-4.1-mini-2025-04-14), and rollout accounting for both Maestro and the three baselines; if Maestro's margins shrink to noise or Maestro needs more rollouts than GEPA, the paper's central claim would be falsified.

Watch

Extended reading notes

Core claim

Maestro casts agent design as maximizing expected utility over graph G and configuration C under rollout, cost, and structure budgets, then alternates a C-step (configuration search with the graph fixed) and a G-step (small structural edits, such as adding, removing, or rewiring nodes, with configuration fixed), using textual trace feedback to choose where to edit. On HotpotQA and IFBench, config-only Maestro reaches 70.33% and 56.12%, slightly above GEPA and GEPA+Merge; graph+config reaches 72.33% and 59.18%, with far fewer rollouts. In two applications, graph edits add an explicit state variable (interviewer completions from 2% to 92%) and a numeric tool plus retrieval guidance (RAG score

Load-bearing premise

The headline margins rest on comparing against GEPA, MIPROv2, and GEPA+Merge numbers reported in GEPA's own paper under GEPA's exact protocol, without re-running those baselines in Maestro's environment; if slight differences in evaluator prompts, LLM versions, or data sampling make the baselines non-comparable, the gains would not be established.

Editorial extensions

If this is right

  • On the two public benchmarks, Maestro's graph+config mode beats reported GEPA/MIPROv2/GEPA+Merge numbers while spending a small fraction of the rollouts, suggesting prompt-only search leaves a measurable accuracy margin on the table.
  • The interviewer result (2% to 92% complete rate) and the RAG result (39.1 to 80.4) show structural additions can turn unreliable agents into usable ones, a class of gains inaccessible to prompt tuning.
  • Since Maestro is framework-agnostic—it registers optimizable configurations and tracks traces without requiring a specific agent implementation—the same optimizer could apply across different agent frameworks and evolving applications.
  • Joint optimization with explicit rollout and token budgets gives a practical way to trade accuracy against latency and cost rather than optimizing quality alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this joint search generalizes beyond the four case studies, hand-fixing agent graphs and tuning prompts separately looks inefficient; agent development workflows could shift toward budgeted, feedback-driven joint search.
  • The interviewer agent's improvement suggests many so-called LLM mistakes in long multi-turn dialogues are really missing-state problems, so adding explicit memory registers may be a cheaper fix than fine-tuning—testable by applying the branches_done trick to other multi-turn agents.
  • A natural stress test is to weaken the evaluator's textual feedback (for example, switch to scalar-only scores); if Maestro's sample-efficiency advantage shrinks, reflective feedback is the load-bearing ingredient, and optimizers that also consume text might close the gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Maestro, a claimed framework-agnostic optimizer that jointly searches over an agent's computation graph and its node configurations (prompts, models, tools, hyperparameters), under rollout and cost budgets. It formalizes agents as typed stochastic computation graphs and proposes a block-coordinate scheme alternating a C-step (configuration optimization) and a G-step (graph structural edits), guided by both numeric evaluation scores and textual feedback from execution traces. The empirical sections report results on HotpotQA and IFBench using the evaluation protocol of GEPA/MIPROv2, plus two in-house applications (interviewer and RAG agents). The central claims are that Maestro outperforms prompt-only optimizers MIPROv2, GEPA, and GEPA+Merge on the public benchmarks, with further gains from graph edits (e.g., HotpotQA 70.33% config-only vs. 72.33% graph+config; IFBench 56.12% vs. 59.18%), and that structural changes such as entity extraction, a validation module, an explicit state variable, and a numeric compute tool fix failure modes that prompt tuning cannot.

Significance. If substantiated, the contribution would be significant: it extends agent optimization beyond prompt tuning to a structured, budgeted search over agent topology, and it uses textual trace feedback in a principled way. The formal graph/config formulation in Section 2 is a useful unifying language, and the public-benchmark experiments target a question of current interest. However, the empirical validation is not yet at the standard needed to support the headline quantitative claims. The comparisons to GEPA/MIPROv2 rely on previously reported numbers rather than matched re-runs, the IFBench agent is re-implemented rather than reused, no error bars or multiple seeds are provided, and the proprietary nature of the optimizer (Section 4) means the graph-edit results cannot be independently audited. The paper also gives credit-worthy concrete applications and reproducible-looking prompts in the appendix, but the lack of released code or search traces prevents verification of the central 'joint graph+config search discovers structural fixes' narrative.

major comments (3)
  1. [Sections 5.1.1, 5.2.1 and Figures 4–5] The headline gains against MIPROv2, GEPA, and GEPA+Merge are not established because the baselines are cited from Agrawal et al. (2025) rather than re-run in the same environment. The paper states 'we follow exactly the evaluation protocol of GEPA' and 'evaluated and reported by Agrawal et al.' but does not re-run those baselines with the same LLM API version, data sampling, evaluator prompts, or agent implementation. For IFBench the agent is explicitly 're-implemented' (Section 5.2.1), so even the underlying agent differs. Since the claimed margins are small—e.g., IFBench config-only 56.12% vs. GEPA+Merge 55.95% (a 0.17pp difference), and HotpotQA config-only 70.33% vs. GEPA 69.00% (1.33pp)—environmental differences could reverse or eliminate the advantage. The paper must either re-run the baselines under identical conditions or report the comparison as preliminary, with explicit caveat
  2. [Sections 5.1.2, 5.2.2, 5.3.2, 5.4.2] No variance information is reported for any result. All headline numbers are single-point estimates without standard errors, confidence intervals, or multiple seeds. On HotpotQA, the difference between Maestro graph+config (72.33%) and GEPA (69.00%) is close to what could plausibly arise from seed/rollout noise on a 300-example test set. Similarly, the interviewer agent's '2% to 92%' improvement rests on 50 test trajectories generated from one batch of personas, and the RAG agent uses a custom judge without inter-annotator agreement or variance estimates. At minimum, the paper should provide standard errors or bootstrapped CIs for all central claims, and ideally multiple independent runs of the optimizer.
  3. [Section 4 and Appendix A] The G-step is described only at a high level, with the footnote 'technical details remain proprietary.' The graph edits (extract_entities, validate_constraints, branches_done, numeric_compute) are presented as final outputs, but no search logs, candidate graph trajectories, or ablation of the G-step are provided. This makes it impossible to verify that these edits were discovered by Maestro's search rather than selected post hoc by the authors. Since the central narrative—that joint graph search is necessary for certain failure modes—depends on the provenance of these edits, the paper should release the search trace or, failing that, provide a reproducibility protocol (e.g., a publicly runnable minimal version with a documented random seed and search budget).
minor comments (5)
  1. [Section 5.3.1, 'LLM' bullet] Typo: 'confured' should be 'configured'.
  2. [Figures 3a and 3b] The two panels appear visually identical in the provided text; the 'highlighted in blue' distinction is not visible in the reproduction. Please ensure the figure renders the added nodes/edges clearly, and consider labeling the diff explicitly.
  3. [Abstract and Section 5.2.2] The abstract claims Maestro 'surpasses' MIPROv2/GEPA/GEPA+Merge by average margins of 12%, 4.9%, and 4.86%, but the per-benchmark numbers in Sections 5.1.2 and 5.2.2 do not obviously yield these averages, and the exact aggregation is unspecified. Clarify whether the reported percentages are per-benchmark average improvement or pooled across benchmarks.
  4. [Section 2.2] The formulation uses a structure regularizer Ω(G) and a constraint Ω(G) ≤ τ, but the practical choice of τ and how it interacts with the trust-region radius r_t in Section 4 (both bound graph change) is not discussed. A brief clarification would help.
  5. [Appendix A.4] The numeric_compute tool code is presented as a 'suggested' addition, but the appendix does not show whether the tool was actually invoked in the evaluation or only in the optimized agent. Please state explicitly which tools were used in the reported runs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains come from held-out test evaluations; the main issues are external comparability and proprietary provenance, not circular reasoning.

full rationale

Maestro's central claim is empirical: optimized configurations and graph edits achieve higher held-out test scores than MIPROv2/GEPA/GEPA+Merge on separate test splits (300 HotpotQA, 294 IFBench). The formal objective in Section 2 is not defined in terms of Maestro's outputs, and the test metrics are not used as training targets. The graph edits (extract_entities, validate_constraints, branches_done, numeric_compute) are presented as search outputs and evaluated on held-out data; even though the search is opaque ('this section presents the general formulation of the Maestro optimizer; technical details remain proprietary', Sec. 4), a proprietary optimizer is not evidence of circularity. The baselines are cited from Agrawal et al. (2025) rather than re-run, and the IFBench agent is 're-implemented' (Sec. 5.2.1), so the headline margins may be fragile or non-comparable; but that is an auditability and external-validity limitation, not a circular derivation. There are no load-bearing self-citations: the cited prior works (GEPA, MIPROv2, MAAS, Reflexion) are external, and the RELAI sandbox/platform references are ancillary benchmark-generation details. No equation, fitted parameter, or defined quantity reduces to the target result by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 3 invented entities

The paper's empirical claims rest on a set of domain assumptions about agent modeling, benchmark comparability, and judge reliability, plus several undisclosed optimizer hyperparameters. No code or data accompany the report, so the optimizer's internal free parameters cannot be audited.

free parameters (5)
  • Rollout budget split (B_t and B'_t per outer iteration) = Not reported
    Controls the reported 240/420/700/900 rollout counts; no values or scheduling rules are given (Section 4).
  • Graph trust-region radius r_t in G-step = Not reported
    Limits how far each graph mutation can move; affects which structural edits are reachable (Section 4).
  • Improvement tolerance xi_t in acceptance rule = Not reported
    Determines whether a proposed graph/config is accepted; directly shapes the reported monotone improvements (Section 4).
  • Structure budget tau = Not reported
    Penalizes graph complexity in the objective (Section 2.2); its value changes how aggressively graphs are pruned.
  • Evaluation mini-batch sizes for bJ estimates = Not reported
    The score estimator bJ uses mini-batches; the sizes affect noise and the acceptance decisions (Section 4).
assumptions (5)
  • domain assumption LLM agents can be modeled as stochastic functions on a typed DAG with adapters and merge operators
    Section 2 defines the computation graph formalism; the optimization objective assumes this representation captures relevant agent behavior.
  • domain assumption The evaluation protocol matches GEPA exactly, so previously reported baseline scores are comparable
    Sections 5.1.1 and 5.2.1 state 'we follow exactly the evaluation protocol of GEPA' and rely on reported GEPA/MIPROv2 numbers without re-running them.
  • domain assumption Reflective textual feedback from traces can be distilled into actionable graph/config edits that improve the objective
    The reflective guidance mechanism in Section 4 assumes critiques and rubric text carry signal that can be converted into targeted search moves.
  • domain assumption HotpotQA, IFBench, and the internal interviewer/RAG benchmarks are reliable proxies for agent reliability
    The conclusion generalizes from these benchmarks to 'reliable and efficient AI agents'; the internal benchmarks are generated by the authors and not externally validated.
  • domain assumption LLM judge scores (o4-mini for interviewer, custom rubric judge for RAG) reflect true task quality
    Sections 5.3.1 and 5.4.2 use LLM judges as the evaluation metric; their reliability is asserted, not independently calibrated.
invented entities (3)
  • External state variable branches_done independent evidence
    purpose: Tracks completed branches of the interviewer agent to prevent re-asking and skipping
    Introduced in the optimized graph (Section 5.3.2, Appendix A.3.3). Its benefit is evidenced by the jump from 66% to 92% on the held-out test trajectories, a falsifiable handle outside the training fit.
  • validate_constraints module independent evidence
    purpose: Checks the candidate answer against original instructions and triggers one additional rewrite if needed
    Added in the IFBench optimized graph (Figure 6b). The module is evaluated on the held-out test set; whether it generalizes is testable, so it has an independent handle.
  • numeric_compute tool independent evidence
    purpose: Performs avg/std/growth computations that the RAG agent previously did inside the LLM
    Added in the RAG optimized graph (Figure 10b). It is testable on the quantitative benchmark category, providing an independent falsifiable outcome.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Maestro: Joint Graph & Config Optimization for Reliable AI Agents." pith.science (2026). https://pith.science/paper/67AFTUFA

@misc{pith2026250904642,
  author       = {Pith},
  title        = {Pith review of: Maestro: Joint Graph & Config Optimization for Reliable AI Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/67AFTUFA}},
  note         = {Machine review of arXiv:2509.04642}
}
read the original abstract

Building reliable LLM agents requires decisions at two levels: the graph (which modules exist and how information flows) and the configuration of each node (models, prompts, tools, control knobs). Most existing optimizers tune configurations while holding the graph fixed, leaving structural failure modes unaddressed. We introduce Maestro, a framework-agnostic holistic optimizer for LLM agents that jointly searches over graphs and configurations to maximize agent quality, subject to explicit rollout/token budgets. Beyond numeric metrics, Maestro leverages reflective textual feedback from traces to prioritize edits, improving sample efficiency and targeting specific failure modes. On the IFBench and HotpotQA benchmarks, Maestro consistently surpasses leading prompt optimizers--MIPROv2, GEPA, and GEPA+Merge--by an average of 12%, 4.9%, and 4.86%, respectively; even when restricted to prompt-only optimization, it still leads by 9.65%, 2.37%, and 2.41%. Maestro achieves these results with far fewer rollouts than GEPA. We further show large gains on two applications (interviewer & RAG agents), highlighting that joint graph & configuration search addresses structural failure modes that prompt tuning alone cannot fix.

Figures

Figures reproduced from arXiv: 2509.04642 by the authors.

Figure 1
Figure 1. Maestro outperforms leading prompt optimizers with far fewer rollouts. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Maestro’s two complementary optimization steps. C-step (config update): keep the graph fixed and tune the configuration under a rollout budget. G-step (graph update): keep the configuration fixed and try small graph edits under a graph budget. Lightweight textual feedback from execution traces helps decide what to change next. which intermediate artifacts are visible to which nodes; and (iv) multi-model, multi-tool … view at source ↗
Figure 3
Figure 3. Comparison of the agent graphs on HotpotQA before and after optimization using Maestro. Nodes and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Evaluation on HotpotQA. In [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Evaluation on IFBench. In [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of the agent graphs on IFBench before and after optimization using Maestro. Nodes and edges [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 8
Figure 8. Figure 8: Comparison of the design of the interviewer agent before and after optimization using Maestro. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 7
Figure 7. Figure 7: Results of optimizing interviewer agent with [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 9
Figure 9. Figure 9: Results of optimizing RAG Agent with Maestro [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Comparison of the design of the RAG agent before and after optimization using Maestro. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference

    cs.SE 2026-07 conditional novelty 6.5 of 10

    FlowFixer uses symbolic inference of node behavioral specs to diagnose and repair agentic workflows, reaching 71.3% repair success and higher attribution accuracy than baselines on Dify/Coze/n8n failures.

  2. From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws

    cs.SE 2026-06 unverdicted novelty 6.0 of 10

    HarnessFix is a trace-guided framework that diagnoses failures in LLM agent harnesses via a new intermediate representation and applies scoped repairs, yielding 15-50% gains on four benchmarks.

  3. Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

    cs.AI 2026-07 conditional novelty 5.0 of 10

    In a two-phase Terminal-Bench evaluation, only regression-aware RELAI-VCL compounded optimization gains, reaching the highest pass rate at every stage.

  4. From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws

    cs.SE 2026-06 unverdicted novelty 5.0 of 10

    HarnessFix diagnoses harness flaws from agent traces via HTIR, maps them to repair operators, and improves benchmark performance by 6.3-18.4% over baselines.

  5. Automated Instruction Revision (AIR): A Structured Comparison of Task Adaptation Strategies for LLM

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    AIR excels on label-remapping classification tasks while KNN retrieval leads on closed-book QA and fine-tuning leads on structured extraction and event-order reasoning, showing task-dependent adaptation performance.

Reference graph

Works this paper leans on

68 extracted references · 56 canonical work pages · cited by 4 Pith papers

  1. [1]

    L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, et al. Gepa: Reflective prompt evolution can outperform reinforcement learning. arXiv preprint arXiv:2507.19457, 2025

    arXiv 2025

  2. [2]

    S. Altman. Planning for agi and beyond. https://openai.com/index/ planning-for-agi-and-beyond/ , 2023. OpenAI Blog. Accessed 2025-08-08

    work page 2023

  3. [3]

    H. Chase. Langchain, 2022. URL https://github.com/langchain-ai/langchain. Software. Accessed 2025-08-08

    work page 2022

  4. [4]

    Haystack, 2024

    deepset-ai. Haystack, 2024. URL https://github.com/deepset-ai/haystack. Software. Accessed 2025-08-08

    work page 2024

  5. [5]

    D. Gao, Z. Li, X. Pan, W. Kuang, Z. Ma, B. Qian, F. Wei, W. Zhang, Y . Xie, D. Chen, L. Yao, H. Peng, Z. Zhang, L. Zhu, C. Cheng, H. Shi, Y . Li, B. Ding, and J. Zhou. Agentscope: A flexible yet robust multi- agent platform. arXiv preprint arXiv:2402.14034 , 2024. doi: 10.48550/arXiv.2402.14034. URL https: //arxiv.org/abs/2402.14034

    arXiv doi:10.48550/arxiv.2402.14034 2024

  6. [6]

    Khattab, A

    O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts. Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714, 2023. doi: 10.48550/arXiv.2310.03714. URL https://arxiv.org/abs/2310.03714

    arXiv doi:10.48550/arxiv.2310.03714 2023

  7. [7]

    S. Liu, C. Le Clair, and C. McAllister. The state of ai agents, 2024, 2024. URL https://www.forrester. com/report/the-state-of-ai-agents-2024/RES181564 . Accessed 2025-08-08

    work page 2024

  8. [8]

    Opsahl-Ong, M

    K. Opsahl-Ong, M. J. Ryan, J. Purtell, D. Broman, C. Potts, M. Zaharia, and O. Khattab. Optimizing instructions and demonstrations for multi-stage language model programs. arXiv preprint arXiv:2406.11695 , 2024. doi: 10.48550/arXiv.2406.11695. URL https://arxiv.org/abs/2406.11695

    arXiv doi:10.48550/arxiv.2406.11695 2024

Show all 68 references
  1. [9]

    Packer, S

    C. Packer, S. Wooders, K. Lin, V . Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023. doi: 10.48550/arXiv.2310.08560. URL https: //arxiv.org/abs/2310.08560

    2023 doi arXiv

  2. [10]

    Pyatkin, S

    V . Pyatkin, S. Malik, V . Graf, H. Ivison, S. Huang, P. Dasigi, N. Lambert, and H. Hajishirzi. IF-RLVR-Train, July

  3. [11]

    Pyatkin, S

    V . Pyatkin, S. Malik, V . Graf, H. Ivison, S. Huang, P. Dasigi, N. Lambert, and H. Hajishirzi. Generalizing verifiable instruction following. arXiv preprint arXiv:2507.02833, 2025. 13 Technical Report

    2025 arXiv

  4. [12]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, and D. Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. doi: 10.48550/arXiv.2402.03300. URL https://arxiv.org/abs...

    2024 doi arXiv

  5. [13]

    Shinn, F

    N. Shinn, F. Cassano, E. Berman, A. Gopinath, K. Narasimhan, and S. Yao. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 2023. doi: 10.48550/arXiv.2303.11366. URL https://arxiv.org/abs/2303.11366

    2023 doi arXiv

  6. [15]

    H. Yang, S. Yue, and Y . He. Auto-gpt for online decision making: Benchmarks and additional opinions. arXiv preprint arXiv:2306.02224, 2023. doi: 10.48550/arXiv.2306.02224. URL https://arxiv.org/abs/ 2306.02224

    2023 doi arXiv

  7. [16]

    Z. Yang, P. Qi, S. Zhang, Y . Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600, 2018

    2018 arXiv

  8. [17]

    H. Zhou, X. Wan, R. Sun, H. Palangi, S. Iqbal, I. Vuli ´c, A. Korhonen, and S. ¨O. Arık. Multi-agent design: Optimizing agents with better prompts and topologies. arXiv preprint arXiv:2502.02533, 2025. doi: 10.48550/ arXiv.2502.02533. URL https://arxiv.org/abs/2502.02533. 14 T...

    2025 doi

  9. [20]

    Extract the raw summary text from summary_1, discarding any JSON wrapper

  10. [21]

    record company, nickname, birth date, spouse)

    Identify the target attribute or information requested by the question (e.g. record company, nickname, birth date, spouse)

  11. [22]

    Identify all proper-noun entities mentioned in the question

  12. [23]

    If the summary already supplies that attribute for the entity, do not include it

    For each entity: a. If the summary already supplies that attribute for the entity, do not include it . b. If the summary names the entity but does not supply the attribute, include the entity name and attribute keyword(s) in your query

  13. [24]

    the first single,

    If the question refers to a related sub-entity (e.g. "the first single," "the polesitter," "the singer") that is not named in the summary, combine: * the primary entity name * the sub-entity relation (e.g. "first single," "polesitter") * the attribute keyword

  14. [25]

    If the summary introduces a new proper noun essential to the question, and that new entity lacks the target attribute, include that entity and the attribute keyword

  15. [26]

    query":

    Combine all selected keywords (entity names and attribute terms) into one space- separated query string. Do not use full sentences or punctuation. Output exactly, with no extra fields or text: {"query": "<your search query>"} Examples Input: question: "What is the nickname of ...

    2005

  16. [27]

    Mentions an entity name explicitly named in the question

  17. [28]

    first single,

    Contains the key relation or attribute the question asks about (for example, "first single," "release," "nickname," etc.), even if that introduces a new proper-noun entity (e.g., a song or album title). 16 Technical Report. Rules: - Copy the fact exactly as it appears after th...

    1977

  18. [29]

    first single

    Determine the **target relation or attribute ** the question asks about (e.g. "first single", "record company", "death year", "nickname", etc.)

  19. [30]

    in the context summary, or b

    Identify all **EntityNames** that appear either a. in the context summary, or b. in passages whose fact text contains any keywords or phrases from the question ( including the target attribute terms)

  20. [31]

    - If a passage corrects or expands a context fact, **prefer** the passage text

    For each such EntityName, in the order they first appear in the question or passages, gather all relevant fact fragments: - From context and/or passages, verbatim. - If a passage corrects or expands a context fact, **prefer** the passage text

  21. [32]

    For each EntityName, join its fragments with semicolons into one entry of the form: EntityName | <fact1>; <fact2>

  22. [33]

    Join different EntityName entries with semicolons and a space

  23. [34]

    What record company released the first single off the album Ol’ Waylon?

    Do **not** add, infer, paraphrase, or omit any words; do **not** include commentary , labels, or extra fields. Example Input: question: "What record company released the first single off the album Ol’ Waylon?" context: "Ol’ Waylon | Ol’ Waylon is an album by Waylon Jennings, r...

    1977

  24. [35]

    Subject entities: every unique exact term inside square brackets in summary_1, listed in order of first appearance

  25. [36]

    or" or "and

    Answer-choice options: any proper names or candidate terms explicitly mentioned in the question (for example, items joined by "or" or "and") that are not already included from summary_1, listed in order of appearance. Exclude all other text (dates, descriptive phrases, job tit...

  26. [37]

    or"). b. All multi-word key concepts (e.g. event names, date references, album titles). c. The primary relation or attribute phrase needed to answer (e.g

    From the question, extract: a. All proper names and answer-choice options (items joined by "or"). b. All multi-word key concepts (e.g. event names, date references, album titles). c. The primary relation or attribute phrase needed to answer (e.g. "producer", " birth date", "he...

  27. [38]

    Check whether summary_1 verbatim contains each extracted proper name/answer-choice and key concept, and whether it includes the required relation phrase

  28. [39]

    If summary_1 contains all those terms and states the needed relation, output an empty string

  29. [40]

    missing answer-choice options or proper names b

    Otherwise, select the single most important missing term in this order: a. missing answer-choice options or proper names b. missing relation or attribute phrase c. missing key concepts

  30. [41]

    Starting with the missing term b

    Compose a concise **keyword** search query by: a. Starting with the missing term b. Appending up to two disambiguating words drawn exactly from the question or summary_1

  31. [42]

    yes" or

    Output only the final query text (no punctuation, full sentences, or explanations). Optimized prompt for final_answer: You are the final answer generation component in a multi-hop QA system. Given the following inputs: * question - the original user question * summary_1 - the ...

  32. [43]

    compute 8+12+5,

    Fast path If the query asks only for a calculation or filling a known fixed template (e.g. " compute 8+12+5," "list memory usage"), skip analysis and under <output> provide the exact result

  33. [44]

    Fibonacci spiral

    Complex tasks Under <think> tags: a. Extract each explicit requirement verbatim; label them Requirement 1, Requirement 2, etc. Do not split or merge. b. Recognize common domain patterns (e.g. "Fibonacci spiral" ⇒ quarter-circles of radius equal to Fibonacci numbers) and apply ...

  34. [45]

    Final output 21 Technical Report. Under <output> tags: Provide only the final answer that verbatim satisfies every requirement-using exact formatting, counts, placeholders, computed values, and standard domain interpretations-without any commentary outside <think> and <output>...

  35. [46]

    Do not split, merge, or infer extra requirements

    Extract each explicit requirement from <query>, verbatim, and label them Requirement 1, Requirement 2, etc. Do not split, merge, or infer extra requirements

  36. [47]

    Assign each requirement a type tag in brackets: [compute], [count], [format], [ structure], [pattern], or [constraint]

  37. [48]

    paragraph 5

    Detect minor contradictions (e.g. "paragraph 5" when only 4 paragraphs are requested). Apply a best-effort fallback (e.g. treat as paragraph 1) and note that fallback in your evidence rather than refusing

  38. [49]

    Fibonacci spiral

    Recognize standard domain patterns (e.g. "Fibonacci spiral" ⇒ quarter-circle arcs of radius equal to Fibonacci numbers unless arc-length is specified)

  39. [50]

    - [count]: use text.count(...) or regex to compare actual vs

    For each requirement, plan how to satisfy it and perform validation: - [compute]: show formulas and compute results. - [count]: use text.count(...) or regex to compare actual vs. required. - [format]/[structure]: apply regex or parsing to confirm separators, placeholders, JSON...

  40. [51]

    For each: Requirement N [type]: PASS - [evidence] or Requirement N [type]: FAIL - [evidence and minimal fix]

    Run all validations in a single pass. For each: Requirement N [type]: PASS - [evidence] or Requirement N [type]: FAIL - [evidence and minimal fix]

  41. [52]

    Repeat until all PASS

    If any requirement FAILs, apply the minimal fix or fallback, then revalidate that requirement. Repeat until all PASS

  42. [53]

    prompt_validate_constraints

    If after fallback a requirement remains impossible or truly contradictory, immediately output exactly: Clarification required: [brief explanation] and stop. Under <output> tags: Provide only the final answer that verbatim and exactly satisfies every requirement- correct comput...

  43. [54]

    then output exactly: OK and stop

    If <response> is exactly: I’m sorry, but I can’t assist with that request. then output exactly: OK and stop

  44. [55]

    Only enforce rules the user actually specified

    Extract every explicit formatting or content constraint from <query>. Only enforce rules the user actually specified. Possible constraint types include (but are not limited to): - First-word and last-word requirements (exact, case-sensitive) - Paragraph count and exact two-new...

  45. [56]

    exactly 2 paragraphs

    Check for logical contradictions among extracted constraints (e.g. "exactly 2 paragraphs" vs. "3 bullets in separate paragraphs"). If any exist, output exactly one bullet: - Contradiction: [brief description] and stop

  46. [57]

    Otherwise, verify each extracted constraint against <response> using these precise definitions: - Paragraphs = blocks separated by exactly two newline characters (‘\n\n‘), with no extra blank lines at start/end - Sentences = substrings ending in ‘.‘, ‘?‘, or ‘!‘ - Words = cont...

  47. [58]

    Optimized prompt for generate_response: You are a precise, policy-compliant, rule-driven assistant

    If and only if there are no violations, output exactly: OK Output only the violation bullets or OK, and nothing else. Optimized prompt for generate_response: You are a precise, policy-compliant, rule-driven assistant. For each user query:

  48. [59]

    Output only: <output>I’m sorry, but I can’t assist with that request.</output> - Do NOT output any <think> block when refusing

    Policy check - If the request violates content policy (hate, harassment, violence, illegal instructions, self-harm, defamation, etc.), immediately refuse. Output only: <output>I’m sorry, but I can’t assist with that request.</output> - Do NOT output any <think> block when refusing

  49. [60]

    sentences must be joined by hyphens with no spaces

    Constraint extraction - Parse the user’s prompt and extract every explicit requirement, including but not limited to: * First-word requirement (exact, case-sensitive) * Last-word requirement (exact, case-sensitive) * Sentence-connection rules (e.g. "sentences must be joined by...

  50. [61]

    exactly 2 paragraphs

    Consistency check - If any extracted constraints conflict or are impossible to satisfy together (e.g. "exactly 2 paragraphs" vs. "include 3 bullets in separate paragraphs"), immediately output only: <output>I’m sorry, but your instructions are contradictory: [brief description...

  51. [62]

    Word count: 28 ≤ 30 → PASS

    Planning & self-validation - Otherwise, in a single <think>...</think> block: a. Confirm the policy check passed. b. List each extracted constraint with its exact parameters. c. Define your parsing and matching rules (to be used both here and by downstream validators): - Parag...

  52. [63]

    Ensure the <output> block contains the verbatim text for downstream validation

    Final answer - In a single <output>...</output> block, provide ONLY the fully formatted answer, exactly meeting every requirement, with no additional text, commentary, or tags. Ensure the <output> block contains the verbatim text for downstream validation. Optimized prompt for...

  53. [64]

    then output exactly: <output>I’m sorry, but I can’t assist with that request.</output> and stop

    If <response> is exactly: I’m sorry, but I can’t assist with that request. then output exactly: <output>I’m sorry, but I can’t assist with that request.</output> and stop. Do NOT output a <think> section

  54. [65]

    Otherwise, in a single <think>...</think> block: 25 Technical Report. a. Extract every explicit formatting or content constraint from <query> (e.g. first -word, last-word, exact paragraph count and ‘\n\n‘ separators, no extra blank lines, bullet style/count, wrap with quotes, ...

  55. [66]

    Budgeting

    In a single <output>...</output> block: - If the <think> block contains only OK, reproduce the original <response> exactly, preserving all whitespace and formatting. - Otherwise, provide a minimally edited, fully corrected version of the answer that fixes all listed violations...

    2024

  56. [67]

    On the user’s answer to Q1: - record answers["Q1"] - determine branch; visited_branches.append(branch) - call ask_branch(branch)

  57. [68]

    ask_branch(branch): follow that branch’s exact sequence, recording each answer in answers

  58. [69]

    [SUMMARY]

    After a branch ends: if visited_branches.length < 5: next_branch = first entry in branch_order not in visited_branches visited_branches.append(next_branch) call ask_branch(next_branch) else: ask Q30; record; ask Q31; record; ask Q32; record then output: [SUMMARY] <concise para...

    2024

  59. [2023]

    URL https://arxiv.org/abs/2308.11432

    arXiv

  60. [2025]

    URL https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.