Pith. sign in

REVIEW 5 major objections 5 minor 38 references

SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-agent feedback loop can generate system prompts that perform nearly as well as opaque soft prompts while staying human-readable.

desk verdict A cleanly framed prompt-optimization framework with an unverifiable central empirical claim; the self-referential judge issue is real and load-bearing. read the letter →

arxiv 2507.03223 v1 pith:KPNCDLRS submitted 2025-07-03 cs.AI cs.LG

Jeshwanth Challagundla , Mantek Singh , Siddharth Raina , Smarth Behl , FNU Harsh , Jasmin Jarsania This is my paper
classification cs.AIcs.LG
keywords systeminstructionspromptspromptoptimizationlargelanguagemodelsmulti-agentsystemshuman-readablefeedback-drivenLLM-as-a-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that writing system instructions for a large language model can be automated by a trio of cooperating agents: one drafts the instruction, one executes the task under it, and one scores the result and optionally scores how readable the instruction is. The claim is that this feedback loop converges on instructions that are both effective on the task and genuinely readable by humans, giving users much of the benefit of automatic prompt optimization without losing the ability to inspect, debug, and trust the prompt. The authors validate this on reasoning, coding, style-transfer, and tool-use benchmarks, comparing against manual prompts, two readable-prompt automation baselines, and one non-readable soft-prompt baseline. Their headline finding is that the agent-written prompts land close to the soft-prompt baseline in task accuracy while scoring higher on readability than manual prompts and the readable baselines. The paper also states plainly that the whole scheme depends on the reliability of the feedback/evaluation agent and that computational cost is high.

What carries the argument

The load-bearing mechanism is the iterative feedback loop itself, run by three specialized agents. The Instructor Agent proposes and revises discrete natural-language SIs; the Instruction Follower Agent is the target LLM treated as a black box; the Feedback/Reward Agent scores the follower's output on task metrics and optionally scores the SI's readability using formulaic metrics such as Flesch Reading Ease or a second language model asked to rate quality. The feedback signal, whether scores, critiques, or preference labels, is fed back into the Instructor's refinement strategy, so optimization pressure is applied to the text of the instruction rather than to model weights or continuous vectors. Readability enters the objective explicitly when the Feedback Agent returns a readability component, and the Instructor can weight it together with task performance.

What would settle it

Run SI-Agent twice on the same task, once with the Feedback Agent's real scores and once with those scores replaced by random values matched in distribution; if the random-feedback run produces SIs with the same or better task accuracy and readability after the same number of iterations, then the feedback loop is not the cause of the reported gains. A more direct check is to collect human readability ratings and task-quality labels for a sample of intermediate SIs and measure whether the Feedback Agent's scores predict those labels; near-zero correlation would falsify the claim that feedback guides refinement.

Watch

Extended reading notes

Core claim

SI-Agent is a three-agent loop in which an Instructor Agent maintains a natural-language system instruction, an Instruction Follower Agent runs the task under that instruction, and a Feedback/Reward Agent returns a structured signal combining task performance and, when enabled, readability. The Instructor consumes this signal through an LLM-based refinement meta-prompt (the implementation used in the main experiments) and produces the next instruction candidate. The paper reports that on GSM8K this loop reaches 79.5% accuracy with a Flesch Reading Ease of 67.4, compared with 74.2% and 62.3 for a manually written prompt, 78.5% and 58.1 for the readable-prompt baselines, and 82.1% for the non-readable prompt-tuning baseline; on HumanEval it reaches 60.8% pass@1 with a human readability rating of 4.3 out of 5. The claim is that these numbers demonstrate a favorable performance-interpretability trade-off: the framework sacrifices a little peak accuracy relative to continuous prompts in exchange for instructions that are more readable than both manual SIs and the automated readable baselines.

Load-bearing premise

The load-bearing premise is that the Feedback Agent's scores for task quality and readability are reliable enough to push the Instructor's edits in the right direction; the paper itself says the framework's success depends on this feedback and acknowledges LLM-judge biases and metric imperfections, but it does not measure feedback reliability directly.

Editorial extensions

If this is right

  • System-instruction tuning becomes an automated, inspectable step: the artifact being optimized is prose, so a human can read, edit, and audit the final instruction.
  • Readability can be treated as an explicit optimization objective rather than an afterthought; the paper's ablations indicate that removing readability feedback lowers readability scores.
  • The modular design means the Instructor's search strategy and the Feedback mechanism can be swapped independently, for example LLM-based editing, evolutionary search, automated metrics, LLM judging, or human ratings, without changing the loop's structure.
  • Because the follower is treated as a black box, the same loop should transfer to API-based models where gradients are unavailable.
  • As with other prompt optimizers, the resulting SIs are tied to the model used during optimization; the paper reports only moderate transfer to other follower LLMs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if judge reliability is the bottleneck, a cheap robustness test is to shuffle or randomize the feedback scores during optimization; a loop that still converges would show the Instructor is mostly guessing rather than responding to signal.
  • Editorial extension: the same generate-execute-evaluate loop could be pointed at other artifacts meant to be read by humans, such as few-shot exemplars, output-format specifications, or tool-use protocols.
  • Editorial extension: readability formulas such as Flesch Reading Ease are shallow proxies; a practical variant could build a small set of human-rated preference pairs and use them to calibrate the readability signal before running the loop, which the paper does not do.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SI-Agent, a multi-agent framework for automatically generating and iteratively refining human-readable system instructions (SIs) for LLMs. The framework consists of an Instructor Agent that proposes textual SIs, an Instruction Follower Agent (the target LLM) that executes tasks under a given SI, and a Feedback/Reward Agent that scores task output and optionally SI readability; the loop repeats until a stopping criterion. The authors describe the architecture, discuss related prompt-optimization methods, and present Table III as experimental evidence that SI-Agent achieves competitive task performance while producing more readable SIs than manual prompting, APE/OPRO, and a continuous prompt-tuning baseline. The paper also lists hypotheses H1-H5, discusses implementation details, and acknowledges limitations around computational cost and feedback reliability.

Significance. If the central claim were empirically established, SI-Agent would be a useful contribution: it targets a real gap in automated prompt optimization, namely producing discrete, human-readable instructions that maintain competitive task performance, and the modular three-agent design is a reasonable organizing principle. The paper also provides a broad and mostly accurate organization of the related work (soft prompts, discrete optimization, evolutionary methods, RL-based methods, and agentic frameworks). However, the significance of the contribution is entirely contingent on the experimental validation, which as presented is not sufficient to support the headline trade-off claim. The paper currently reads as a detailed proposal with a placeholder result table rather than a complete empirical study, and the missing artifacts (code, data, prompts, error bars, statistical details) make the central claim impossible to verify or falsify from the manuscript alone.

major comments (5)
  1. [Section VI-B, Table III] Table III reports single scalar values for each method and metric with no standard deviations, confidence intervals, number of runs, seeds, or evaluation subset sizes. The central claims of hypothesis H1 (effectiveness), H2 (readability), and H3 (favorable comparative trade-off) rest entirely on this table, so without variance information the reader cannot tell whether the differences (e.g., GSM8K accuracy 79.5 for SI-Agent vs. 78.5 for APE/OPRO, or FRE 67.4 vs. 62.3 for Manual SI) are meaningful or within run-to-run noise. The manuscript must report these statistics or clearly state that only one run was performed, in which case the empirical claim is not substantiated.
  2. [Section VI-C] The Analysis Plan asserts that 'appropriate statistical tests confirmed the significance of key differences' but provides no test names, test statistics, p-values, effect sizes, or correction for multiple comparisons. Similarly, 'scatter plots (not shown here)', 'Learning curves showed convergence', and 'Ablations ... generally led to lower readability scores' are referenced as if they were results, but they are not included in the paper. These statements are not evidence; the actual plots, learning curves, statistical outputs, and ablation tables must be presented in the manuscript or in a supplementary document.
  3. [Section VII-B and Table III] The Feedback Agent is instantiated from the Llama 3 family, the same model family as the Follower Agent, and the framework's success is explicitly conceded to depend on the Feedback Agent. The paper reports no measurement of judge reliability, such as agreement with human readability ratings, inter-judge consistency, calibration, or sensitivity to prompt design. Consequently, the readability gains in Table III (e.g., FRE 67.4 vs. 62.3 on GSM8K; human-style 4.3 vs. 3.9 on HumanEval) and the near-baseline task accuracy are exactly the pattern that would arise from optimizing the Instructor's outputs toward the judge's stylistic preferences rather than toward genuine human readability or task quality. The authors need to provide human evaluation of the final SIs on a held-out set, or at minimum evaluate with an independently sourced judge model, to rule out self-referential overfitting.
  4. [Section V-D, Section V-E, and Table III] APE and OPRO are merged into a single column 'APE/OPRO' in Table III with no separate results for each baseline, no description of how the methods were adapted to produce the same SI representation, and no reporting of their optimization budgets (iterations, number of samples, cost). The Prompt Tuning baseline is listed but the implementation is not described beyond 'not implemented in this phase' in Section V-D (which appears to contradict its presence in Table III); moreover, the paper does not explain how a continuous soft prompt is evaluated for the task metrics while readability is marked N/A. Without a clear and comparable baseline protocol, the comparative claims in H3 cannot be evaluated.
  5. [Section VI-B and Section VI-C (H4, H5)] Hypothesis H4 (efficiency) and H5 (ablation component impact) are listed as tested, but the paper reports no efficiency numbers (wall-clock time, token usage, iterations to convergence) and no ablation results beyond the unsupported statement that removing readability feedback 'generally led to lower readability scores'. Since Sections VII-B and VIII-A further depend on these claims (e.g., 'efficiency gains over manual tuning were observed'), the absence of the underlying data makes those conclusions unfounded. The ablation and efficiency results must be reported in detail or removed from the claims.
minor comments (5)
  1. [Table III caption] The table caption contains the typo 'GY AFC' for the GYAFC dataset; please correct it.
  2. [Section III-A and Figure 1] Figure 1 is described in the text but appears to be missing from the manuscript or is not clearly placed; ensure the figure is present and referenced in Section III-A where the architecture is introduced.
  3. [Section V-E] The implementation details state that optimization ran for a fixed number of iterations (e.g., 20-50), but Table III and the results section do not report the actual number of iterations, batch sizes, or total LLM calls used for each task; these details are needed to interpret the efficiency claims.
  4. [References] Several references are non-archival blog posts or documentation pages (e.g., [3], [4], [7], [11], [15], [17], [20]); for a journal submission, please prefer primary sources or provide archived versions of these resources.
  5. [Throughout the manuscript] There are formatting issues such as 'F eedbacki', 'SIi', and 'T askinput' with spurious spaces; please run a LaTeX pass to fix these token spacing artifacts.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: reported task and readability metrics are external to the optimization loop.

full rationale

SI-Agent is an empirical framework paper rather than a formal derivation. The central claim (favorable performance/readability trade-off) is supported by Table III, whose task metrics (accuracy, pass@1, BLEU, EM) are standard external benchmarks evaluated on held-out test sets. Readability is reported via Flesch Reading Ease and human ratings, both of which are distinct from the LLM-as-a-Judge readability proxy used inside the optimization loop (Section V-E); the paper does not substitute its own optimization target as the final evidence. The only self-citations ([12], [19], [28]) appear in related-work and future-work contexts and are not load-bearing for the main claim. The acknowledged dependence on the Feedback Agent (Section VII-B) raises a validity and reproducibility concern about optimizing toward a Llama-3 judge's preferences, but no reported equation or metric is defined in terms of the quantity it is used to establish. Therefore there is no demonstrated circular step.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical, so the ledger records the tuning choices and domain assumptions the reported results depend on; all are unquantified in the paper.

free parameters (3)
  • optimization iterations = unreported (20-50 depending on task)
    The number of refinement cycles is a free choice that affects the final SI and reported performance; no sensitivity analysis is provided.
  • feedback combination weight = unreported
    Section V.E says feedback combined automated metrics and LLM-as-a-Judge scores using a 'simple weighted average where applicable', but the weights are not specified.
  • evaluation subset size = unreported
    The number of examples used per feedback step and for the final evaluation is not stated, making the reported metric values uninterpretable.
assumptions (4)
  • domain assumption Validation-set task performance is a reliable proxy for held-out test performance.
    The optimization loop selects SIs based on validation performance and then reports test values; no validation-to-test correlation is shown.
  • domain assumption Flesch Reading Ease and LLM-as-a-Judge scores are valid measures of human readability.
    The paper uses FRE and LLM judge scores to claim readability (Section V.C), but Section VII.B admits 'Formulaic metrics like FRE provided a useful signal but qualitative human assessment remains important'.
  • ad hoc to paper The Feedback Agent's signal is reliable enough to drive iterative improvement.
    The paper relies on this without measuring reliability, while Section VII.B says 'The framework's success depends on the Feedback Agent' and admits sensitivity to judge limitations.
  • domain assumption The target LLM is treated as a stable black-box function during optimization.
    The framework assumes the Follower LLM's behavior is stationary across iterations so that score changes are attributable to SI changes; no variance control is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Models." pith.science (2026). https://pith.science/paper/KPNCDLRS

@misc{pith2026250703223,
  author       = {Pith},
  title        = {Pith review of: SI-Agent: An Agentic Framework for Feedback-Driven Generation and Tuning of Human-Readable System Instructions for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KPNCDLRS}},
  note         = {Machine review of arXiv:2507.03223}
}
read the original abstract

System Instructions (SIs), or system prompts, are pivotal for guiding Large Language Models (LLMs) but manual crafting is resource-intensive and often suboptimal. Existing automated methods frequently generate non-human-readable "soft prompts," sacrificing interpretability. This paper introduces SI-Agent, a novel agentic framework designed to automatically generate and iteratively refine human-readable SIs through a feedback-driven loop. SI-Agent employs three collaborating agents: an Instructor Agent, an Instruction Follower Agent (target LLM), and a Feedback/Reward Agent evaluating task performance and optionally SI readability. The framework utilizes iterative cycles where feedback guides the Instructor's refinement strategy (e.g., LLM-based editing, evolutionary algorithms). We detail the framework's architecture, agent roles, the iterative refinement process, and contrast it with existing methods. We present experimental results validating SI-Agent's effectiveness, focusing on metrics for task performance, SI readability, and efficiency. Our findings indicate that SI-Agent generates effective, readable SIs, offering a favorable trade-off between performance and interpretability compared to baselines. Potential implications include democratizing LLM customization and enhancing model transparency. Challenges related to computational cost and feedback reliability are acknowledged.

Figures

Figures reproduced from arXiv: 2507.03223 by the authors.

Figure 1
Figure 1. Architecture of the SI-Agent Framework. The Instructor Agent generates SI [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 15 canonical work pages

  1. [1]

    Wei, et al., ”Chain-of-thought prompting elicits reasoning in large language models,” arXiv preprint arXiv:2201.11903 , 2022

    J. Wei, et al., ”Chain-of-thought prompting elicits reasoning in large language models,” arXiv preprint arXiv:2201.11903 , 2022

    arXiv 2022

  2. [2]

    Brown, et al., ”Language models are few-shot learners,” Advances in neural information processing systems , vol

    T. Brown, et al., ”Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877-1901, 2020

    work page 1901

  3. [3]

    Available: https: //cloud.google.com/discover/what-is-prompt-engineering

    Google Cloud, ”What is prompt engineering?” [Online]. Available: https: //cloud.google.com/discover/what-is-prompt-engineering

    work page

  4. [4]

    Avail- able: https://portkey.ai/blog/the-complete-guide-to-prompt-engineering

    Portkey.ai, ”The Complete Guide to Prompt Engineering,” [Online]. Avail- able: https://portkey.ai/blog/the-complete-guide-to-prompt-engineering

    work page

  5. [5]

    Available: https://latitude-blog.ghost.io/blog/ common-llm-prompt-engineering-challenges-and-solutions/

    Latitude Blog, ”Common LLM Prompt Engineering Challenges and Solutions,” [Online]. Available: https://latitude-blog.ghost.io/blog/ common-llm-prompt-engineering-challenges-and-solutions/

    work page

  6. [6]

    Yang, et al., ”Harnessing the power of LLMs in planning: A survey of techniques and applications,” arXiv preprint arXiv:2311.05661 , 2023

    J. Yang, et al., ”Harnessing the power of LLMs in planning: A survey of techniques and applications,” arXiv preprint arXiv:2311.05661 , 2023

    arXiv 2023

  7. [7]

    Avail- able: https://portkey.ai/blog/what-is-automated-prompt-engineering

    Portkey.ai, ”What is Automated Prompt Engineering?” [Online]. Avail- able: https://portkey.ai/blog/what-is-automated-prompt-engineering

    work page

  8. [8]

    Fernando, et al., ”Promptbreeder: Self-referential self-improvement via prompt evolution,” arXiv preprint arXiv:2309.16797 , 2023

    C. Fernando, et al., ”Promptbreeder: Self-referential self-improvement via prompt evolution,” arXiv preprint arXiv:2309.16797 , 2023

    arXiv 2023

Show all 38 references
  1. [9]

    Zhou, et al., ”Large language models are human-level prompt engineers,” arXiv preprint arXiv:2211.01910 , 2022

    Y . Zhou, et al., ”Large language models are human-level prompt engineers,” arXiv preprint arXiv:2211.01910 , 2022

    2022 arXiv

  2. [10]

    Yang, et al., ”Large language models as optimizers,” arXiv preprint arXiv:2309.03409, 2023

    C. Yang, et al., ”Large language models as optimizers,” arXiv preprint arXiv:2309.03409, 2023

    2023 arXiv

  3. [11]

    Avail- able: https://www.promptingguide.ai/techniques/ape

    Prompting Guide, ”Automatic Prompt Engineer (APE),” [Online]. Avail- able: https://www.promptingguide.ai/techniques/ape

  4. [12]

    Challagundla, K

    J. Challagundla, K. Tyagi, T. Chugh, and M. Manry, ”Multiple Gain Adap- tations for Improved Neural Networks Training,” in 2024 IEEE 14th An- nual Computing and Communication Workshop and Conference (CCWC) , 2024, pp. 0414-0420. doi: 10.1109/CCWC60891.2024.10427781

    2024

  5. [13]

    Chang, et al., ”Efficient Prompting Methods for Large Language Models: A Survey,” arXiv preprint arXiv:2404.01077 , 2024

    K. Chang, et al., ”Efficient Prompting Methods for Large Language Models: A Survey,” arXiv preprint arXiv:2404.01077 , 2024

    2024 arXiv

  6. [14]

    X. L. Li and P. Liang, ”Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190 , 2021

    2021 arXiv

  7. [15]

    Available: https:// learnprompting.org/docs/trainable/prefix-tuning

    Learn Prompting, ”Prefix-Tuning,” [Online]. Available: https:// learnprompting.org/docs/trainable/prefix-tuning

  8. [16]

    Liu, et al., ”P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv preprint arXiv:2110.07602, 2021

    X. Liu, et al., ”P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv preprint arXiv:2110.07602, 2021

    2021 arXiv

  9. [17]

    Available: https://aclanthology.org/2021.emnlp-main

    ACL Anthology, ”The Power of Scale for Parameter-Efficient Prompt Tuning,” [Online]. Available: https://aclanthology.org/2021.emnlp-main. 243/. (Corresponds to Lester et al., 2021)

    2021

  10. [18]

    Deng, et al., ”RLPrompt: Optimizing discrete text prompts with reinforcement learning,” arXiv preprint arXiv:2205.12548 , 2022

    M. Deng, et al., ”RLPrompt: Optimizing discrete text prompts with reinforcement learning,” arXiv preprint arXiv:2205.12548 , 2022

    2022 arXiv

  11. [20]

    Available: https://blog.ml.cmu.edu/2023/02/24/ rlprompt-optimizing-discrete-text-prompts-with-reinforcement-learning/

    CMU ML Blog, ”RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning,” [On- line]. Available: https://blog.ml.cmu.edu/2023/02/24/ rlprompt-optimizing-discrete-text-prompts-with-reinforcement-learning/. Accessed: Feb. 24, 2023

    2023

  12. [21]

    S. S. S. Das, et al., ”GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers,” arXiv preprint arXiv:2412.09722, 2024

    2024 arXiv

  13. [22]

    Wang, et al., ”A survey on large language model based autonomous agents,” arXiv preprint arXiv:2308.11432 , 2023

    L. Wang, et al., ”A survey on large language model based autonomous agents,” arXiv preprint arXiv:2308.11432 , 2023

    2023 arXiv

  14. [23]

    Wu, et al., ”AutoGen: Enabling next-gen LLM applications via multi- agent conversation framework,” arXiv preprint arXiv:2308.08155 , 2023

    Q. Wu, et al., ”AutoGen: Enabling next-gen LLM applications via multi- agent conversation framework,” arXiv preprint arXiv:2308.08155 , 2023

    2023 arXiv

  15. [24]

    Available: https://microsoft.github.io/autogen/0.2/docs/Use-Cases/agent chat/

    Microsoft AutoGen Docs, ”Agent Chat Use Cases,” [Online]. Available: https://microsoft.github.io/autogen/0.2/docs/Use-Cases/agent chat/

  16. [25]

    Singla, et al., ”Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models,” arXiv preprint arXiv:2411.08733, 2024

    S. Singla, et al., ”Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models,” arXiv preprint arXiv:2411.08733, 2024

    2024 arXiv

  17. [26]

    Bai, et al., ”Constitutional AI: Harmlessness from AI feedback,” arXiv preprint arXiv:2212.08073, 2022

    Y . Bai, et al., ”Constitutional AI: Harmlessness from AI feedback,” arXiv preprint arXiv:2212.08073, 2022

    2022 arXiv

  18. [27]

    Kojima, et al., ”Large language models are zero-shot reasoners,” Advances in neural information processing systems , vol

    T. Kojima, et al., ”Large language models are zero-shot reasoners,” Advances in neural information processing systems , vol. 35, pp. 22199- 22213, 2022

    2022

  19. [28]

    Challagundla, M

    J. Challagundla, M. Singh, V . Tiwari, and S. Raina, ”Second Order Neural Network Optimization: Meta Analysis,” in 2024 International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) , 2024, pp. 1865-1874. doi: 10.1109/ICICML63543.2024.10958004

    2024

  20. [29]

    Guo, et al., ”Connecting large language models with evolution- ary algorithms yields powerful prompt optimizers,” arXiv preprint arXiv:2309.08532, 2023

    Q. Guo, et al., ”Connecting large language models with evolution- ary algorithms yields powerful prompt optimizers,” arXiv preprint arXiv:2309.08532, 2023

    2023 arXiv

  21. [30]

    Tang, et al., ”STRAGO: Strategy-Guided Reflection for Effective Prompt Optimization,” arXiv preprint arXiv:2410.08601 , 2024

    Y . Tang, et al., ”STRAGO: Strategy-Guided Reflection for Effective Prompt Optimization,” arXiv preprint arXiv:2410.08601 , 2024

    2024 arXiv

  22. [31]

    Pryzant, et al., ”Automatic prompt optimization with large language models,” arXiv preprint arXiv:2305.03495 , 2023

    R. Pryzant, et al., ”Automatic prompt optimization with large language models,” arXiv preprint arXiv:2305.03495 , 2023

    2023 arXiv

  23. [32]

    Honovich, et al., ”Instruction induction: From few examples to natural language task descriptions,” arXiv preprint arXiv:2205.10782 , 2022

    O. Honovich, et al., ”Instruction induction: From few examples to natural language task descriptions,” arXiv preprint arXiv:2205.10782 , 2022

    2022 arXiv

  24. [33]

    Hsieh, et al., ”Automatic Prompt Engineering for Large Language Models with Human-in-the-Loop,” arXiv preprint arXiv:2311.13123 , 2023

    C. Hsieh, et al., ”Automatic Prompt Engineering for Large Language Models with Human-in-the-Loop,” arXiv preprint arXiv:2311.13123 , 2023

    2023 arXiv

  25. [34]

    Available: https://aclanthology.org/ 2022.emnlp-main.222.pdf

    ACL Anthology, ”RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning,” [Online]. Available: https://aclanthology.org/ 2022.emnlp-main.222.pdf. (Corresponds to Deng et al., 2022)

    2022

  26. [35]

    Available: https://arxiv.org/ abs/2309.06553

    arXiv, ”Prompt-OIRL: Query-Dependent Prompt Optimization via Offline Inverse Reinforcement Learning,” [Online]. Available: https://arxiv.org/ abs/2309.06553

    arXiv

  27. [36]

    Available: https://openreview.net/forum?id=fWRBheSJth

    OpenReview, ”GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers,” [Online]. Available: https://openreview.net/forum?id=fWRBheSJth

  28. [37]

    Kim, et al., ”Prefix-Tuning Preserves Representation Space,” arXiv preprint arXiv:2411.00029, 2024

    J. Kim, et al., ”Prefix-Tuning Preserves Representation Space,” arXiv preprint arXiv:2411.00029, 2024

    2024 arXiv

  29. [38]

    Han, et al., ”LLM Multi-Agent Systems: Challenges and Open Problems,” arXiv preprint arXiv:2402.03578 , 2024

    S. Han, et al., ”LLM Multi-Agent Systems: Challenges and Open Problems,” arXiv preprint arXiv:2402.03578 , 2024

    2024 arXiv

  30. [39]

    Tian, et al., ”AutoAgents: A Framework for Automatic Agent Generation,” arXiv preprint arXiv:2309.17288 , 2023

    Y . Tian, et al., ”AutoAgents: A Framework for Automatic Agent Generation,” arXiv preprint arXiv:2309.17288 , 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.