Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Eliciting Language Model Behaviors with Investigator Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that training "investigator" language models with supervised fine-tuning, DPO, and an iterative Frank-Wolfe diversity objective turns prompt search into an amortized inference problem, yielding natural-language prompts…

desk verdict A solid empirical method for automated red-teaming with a clever diversity trick, but the Frank-Wolfe theory is not actually established; treat the method as a heuristic and the paper as worth revising. read the letter →

arxiv 2502.01236 v1 pith:V2AGDCTI submitted 2025-02-03 cs.LG cs.AIcs.CL

Xiang Lisa Li , Neil Chowdhury , Daniel D. Johnson , Tatsunori Hashimoto , Percy Liang , Sarah Schwettmann , Jacob Steinhardt This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords behaviorelicitationinvestigatormodelsamortizedinferencedirectpreferenceoptimizationFrank-Wolfejailbreakinghallucinationredteaming
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the hard problem of finding prompts that make a language model exhibit a specified behavior can be amortized: instead of optimizing a new prompt for every target, one trains a single "investigator" model to map any target behavior to a distribution of effective prompts. The authors argue that this is essentially posterior inference over prompts, and they build a three-stage pipeline—supervised fine-tuning on inverted prompt–response pairs, DPO for reward, and a Frank-Wolfe-style iterative step that penalizes already-discovered strategies—to make the search both effective and diverse. If the claim holds, automated red-teaming tools could move from per-instance adversarial searches toward general-purpose investigator agents that surface human-readable jailbreaks, hallucinations, and other failures across many models. The headline results are a 100% attack success rate on a 50-item subset of AdvBench Harmful Behaviors, 98.8% on harmful strings, an 85% hallucination elicitation rate, and 81% on aberrant-behavior rubrics.

What carries the argument

The load-bearing objects are the investigator model $p_\theta(x|y)$ and its objective $\mathcal{L}(p_\theta)=R(p_\theta)+\beta_1 H(p_\theta)-\beta_2 H(p_\theta,p_m)$, where the reward $R(p_\theta)$ is the expected log-probability the target model assigns to the desired behavior. The paper's novel algorithmic piece is the iterative Frank-Wolfe step: at round $i$, a fresh model $q^{(i)}$ is trained by DPO with reward $\log p_m(y|x)-\lambda\log p_\theta^{(i-1)}(x|y)$, and the aggregate $p_\theta^{(i)}$ is a convex combination of the previous aggregate and $q^{(i)}$; this is the mechanism that penalizes strategies already found and forces new elicitation strategies to appear. Rubric-based elicitation is reduced to the same machinery by splitting it into two string-elicitation stages: first infer a response $y$ that satisfies a rubric $(R,a)$ under a verifier $p_v(a|R,y)$, then infer a prompt $x$ that elicits that $y$ from the target.

What would settle it

Run the pipeline with the diversity penalty coefficient $\lambda$ set to zero (so $\beta_1=\beta_2$) while keeping the aggregation step; if the diversity of sampled prompts still increases across iterations, the improvement cannot be attributed to the penalization mechanism the Frank-Wolfe derivation relies on. A second direct check is to track the penalty term $\log p_\theta^{(i-1)}(x|y)$ on prompts sampled from the next iterate: if it does not decrease relative to the previous iterate, the iterative procedure is not actually implementing the claimed regularization.

Watch

Extended reading notes

Core claim

The central discovery is that behavior elicitation can be cast as amortized Bayesian inference: for a target response $y$, the optimal investigator distribution $p_\theta(x|y)$ should approximate the target model's posterior $p_m(x|y) \propto p_m(y|x)p_m(x)$, so prompts are sampled rather than optimized per instance. Training starts from a forward-data SFT stage, where $(x,y)$ pairs drawn from the target model teach the investigator to invert generation; DPO then pushes sampled prompts toward high $p_m(y|x)$; and an iterative Frank-Wolfe update replaces the usual KL regularizer with a penalty against the aggregate of past investigator models, forcing each iteration to find qualitatively new strategies. The paper shows this pipeline on Llama-3.1 8B and reports near-perfect or very high success on four elicitation tasks, with the Frank-Wolfe aggregate dominating both SFT and plain DPO in the joint metric of reward plus diversity.

Load-bearing premise

The claimed diversity guarantee rests on the assumption that the preference-training update with the modified reward closely approximates the mathematically derived regularized search step, so that each iteration really is a Frank-Wolfe step rather than just another round of preference training.

Editorial extensions

If this is right

  • Different Frank-Wolfe iterations yield qualitatively different elicitation strategies (repetition, topic-heading, continuation, role-play), so the aggregate investigator covers a wider strategy space than a single DPO model.
  • The same trained investigator generalizes to new target behaviors without per-behavior optimization; only the behavior distribution $P_{RL}$ and the reward signal change.
  • Attack success can be further boosted by best-of-$k$ reranking over sampled prompts, and by aggregating across iterations (hallucination ASR rises from 85% to 93% with 75 total examples).
  • The learned prompts transfer beyond the exact target model, including partial transfer to closed-weight models such as GPT-4o and Claude 3.5, suggesting that discovered strategies are not overfit to one model's surface form.
  • The method works with a 1B-parameter investigator against an 8B target, so the approach scales down in investigator size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the amortization claim generalizes, the same recipe could turn any generative model with tractable forward sampling into an elicitation target, so the approach might be applied to multimodal or code-generation models rather than only text LMs.
  • A testable extension is to use the investigator loop as an automatic benchmark generator: iteratively eliciting failures from a frontier model could map its capability and alignment boundary more completely than fixed datasets, provided the verifier is made robust against reward hacking.
  • The paper's observed reward hacking against LM judges suggests that verifier quality, not prompt-search power, may become the bottleneck as investigators improve; substituting tool-augmented or externally grounded verifiers should reduce ASR if the hacking is the source of the high scores.
  • The Frank-Wolfe diversity mechanism is not specific to elicitation; the same penalty against an aggregate policy could be dropped into alignment pipelines to preserve output diversity during DPO training, which would be a direct test of whether the diversity guarantee holds outside this task.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies behavior elicitation: training an 'investigator' language model to propose prompts that induce a specified target behavior (exact string match or rubric-satisfying response) from a fixed target language model. Training proceeds through supervised fine-tuning on inverted prompt-response pairs, DPO on elicitation reward, and an iterative diversity penalty that the paper claims is equivalent to Frank-Wolfe optimization and asymptotically approximates Bayesian posterior sampling. The experiments cover pretraining string elicitation, AdvBench harmful strings and behaviors, TruthfulQA-based hallucination elicitation, and DSM-5-based aberrant behavior elicitation, reporting best-of-25 attack success rates of 98.8% and 100% on the two AdvBench tasks, 85% on hallucinations, and 81% on aberrant behaviors, with transfer results to other models.

Significance. The task is timely and the empirical design is ambitious: amortized training over many target behaviors, human-interpretable natural-language prompts, and evaluation against external benchmarks are all strengths. The paper also includes useful ablations (SFT data source, model size, aggregation schedule, number of iterations) and reports qualitative examples that support the claim that different Frank-Wolfe iterations surface qualitatively different strategies. If the Frank-Wolfe equivalence and the posterior-sampling statement were rigorously established, the paper would contribute a principled diversity mechanism for prompt search. As written, however, the central methodological derivation is internally inconsistent with the implemented algorithms, so the paper's main theoretical framing is not currently supported; the empirical results may stand as an effective heuristic, but the paper needs a substantive revision to either repair the derivation or explicitly reframe the method as a heuristic.

major comments (3)
  1. [§3.3, Eq. (5), Algorithm 3] The implemented DPO regularization strength does not match the derivation. Equation (5) specifies q(i) = DPO(r = log pm(y|x) - λ log p_agg(x|y), β = β2), but Algorithm 3 line 4 calls DPO(pSFT_theta, PRL, r, λ, TDPO), so β is set to λ = β1 - β2 rather than β2. In the §5.1 setting β1 = 0.6 and β2 = 0.1, so the algorithm uses β = 0.5 while the derivation calls for β = 0.1. The update actually run is therefore not the Frank-Wolfe step derived in the text.
  2. [§3.2, Algorithm 2, Eq. (4)] The DPO reference policy is not the target model pm, despite the derivation requiring KL(q || pm). Algorithm 2 line 5 states that the reference policy pref is pθ, and Algorithm 3 initializes DPO from pSFT_theta. The DPO objective therefore regularizes toward the current investigator model, not toward the target model pm. Consequently, the claim in §3.2 that DPO approximately optimizes E[log pm(y|x)] - β KL(pθ(x|y) || pm(x)) and is equivalent to L(pθ) with β1 = β2 = β is not correct as stated; the implicit KL is to pref, not to pm. This also breaks the derivation of Eq. (5), which requires the closed-form minimizer q* ∝ pm(x) exp((log pm - λ log p_agg)/β2).
  3. [§3.3 and Section 1] The statement that the iterative method 'asymptotically approximates Bayesian posterior sampling' is not supported by any theorem or argument in the paper. Even if Algorithm 3 matched Eq. (5), convergence of Frank-Wolfe to a stationary point of L(pθ) would not by itself imply convergence to the posterior pm(x | y). Additionally, the derivation in §3.3 writes each Frank-Wolfe iteration as an argmin, while the surrounding algebra and the DPO connection show the method is maximizing the linearized objective; this sign convention should be corrected or clarified.
minor comments (5)
  1. [§5.2] The text says the 2nd (out of 15) DPO iteration is used as the regularizer for future Frank-Wolfe rounds because later iterates have very low entropy; this checkpoint-selection heuristic is not described in Algorithm 2 or Algorithm 3 and should be made explicit in the method description.
  2. [Table 4] The 'Adaptive Attacks' row reports 100% but the table does not give a citation or a precise statement of the protocol used to obtain that number; please add a reference and clarify the comparison.
  3. [Figure 6 and Table 3] There are several typos in the text and figures: 'ηi = 2/u1' in the Figure 6 caption, 'propmts' in §5.2, 'synethetically' in §5.2, 'abberant' in Table 1, and malformed attack strings in Table 3 (e.g., 'excepwiseley', 'Ggive', 'exaclty'). These should be cleaned up.
  4. [Algorithm 5] The 'Ensure' line reads 'q, pθ .' with an awkward spacing and no return statement; please format the pseudocode consistently with the other algorithms.
  5. [§5.3] In the hallucination experiments both the target model pm and the verifier pv are Llama-3.1 8B-Instruct; the paper should discuss whether this shared identity inflates reported success rates or risks reward hacking, beyond the general reward-hacking caveat already in the Limitations section.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main results are empirical evaluations against external benchmarks, and the questionable FW/DPO equivalence is an implementation mismatch rather than a circular reduction.

full rationale

The paper's load-bearing results are empirical: investigator models are trained and then measured against external corpora (FineWeb, AdvBench, TruthfulQA, DSM-5-derived rubrics), so the headline attack-success and hallucination numbers are not derived from the assumptions. The theoretical Section 3.3 is an algebraic decomposition of the objective in Eq. (2) into f and g, and the reduction of the Frank-Wolfe argmin to Eq. (5) follows from the cited DPO/RL equivalence of Rafailov et al.; no step in that chain is defined in terms of the conclusion it is used to support, and the paper does not rely on its own prior work to force the method. Two concerns do arise but they are not circularity. First, Eq. (5) specifies beta=beta2 while Algorithm 3 passes lambda=beta1-beta2 to DPO, and DPO's KL is taken with respect to pSFT rather than pm; this means the claimed equivalence to Frank-Wolfe is not actually established by the paper's equations, but an unsupported approximation is a correctness risk, not an input redefined as an output. Second, the DPO augmentation in Appendix E.2 seeds generation with examples sampled from the AdvBench test set, which is a possible data-contamination issue, but the attack success still requires the investigator to produce effective prompts and is not forced by construction. The rubric experiments also use the Llama family for target, verifier, and judge, making the 'success' labels family-internal, but this affects external validity rather than creating a circular derivation. Because no prediction reduces to its own inputs by definition or by fitted parameters, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard optimization background and on domain assumptions about how to measure elicitation. The main paid-in assumptions are that greedy decoding is a sufficient behavior probe, that the LLM judge's rubric score is faithful, and that DPO can stand in for the Frank-Wolfe argmin. Hyperparameters beta1, beta2, lambda, eta, and candidate count k are chosen per task and directly affect the reported success rates.

free parameters (4)
  • beta1, beta2 (entropy and fluency regularization coefficients) = beta1=0.6, beta2=0.1 for pretraining strings; lambda=2.0 for harmful responses; lambda=0.1 for rubric settings
    These hyperparameters set the balance between elicitation reward, diversity, and fluency in Eq. (2), and are chosen per task by performance in the experiments.
  • eta_i (aggregation schedule) = eta_i = 1/i, or 2/(i+1) in the ablation
    Determines how iteration models are mixed into the aggregate policy; the paper selects between two schedules via ablation.
  • k (candidate count for best-of-k reranking) = 25
    All ASR numbers are best-of-25 with reranking by elicitation log-probability, so the reported success rates are not average sampling performance.
  • TDPO and TFW (DPO and Frank-Wolfe iteration counts, plus checkpoint selection) = e.g., 15 DPO iterations per FW round with the 2nd kept as regularizer; FW runs 4 iterations
    Chosen by stability: the paper notes too many DPO iterations degrade performance and that the 2nd checkpoint is a better regularizer.
assumptions (5)
  • standard math DPO optimizes a KL-regularized RL objective with respect to an implicit reward.
    Invoked in Section 3.2 to connect DPO to the loss L(p_theta) of Eq. (2), following Rafailov et al. (2023).
  • domain assumption Greedy decoding from the target model is a valid probe of whether a prompt elicits a behavior.
    Used throughout: the elicitation reward is log pm(y | x), and every ASR metric is based on greedy decoding of the target model.
  • domain assumption The verifier model p_v(a | R, y) faithfully measures rubric satisfaction.
    This is the reward for rubric-based elicitation in Section 2.2. The authors themselves report observed reward hacking against LM judges, which weakens this assumption.
  • ad hoc to paper Running DPO on reward log pm(y|x) minus lambda times log p_previous is a good approximation of the Frank-Wolfe argmin in Eq. (5).
    Section 3.3 relies on this to claim equivalence to Frank-Wolfe. The paper only says the argmin is 'approximately optimized by DPO', and the algorithm pseudocode conflicts with the equation.
  • domain assumption The harmfulness judge used for AdvBench Harmful Behaviors captures genuine harmfulness.
    The DPO reward is augmented with a Llama-3.1 8B judge (Appendix F), and the evaluation uses a GPT-4 autograder with acknowledged false positives.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Eliciting Language Model Behaviors with Investigator Agents." pith.science (2026). https://pith.science/paper/V2AGDCTI

@misc{pith2026250201236,
  author       = {Pith},
  title        = {Pith review of: Eliciting Language Model Behaviors with Investigator Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V2AGDCTI}},
  note         = {Machine review of arXiv:2502.01236}
}
read the original abstract

Language models exhibit complex, diverse behaviors when prompted with free-form text, making it difficult to characterize the space of possible outputs. We study the problem of behavior elicitation, where the goal is to search for prompts that induce specific target behaviors (e.g., hallucinations or harmful responses) from a target language model. To navigate the exponentially large space of possible prompts, we train investigator models to map randomly-chosen target behaviors to a diverse distribution of outputs that elicit them, similar to amortized Bayesian inference. We do this through supervised fine-tuning, reinforcement learning via DPO, and a novel Frank-Wolfe training objective to iteratively discover diverse prompting strategies. Our investigator models surface a variety of effective and human-interpretable prompts leading to jailbreaks, hallucinations, and open-ended aberrant behaviors, obtaining a 100% attack success rate on a subset of AdvBench (Harmful Behaviors) and an 85% hallucination rate.

Figures

Figures reproduced from arXiv: 2502.01236 by the authors.

Figure 1
Figure 1. To surface specific behaviors from the target language model (e.g. an SQL injection error), we train an investigator model to search for prompts that elicit responses satisfying example-specific criteria, including exact string matches and natural-language rubrics. jailbreaking, our investigators uncover strategies such as repetition, continuation, and prepending summaries, out￾performing the DPO baseline in diversi… view at source ↗
Figure 2
Figure 2. Training pipeline for our investigator model: 1. We first collect (prompt, response) pairs by generating responses from the target model. 2. We perform SFT to predict the prompt from the response. 3. We refine this investigator using DPO to further improve the elicitation probabilities. 4. We apply the Frank-Wolfe algorithm to discover new strategies that were not revealed by previous iterations. Algorithm 2 DPO (pθ… view at source ↗
Figure 3
Figure 3. We cast rubric-based elicitation as a two-stage problem: the first stage searches for the target response y, and the second stage searches for the prompt x that elicits the target response. Both stages can be solved using methods for string elicitation. appropriate objective, which we will show at the end of this section. For the aggregate model pθ, we initialize p (0) θ as the uniform distribution and η1 = 1. For e… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Elicitation log-probability and fluency first drops at the second iteration then gradually increases across the remaining iterations. Diversity initially increases and then slightly decreases over the remaining iterations. A. Ablation Studies Our elicitation method int…
Figure 5
Figure 5. Figure 5: Performance of WildChat and UltraChat across DPO iterations for AdvBench (Harmful Strings). Elicitation log-probability initially improves but can decrease over the course of training. A.2. The number of Frank-Wolfe iterations We study the effect of the number of FW it…
Figure 6
Figure 6. Figure 6: We compare the elicitation, fluency, diversity, and overall objective across two choices of ηi. ηi = 1 i converges to a better objective L. It outperforms ηi = 2 i+1 in terms of diversity score, while performing slightly worse on fluency and elicitation Investigator It…
Figure 7
Figure 7. Figure 7: The three iterations of FW cover different levels of repetition rates, and their aggregate distribution not only covers the full range of repetition rates, but also Pareto dominates SFT and DPO. C. Additional Related Work Language Model Inversion. Language models gener…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Data Swarms: Optimizable Generation of Synthetic Evaluation Data

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Data Swarms uses particle swarm optimization over data-generator LLM weights to produce synthetic evaluation data that scores higher on five quantitative evaluation objectives than eight baselines.

Reference graph

Works this paper leans on

33 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Jailbreaking leading safety-aligned llms with simple adaptive attacks

    Andriushchenko, M., Croce, F., and Flammarion, N. Jailbreaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint arXiv:2404.02151, 2024. URL https://arxiv.org/abs/2404.02151

    arXiv 2024

  3. [3]

    Diagnostic and Statistical Manual of Mental Disorders (DSM-5 )

    APA. Diagnostic and Statistical Manual of Mental Disorders (DSM-5 ) . American Psychiatric Publishing, 2013. ISBN 9780890425572. URL https://books.google.com/books?id=-JivBAAAQBAJ

    work page 2013

  4. [4]

    Infinite backrooms: Dreams of an electric mind, 2024

    Ayrey, A. Infinite backrooms: Dreams of an electric mind, 2024. URL https://dreams-of-an-electric-mind.webflow.io/. [Online; accessed 3-February-2025]

    work page 2024

  5. [5]

    Bach, F. R. Duality between subgradient and conditional gradient methods. CoRR, abs/1211.6302, 2012. URL http://arxiv.org/abs/1211.6302

    work page Pith review arXiv 2012

  6. [6]

    Constitutional ai: Harmlessness from ai feedback

    Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022

    arXiv 2022

  7. [7]

    M., Kucukelbir, A., and McAuliffe, J

    Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112 0 (518): 0 859--877, 2017. doi:10.1080/01621459.2017.1285773. URL https://doi.org/10.1080/01621459.2017.1285773

    work page arXiv 2017

  8. [8]

    J., and Wong, E

    Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., and Wong, E. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419, 2023. URL https://arxiv.org/abs/2310.08419

    arXiv 2023

Show all 33 references
  1. [9]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ding, N., Chen, Y., Xu, B., Qin, Y., Zheng, Z., Hu, S., Liu, Z., Sun, M., and Zhou, B. Enhancing chat language models by scaling high-quality instructional conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 3029--3051...

    2023

  2. [10]

    Enabling language models to fill in the blanks

    Donahue, C., Lee, M., and Liang, P. Enabling language models to fill in the blanks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.\ 2492--2501. Association for Computational Linguistics, 2020. URL https://aclanthology.org/2020.a...

    2020

  3. [11]

    and Wolfe, P

    Frank, M. and Wolfe, P. An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3: 0 95--110, 1956. URL https://api.semanticscholar.org/CorpusID:122654717

    1956

  4. [12]

    R., Srivastava, A., and Agrawal, P

    Hong, Z.-W., Shenfeld, I., Wang, T.-H., Chuang, Y.-S., Pareja, A., Glass, J. R., Srivastava, A., and Agrawal, P. Curiosity-driven red-teaming for large language models. In Proceedings of the Twelfth International Conference on Learning Representations, 2024. URL https://openre...

    2024

  5. [13]

    I., Ghahramani, Z., Jaakkola, T

    Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. An introduction to variational methods for graphical models. Machine Learning, 37: 0 183--233, 1999

    1999

  6. [14]

    Llm defenses are not robust to multi-turn human jailbreaks yet

    Li, N., Han, Z., Steneker, I., Primack, W., Goodside, R., Zhang, H., Wang, Z., Menghini, C., and Yue, S. Llm defenses are not robust to multi-turn human jailbreaks yet. arXiv preprint arXiv:2408.15221, 2024

    2024 arXiv

  7. [15]

    Truthfulqa: Measuring how models mimic human falsehoods

    Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 3214--3252. Association for Computational Linguistics, 2022. URL...

    2022

  8. [16]

    AutoDAN : Generating stealthy jailbreak prompts on aligned large language models

    Liu, X., Xu, N., Chen, M., and Xiao, C. AutoDAN : Generating stealthy jailbreak prompts on aligned large language models. In Proceedings of the Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=7Jwpw4qKkb

    2024

  9. [17]

    X., Zhao, W., Chiu, J

    Morris, J. X., Zhao, W., Chiu, J. T., Shmatikov, V., and Rush, A. M. Language model inversion. arXiv preprint arXiv:2311.13647, 2023. URL https://arxiv.org/abs/2311.13647

    2023 arXiv

  10. [18]

    Chenghaomou/text-dedup: Reference snapshot, September 2023

    Mou, C., Ha, C., Enevoldsen, K., and Liu, P. Chenghaomou/text-dedup: Reference snapshot, September 2023. URL https://doi.org/10.5281/zenodo.8364980

    2023 doi

  11. [19]

    Openai model specification

    OpenAI. Openai model specification. Technical report, 5 2024. URL https://cdn.openai.com/spec/model-spec-2024-05-08.html. Version 2024-05-08

    2024

  12. [20]

    The fineweb datasets: Decanting the web for the finest text data at scale

    Penedo, G., Kydlíček, H., Ben Allal, L., Lozhkov, A., Mitchell, M., Raffel, C., Von Werra, L., and Wolf, T. The fineweb datasets: Decanting the web for the finest text data at scale. arXiv preprint arXiv:2406.17557, 2024. URL https://arxiv.org/abs/2406.17557

    2024 arXiv

  13. [21]

    Red teaming language models with language models

    Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., and Irving, G. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp.\ 3419--3448. Association ...

    2022

  14. [22]

    Eliciting language model behaviors using reverse language models

    Pfau, J., Infanger, A., Sheshadri, A., Panda, A., Michael, J., and Huebner, C. Eliciting language model behaviors using reverse language models. arXiv preprint arXiv:2310.11689, 2023. URL https://openreview.net/forum?id=m6xyTie61H

    2023 arXiv

  15. [23]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023

    2023 arXiv

  16. [24]

    D., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023. URL https://arxiv.org/abs/2305.18290

    2023 arXiv

  17. [25]

    A conversation with bing’s chatbot left me deeply unsettled

    Roose, K. A conversation with bing’s chatbot left me deeply unsettled. The New York Times, February 2023. URL https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html

    2023

  18. [26]

    Toolformer: Language models can teach themselves to use tools

    Schick, T., Dwivedi-Yu, J., Dess \` , R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36: 0 68539--68551, 2023

    2023

  19. [27]

    R., Schwettmann, S., Wang, F., Rajaram, A., Hernandez, E., Andreas, J., and Torralba, A

    Shaham, T. R., Schwettmann, S., Wang, F., Rajaram, A., Hernandez, E., Andreas, J., and Torralba, A. A multimodal automated interpretability agent. In Forty-first International Conference on Machine Learning, 2024

    2024

  20. [28]

    D., and Su, W

    Song, Z., Cai, T., Lee, J. D., and Su, W. J. Reward collapse in aligning large language models, 2023. URL https://arxiv.org/abs/2305.17608

    2023

  21. [29]

    Trl: Transformer reinforcement learning

    von Werra, L., Belkada, Y., Tunstall, L., Beeching, E., Thrush, T., Lambert, N., Huang, S., Rasul, K., and Gallouédec, Q. Trl: Transformer reinforcement learning. https://github.com/huggingface/trl, 2020

    2020

  22. [30]

    A., Liu, F., Georgiev, G., Das, R

    Wang, Y., Wang, M., Manzoor, M. A., Liu, F., Georgiev, G., Das, R. J., and Nakov, P. Factuality of large language models: A survey, 2024. URL https://arxiv.org/abs/2402.02420

    2024 arXiv

  23. [31]

    Wildchat: 1m chat GPT interaction logs in the wild

    Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., and Deng, Y. Wildchat: 1m chat GPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Bl8u7ZRlbM

    2024

  24. [32]

    D., Maas, A

    Ziebart, B. D., Maas, A. L., Bagnell, J. A., and Dey, A. K. Maximum entropy inverse reinforcement learning. In Association for the Advancement of Artificial Intelligence (AAAI), 2008

    2008

  25. [33]

    Z., and Fredrikson, M

    Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. URL https://arxiv.org/abs/2307.15043

    2023 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.