IHEval: Evaluating Language Models on Following the Instruction Hierarchy

Bing Yin; Haodong Wang; Haoming Jiang; Meng Jiang; Qingyu Yin; Shiyang Li; Xianfeng Tang; Xin Liu; Yichuan Li; Yifan Gao

REVIEW 2 major objections 7 minor 1 cited by

IHEval: Evaluating Language Models on Following the Instruction Hierarchy

T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current language models are not optimized for the instruction hierarchy: when system, user, history, and tool instructions conflict, the best open-source model resolves only 48% of cases.

desk verdict A well-built programmatic benchmark with a clear normative framing: the conflict gaps are real, but the '48%' number measures a stipulated priority order that not every deployment would choose. read the letter →

arxiv 2502.08745 v2 pith:3QDUQG67 submitted 2025-02-12 cs.CL

Zhihan Zhang , Shiyang Li , Zixuan Zhang , Xin Liu , Haoming Jiang , Xianfeng Tang , Yifan Gao , Zheng Li

show 6 more authors

Haodong Wang Zhaoxuan Tan Yichuan Li Qingyu Yin Bing Yin Meng Jiang

This is my paper · ORCID

classification cs.CL

keywords instructionhierarchybenchmarksystemmessagesfollowingconflictinginstructionspromptinjectionsafetyevaluationlargelanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Language models are often deployed with a hierarchy of instructions: developer-set system messages should outrank user messages, which should outrank conversation history, which should outrank tool outputs. This paper introduces IHEval, a 3,538-example benchmark spanning nine tasks in four scenarios (rule following, task execution, safety defense, and tool use), to measure whether models actually respect that priority order when instructions align or conflict. The central finding is that they do not: every model tested drops sharply in the conflict setting, with the strongest open-source model, Qwen-2 72B, averaging 47.8% accuracy, Llama-3.1 70B collapsing to 14.0%, and GPT-4o reaching 70.0%. The paper argues this exposes a real deployment risk, since models that cannot prioritize higher-level instructions may misbehave or act unsafely when low-priority inputs carry conflicting requests.

What carries the argument

The central object is the instruction hierarchy itself, a fixed priority ordering (system messages highest, then user messages, conversation history, and tool outputs lowest) first proposed in prior work (Wallace et al., 2024). The benchmark operationalizes it through three paired settings per task: a reference setting with a single merged instruction, an aligned setting with hierarchical inputs that agree, and a conflict setting where a low-priority input demands something incompatible with the high-priority main instruction. Every task is scored programmatically by whether the model completes the main instruction, with strict and loose metrics adopted from IFEval; the aligned-versus-reference difference isolates hierarchical-input processing, and the conflict-versus-reference difference isolates hierarchy-following. Task design also varies the imperative strictness of both the main and conflicting instructions to test whether models respond to priority or to wording.

What would settle it

Present the same IHEval conflict pairs with the priority labels reversed, for example a tool output demanding format A while a system message demands format B, so that recency and hierarchy point in opposite directions; if models' scores track the stated priority order regardless of which slot carries the instruction, the hierarchy itself is being encoded, whereas if scores track recency or emphasis instead, the reported conflict gap reflects a superficial cue rather than hierarchy-following.

Watch

Extended reading notes

Core claim

The paper claims that current language models lack a robust capability to follow the instruction hierarchy, defined as the priority order system message > user message > conversation history > tool output. On IHEval, models perform near their baseline when hierarchical inputs align, but conflict settings cause large, consistent drops across all thirteen models tested, with scores falling 22 to 78 points below the reference setting. The benchmark is constructed so that any execution of the low-priority conflicting instruction hurts the score, and the ideal behavior is to ignore it entirely and complete only the high-priority task. The authors further show that even aligned hierarchical inputs destabilize some models, that conflict resolution is swayed by the superficial strictness of wording rather than by priority, that an explicit prompt stating the hierarchy does not improve performance, and that the ability to resolve conflicts does not scale reliably with model size. Their conclusion is that dedicated training on hierarchy-following, not prompt engineering, is needed.

Load-bearing premise

The benchmark assumes one fixed, universally correct priority order (system messages above user messages above conversation history above tool outputs) and assumes that silently executing the highest-priority instruction, rather than asking for clarification, partially complying, or following the most recent user request, is always the correct behavior.

Editorial extensions

If this is right

Model developers should treat hierarchy-following as a distinct capability to be measured and trained, separate from general instruction-following, since reference and aligned scores do not predict conflict-setting performance.
Deployments that layer system-message constraints over user-facing tools cannot rely on prompt wording: stating the priority order explicitly did not improve any tested model's conflict resolution.
Safety-critical settings such as password gates are at risk: Qwen-2 72B successfully defends only about one-third of hijack and extraction attacks, and several strong models fail them far more often.
Compared with the concurrent SysBench system-message benchmark, IHEval reports a larger aligned-versus-conflict gap, meaning hierarchy evaluation reveals weaknesses that system-prompt-only benchmarks miss.
Scaling alone will not fix the problem; some model families show inverse scaling on conflict tasks, where the smaller model outperforms the larger one.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test the paper does not run: fine-tune one small open model on conflict pairs whose correct label is 'ignore the lower-priority instruction'; if its IHEval conflict score jumps toward its reference score, the deficit is a training gap, and if it stays flat, hierarchy-following resists this kind of data.
The paper's recency evidence, where models do follow a new user request after a conflicting turn, suggests deployments should assume models weight recency, and any training fix must explicitly break that heuristic; a benchmark that varies instruction order while holding content fixed would isolate recency from hierarchy.
Because the paper scores only strict adherence to the top-priority instruction, a complementary benchmark that credits clarification or 'I cannot do X' responses would separate genuine priority awareness from a learned refusal style; the paper's error analysis already attributes most failures to following the conflicting instruction rather than to refusing.
In production, the hierarchy order itself is a design choice, not a fact; the assumption that tool outputs always rank lowest implies agents must never obey tool-retrieved content, which would forbid legitimate web-based workflows, a policy question the benchmark implicitly resolves in favor of strict developer control.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take on IHEval. It does what it says: a 3,538-example, nine-task benchmark covering four input types, scored programmatically, run on 13 popular models. The core empirical finding—performance collapses when low-priority instructions conflict with high-priority ones—is real and consistent across every model. GPT-4o at 70.0 and Qwen-2 at 47.8 on the conflict setting, Llama-3.1-70B at 14.0. Those gaps hold up as a description of how these models behave under the paper's scoring rule. The detailed error analysis (Figure 6, task cards) is genuinely useful; the distinction between 'followed the conflicting instruction' and 'synthesized both' is informative.

The main caveat, which the stress-test note gets right, is that the scoring rule in §3 and Appendix B encodes a normative policy: system over user over history over tool, and only strict execution of the highest-priority instruction counts. Anything else—clarification, partial compliance, answering the latest user request—is scored as a failure. That is a legitimate evaluation design for developer-controlled system messages in safety-critical deployments. But it is not self-evidently the right policy for all nine tasks; the translation-vs-math example is a case where following the user is arguably the helpful response. Table 4 also shows models do follow the most recent user instruction when history conflicts, which is a rational policy for ordinary chat. So the headline '48%' is a real number, but it measures adherence to a stipulated priority order, not a universal capability. The paper is fairly transparent about this in the footnote and Appendix B, but the abstract and intro treat the hierarchy as if it's given by nature.

Other soft spots: the reference setting merges all instructions into one user message, changing prompt structure as well as hierarchy, so some of the aligned-vs-reference drop is a prompt-format effect. Conflict instructions were chosen by heuristics and trials, with excluded variants not reported, so there is selection risk. No error bars or significance tests is minor for a deterministic eval with 13 models, though some task sizes are small (100 examples). Appendix B's use of GPT-4o's high score as evidence that the criteria match industry practice is mildly circular, but it is supplemental.

All that said, the central claim holds for the safety tasks (hijack, extraction), where developer priority is clearly right, and the benchmark is a solid contribution. I'd cite it and send it to review. The authors should release data and code explicitly and discuss the normative assumption more carefully. Recommendation: send to a serious referee; ask them to engage with the scoring rule and the reference-setting confound.

Referee Report

2 major / 7 minor

Summary. IHEval introduces a 3,538-example benchmark, spanning nine tasks in four categories (rule following, task execution, safety defense, tool use), for measuring how well language models follow the instruction hierarchy (system > user > conversation history > tool outputs). For each task the authors construct three settings: a reference setting with all instructions merged into a single user message, an aligned setting with hierarchical inputs whose priorities agree, and a conflict setting in which a lower-priority instruction contradicts the higher-priority one. They evaluate 13 models from five families under deterministic decoding, using programmatic metrics (IFEval-style strict and loose rules, F1, ROUGE-L, accuracy, defense-success criteria). The central findings are that models perform inconsistently between the reference and aligned settings, that all models drop sharply in the conflict setting (GPT-4o at 70.0, Qwen-2 72B at 47.8, Llama-3.1 70B at 14.0 aggregate), that results are sensitive to the strictness of instruction wording, that an explicit instruction-priority prompt does not restore performance, and that larger scale helps aligned but not conflict performance. The paper concludes that current LMs are not sufficiently optimized for the instruction hierarchy.

Significance. The benchmark fills a genuine gap: SysBench covers only system-message adherence, and Wallace et al. (2024) evaluated only GPT models on proprietary safety data, whereas IHEval covers all four hierarchy levels with open, programmatic evaluation. The strengths are concrete: fully deterministic scoring with no LLM-as-judge, 13 models across five families, per-task breakdowns in Tables 5-10, task cards in Appendix D, a useful negative result on prompt-based intervention (section 4.4), and an informative error-type decomposition (Figure 6) showing that models often combine or partially execute both instructions. If the measurement is valid, the consistent conflict-setting drops constitute an important signal for training and evaluation. The main validity risks, developed in the major comments, concern the normative scoring rule and the reference-setting confound; both affect the interpretation of magnitudes rather than the existence of the phenomenon.

major comments (2)

[§3 (Evaluation), Appendix B; Table 1] The scoring rule in §3 ('Evaluation' paragraph) and Appendix B counts only strict execution of the highest-priority instruction as correct, so clarification, refusal, partial compliance, and following the most recent user instruction are all scored as errors. The headline conflict-setting numbers (Table 1: 14.0% for Llama-3.1-70B, 47.8% for Qwen-2 72B, 70.0% for GPT-4o) are therefore measurements of adherence to one stipulated policy, not a model-neutral measure of 'hierarchy awareness.' Since §4.5 and Table 4 show that models are trained to follow the most recent user instruction, recency-following is a defensible policy in ordinary chat deployments, and in at least one task (Task Execution-Generation, where the system demands translation but the user asks for a math solution) following the user is arguably the more helpful behavior. The abstract, §1, and §5 state that models 'fail' to resolve conflicts and require 'targeted optimization'; these claims should be scoped to the stated deployment model (developer-set system messages, footnote 1) and supported by either per-task arguments for the normative priority or a robustness analysis that scores alternative behaviors separately—the error-type taxonomy in Figure 6 already provides the machinery. Without this, the magnitude of the reported failure is not separable from the normative stipulation.
[§3 (Task Settings); §4.1] The reference setting in §3 ('Task Settings') does not isolate 'original task performance' as claimed: merging all hierarchical instructions into a single user message removes the hierarchy dimension while simultaneously moving the instruction from the system channel to the user channel and shortening the input structure. The aligned-versus-reference gaps in §4.1 (e.g., Llama-3.1-70B's average 13-point drop) therefore confound hierarchy-following with instruction-position and phrasing sensitivity. This weakens the paper's first key finding that models are 'less optimized for hierarchical inputs.' I suggest a control that holds the instruction channel fixed across conditions (e.g., a reference condition with the same instruction placed in a system message and an instruction-free user message), or a per-task decomposition of the channel effect. The aligned-versus-conflict comparison used for the headline claim is not affected by this confound.

minor comments (7)

[Appendix B] The sentence 'GPT-4o's strong performance on IHEval tasks further supports that our criteria reflect industry practices' is circular, because GPT-4o is the model whose training target (Wallace et al., 2024) is the hierarchy under test; its high score cannot independently validate the criteria, so this sentence should be removed or reframed.
[§3 (Task Design)] Please document the screening procedure and the excluded candidates behind the statement that conflicting instructions were drawn 'based on heuristics and trials on sample data,' since this affects the benchmark's difficulty calibration and the reproducibility of the reported drop magnitudes.
[§3 (Evaluation)] The claim that asking for clarification gives instructions 'the same level of priority' is not self-evident, because clarification is often a reasonable strategy for resolving conflicting requests; please provide an argument or report the frequency and treatment of clarification-only responses.
[§4.1] The comparison stating that IHEval 'reveals a larger performance gap between aligned and conflict inputs' than SysBench is made without running SysBench on the same model set; please add the head-to-head results or soften the claim.
[Figure 6] The caption states that error proportions are computed over 'all models in Figure 1,' but Figure 1 is the hierarchy diagram rather than a results table; the intended reference is presumably the models in Table 1.
[§4.2] The inverse-scaling observation (Claude-Haiku outperforming Claude-Sonnet on 5 of 9 tasks) and the monotone-scaling claim rest on a single model pair per family without significance testing; consider a sign test over the five families or softer wording.
[Table 4] The caption defines 'All Models' as the average over the models in Table 1, which is a selected subset of six rather than all 13 evaluated models; please clarify to avoid confusing the reader.

Circularity Check

1 steps flagged · score 1.0 of 10

Mild circularity in Appendix B's GPT-4o validation; the benchmark's main comparison is self-contained.

other [Appendix B (Evaluation Criteria), final paragraph]
"Moreover, GPT-4o's strong performance on IHEval tasks further supports that our criteria reflect industry practices."

The paper uses GPT-4o's high IHEval score to validate its scoring criterion, but the paper's own Section 4.1 explains that GPT-4o's advantage 'likely reflect[s] OpenAI's fine-tuning efforts on the instruction hierarchy as described in Wallace et al. (2024)'. GPT-4o was therefore trained to satisfy exactly the strict-priority rule that IHEval scores; using its strong performance as evidence that the criterion matches industry practice is a mild confirmation loop. The central cross-model comparison and the reference/conflict score differences do not depend on this argument, so the circularity is local and non-load-bearing.

full rationale

IHEval is a benchmark rather than a fitted model; all nine tasks are scored by external programs (IFEval script, F1, ROUGE-L, accuracy, and defense-success rules), with no parameters fitted to model outputs. The central comparison among reference, aligned, and conflict settings is therefore not circular: conflict-setting scores are computed under a stated priority rule, and the conclusion that models fail to follow the instruction hierarchy is a direct consequence of those scores, not a disguised reproduction of the rule. The hierarchy ordering is attributed to Wallace et al. (2024), a non-overlapping external source, and no uniqueness theorem from the present authors is invoked. The only mild circular passage is Appendix B's validation of the evaluation criteria by GPT-4o's high score, which is explained in Section 4.1 as reflecting OpenAI's hierarchy-specific training, making it a local and non-load-bearing confirmation loop. The normative choice that system messages should outrank user messages, history, and tool outputs is a stated design assumption, not a circular derivation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The benchmark rests on the normative priority ordering and on the choice of strict main-instruction execution as the success criterion. It also assumes the reference setting isolates IH-following from task difficulty. No numerical free parameters are fitted and no new physical or conceptual entities are postulated.

assumptions (4)

domain assumption The priority order system message > user message > conversation history > tool output is the correct normative hierarchy and should be followed in all IHEval scenarios.
Adopted from Wallace et al. (2024) in Section 1 and Section 3 Definition; the paper does not justify why this order should hold universally, and footnote 1 limits scope to cases where users cannot modify developer-set system messages.
domain assumption Successful hierarchy-following means strictly completing only the highest-priority instruction; clarification, partial compliance, or answering both instructions counts as failure.
Appendix B states that overly interactive responses are discouraged and that avoiding responses to potential misuse aligns with standard LM practices.
ad hoc to paper The reference setting, with all hierarchical instructions merged into a single user message, isolates original task performance, so differences in aligned and conflict settings measure IH-following ability.
Section 3 Task Design introduces this setting and assumes system-message placement and added text do not change task difficulty or model behavior independently of hierarchy.
domain assumption Programmatic metrics including F-1, ROUGE-L, accuracy, and defense success rate faithfully measure whether the model completed the highest-priority instruction.
Section 3 Evaluation and Figure 4 specify the metrics; no human validation of metric agreement is reported for the conflict settings.

how reviews work

0 comments

Cite this review

Pith. "Pith review of IHEval: Evaluating Language Models on Following the Instruction Hierarchy." pith.science (2026). https://pith.science/paper/3QDUQG67

@misc{pith2026250208745,
  author       = {Pith},
  title        = {Pith review of: IHEval: Evaluating Language Models on Following the Instruction Hierarchy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3QDUQG67}},
  note         = {Machine review of arXiv:2502.08745}
}

read the original abstract

The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs.

Figures

Figures reproduced from arXiv: 2502.08745 by the authors.

**Figure 1.** Four categories of the instruction hierarchy and the corresponding priority orders of instructions. Conflict [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Results of mainstream LMs on IHEval. The reference setting represents original task performance without hierarchical inputs. We observe large performance drops when models face conflicting hierarchical instructions. no comprehensive benchmark to evaluate how well different LMs adhere to the instruction hierarchy. In order to bridge this gap and highlight the vital role of the instruction hierarchy, we create IHEval,… view at source ↗

**Figure 3.** IHEval covers four categories and nine tasks. Detailed examples and instructions are in Figures [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (13 more)

**Figure 4.** Figure 4: The original data source, the evaluation metric, and the data size of each task. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: The trend of IHEval performance by model scale. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Error types when facing instruction conflicts [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: The input configuration of different settings in § [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 8.** Figure 8: Task card of the single-turn conversation task in Rule Following category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

**Figure 9.** Figure 9: Task card of the multi-turn conversation task in Rule Following category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

**Figure 10.** Figure 10: Task card of the extraction task in Task Execution category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

**Figure 11.** Figure 11: Task card of the generation task in Task Execution category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

**Figure 12.** Figure 12: Task card of the classification task in Task Execution category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

**Figure 13.** Figure 13: Task card of the Hijack task in Safety Defense category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]

**Figure 14.** Figure 14: Task card of the Extraction task in Safety Defense category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

**Figure 15.** Figure 15: Task card of the Intrinsic Instruction task in Tool Use category. The main instruction is in [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

**Figure 16.** Figure 16: Task card of the Injected Instruction task in Tool Use category. This task slightly differs from others [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Reasoning Up the Instruction Ladder for Controllable Language Models
cs.CL 2025-10 conditional novelty 6.0 of 10

RLVR on ~7K verifiable system/user conflict examples teaches LLMs to prioritize higher-priority instructions, improving instruction-hierarchy and safety benchmarks.

Reference graph

Works this paper leans on

44 extracted references · 16 canonical work pages · cited by 1 Pith paper

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Anthropic. 2024 a . https://www.anthropic.com/news/claude-3-family Introducing the next generation of claude . News accouncement by Anthropic

work page 2024
[4]

Anthropic. 2024 b . https://docs.anthropic.com/en/release-notes/system-prompts/ System prompts of claude models . Release Notes by Anthropic

work page 2024
[5]

Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, and et al. 2024. https://doi.org/10.48550/arXiv.2401.02954 Deepseek LLM: scaling open-source language models with longtermism . Arxi...

arXiv doi:10.48550/arxiv.2401.02954 2024
[6]

Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer - Kellner, Marc Fischer, and Florian Tram \` e r. 2024. https://doi.org/10.48550/arXiv.2406.13352 Agentdojo: A dynamic environment to evaluate attacks and defenses for LLM agents . Arxiv preprint, 2406.13352

arXiv doi:10.48550/arxiv.2406.13352 2024
[7]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aur \' e lien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozi \` e...

arXiv doi:10.48550/arxiv.2407.21783 2024
[8]

Hashimoto

Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/5fc47800ee5b30b8777fdd30abcaaf3b-Abstract-Conference.html Alpacafarm: A simulation framework for methods that learn from human feedback . In Annual Conferenc...

work page 2023

Show all 44 references

[9]

Saiful Islam, Kazi Samin Mubasshir, Yuan - Fang Li, Yong - Bin Kang, M

Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Samin Mubasshir, Yuan - Fang Li, Yong - Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. https://doi.org/10.18653/v1/2021.findings-acl.413 Xl-sum: Large-scale multilingual abstractive summarization for 44 languages...

2021 doi
[10]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In International Conference on Learning Representations, ICLR 2021

2021
[11]

Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A

Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew E. Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. https://doi.org/10.48550/arXiv.2311.10702 Camels in a changing climate: Enhancing LM adaptation...

2023 doi arXiv
[12]

Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. 2023. https://doi.org/10.48550/arXiv.2310.20410 Followbench: A multi-level fine-grained constraints following benchmark for large language models . Arxiv ...

2023 doi arXiv
[13]

Seongyun Lee, Sue Hyun Park, Seungone Kim, and Minjoon Seo. 2024. https://doi.org/10.48550/arXiv.2405.17977 Aligning to thousands of preferences via system message generalization . Arxiv preprint, 2405.17977

2024 doi arXiv
[14]

Shiyang Li, Jun Yan, Hai Wang, Zheng Tang, Xiang Ren, Vijay Srinivasan, and Hongxia Jin. 2023. https://doi.org/10.48550/arXiv.2307.10558 Instruction-following evaluation through verbalizer manipulation . Arxiv Preprint, 2307.10558

2023 doi arXiv
[15]

Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out. Association for Computational Linguistics

2004
[16]

Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han, and Yongbin Li. 2024. https://aclanthology.org/2024.findings-acl.424 Sofa: Shielded on-the-fly alignment via priority rule following . In Findings of the Association for Computational Linguistics, ACL 2024

2024
[17]

Mistral. 2024. https://mistral.ai/news/mistral-large-2407 Large enough | mistral ai | frontier ai in your hands . News accouncement by Mistral AI

2024
[18]

Norman Mu, Sarah Chen, Zifan Wang, Sizhe Chen, David Karamardian, Lulwa Aljeraisy, Dan Hendrycks, and David A. Wagner. 2023. https://doi.org/10.48550/arXiv.2311.04235 Can llms follow simple rules? Arxiv Preprint, 2311.04235

2023 doi arXiv
[19]

Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. https://doi.org/10.48550/arXiv.2306.02707 Orca: Progressive learning from complex explanation traces of GPT-4 . Arxiv preprint, 2306.02707

2023 doi arXiv
[20]

OpenAI. 2023. https://doi.org/10.48550/arXiv.2303.08774 GPT-4 technical report . Arxiv preprint, 2303.08774

2023 doi arXiv
[21]

OpenAI . 2024. https://cdn.openai.com/spec/model-spec-2024-05-08.html Model spec: Overview . OpenAI Website

2024
[22]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. https://aclanthology.org/P02-1040/ Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, ACL 2022

2002
[23]

Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Bj \" o rkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. https://aclanthology.org/W13-3516/ Towards robust linguistic analysis using ontonotes . In Proceedings of the Seventeenth Conference on Com...

2013
[24]

Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, Weipeng Chen, Zenan Zhou, Wentao Zhang, and Bin Cui. 2024 a . https://doi.org/10.48550/arXiv.2408.10943 Sysbench: Can large language models follow system messages? Arxiv preprint, 2...

2024 doi
[25]

Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024 b . https://doi.org/10.48550/arXiv.2401.03601 Infobench: Evaluating instruction following ability in large language models . Arxiv Preprint, 2401.03601

2024 doi arXiv
[26]

Sunil Ramlochan. 2024. https://promptengineering.org/system-prompts-in-large-language-models/ System prompts in large language models . Blog on promptengineering.org

2024
[27]

Morgane Rivi \` e re, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \' e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \' e , Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le ...

2024 doi arXiv
[28]

Boyd - Graber

Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis - Fran c ois Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan L. Boyd - Graber. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.302 Ignore this title and hackaprompt: Ex...

2023 doi
[29]

Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2023. https://openreview.net/forum?id=fR3wGCk-IXp Language models are multilingual chain-of-thought reasone...

2023
[30]

Xiaoshuai Song, Zhengyang Wang, Keqing He, Guanting Dong, Yutao Mou, Jinxu Zhao, and Weiran Xu. 2024. https://doi.org/10.48550/ARXIV.2402.08631 Knowledge editing on black-box large language models . CoRR, abs/2402.08631

2024 doi arXiv
[31]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, and et al....

2023 doi arXiv
[32]

Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, Alan Ritter, and Stuart Russell. 2024. https://openreview.net/forum?id=fsW7wJGLBd Tensor trust: Interpretable prompt injecti...

2024
[33]

Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. https://doi.org/10.48550/arXiv.2404.13208 The instruction hierarchy: Training llms to prioritize privileged instructions . Arxiv preprint, 2404.13208

2024 doi arXiv
[34]

Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. 2024. https://doi.org/10.48550/arXiv.2407.03978 Benchmarking complex instruction-following with multiple con...

2024 doi arXiv
[35]

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and et al. 2024. https://doi.org/10.48550...

2024 doi arXiv
[36]

Jingwei Yi, Yueqi Xie, Bin Zhu, Keegan Hines, Emre Kiciman, Guangzhong Sun, Xing Xie, and Fangzhao Wu. 2023. https://doi.org/10.48550/arXiv.2312.14197 Benchmarking and defending against indirect prompt injection attacks on large language models . Arxiv Preprint, 2312.14197

2023 doi arXiv
[37]

Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. 2024. https://doi.org/10.48550/arXiv.2407.04295 Jailbreak attacks and defenses against large language models: A survey . Arxiv Preprint, 2407.04295

2024 doi arXiv
[38]

Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. https://aclanthology.org/2024.findings-acl.624 Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents . In Findings of the Association for Computational Linguistics, ACL 2024

2024
[39]

Weinberger, and Yoav Artzi

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with BERT . In 8th International Conference on Learning Representations, ICLR 2020

2020
[40]

Xing, Hao Zhang, Joseph E

Lianmin Zheng, Wei - Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023 a . http://papers.nips.cc/paper\_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Ab...

2023
[41]

a helpful assistant

Mingqian Zheng, Jiaxin Pei, and David Jurgens. 2023 b . https://doi.org/10.48550/arXiv.2311.10054 Is "a helpful assistant" the best role for large language models? A systematic evaluation of social roles in system prompts . Arxiv preprint, 2311.10054

2023 doi arXiv
[42]

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. https://doi.org/10.48550/arXiv.2311.07911 Instruction-following evaluation for large language models . Arxiv Preprint, 2311.07911

2023 doi arXiv
[43]

Xiaotian Zou, Yongkang Chen, and Ke Li. 2024. https://doi.org/10.48550/arXiv.2402.14857 Is the system message really important to jailbreaks in large language models? Arxiv preprint, 2402.14857

2024 doi arXiv
[44]

Egor Zverev, Sahar Abdelnabi, Mario Fritz, and Christoph H. Lampert. 2024. https://doi.org/10.48550/arXiv.2403.06833 Can llms separate instructions from data? and what do we even mean by that? Arxiv Preprint, 2403.06833

2024 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Anthropic. 2024 a . https://www.anthropic.com/news/claude-3-family Introducing the next generation of claude . News accouncement by Anthropic

work page 2024

[4] [4]

Anthropic. 2024 b . https://docs.anthropic.com/en/release-notes/system-prompts/ System prompts of claude models . Release Notes by Anthropic

work page 2024

[5] [5]

Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, and et al. 2024. https://doi.org/10.48550/arXiv.2401.02954 Deepseek LLM: scaling open-source language models with longtermism . Arxi...

arXiv doi:10.48550/arxiv.2401.02954 2024

[6] [6]

Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer - Kellner, Marc Fischer, and Florian Tram \` e r. 2024. https://doi.org/10.48550/arXiv.2406.13352 Agentdojo: A dynamic environment to evaluate attacks and defenses for LLM agents . Arxiv preprint, 2406.13352

arXiv doi:10.48550/arxiv.2406.13352 2024

[7] [7]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aur \' e lien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozi \` e...

arXiv doi:10.48550/arxiv.2407.21783 2024

[8] [8]

Hashimoto

Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/5fc47800ee5b30b8777fdd30abcaaf3b-Abstract-Conference.html Alpacafarm: A simulation framework for methods that learn from human feedback . In Annual Conferenc...

work page 2023