Towards Efficient and Effective Alignment of Large Language Models

Yuxin Jiang

REVIEW 4 major objections 6 minor 240 references

Towards Efficient and Effective Alignment of Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This thesis argues that LLM alignment improves at every stage—data, training, and evaluation—when each stage is redesigned around model weaknesses and fine-grained signals, and it introduces five frameworks (Lion, WebR, LTE, BMC, and…

desk verdict A five-chapter thesis with genuinely new mechanisms and broad empirical work, but the instruction-following claims rest on a GPT-4 generation/evaluation loop that needs cross-judge validation. read the letter →

arxiv 2506.09329 v1 pith:35WBA2VZ submitted 2025-06-11 cs.CL

Yuxin Jiang This is my paper

classification cs.CL

keywords LLMalignmentinstructiontuningadversarialdistillationpreferenceoptimizationdirectknowledgeeditingwebdatasynthesisconstraint-followingbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis sets out to show that aligning large language models can be made both more efficient and more effective by intervening at all three stages: synthesizing training data, optimizing preference training, and evaluating instruction following. It claims that its five frameworks—adversarial distillation (Lion), web reconstruction (WebR), learning to edit (LTE), bridged and modeled correlations for DPO (BMC), and FollowBench—each outperform their state-of-the-art baselines. If true, alignment would no longer depend on expensive human-curated datasets, memorization-based knowledge updates, or coarse response-quality evaluation. A sympathetic reader should care because the thesis offers a complete, testable pipeline for building and measuring instruction-following models.

What carries the argument

The load-bearing machinery is a set of feedback loops and pair-correlation structures. Lion's adversarial distillation loop has three stages—imitation, discrimination, and generation—where a proprietary LLM plays teacher, referee, and generator, and the referee's score gap identifies hard instructions that the generator expands. WebR's dual-perspective reconstruction assigns each raw web document either as an instruction (concatenated with a synthesized rewrite request) or as a response (with latent-instruction rollout and refinement), using a 2:1 ratio that balances reasoning and instruction-following abilities. LTE uses parallel data construction (with and without an edit prompt, for in-scope and out-of-scope queries) plus retrieval of edit descriptors from a memory bank. BMC uses targeted modification with the winning response as reference to synthesize pseudo-winning responses, then computes token-level reward weights from the policy model's own confidence via Levenshtein alignment. FollowBench's multi-level mechanism starts from a base instruction and adds one constraint per level across five constraint types, evaluated by prompt-based judges that follow the constraint-evolution path.

What would settle it

Take a fixed set of instructions, train two student models with identical data and hyperparameters but evaluate one with a judge from the same model family that generated the data and the other with an independent judge or human annotators; if the gap between BMC and DPO, or between WebR-trained and baseline models, shrinks dramatically under the independent judge, the central claim of general alignment improvement fails. A cheaper check is to sample the pseudo-winning responses from the Bridging Phase and measure whether the 'wins' are mostly stylistic changes (length, formatting, politeness) rather than content corrections.

Watch

Extended reading notes

Core claim

The central claim is that alignment failures in LLMs stem from three correctable design gaps: unidirectional data distillation that never targets student weaknesses, preference data whose winning and losing responses are generated in isolation, and evaluation that measures response quality but not explicit constraint adherence. The thesis proposes closing each gap: Lion runs an imitation-discrimination-generation loop so the teacher LLM generates hard instructions tailored to the student's errors; WebR reconstructs raw web documents through two complementary routes (web as instruction and web as response) without human annotation; LTE trains models to apply updated knowledge at inference time rather than memorize edits; BMC bridges preference pairs by rewriting losing responses into pseudo-winning ones and then models token-level correlations via policy confidence; FollowBench incrementally adds constraints to instructions to measure fine-grained adherence. Across the thesis's experiments, each framework claims consistent gains over its respective baselines.

Load-bearing premise

The methods assume that the API-accessible LLMs used as data generators, referees, and judges are unbiased enough that improving against them is the same as improving real human alignment; if those models impose systematic stylistic or quality biases, the measured gains may reflect better imitation of the judge rather than general alignment.

Editorial extensions

If this is right

If Lion works as claimed, a proprietary teacher's knowledge can be transferred to a compact open model using far fewer human-curated data points, and hard-instruction mining becomes a reusable loop.
If WebR works, high-quality instruction-tuning data can be produced from arbitrary web documents at low cost, and domain adaptation reduces to re-weighting source web domains.
If LTE works, knowledge editing becomes an inference-time, retrieval-augmented operation that handles batch and sequential edits with little degradation and no per-edit parameter updates.
If BMC works, preference optimization gains from correlated pairs and token-level credit assignment, and the same recipe transfers to DPO variants such as IPO, ORPO, R-DPO, and SimPO.
If FollowBench works, instruction-following evaluation can be decomposed by constraint type and difficulty level, exposing which constraints models fail rather than reporting a single quality score.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: The shared trust in black-box LLM judges means all five reported gains are measured against the same family of models that generated the data; a natural extension is to measure alignment gains with independent human judgments or cross-model judges to separate general alignment from judge-specific conformity.
Editorial inference: The adversarial loop's equilibrium claim suggests a possible curriculum: if hard instructions are generated from the student's current errors, the data distribution itself becomes a function of the student, which could be adapted to continual learning or personalized alignment.
Editorial inference: WebR's domain adaptability implies a practical recipe for low-resource domains: collect raw web documents in the target domain, adjust their proportion, and synthesize instruction data without any seed examples—testable on languages or specialties where no instruction datasets exist.
Editorial inference: BMC's length-normalization observation hints that correlated preference pairs may naturally suppress verbosity bias; a direct test is whether the same effect appears in human preference judgments, not just in model-based win rates.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Fair: this is a PhD thesis that reads as five independent, method-focused chapters rather than a single tightly-argued paper. The novel mechanisms are real: Lion's adversarial distillation loop, WebR's web-as-instruction/as-response reconstruction, LTE's learn-to-edit with retrieval, BMC's Bridging/Modeling correlation framework, and FollowBench's multi-level constraint benchmark. Each chapter compares against relevant baselines and reports on external benchmarks like AGIEval, BBH, GSM8K, MMLU, HumanEval, and MATH. WebR is the strongest chapter: five-seed averaging, clear ablations, cost estimates, and a consistent linear improvement with data scale. BMC also has decent analysis, including gradient derivations and token-level reward visualization.

The soft spots are mostly about evidence hygiene, not the core ideas. Most tables report point estimates without error bars or significance tests, the hard-sample upper-bound argument in Chapter 3 is asserted rather than proven, and the thesis ships no code or data. The abstract also oversells Lion: the body shows Lion-13B at about 72.5% of ChatGPT on AGIEval, which is 'better than Vicuna' but not 'state-of-the-art zero-shot reasoning' in any absolute sense.

The stress-test concern about the GPT-4 closed loop is legitimate and it lands hardest on the instruction-following results. Lion, BMC, and FollowBench all use GPT-4-class models as either data generators or judges, and the headline instruction-following metrics (AlpacaEval 2, Arena-Hard, MT-Bench) use GPT-4 as the evaluator. So the measured gains could partly reflect alignment to GPT-4's stylistic preferences rather than general alignment. That is a real risk. It is not fatal, because the reasoning and knowledge benchmarks are objective, and WebR's gains on IFEval and downstream tasks like MMLU/MATH/HumanEval do not reduce to judge bias. But the central alignment claim would be much stronger with a cross-judge check (e.g., human ratings or a non-OpenAI judge) on the instruction-following benchmarks.

Who is this for? Someone working on alignment data synthesis, DPO variants, knowledge editing, or instruction-following evaluation will find at least one chapter useful. As a thesis it is not a single citable artifact, but the individual chapters deserve serious referee attention in the relevant venues. I would send it to peer review rather than desk reject; the requests would be for artifacts, error bars, cross-judge evaluation, and a more measured abstract.

Referee Report

4 major / 6 minor

Summary. The thesis presents five contributions spanning the three stages of LLM alignment: Lion, an adversarial distillation framework for data synthesis; WebR, a fully automated web-based instruction-tuning data generator; LTE, a learning-to-edit framework for knowledge editing; BMC, a refinement of DPO that bridges and models token-level correlations in preference data; and FollowBench, a multi-level benchmark for constraint following. Each chapter reports experiments on standard benchmarks and claims state-of-the-art or significantly improved performance. The thesis also includes cost analyses, ablations, and some reproducibility details such as five-seed averaging for WebR and human agreement for FollowBench's model-based evaluation.

Significance. If the central claim is correct, the thesis offers several practical tools that could be useful to the alignment community: a data-synthesis framework that beats existing synthetic and human-crafted data, a fast knowledge-editing method with strong portability, a DPO variant with consistent gains over strong baselines, and a fine-grained instruction-following benchmark. The manuscript is strongest where it reports objective benchmarks (AGIEval, BBH, GSM8K, MATH, HumanEval) rather than LLM-judged open-ended metrics, and it makes a genuine effort to report costs and to show that some components can be replaced by open-source models. However, the thesis's headline claim of 'significantly improve alignment across all stages' rests heavily on instruction-following evaluations that use GPT-4 both as the training-data generator and as the judge, with no independent judge or human validation. The lack of error bars or significance tests in most tables further weakens the repeated 'significantly' and 'consistently' statements. These issues are fixable but require additional experiments or more cautious wording.

major comments (4)

[Chapters 3, 6, 7; Tables 3.1, 6.2, 7.3]
[Tables 3.2, 3.3, 5.1, 6.1, 6.2, 7.2]
[Section 3.2, first paragraph]
[Table 4.1]

minor comments (6)

[Section 3.2.5]
[Table 5.1]
[Section 6.2.2, after Eq. (6.6)]
[Table 4.7]
[Section 3.3.1]
[Figure 3.5]

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central claims are validated on external, independently scored benchmarks and no derivation reduces to its own inputs.

full rationale

The thesis's proposed methods (Lion, WebR, LTE, BMC, FollowBench) are each evaluated against external benchmarks with fixed answers or independent protocols, rather than against quantities fitted by the methods themselves. Lion is tested on AGIEval and BIG-Bench Hard with exact-match accuracy; WebR is tested on AlpacaEval 2, Arena-Hard, MT-Bench, IFEval, MMLU, ARC, GSM8K, MATH, and HumanEval; LTE is tested on KnowEdit benchmarks with rule-based edit-success, portability, locality, and fluency metrics; BMC is tested on QA and math tasks with accuracy plus AlpacaEval/Arena-Hard; FollowBench reports human-agreement validation for its model-based evaluation. The hyperparameters and ablation choices (e.g., the threshold tau in Lion, the ratio in WebR, and delta in BMC) are disclosed as ablations and are not disguised as predictions. The use of GPT-4-family models for both data synthesis and some instruction-following judgments creates a potential evaluation-bias risk, but this is not an equivalence by construction: the synthesized data is not the same object as the evaluation signal, and the main reasoning, math, and knowledge claims are supported by non-GPT-4 metrics. The paper itself acknowledges limitations of GPT-4-based automated evaluation in Section 3.5.2. No load-bearing self-citation or uniqueness theorem is invoked to force the conclusions. Therefore, no circular step is exhibited under the required standard.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claims rest mainly on assumptions about the reliability of LLM-generated data and LLM judges, plus standard preference-optimization assumptions. No free parameter appears to be fit to the target result; the listed parameters are hyperparameters tuned by ablation.

free parameters (6)

Lion threshold tau = 1.0 (optimal range 1.0 to 1.5 in Table 3.4)
Classifies hard versus easy instructions in the discrimination stage; ablation shows performance varies with tau.
Lion hard-to-easy ratio r = 1:1 (optimal in Table 3.5)
Controls the proportion of newly generated hard versus easy instructions; 1:1 gives the best average scores.
WebR instruction-to-response ratio = 2:1 (optimal in Table 4.3)
Balances Web as Instruction and Web as Response branches; ablation shows 2:1 is best overall.
WebR MinHash similarity threshold = 0.7
Used for deduplication; removing MinHash deduplication degrades performance.
LTE retrieval top-k = 3
Number of edit descriptors retrieved at inference; ablation shows k=3 slightly outperforms k=1 and k=2.
BMC emphasis upper bound delta = 3.0
Caps token-level reward emphasis in Equations 6.5 and 6.6; ablation in Figure 6.4 shows delta near 3.0 is best.

assumptions (6)

domain assumption API-access LLMs provide faithful teacher, referee, and generator signals.
Lion prompts ChatGPT to identify hard instructions and generate new ones; LTE, WebR, and BMC use GPT-4 or GPT-4o-mini to synthesize or refine data. The claims inherit any bias in these models.
domain assumption GPT-4-as-judge scores are a valid proxy for human preference in open-ended evaluation.
Chapters 3, 4, 6, and 7 rely on LLM judges for Vicuna-80, AlpacaEval, MT-Bench, and FollowBench; human agreement is validated only for FollowBench, not for all evaluations.
domain assumption Optimizing the student on hard samples reduces expected teacher-student discrepancy.
Section 3.2 states this upper-bound rationale without a formal derivation; the equilibrium claim in Section 3.2.5 is heuristic.
domain assumption Token-level DPO reward weighting by inverse policy confidence improves alignment.
Section 6.2.2 motivates Equations 6.5 and 6.6 empirically; no formal guarantee that this weighting is optimal or monotone.
domain assumption Web documents contain enough latent instruction-response structure to reconstruct useful training data.
WebR assumes raw web content can be assigned as instruction or response and reconstructed; the method's success depends on this assumption.
standard math The Bradley-Terry model and DPO's implicit reward parameterization hold for preference data.
Chapter 6 builds on the closed-form DPO reward in Equation 2.11 and the Bradley-Terry objective in Equation 2.12; these are standard assumptions in the cited preference-optimization literature.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Efficient and Effective Alignment of Large Language Models." pith.science (2026). https://pith.science/paper/35WBA2VZ

@misc{pith2026250609329,
  author       = {Pith},
  title        = {Pith review of: Towards Efficient and Effective Alignment of Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/35WBA2VZ}},
  note         = {Machine review of arXiv:2506.09329}
}

read the original abstract

Large language models (LLMs) exhibit remarkable capabilities across diverse tasks, yet aligning them efficiently and effectively with human expectations remains a critical challenge. This thesis advances LLM alignment by introducing novel methodologies in data collection, training, and evaluation. We first address alignment data collection. Existing approaches rely heavily on manually curated datasets or proprietary models. To overcome these limitations, we propose Lion, an adversarial distillation framework that iteratively refines training data by identifying and generating challenging instructions, enabling state-of-the-art zero-shot reasoning. Additionally, we introduce Web Reconstruction (WebR), a fully automated framework that synthesizes instruction-tuning data directly from raw web documents, significantly improving data diversity and scalability over existing synthetic data methods. Next, we enhance alignment training through novel optimization techniques. We develop Learning to Edit (LTE), a framework that enables LLMs to efficiently integrate new knowledge while preserving existing information. LTE leverages meta-learning to improve both real-time and batch knowledge updates. Furthermore, we introduce Bridging and Modeling Correlations (BMC), a refinement of Direct Preference Optimization (DPO) that explicitly captures token-level correlations in preference data, leading to superior alignment across QA and mathematical reasoning tasks. Finally, we tackle the challenge of evaluating alignment. Existing benchmarks emphasize response quality but overlook adherence to specific constraints. To bridge this gap, we introduce FollowBench, a multi-level, fine-grained benchmark assessing LLMs' ability to follow complex constraints across diverse instruction types. Our results expose key weaknesses in current models' constraint adherence, offering insights for future improvements.

Figures

Figures reproduced from arXiv: 2506.09329 by the authors.

**Figure 5.2.** 151 C.2 Training hyperparameters for both LLaMA2-Chat-7B and Qwen-Chat-7B. 152 D.1 Statistics of the training and evaluation datasets. 153 D.2 Various preference optimization objectives and hyperparameter search range. 156 D.3 Hyperparameter values for diverse training settings in DPO-BMC. 156 D.4 Performance comparison across different base models. 157 E.1 Answer template of Example Constraints. 161 xx [PITH_FULL_… view at source ↗

**Figure 1.1.** Roadmap of the thesis. 1.2 Thesis Organization As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p026_1_1.png] view at source ↗

**Figure 2.1.** Demonstration of RL optimization algorithms: DPO, PPO, and GRPO. [PITH_FULL_IMAGE:figures/full_fig_p035_2_1.png] view at source ↗

Figures from the paper (30 more)

**Figure 3.1.** Figure 3.1: An illustration of the distinction between our approach and earlier ones. Previ [PITH_FULL_IMAGE:figures/full_fig_p042_3_1.png]

**Figure 3.2.** Figure 3.2: The overview of our adversarial distillation framework, where we craft a com [PITH_FULL_IMAGE:figures/full_fig_p044_3_2.png]

**Figure 3.** Figure 3: demonstrates that the discrimination stage starts from the Cache Pool, denoted [PITH_FULL_IMAGE:figures/full_fig_p045_3.png]

**Figure 3.3.** Figure 3.3: The top 20 most common root verbs (inner circle) and their top 4 direct noun [PITH_FULL_IMAGE:figures/full_fig_p046_3_3.png]

**Figure 3.4.** Figure 3.4: Relative response quality against ChatGPT on diverse task categories of [PITH_FULL_IMAGE:figures/full_fig_p051_3_4.png]

**Figure 3.5.** Figure 3.5: Performance of Lion-7B and Lion-13B on AGIEval and BBH through the train [PITH_FULL_IMAGE:figures/full_fig_p054_3_5.png]

**Figure 4.1.** Figure 4.1: Our proposed Web Reconstruction method surpasses previous techniques by [PITH_FULL_IMAGE:figures/full_fig_p058_4_1.png]

**Figure 4.2.** Figure 4.2: Overview of the proposed Web Reconstruction (WebR) framework. Leveraging an off-the-shelf LLM, WebR transforms raw web documents into high-quality instruction-response pairs. It strategically assigns each document as either an instruction or a response to trigger t…

**Figure 4.3.** Figure 4.3: Statistics of instruction quality and difficulty. [PITH_FULL_IMAGE:figures/full_fig_p067_4_3.png]

**Figure 4.** Figure 4: illustrates the impact of training data scale on model performance. The results [PITH_FULL_IMAGE:figures/full_fig_p068_4.png]

**Figure 4.4.** Figure 4.4: The impact of training data scale on the average instruction-following perfor [PITH_FULL_IMAGE:figures/full_fig_p069_4_4.png]

**Figure 5.1.** Figure 5.1: Previous knowledge editing methods primarily rely on first memorizing [PITH_FULL_IMAGE:figures/full_fig_p073_5_1.png]

**Figure 5.** Figure 5: , changing the origin city of [PITH_FULL_IMAGE:figures/full_fig_p077_5.png]

**Figure 5.3.** Figure 5.3: Averaged Batch Editing performance on four benchmarks against batch numbers in [1, 10, 100, 1000]. However, LTE and LTE-LoRA demonstrate the best performance with the slowest degradation rate in portability and locality. These results underscore the enhanced robust…

**Figure 5.4.** Figure 5.4: Averaged Sequential Editing performance on four knowledge editing benchmarks against data stream size (log-scale) in [1, 10, 100, 500, 1000]. ment Phase of LTE, which alters the parameters of the initial model, inadvertently compromises the model’s competence in un…

**Figure 6.1.** Figure 6.1: Overview of our proposed BMC framework. (1) In the Bridging Phase, we uti [PITH_FULL_IMAGE:figures/full_fig_p091_6_1.png]

**Figure 6.** Figure 6: , we utilize an off-the-shelf LLM to make targeted modification of [PITH_FULL_IMAGE:figures/full_fig_p092_6.png]

**Figure 6.2.** Figure 6.2: We aggregate varied tokens in ˜yw or yl into more coarser-grained spans. During the DPO training on D˜ , we compute the averaged − log(p) of tokens in different positions of spans. span. Therefore, we adjust the reward factor to also be inversely proportional to th…

**Figure 6.3.** Figure 6.3: Ablation study on data modification proportion in the Bridging Phase. Influence of Data Modification Proportion [PITH_FULL_IMAGE:figures/full_fig_p101_6_3.png]

**Figure 6.4.** Figure 6.4: Ablation study on δ in the Modeling Phase. The average accuracy is presented as the QA performance. all tested values of δ, our method consistently outperforms the DPO baseline, indicating its robustness and effectiveness in stabilizing the learning process. 6.4 Anal…

**Figure 6.5.** Figure 6.5: We segment the 60k training data of UltraFeedback into six equal-sized splits [PITH_FULL_IMAGE:figures/full_fig_p105_6_5.png]

**Figure 6.** Figure 6: depicts the token-level reward assignment [PITH_FULL_IMAGE:figures/full_fig_p105_6.png]

**Figure 6.6.** Figure 6.6: Visualization of token-level rewards assigned by DPO and our method. The [PITH_FULL_IMAGE:figures/full_fig_p106_6_6.png]

**Figure 7.1.** Figure 7.1: FollowBench covers five fine-grained constraint categories and is constructed based on the Multi-level mechanism, which increasingly adds a single constraint to straightforward instructions. On the right, the model that can follow instructions with more constraints i…

**Figure 7.2.** Figure 7.2: FollowBench covers five fine-grained categories of constraints. Within each constraint type, we construct a range of Multi-level instructions by incrementally adding constraints (highlighted in red). There are five levels in total; however, we only display the first …

**Figure 7.** Figure 7: , which sets specific criteria for the retrieved object. Ensuring that LLMs adhere [PITH_FULL_IMAGE:figures/full_fig_p112_7.png]

**Figure 7.4.** Figure 7.4: Prompt template for modelbased evaluation. 7.2.2 Evaluation Protocol Given that nearly half of instructions in FollowBench are open-ended without reference answers, devising a rule-based program to assess the outputs is extremely challenging. To overcome this, inspi…

**Figure 7.5.** Figure 7.5: HSR (%) results in diverse constraint categories. For each category, we com [PITH_FULL_IMAGE:figures/full_fig_p118_7_5.png]

**Figure 7.6.** Figure 7.6: The effect of varying the temperature parameter [PITH_FULL_IMAGE:figures/full_fig_p122_7_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

240 extracted references · 17 canonical work pages

[1]

Explanations for commonsenseqa: New dataset and models

Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khan- delwal, Parag Singla, and Dinesh Garg. Explanations for commonsenseqa: New dataset and models. In Annual Meeting of the Association for Computational Linguis- tics, pages 3050–3065, 2021

2021
[2]

Perplexed by perplexity: Perplexity-based data pruning with small reference models

Zachary Ankner, Cody Blakeney, Kartik Sreenivasan, Max Marion, Matthew L Leav- itt, and Mansheej Paul. Perplexed by perplexity: Perplexity-based data pruning with small reference models. arXiv preprint arXiv:2405.20541, 2024

arXiv 2024
[3]

arxiv dataset, 2023

arXiv.org submitters. arxiv dataset, 2023

2023
[4]

A general language assistant as a laboratory for alignment

Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021

arXiv 2021
[5]

Program synthesis with large language models

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 , 2021

arXiv 2021
[6]

A general theoretical paradigm to understand learning from human preferences

Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Rémi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human preferences. In International Conference on Arti- ficial Intelligence and Statistics, volume 238, pages 4447–4455, 2024

2024
[7]

Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Ni- hal V . Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Ur- mish Thakker, Khalid Almubarak, Xiangru T...

2022
[8]

Qwen technical report

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

arXiv 2023

Show all 240 references

[9]

Training a helpful and harmless assistant with reinforcement learning from human feedback

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Das- Sarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

2022 arXiv
[10]

Constitutional ai: Harmlessness from ai feedback

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. In arXiv, 2022

2022
[11]

Baichuan 2: Open large-scale language models

Baichuan. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305, 2023

2023 arXiv
[12]

PIQA: reasoning about physical commonsense in natural language

Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. PIQA: reasoning about physical commonsense in natural language. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Ap- plications of Artificial Intel...

2020
[13]

On the opportunities and risks of foundation models

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Syd- 109 ney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brun- skill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

2021 arXiv
[14]

Rank analysis of incomplete block designs: I

Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952

1952
[15]

On the resemblance and containment of documents

Andrei Z Broder. On the resemblance and containment of documents. In Proceed- ings. Compression and Complexity of SEQUENCES 1997 (Cat. No. 97TB100171) , pages 21–29. IEEE, 1997

1997
[16]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Pra- fulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey...

1901
[17]

Data diversity matters for robust instruction tuning

Alexander Bukharin and Tuo Zhao. Data diversity matters for robust instruction tuning. arXiv preprint arXiv:2311.14736, 2023

2023 arXiv
[18]

Drlc: Reinforcement learning with dense rewards from llm critic

Meng Cao, Lei Shu, Lei Yu, Yun Zhu, Nevan Wichers, Yinxiao Liu, and Lei Meng. Drlc: Reinforcement learning with dense rewards from llm critic. In arXiv, 2024

2024
[19]

Instruction mining: High-quality instruc- tion data selection for large language models.arXiv preprint arXiv:2307.06290, 1(3):6, 2023

Yihan Cao, Yanbin Kang, and Lichao Sun. Instruction mining: High-quality instruc- tion data selection for large language models.arXiv preprint arXiv:2307.06290, 1(3):6, 2023

2023 arXiv
[20]

Wit3: Web inventory of transcribed and translated talks

Mauro Cettolo, Christian Girardi, and Marcello Federico. Wit3: Web inventory of transcribed and translated talks. In Proceedings of the Conference of European Associa- tion for Machine Translation (EAMT), pages 261–268, 2012

2012
[21]

Dense reward for free in reinforcement learning from human feedback

Alex J Chan, Hao Sun, Samuel Holt, and Mihaela van der Schaar. Dense reward for free in reinforcement learning from human feedback. In arXiv, 2024. 110

2024
[22]

Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning

Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuetao Ma, Yi- fan Yanggong, and Junbo Zhao. Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning. arXiv preprint arXiv:2305.09246, 2023

2023 arXiv
[23]

Controllable text generation with language constraints

Howard Chen, Huihan Li, Danqi Chen, and Karthik Narasimhan. Controllable text generation with language constraints. arXiv preprint arXiv:2212.10466, 2022

2022 arXiv
[24]

Alpagasus: Training a better alpaca with fewer data

Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.08701, 2023

2023 arXiv
[25]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G...

2021
[26]

DoG- instruct: Towards premium instruction-tuning data via text-grounded instruction wrapping

Yongrui Chen, Haiyun Jiang, Xinting Huang, Shuming Shi, and Guilin Qi. DoG- instruct: Towards premium instruction-tuning data via text-grounded instruction wrapping. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North Americ...

2024
[27]

Improving large language models via fine-grained 111 reinforcement learning with minimum editing constraint

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, and Ji-Rong Wen. Improving large language models via fine-grained 111 reinforcement learning with minimum editing constraint. In Findings of the Associa- tion for Computational Linguistics, pages 569...

2024
[28]

Low-redundant optimization for large language model alignment

Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. Low-redundant optimization for large language model alignment. In arXiv, 2024

2024
[29]

Replacing language model for style transfer

Pengyu Cheng and Ruineng Li. Replacing language model for style transfer. arXiv preprint arXiv:2211.07343, 2022

2022 arXiv
[30]

Siyuan Cheng, Bozhong Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, and Ningyu Zhang. Can we edit multimodal large language models? In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc...

2023
[31]

Gonzalez, Ion Stoica, and Eric P

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lian- min Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P . Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023

2023
[32]

Learning phrase representa- tions using rnn encoder-decoder for statistical machine translation

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representa- tions using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014

2014 arXiv
[33]

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebas- tian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vi...

2023
[34]

Christiano, Jan Leike, Tom B

Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, pages 4299–4307, 2017

2017
[35]

All that’s’ human’is not gold: Evaluating human evaluation of generated text

Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A Smith. All that’s’ human’is not gold: Evaluating human evaluation of generated text. arXiv preprint arXiv:2107.00061, 2021

2021 arXiv
[36]

Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

2018
[37]

The future landscape of large language models in medicine

Jan Clusmann, Fiona R Kolbinger, Hannah Sophie Muti, Zunamys I Carrero, Jan- Niklas Eckardt, Narmin Ghaffari Laleh, Chiara Maria Lavinia Löffler, Sophie- Caroline Schwarzkopf, Michaela Unger, Gregory P Veldhuizen, et al. The future landscape of large language models in medicin...

2023
[38]

Training verifiers to solve math word prob- lems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word prob- lems. arXiv preprint arXiv:2110.14168, 2021

arXiv
[39]

Evaluating the ripple effects of knowledge editing in language models

Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. Evaluating the ripple effects of knowledge editing in language models. CoRR, abs/2307.12976, 2023

2023 arXiv
[40]

Redpajama: An open source recipe to reproduce llama training dataset, 2023

Together Computer. Redpajama: An open source recipe to reproduce llama training dataset, 2023. 113

2023
[41]

Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023

Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023

2023
[42]

Opencompass: A universal evaluation platform for foundation models

OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass, 2023

2023
[43]

Ultrafeedback: Boosting language models with high-quality feedback

Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with high-quality feedback. In arXiv, 2023

2023
[44]

Knowl- edge neurons in pretrained transformers

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowl- edge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Associ- ation for Computational Linguistic...

2022
[45]

Editing factual knowledge in language models

Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen- tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6491–...

2021
[46]

Bert: Pre- training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. In Proceed- ings of the 2019 conference of the North American chapter of the association for computa- tional linguistics: human ...

2019
[47]

Enhancing chat language models by scaling high-quality instructional conversations

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. In Conference on Empirical Methods in Natural Language Processing, pages 3029–3051, 2023. 114

2023
[48]

A survey on in-context learning

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022

2022 arXiv
[49]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arX...

2010 arXiv
[50]

Glm: General language model pretraining with autoregressive blank infilling

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

2022
[51]

The llama 3 herd of models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. In arXiv, 2024

2024
[52]

Length- controlled alpacaeval: A simple way to debias automatic evaluators

Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length- controlled alpacaeval: A simple way to debias automatic evaluators. In arXiv, 2024

2024
[53]

Fact-checking the output of large language models via token-level uncertainty quantification

Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. Fact-checking the output of large language models via token-level uncert...

2024
[54]

Hierarchical neural story generation

Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898, Melbourne, Australia, July 2018. Association for Computational Linguistics

2018
[55]

Data-free adversarial distillation

Gongfan Fang, Jie Song, Chengchao Shen, Xinchao Wang, Da Chen, and Mingli Song. Data-free adversarial distillation. CoRR, abs/1912.11006, 2019. 115

1912 arXiv
[56]

Gptscore: Evaluate as you desire

Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. Gptscore: Evaluate as you desire. In Conference of the North American Chapter of the Association for Computa- tional Linguistics, pages 6556–6576, 2024

2024
[57]

Preference learning and ranking by pairwise comparison

Johannes Fürnkranz and Eyke Hüllermeier. Preference learning and ranking by pairwise comparison. In Preference learning, pages 65–82. Springer, 2010

2010
[58]

Scaling synthetic data creation with 1,000,000,000 personas

Tao Ge, Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094, 2024

2024 arXiv
[59]

Koala: A dialogue model for academic research

Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. Koala: A dialogue model for academic research. Blog post, April, 1, 2023

2023
[60]

Did aristotle use a laptop? A question answering benchmark with implicit rea- soning strategies

Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Be- rant. Did aristotle use a laptop? A question answering benchmark with implicit rea- soning strategies. Transactions of the Association for Computational Linguistics, 9:346– 361, 2021

2021
[61]

Chatgpt outperforms crowd- workers for text-annotation tasks

Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowd- workers for text-annotation tasks. arXiv preprint arXiv:2303.15056, 2023

2023 arXiv
[62]

SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization

Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70–79, Hong Kong, China, November 2019. Association for...

2019
[64]

English gigaword

David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. English gigaword. Lin- guistic Data Consortium, Philadelphia, 4(1):34, 2003

2003
[65]

A survey on llm-as-a-judge

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594, 2024. 116

2024 arXiv
[66]

The false promise of imitating proprietary llms

Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. The false promise of imitating proprietary llms. CoRR, abs/2305.15717, 2023

2023 arXiv
[67]

Deepseek-r1: Incentivizing rea- soning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing rea- soning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

2025 arXiv
[68]

Beyond imita- tion: Leveraging fine-grained quality signals for alignment

Geyang Guo, Ranchi Zhao, Tianyi Tang, Xin Zhao, and Ji-Rong Wen. Beyond imita- tion: Leveraging fine-grained quality signals for alignment. In International Confer- ence on Learning Representations, 2024

2024
[69]

Coopera- tive inverse reinforcement learning

Dylan Hadfield-Menell, Stuart J Russell, Pieter Abbeel, and Anca Dragan. Coopera- tive inverse reinforcement learning. Advances in neural information processing systems, 29, 2016

2016
[70]

Aging with grace: Lifelong model editing with discrete key- value adaptors

Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with grace: Lifelong model editing with discrete key- value adaptors. In Advances in Neural Information Processing Systems, 2023

2023
[71]

Does localization inform editing? surprising differences in causality-based localization vs

Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models, 2023

2023
[72]

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview....

2021
[73]

Measuring mathematical problem solving with the math dataset

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

2021
[74]

Knowledge distil- lation with adversarial samples supporting decision boundary

Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi. Knowledge distil- lation with adversarial samples supporting decision boundary. In The Thirty-Third 117 AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Ap- plications of Artificial In...

2019
[75]

Orpo: Monolithic preference optimiza- tion without reference model

Jiwoo Hong, Noah Lee, and James Thorne. Orpo: Monolithic preference optimiza- tion without reference model. In arXiv, 2024

2024
[76]

Parameter- efficient transfer learning for nlp

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter- efficient transfer learning for nlp. In International conference on machine learning , pages 2790–2799. PMLR, 2019

2019
[77]

LoRA: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022

2022
[78]

Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech

F Huang, H Kwak, and J An. Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech. arxiv, 2023

2023
[79]

Deep q-networks

Yanhua Huang. Deep q-networks. Deep reinforcement learning: fundamentals, research and applications, pages 135–160, 2020

2020
[80]

C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models

Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, et al. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. Advances in Neural Information Processing Systems, 36:6...

2023
[81]

Smith, Iz Beltagy, and Han- naneh Hajishirzi

Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Pe- ters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Han- naneh Hajishirzi. Camels in a changing climate: Enhancing lm adaptation with tulu 2, 2023

2023
[82]

Aligner: Achieving efficient alignment through weak-to-strong correction

Jiaming Ji, Boyuan Chen, Hantao Lou, Donghai Hong, Borong Zhang, Xuehai Pan, Juntao Dai, and Yaodong Yang. Aligner: Achieving efficient alignment through weak-to-strong correction. In arXiv, 2024. 118

2024
[83]

Mistral 7b

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Deven- dra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, et al. Mistral 7b. In arXiv, 2023

2023
[84]

What disease does this patient have? a large-scale open domain ques- tion answering dataset from medical exams

Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain ques- tion answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021

2021
[85]

Plus disease in retinopathy of prematurity: im- proving diagnosis by ranking disease severity and using quantitative image analy- sis

Jayashree Kalpathy-Cramer, J Peter Campbell, Deniz Erdogmus, Peng Tian, Dha- ranish Kedarisetti, Chace Moleta, James D Reynolds, Kelly Hutcheson, Michael J Shapiro, Michael X Repka, et al. Plus disease in retinopathy of prematurity: im- proving diagnosis by ranking disease sev...

2016
[86]

Scaling laws for neural language models

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

2001 arXiv
[87]

A survey of reinforcement learning from human feedback

Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. A survey of reinforcement learning from human feedback. arXiv preprint arXiv:2312.14925, 10, 2023

2023
[88]

QASC: A dataset for question answering via sentence composition

Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. QASC: A dataset for question answering via sentence composition. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8082–8090, 2020

2020
[89]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In arXiv, 2014

2014
[90]

MAWPS: A math word problem repository

Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. MAWPS: A math word problem repository. In Conference of the North American Chapter of the Association for Computational Linguistics , pages 1152–1157, 2016

2016
[91]

Openassistant conversations-democratizing large language model alignment

Andreas Köpf, Yannic Kilcher, Dimitri Von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, 119 et al. Openassistant conversations-democratizing large language model alignment. Advances in Neural Information ...

2023
[92]

Spoc: Search-based pseudocode to code

Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy S Liang. Spoc: Search-based pseudocode to code. Advances in Neural Information Processing Systems, 32, 2019

2019
[93]

Mt-eval: A multi-turn capabilities evaluation benchmark for large language models

Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, and Kam-Fai Wong. Mt-eval: A multi-turn capabilities evaluation benchmark for large language models. CoRR, abs/2401.16745, 2024

2024 arXiv
[94]

Large language models in law: A survey

Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and Philip S Yu. Large language models in law: A survey. AI Open, 2024

2024
[95]

Ds-1000: A natural and reliable benchmark for data science code generation

Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettle- moyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pages 18319–18345....

2023
[96]

RLAIF vs

Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. RLAIF vs. RLHF: scaling reinforcement learning from human feedback with AI feedback. In International Confere...

2024
[97]

Scalable agent alignment via reward modeling: a research direction

Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv:1811.07871, 2018

2018 arXiv
[98]

Zero-shot relation extraction via reading comprehension

Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Roger Levy and Lucia Specia, editors, Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3-...

2017
[99]

Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for 120 knowledge-intensive NLP tasks. In Hugo L...

2020
[100]

Cmmlu: Measuring massive multitask language un- derstanding in chinese

Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. Cmmlu: Measuring massive multitask language un- derstanding in chinese. arXiv preprint arXiv:2306.09212, 2023

2023 arXiv
[101]

Synthetic data (almost) from scratch: Generalized instruction tuning for language models

Haoran Li, Qingxiu Dong, Zhengyang Tang, Chaojun Wang, Xingxing Zhang, Haoyang Huang, Shaohan Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, et al. Synthetic data (almost) from scratch: Generalized instruction tuning for language models. arXiv preprint arXiv:2402.13064, 2024

2024 arXiv
[102]

From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning

Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jian- zong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning. In Kevin Duh, Helena Gomez, and Steven Bethard, editor...

2024
[103]

Document-level event argument extraction by conditional generation

Sha Li, Heng Ji, and Jiawei Han. Document-level event argument extraction by conditional generation. In Kristina Toutanova, Anna Rumshisky, Luke Zettle- moyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors,Proceedin...

2021
[104]

From live data to high-quality benchmarks: The arena-hard pipeline, 2024

Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Banghua Zhu, Joseph E Gon- zalez, and Ion Stoica. From live data to high-quality benchmarks: The arena-hard pipeline, 2024

2024
[105]

Self-alignment with instruction backtranslation

Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Ja- 121 son E Weston, and Mike Lewis. Self-alignment with instruction backtranslation. In The Twelfth International Conference on Learning Representations, 2024

2024
[106]

Prefix-tuning: Optimizing continuous prompts for generation

Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021

2021 arXiv
[107]

Hashimoto

Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/ alpaca_eval, 2023

2023
[108]

Large language models in finance: A survey

Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen. Large language models in finance: A survey. In Proceedings of the fourth ACM international conference on AI in finance, pages 374–382, 2023

2023
[109]

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004

2004
[110]

OpenSubtitles2016: Extracting large parallel cor- pora from movie and TV subtitles

Pierre Lison and Jörg Tiedemann. OpenSubtitles2016: Extracting large parallel cor- pora from movie and TV subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16) , pages 923–929, Portorož, Slovenia, May 2016. European Lang...

2016
[111]

M3ke: A massive multi-level multi-subject knowledge evaluation benchmark for chinese large lan- guage models

Chuang Liu, Renren Jin, Yuqi Ren, Linhao Yu, Tianyu Dong, Xiaohan Peng, Shut- ing Zhang, Jianxiang Peng, Peiyi Zhang, Qingqing Lyu, et al. M3ke: A massive multi-level multi-subject knowledge evaluation benchmark for chinese large lan- guage models. arXiv preprint arXiv:2305.10...

2023 arXiv
[112]

Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems , 36:21558– 21572, 2023

2023
[113]

Statistical rejection sampling improves preference optimization

Tianqi Liu, Yao Zhao, Rishabh Joshi, Misha Khalman, Mohammad Saleh, Peter J Liu, and Jialu Liu. Statistical rejection sampling improves preference optimization. In International Conference on Learning Representations, 2024. 122

2024
[114]

Agentbench: Evaluating llms as agents

Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688, 2023

2023 arXiv
[115]

Roberta: A robustly optimized bert pretraining approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019

1907 arXiv
[116]

Swin transformer: Hierarchical vision transformer using shifted windows

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012–10022, 2021

2021
[117]

Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning

Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Ra- jpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In International Conference on Learning Representations, 2023

2023
[118]

Inverse scaling: When bigger isn’t better

Ian R McKenzie, Alexander Lyzhov, Michael Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Ross, Alisa Liu, et al. Inverse scaling: When bigger isn’t better. arXiv preprint arXiv:2306.09479, 2023

2023 arXiv
[119]

Locating and editing factual associations in GPT

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Pro- cessing Systems 35: Annual Conference on Ne...

2022
[120]

Andonian, Yonatan Belinkov, and David Bau

Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenRe- view.net, 2023

2023
[121]

Simpo: Simple preference optimization with a reference-free reward

Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. In arXiv, 2024. 123

2024
[122]

Paul Micaelli and Amos J. Storkey. Zero-shot knowledge transfer via adversarial belief matching. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Flo- rence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Ann...

2019
[123]

Can a suit of ar- mor conduct electricity? A new dataset for open book question answering

Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of ar- mor conduct electricity? A new dataset for open book question answering. In Con- ference on Empirical Methods in Natural Language Processing, pages 2381–2391, 2018

2018
[124]

George A. Miller. WordNet: A lexical database for English. In Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992, 1992

1992
[125]

Cross- mishra-etal-2022-cross

Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross- mishra-etal-2022-cross. In Smaranda Muresan, Preslav Nakov, and Aline Villavi- cencio, editors, Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: L...

2022
[126]

Man- ning

Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Man- ning. Fast model editing at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022

2022
[127]

Manning, and Chelsea Finn

Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory-based model editing at scale. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, Interna- tional Conference on Machine Learning,...

2022
[128]

A corpus and cloze evaluation for deeper understanding of commonsense stories

Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Ba- tra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 124 2016 Conference of the North American C...

2016
[129]

Orca: Progressive learning from complex explanation traces of GPT-4

Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Hassan Awadallah. Orca: Progressive learning from complex explanation traces of GPT-4. CoRR, abs/2306.02707, 2023

2023 arXiv
[130]

Abstractive text summarization using sequence-to-sequence rnns and beyond

Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023, 2016

2016 arXiv
[131]

Cohen, and Mirella Lapata

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summariza- tion. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018 Confer...

2018
[132]

Algorithms for inverse reinforcement learning

Andrew Y Ng, Stuart Russell, et al. Algorithms for inverse reinforcement learning. In Icml, volume 1, page 2, 2000

2000
[133]

Better alignment with instruction back-and-forth translation, 2024

Thao Nguyen, Jeffrey Li, Sewoong Oh, Ludwig Schmidt, Jason Weston, Luke Zettle- moyer, and Xian Li. Better alignment with instruction back-and-forth translation, 2024

2024
[134]

The E2E dataset: New chal- lenges for end-to-end generation

Jekaterina Novikova, Ondˇ rej Dušek, and Verena Rieser. The E2E dataset: New chal- lenges for end-to-end generation. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 201–206, Saarbrücken, Germany, August 2017. As- sociation for Computational L...

2017
[135]

GPT-4 technical report

OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023

2023 arXiv
[136]

Chatgpt: Optimizing language models for dialogue

TB OpenAI. Chatgpt: Optimizing language models for dialogue. OpenAI, 2022

2022
[137]

Train- 125 ing language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Train- 125 ing language models to follow instructions with human feedback. Advances in Neu- ral Information Processing Systems,...

2022
[138]

Preference modelling

Meltem Öztürk, Alexis Tsoukiàs, and Philippe Vincke. Preference modelling. Mul- tiple criteria decision analysis: State of the art surveys, 78:27–59, 2005

2005
[139]

West-of-n: Synthetic preference generation for improved reward modeling

Alizée Pace, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Sev- eryn. West-of-n: Synthetic preference generation for improved reward modeling. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models, 2024

2024
[140]

The effects of reward misspecification: Mapping and mitigating misaligned models

Alexander Pan, Kush Bhatia, and Jacob Steinhardt. The effects of reward misspecification: Mapping and mitigating misaligned models. arXiv preprint arXiv:2201.03544, 2022

2022 arXiv
[141]

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

2002
[142]

Disentangling length from quality in direct preference optimization

Ryan Park, Rafael Rafailov, Stefano Ermon, and Chelsea Finn. Disentangling length from quality in direct preference optimization. In arXiv, 2024

2024
[143]

Openweb- math: An open dataset of high-quality mathematical web text

Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. Openweb- math: An open dataset of high-quality mathematical web text. In The Twelfth Inter- national Conference on Learning Representations, 2024

2024
[144]

Faster and smaller n-gram language models

Adam Pauls and Dan Klein. Faster and smaller n-gram language models. In Pro- ceedings of the 49th annual meeting of the Association for Computational Linguistics: Hu- man Language Technologies, pages 258–267, 2011

2011
[145]

The analysis of permutations

Robin L Plackett. The analysis of permutations. Journal of the Royal Statistical Society Series C: Applied Statistics, 24(2):193–202, 1975

1975
[146]

Scaling laws for reward model overop- timization in direct alignment algorithms

Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sushil Sikchi, Joey Hejna, Brad Knox, Chelsea Finn, and Scott Niekum. Scaling laws for reward model overop- timization in direct alignment algorithms. Advances in Neural Information Processing Systems, 37:126207–126242, 2024. 126

2024
[147]

From r to q*: Your lan- guage model is secretly a q-function

Rafael Rafailov, Joey Hejna, Ryan Park, and Chelsea Finn. From r to q*: Your lan- guage model is secretly a q-function. In arXiv, 2024

2024
[148]

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Er- mon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems, 2023

2023
[149]

A comprehensive sur- vey of bias in llms: Current landscape and future directions

Rajesh Ranjan, Shailja Gupta, and Surya Narayan Singh. A comprehensive sur- vey of bias in llms: Current landscape and future directions. arXiv preprint arXiv:2409.16430, 2024

2024 arXiv
[150]

Sentence-bert: Sentence embeddings using siamese bert-networks

Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019

2019
[151]

Wino- grande: An adversarial winograd schema challenge at scale

Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Wino- grande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641, 2019

1907 arXiv
[152]

Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Tae- woon Kim, Gunjan Chhablani, Nihal V . Nayak...

2022
[153]

High-dimensional continuous control using generalized advantage estimation

John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. 127

2015 arXiv
[154]

Proximal policy optimization algorithms

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

2017 arXiv
[155]

Role-play with large lan- guage models

Murray Shanahan, Kyle McDonell, and Laria Reynolds. Role-play with large lan- guage models. arXiv preprint arXiv:2305.16367, 2023

2023 arXiv
[156]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

2024 arXiv
[157]

Industrycorpus2, 2024

Xiaofeng Shi, Lulu Zhao, Hua Zhou, and Donglin Hao. Industrycorpus2, 2024

2024
[158]

Pyrkin, Sergei Popov, and Artem Babenko

Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry V . Pyrkin, Sergei Popov, and Artem Babenko. Editable neural networks. In 8th International Conference on Learning Repre- sentations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020

2020
[159]

Con- trolling style in generated dialogue

Eric Michael Smith, Diana Gonzalez-Rico, Emily Dinan, and Y-Lan Boureau. Con- trolling style in generated dialogue. arXiv preprint arXiv:2009.10855, 2020

2009 arXiv
[160]

Beyond the imitation game: Quantifying and extrapolating the capa- bilities of language models

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga- Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capa- bilities of language models. arXiv preprin...

2022 arXiv
[161]

Le, Ed H

Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V . Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. CoRR, abs/2210.09261, 2022

arXiv
[162]

Common- senseQA: A question answering challenge targeting commonsense knowledge

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Common- senseQA: A question answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter o...

2019
[163]

Struc- bench: Are large language models really good at generating complex structured data? arXiv preprint arXiv:2309.08963, 2023

Xiangru Tang, Yiming Zong, Yilun Zhao, Arman Cohan, and Mark Gerstein. Struc- bench: Are large language models really good at generating complex structured data? arXiv preprint arXiv:2309.08963, 2023

2023 arXiv
[164]

Hashimoto

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Car- los Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/ stanford_alpaca, 2023

2023
[165]

Qwen2.5: A party of foundation models, September 2024

Qwen Team. Qwen2.5: A party of foundation models, September 2024

2024
[166]

Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023

Teknium. Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023

2023
[167]

Parallel data, tools and interfaces in opus

Jörg Tiedemann. Parallel data, tools and interfaces in opus. In Lrec, volume 2012, pages 2214–2218. Citeseer, 2012

2012
[168]

Tjong Kim Sang and Fien De Meulder

Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147, 2003

2003
[169]

Behavioral cloning from observa- tion

Faraz Torabi, Garrett Warnell, and Peter Stone. Behavioral cloning from observa- tion. arXiv preprint arXiv:1805.01954, 2018

2018 arXiv
[170]

Llama: Open and efficient foundation language models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

2023 arXiv
[171]

Llama 2: Open foundation and fine-tuned chat models, 2023

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy...

2023
[172]

Style control for schema- guided natural language generation

Alicia Tsai, Shereen Oraby, Vittorio Perera, Jiun-Yu Kao, Yuheng Du, Anjali Narayan-Chen, Tagyoung Chung, and Dilek Hakkani-Tur. Style control for schema- guided natural language generation. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational A...

2021
[173]

Zephyr: Direct distillation of lm alignment

Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. Zephyr: Direct distillation of lm alignment. In arXiv, 2023

2023
[174]

Wikidata: a free collaborative knowledge- base

Denny Vrandeˇ ci´ c and Markus Krötzsch. Wikidata: a free collaborative knowledge- base. Communications of the ACM, 57(10):78–85, 2014

2014
[175]

Cross- lingual knowledge editing in large language models

Jiaan Wang, Yunlong Liang, Zengkui Sun, Yuxuan Cao, and Jiarong Xu. Cross- lingual knowledge editing in large language models. CoRR, abs/2309.08952, 2023

2023 arXiv
[176]

A survey on data selection for llm instruction tuning

Jiahao Wang, Bolin Zhang, Qianlong Du, Jiajun Zhang, and Dianhui Chu. A survey on data selection for llm instruction tuning. arXiv preprint arXiv:2402.05123, 2024

2024 arXiv
[177]

Large language models are not fair evaluators

Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. CoRR, abs/2305.17926, 2023

2023 arXiv
[178]

130 Easyedit: An easy-to-use knowledge editing framework for large language mod- els

Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Huajun Chen. 130 Easyedit: An easy-to-use knowledge editing framework for large language mod- els. CoRR, abs/2308.07269, 2023

2023 arXiv
[179]

On the essence and prospect: An investigation of alignment approaches for big models

Xinpeng Wang, Shitong Duan, Xiaoyuan Yi, Jing Yao, Shanlin Zhou, Zhihua Wei, Peng Zhang, Dongkuan Xu, Maosong Sun, and Xing Xie. On the essence and prospect: An investigation of alignment approaches for big models. arXiv preprint arXiv:2403.04204, 2024

2024 arXiv
[180]

Smith, Daniel Khashabi, and Hannaneh Hajishirzi

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics...

2023
[181]

Super-NaturalInstructions: Gener- alization via declarative instructions on 1600+ NLP tasks

Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amir- reza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Is- han Purohit, Ishani Mondal, Jacob Anderson, Kirby K...

2022
[182]

Interac- tive natural language processing

Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, et al. Interac- tive natural language processing. arXiv preprint arXiv:2305.13246, 2023

2023 arXiv
[183]

A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more

Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, et al. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. In arXiv, 2024. 131

2024
[184]

Dai, and Quoc V Le

Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022

2022
[185]

Emergent abilities of large language models

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022

2022 arXiv
[186]

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824– 24837, 2022

2022
[187]

Ethical and social risks of harm from language models

Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po- Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359 , 2021

2021 arXiv
[188]

Some moral and technical consequences of automation: As ma- chines learn they may develop unforeseen strategies at rates that baffle their pro- grammers

Norbert Wiener. Some moral and technical consequences of automation: As ma- chines learn they may develop unforeseen strategies at rates that baffle their pro- grammers. Science, 131(3410):1355–1358, 1960

1960
[189]

A sur- vey of preference-based reinforcement learning methods

Christian Wirth, Riad Akrour, Gerhard Neumann, and Johannes Fürnkranz. A sur- vey of preference-based reinforcement learning methods. Journal of Machine Learn- ing Research, 18(136):1–46, 2017

2017
[190]

Style over substance: Evaluation biases for large language models

Minghao Wu and Alham Fikri Aji. Style over substance: Evaluation biases for large language models. arXiv preprint arXiv:2307.03025, 2023

2023 arXiv
[191]

Less: Selecting influential data for targeted instruction tuning

Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402.04333, 2024

2024 arXiv
[192]

On hallucination and predictive uncertainty in conditional language generation

Yijun Xiao and William Yang Wang. On hallucination and predictive uncertainty in conditional language generation. In Conference of the European Chapter of the Associa- tion for Computational Linguistics, pages 2734–2744, 2021. 132

2021
[193]

Finben: An holistic financial benchmark for large language models

Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Kuang, Chenhan Yuan, Kailai Yang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, GUOJUN XIONG, Zhiyang Deng, Yuechen Jiang, Zhiy...

2024
[194]

WizardLM: Empowering large pre-trained lan- guage models to follow complex instructions

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. WizardLM: Empowering large pre-trained lan- guage models to follow complex instructions. In The Twelfth International Conference on Learning Representations, 2024

2024
[195]

Beyond goldfish memory: Long-term open-domain conversation

Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open-domain conversation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors,Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long P...

2022
[196]

Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing

Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing. arXiv preprint arXiv:2406.08464, 2024

2024 arXiv
[197]

Foundation models for decision making: Problems, methods, and opportu- nities

Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, and Dale Schuur- mans. Foundation models for decision making: Problems, methods, and opportu- nities. arXiv preprint arXiv:2303.04129, 2023

2023 arXiv
[198]

React: Synergizing reasoning and acting in language models

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022

2022 arXiv
[199]

Metamath: Bootstrap your 133 own mathematical questions for large language models

Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your 133 own mathematical questions for large language models. In International Conference on Learning Representations, 2024

2024
[200]

Mates: Model-aware data selection for efficient pretraining with data influence models

Zichun Yu, Spandan Das, and Chenyan Xiong. Mates: Model-aware data selection for efficient pretraining with data influence models. Advances in Neural Information Processing Systems, 37:108735–108759, 2024

2024
[201]

Bartscore: Evaluating generated text as text generation

Weizhe Yuan, Graham Neubig, and Pengfei Liu. Bartscore: Evaluating generated text as text generation. In Advances in Neural Information Processing Systems , pages 27263–27277, 2021

2021
[202]

The ai alignment problem: why it is hard, and where to start

Eliezer Yudkowsky. The ai alignment problem: why it is hard, and where to start. Symbolic Systems Distinguished Speaker, 4(1), 2016

2016
[203]

Mammoth2: Scaling in- structions from the web

Xiang Yue, Tuney Zheng, Ge Zhang, and Wenhu Chen. Mammoth2: Scaling in- structions from the web. Advances in Neural Information Processing Systems, 2024

2024
[204]

A normalized levenshtein distance metric

Li Yujian and Liu Bo. A normalized levenshtein distance metric. IEEE transactions on pattern analysis and machine intelligence, 29(6):1091–1095, 2007

2007
[205]

A survey of controllable text generation using transformer-based pre-trained language models

Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. A survey of controllable text generation using transformer-based pre-trained language models. ACM Computing Surveys, 2022

2022
[206]

A comprehensive study of knowledge editing for large language models

Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. ...

2024 arXiv
[207]

Instruction tuning for large lan- guage models: A survey

Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large lan- guage models: A survey. arXiv preprint arXiv:2308.10792, 2023

2023
[208]

Generating informative and diverse conversational responses via adver- 134 sarial information maximization

Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. Generating informative and diverse conversational responses via adver- 134 sarial information maximization. Advances in Neural Information Processing Systems, 31, 2018

2018
[209]

Wildchat: 1m chatGPT interaction logs in the wild

Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatGPT interaction logs in the wild. In The Twelfth International Con- ference on Learning Representations, 2024

2024
[210]

Large language models are effective table-to-text generators, evaluators, and feedback providers

Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, and Arman Cohan. Large language models are effective table-to-text generators, evaluators, and feedback providers. arXiv preprint arXiv:2305.14987, 2023

2023 arXiv
[211]

Gonzalez, and Ion Stoica

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yong- hao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Inf...

2023
[212]

Agieval: A human-centric benchmark for evaluating foundation models

Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models. CoRR, abs/2304.06364, 2023

2023 arXiv
[213]

Manning, Christopher Potts, and Danqi Chen

Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. Mquake: Assessing knowledge editing in language models via multi- hop questions. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods...

2023
[214]

write an abstract about the following method

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large lan- guage models, 2023. 135 APPENDIX A APPENDIX FOR CHAPTER 3 A.1 Data Statistics Table A.1 and Table A.2 show the data statis...

2023
[216]

Make the minimal modifications necessary (changing the fewest words) to correct the Prediction
[217]

Only output the complete Corrected Prediction without saying anything else
[218]

extract one named entity that is a location

If the Prediction is already good enough, simply output 'None'. **Problem:** {𝒙} **Correct Answer:** {𝒚𝒘} **Prediction:** {𝒚𝒍} **Corrected Prediction:** Prompt Template for Question Answering and Mathematical Reasoning Tasks Figure D.1: Prompt template of targeted modification...

2003
[221]

#Given Instruction# {given_instruction} #Rewriting Requirement# Please add one proper content constraint to the #Given Instruction#

You should try your best not to make the #Rewritten Instruction# become verbose, #Rewritten Instruction# can only add 10 to 20 words into #Given Instruction#. #Given Instruction# {given_instruction} #Rewriting Requirement# Please add one proper content constraint to the #Given...
[222]

Add a Subtask or Another Related Question
[223]

Narrow Down the Topic: Instead of a general theme or topic, provide a more specific subset
[224]

Set a Higher Standard: Raise the bar for what's considered acceptable or successful
[225]

Limit Resources: Restrict the number or type of resources someone can use
[226]

Introduce Specific Criteria: Mandate particular components or features that must be included
[227]

answer":

Specifying Sequence: Dictate the order in which certain steps or actions should be taken. #Rewritten Instruction# Prompt Template (Open-ended Question Answering in Content Constraints) Figure E.1: The prompt template for Open-ended Question Answering in Content Con- straints. ...
[230]

#Given Instruction# {given_instruction} #Rewriting Requirement# Please add one proper style constraint that #Given Instruction# does not have

You should try your best not to make the #Rewritten Instruction# become verbose, #Rewritten Instruction# can only add 10 to 20 words into #Given Instruction#. #Given Instruction# {given_instruction} #Rewriting Requirement# Please add one proper style constraint that #Given Ins...
[231]

Tone and Emotion: Specify the desired emotional tone for the response
[232]

Writing Style: Ask the AI to mimic a specific author's writing style
[233]

Contradiction: Ask the AI to provide a response that contradicts the previous statement or take a stance opposite to its prior response
[234]

Ambiguity: Instruct the AI to create responses with intentional ambiguity or double meanings
[235]

Humor or Satire: Request that the response be humorous or satirical, requiring the AI to generate jokes or witty remarks. #Rewritten Instruction# Prompt Template (Open-ended Question Answering in Style Constraints) Figure E.2: The prompt template for Open-ended Question Answer...
[236]

Your rewriting cannot omit the non-text parts such as the table and code in #Given Instruction#
[237]

#Rewritten Instruction# must be reasonable and must be understood and responded by humans
[238]

#Given Instruction# {given_instruction} #Rewriting Requirement# Please add one proper format constraint that #Given Instruction# does not have

You should try your best not to make the #Rewritten Instruction# become verbose, #Rewritten Instruction# can only add 10 to 20 words into #Given Instruction#. #Given Instruction# {given_instruction} #Rewriting Requirement# Please add one proper format constraint that #Given In...
[239]

Length: Imposing constraints on the length of individual words, sentences, or paragraphs
[240]

Hierarchical Instructions: Providing instructions that have a hierarchical structure, where the AI needs to understand and follow a hierarchy of tasks to construct a response
[241]

Special Output Format: Asking the AI to respond by using data format like table, json, HTML, LaTeX, etc
[242]

Morphological Constraints: Asking the AI to avoid or use specific morphemes
[243]

Multi-lingual Constraints: Asking the AI to respond in multiple languages or switch between languages according to complex patterns
[244]

Incorporation of Specific Literary Devices: Requiring the inclusion of specific, and perhaps numerous, literary devices
[245]

Following a Specific Grammatical Structure: Requiring the AI to create responses that strictly follow a particular grammatical structure. #Rewritten Instruction# Prompt Template (Open-ended Question Answering in Format Constraints) Figure E.3: The prompt template for Open-ende...
[2016]

Association for Computational Linguistics

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Explanations for commonsenseqa: New dataset and models

Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khan- delwal, Parag Singla, and Dinesh Garg. Explanations for commonsenseqa: New dataset and models. In Annual Meeting of the Association for Computational Linguis- tics, pages 3050–3065, 2021

2021

[2] [2]

Perplexed by perplexity: Perplexity-based data pruning with small reference models

Zachary Ankner, Cody Blakeney, Kartik Sreenivasan, Max Marion, Matthew L Leav- itt, and Mansheej Paul. Perplexed by perplexity: Perplexity-based data pruning with small reference models. arXiv preprint arXiv:2405.20541, 2024

arXiv 2024

[3] [3]

arxiv dataset, 2023

arXiv.org submitters. arxiv dataset, 2023

2023

[4] [4]

A general language assistant as a laboratory for alignment

Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021

arXiv 2021

[5] [5]

Program synthesis with large language models

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 , 2021

arXiv 2021

[6] [6]

A general theoretical paradigm to understand learning from human preferences

Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Rémi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human preferences. In International Conference on Arti- ficial Intelligence and Statistics, volume 238, pages 4447–4455, 2024

2024

[7] [7]

Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Ni- hal V . Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Ur- mish Thakker, Khalid Almubarak, Xiangru T...

2022

[8] [8]

Qwen technical report

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

arXiv 2023