Pith. sign in

REVIEW 3 major objections 5 minor 72 references

Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning on data curated by Star-Agents—a multi-LLM generation and dual-model selection pipeline—improves instruction-following scores by about 12% over Evol-Instruct data, with up to 40% gains on Fermi tasks.

desk verdict Consistent gains and a sensible ablation make this worth refereeing; the unverified IFD monotonicity and missing implementation details are fixable. read the letter →

arxiv 2411.14497 v1 pith:EOAZUQSY submitted 2024-11-21 cs.CL cs.AI

Hang Zhou , Yehui Tang , Haochen Qin , Yujie Yang , Renren Jin , Deyi Xiong , Kai Han , Yunhe Wang This is my paper · ORCID
classification cs.CLcs.AI
keywords instructiontuningdataoptimizationmulti-agentLLMsdual-modelevaluationIFDdifficultymetricselectioninstruction-followingbenchmarksLLMagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the quality of instruction-tuning data can be improved automatically, without human annotation, by having multiple LLM agents rewrite a seed dataset and then selecting the best samples with a dual-model difficulty filter. The authors report that Pythia-1B and Llama-2-7B fine-tuned on this optimized 'Star Instruct' data beat the same models fine-tuned on Evol-Instruct data by an average of about 12% on MT-bench, Vicuna-bench, and the WizardLM testset, with up to 40% improvement on Fermi tasks. If confirmed, this gives a low-cost recipe for squeezing better instruction-following performance out of small models, using data that is neither too easy nor too hard for them.

What carries the argument

The load-bearing object is the dual-model evaluation score $\pi = \pi_{llm} \cdot \pi_{dual}$, where $\pi_{dual}$ is the normalized difference between the IFD (Instruction-Following Difficulty) scores of the small target model and a larger model, and $\pi_{llm}$ is a judge score comparing each candidate sample against a base sample. IFD measures how much easier a response is to generate when the instruction is provided than when it is not; the paper assumes a stronger model always gives a lower IFD for the same sample. The score is used both to select the best sample per seed instruction and, through an update rule, to evolve the sampling probability of each agent-pair, so that generator pairs that produce high-scoring data are used more often.

What would settle it

Compute IFD scores for a large fixed set of instruction-response pairs using a 1B and a 7B model, and count the fraction of pairs where the 7B model's IFD is not below the 1B model's; a non-negligible violation rate (say, above 5%) would show the assumption behind the dual-model gap is not reliable. A second check is to replace the large evaluator with another large model of similar size but different training and observe whether the selected data, and the fine-tuned model's scores, change materially.

Watch

Extended reading notes

Core claim

The paper claims that an automated data-optimization loop can replace hand-crafted complexity escalation for instruction tuning. Starting from the Evol-Instruct seed set, Star-Agents has multiple LLM agents rewrite instructions and responses in pairs, scores each candidate with a dual-model evaluation that combines an IFD-based difficulty gap (small target model versus a larger model) with an LLM judge's quality verdict, and then evolves the sampling probabilities of the agent pairs toward those that produce high-scoring data. The selected 'Star Instruct' data, when used to fine-tune Pythia-1B and Llama-2-7B, yields an average relative improvement of about 12% over the same models fine-tuned on Evol-Instruct data across MT-bench, Vicuna-bench, and the WizardLM testset, with gains up to 40% on Fermi tasks and smaller but consistent gains on a public leaderboard.

Load-bearing premise

The selection rule works only if a stronger model always assigns a lower IFD score than the weaker model to the same instruction-response pair, so the gap between the two scores marks samples that are hard for the small model but still learnable; if that ordering fails, the chosen data may not be the most beneficial for the target model.

Editorial extensions

If this is right

  • Instruction-tuning data can be selected or generated automatically for a specific target model, reducing the need for expensive human annotation and manual prompt design.
  • Small models (around 1B parameters) benefit more from data of intermediate, tailored difficulty than from uniformly harder data; the paper shows Evol-Instruct's complex samples can actually hurt Pythia-1B relative to Star Instruct.
  • The evolution rule concentrates generation budget on a few effective agent pairs (e.g., Mistral-ChatGPT reaches about 30% sampling probability while Phi2-ChatGPT decays to near zero), so the framework becomes cheaper over iterations.
  • The gains are not confined to the three headline benchmarks: the paper's appendix reports higher average scores on a public leaderboard for Llama-2-7B trained on Star Instruct compared with Evol-Instruct (56.29 vs 55.11).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The dual-model IFD gap is a general-purpose difficulty signal that could be applied beyond instruction tuning, for instance to pretraining data filtering or curriculum learning for any small model paired with a capable teacher.
  • The monotonicity assumption linking model strength to IFD is probably the most fragile link: if the large evaluator is not a capable generalist on the task distribution, the ranking could invert, so the method may need re-calibration for each target/large pair.
  • The paper's own limitations note that multi-turn and domain-specific instruction sets were not tested; a direct next experiment is to check whether the same pipeline improves performance on multi-turn dialogue data.
  • The agent-pair evolution resembles an online learning algorithm over a finite set of generators; analyzing its regret or connecting it to adversarial bandit methods could provide convergence guarantees the current heuristic lacks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Star-Agents, a framework for automatically constructing instruction-tuning data tailored to a target LLM. It has three stages: (i) generating diverse candidate instruction-response pairs by sampling 'agent-pairs' from a pool of LLMs; (ii) scoring each candidate with π = πllm · πdual, where πllm is an LLM-judge quality comparison against a base sample and πdual is the normalized gap between the IFD difficulty scores of the target (small) model and a larger model; and (iii) evolving the agent-pair sampling probabilities based on these scores, with an Instruction Memory Bank for task-specific retrieval. The authors fine-tune Pythia-1B and Llama-2-7B on the Evol-Instruct dataset and a Star-Agents-refined version ('Star Instruct'), and report that the latter improves average scores on MT-Bench, Vicuna-bench, and WizardLM testset by about 12% (absolute ~0.45 points), with up to 40% relative gains on Fermi tasks. Ablations isolate the contributions of diversity, selection, and evolution, and compare the selection metric against IFD and random selection.

Significance. If the reported gains hold, Star-Agents provides a practical, automatic alternative to manual and single-model data curation, and its multi-agent generation and evolutionary selection are plausible contributors. The paper includes a reasonable ablation structure, comparisons against IFD and random baselines, and a small Open LLM Leaderboard check, which are commendable. However, the central dual-model scoring mechanism rests on an unverified monotonicity assumption about IFD, and the empirical evidence consists of single runs without variance reporting. The core idea is interesting and likely reproducible, but the evidence as presented is not sufficient to establish the 12%/40% claims.

major comments (3)
  1. [Section 3.2, Eqs. (4)-(5)] The dual-model score πdual is the core novelty, yet it rests on the unproven assumption stated after Eq. (4) that 'stronger model yields a smaller IFD score'. Since IFD is a ratio of conditional to unconditional perplexity, and model strength typically reduces both, monotonicity is not guaranteed. No empirical evidence is provided for this assumption, and the paper never specifies which model serves as the 'large' model for IFD_large (Section 4.1 lists generator LLMs but not the evaluator). If the assumption fails on a meaningful fraction of candidates, the selected samples will not lie in the intended difficulty corridor, and the claimed advantage of the dual-model metric over plain IFD is unsupported. Please verify the assumption (e.g., report the share of samples where IFD_large < IFD_small across model pairs) and identify the large model.
  2. [Tables 2-4, Section 4.2] All reported improvements are based on single fine-tuning runs and a single GPT-4 judging pass; no error bars, multiple seeds, or variance estimates are provided. The average gains (e.g., 4.17 vs 3.73 in Table 2) are small relative to the known variability of LLM-as-a-judge scores, so the headline claims of 12% and 40% are not yet statistically grounded. Please report means and standard deviations over at least three seeds for the main comparisons and include a human-evaluation subset or judge-agreement statistics.
  3. [Section 4.1 and Table 4] The IFD and Random baselines are underspecified. It is not clear which candidate pool they select from (the same Star-Agents-generated D(Si) or the original Evol-Instruct data), how many samples are selected per seed (one per seed, as Eq. (1) suggests, or a filtered subset), and what the final dataset sizes are. Without this information, Table 4 cannot be interpreted as a controlled comparison of selection metrics. Please describe the exact selection protocol for each baseline and report the resulting dataset sizes.
minor comments (5)
  1. [Abstract] The word 'Finaly' should be 'Finally'.
  2. [Eq. (1) and Eq. (5)] Equation (1) writes arg max over π(D(Si)) rather than π(d), and Eq. (5) uses 'max_{1≤i≤m}' without defining the index set; please fix the notation.
  3. [Section 4.1] The paper states '10 agent-pairs were employed' but does not specify the values of M, β, l, and n from Section 3; please provide these hyperparameters for reproducibility.
  4. [Throughout] There are several typos: 'Performence' in Figure 2, 'Evolutiuon' in Table 3, 'Imapct' in Table 4, 'comparsion' in Section 4.1, 'socres' in Section 3.3, and 'Evol-Instuct' in Related Work.
  5. [Appendix A.5] The computational cost section confuses MACs and FLOPs, and the '5 of 10 LLMs' reduction is not directly tied to the agent-pair sampling described in Section 3.1; please clarify the calculation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are validated by external GPT-4-judged benchmarks, not entailed by the selection equations.

full rationale

The paper's derivation chain is: generate diverse candidate instruction-response pairs via agent-pair sampling (Eqs. 2-3); score each candidate with a composite of an LLM quality comparison (Eq. 6) and a dual-model IFD gap (Eq. 5); select the top candidate per seed instruction (Eq. 1); update agent-pair sampling probabilities with that score (Eq. 8); fine-tune the target model on the selected set; and measure performance on MT-bench, Vicuna-bench, and WizardLM testset using GPT-4 as judge. The final evaluation is external to the selection objective: GPT-4 judging is distinct from the ChatGPT-based quality referee, and the benchmark tasks are not used to compute IFD scores or selection scores. The reported 12% average and 40% Fermi improvements are therefore empirical outcomes that are not derived by construction from the selection metric. The central heuristic, stated after Eq. 4 as 'We assume that for the same sample, stronger model yields a smaller IFD score,' is an unverified assumption that poses a correctness/transfer risk, but it is not circular: an assumption being unproven is different from the conclusion being equivalent to the input. The large model used to compute IFD_large is unspecified, which is a reproducibility concern rather than a circularity concern. The paper's self-citations (e.g., refs. 21, 27, 28, 30, 31, 39) appear in related-work or background contexts and are not load-bearing for the main claim. The Limitations section explicitly notes the absence of multi-turn evaluation, which is a scope limitation, not a circular step. Overall, the framework is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The framework rests on two main assumptions about IFD scores and on several unreported hyperparameters. The selection pipeline uses the target model itself to compute IFD, so the model's pretrained behavior influences which data are chosen. The free parameters (beta, M, l, n, base agent-pairs, and the large model choice) are chosen by hand and not all specified, limiting reproducibility.

free parameters (6)
  • beta (evolution rate)
    Controls how much agent-pair sampling probabilities are updated in Eq. (8); no value is reported, and it affects the evolution dynamics.
  • M (number of sampled agent-pairs per seed) = possibly 10
    In Section 3.1, M is the number of agent-pairs sampled for each seed. Section 4.1 says '10 agent-pairs were employed', but it is unclear if this is M or the total number of pairs available.
  • l (agent-pairs sampled from memory bank)
    In Section 3.3, l is the number of agent-pairs sampled from the Instruction Memory Bank; no value is given.
  • n (top matches in memory bank)
    In Section 3.3, top n closest matches are retrieved from the memory bank; no value is given.
  • choice of large model for IFD
    The dual-model evaluation requires a larger model for IFD computation, but the paper never specifies which model is used. This is a free choice that affects the selection scores.
  • base agent-pairs
    Section 3.1 mentions a fixed set of base agent-pairs used every iteration, but their identities are not listed.
assumptions (5)
  • standard math IFD metric as defined in Eq. (4) measures instruction-following difficulty.
    The formula is taken from Cherry LLM [20] and used without modification.
  • domain assumption A stronger model yields a smaller IFD score for the same sample.
    Stated in Section 3.2 after Eq. (4): 'We assume that for the same sample, stronger model yields a smaller IFD score.' This is not empirically validated.
  • domain assumption A large gap between small-model and large-model IFD scores indicates data of appropriate difficulty for learning.
    This is the basis of the dual-model selection in Section 3.2 and Figure 3; it is assumed without verification.
  • domain assumption ChatGPT can serve as a reliable judge of response quality for selection.
    The LLM judge in Eq. (6) is ChatGPT, and no evidence is given that its quality judgments correlate with downstream performance.
  • standard math Standard probability, log, and normalization operations in the scoring formulas.
    The equations use standard mathematical operations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning." pith.science (2026). https://pith.science/paper/EOAZUQSY

@misc{pith2026241114497,
  author       = {Pith},
  title        = {Pith review of: Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EOAZUQSY}},
  note         = {Machine review of arXiv:2411.14497}
}
read the original abstract

The efficacy of large language models (LLMs) on downstream tasks usually hinges on instruction tuning, which relies critically on the quality of training data. Unfortunately, collecting high-quality and diverse data is both expensive and time-consuming. To mitigate this issue, we propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets through multi-agent collaboration and assessment. The framework adopts a three-pronged strategy. It initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. Subsequently, the generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality. Finaly, the above process evolves in a dynamic refinement phase, where more effective LLMs are prioritized, enhancing the overall data quality. Our empirical studies, including instruction tuning experiments with models such as Pythia and LLaMA, demonstrate the effectiveness of the proposed framework. Optimized datasets have achieved substantial improvements, with an average increase of 12% and notable gains in specific metrics, such as a 40% improvement in Fermi, as evidenced by benchmarks like MT-bench, Vicuna bench, and WizardLM testset.

Figures

Figures reproduced from arXiv: 2411.14497 by the authors.

Figure 1
Figure 1. The diagram of the Star-Agents Framework. Step 1 is designed to gather diverse instructions [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison of varied￾scale models on the Alpaca and Evol-Instruct datasets. The tasks from the Evol-Instruct dataset are more complex than those from Alpaca. IFD Complexity Large model Small model [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Radar plot of detailed scores for Llama-2-7B-star_instrcut against the major baseline on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Evolution of the typical Agent-Pairs. 4.3 Ablation Study Main Components. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 31 canonical work pages

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  2. [2]

    Pythia: A suite for analyzing large language models across training and scaling

    Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pages 2397–2430. PMLR, 2023

    2023

  3. [3]

    Yu, Qiang Yang, and Xing Xie

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. A survey on evaluation of large language models, 2023

    2023

  4. [4]

    Magdi: Structured distillation of multi-agent interaction graphs improves reasoning in smaller language models

    Justin Chih-Yao Chen, Swarnadeep Saha, Elias Stengel-Eskin, and Mohit Bansal. Magdi: Structured distillation of multi-agent interaction graphs improves reasoning in smaller language models. arXiv preprint arXiv:2402.01620, 2024

    arXiv 2024

  5. [5]

    Alpagasus: Training a better alpaca with fewer data

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.08701, 2023

    arXiv 2023

  6. [6]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6, 2023

    2023

  7. [7]

    Free dolly: Introducing the world’s first truly open instruction-tuned llm

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world’s first truly open instruction-tuned llm. Company Blog of Databricks, 2023

    2023

  8. [8]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023

    arXiv 2023

Show all 72 references
  1. [9]

    GLM: General language model pretraining with autoregressive blank infilling

    Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. GLM: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume1: Long Paper...

    2022

  2. [10]

    Evaluating large language models: A comprehensive survey

    Zishan Guo, Renren Jin, Chuang Liu, Yufei Huang, Dan Shi, Linhao Yu, Yan Liu, Jiaxuan Li, Bojian Xiong, Deyi Xiong, et al. Evaluating large language models: A comprehensive survey. arXiv preprint arXiv:2310.19736, 2023

    2023 arXiv

  3. [11]

    Phi-2: The surprising power of small language models

    Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio César Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. Phi-2: The surprising power of small language models. Microsoft Research Blog, 2023

    2023

  4. [12]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  5. [13]

    Lion: Adversarial distillation of closed-source large language model

    Yuxin Jiang, Chunkit Chan, Mingyang Chen, and Wei Wang. Lion: Adversarial distillation of closed-source large language model. arXiv preprint arXiv:2305.12870, 2023

    2023 arXiv

  6. [14]

    UNIFIEDQA: Crossing format boundaries with a single QA system

    Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. UNIFIEDQA: Crossing format boundaries with a single QA system. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1896–1907, Online, No...

    2020

  7. [15]

    Openassistant conversations- democratizing large language model alignment

    Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, et al. Openassistant conversations- democratizing large language model alignment. Advances in Neural Information Pro...

    2024

  8. [16]

    Camel: Communicative agents for" mind" exploration of large scale language model society

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society. 2023

    2023

  9. [17]

    Selective reflection-tuning: Student-selected data recycling for llm instruction-tuning

    Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, and Tianyi Zhou. Selective reflection-tuning: Student-selected data recycling for llm instruction-tuning. arXiv preprint arXiv:2402.10110, 2024

    2024 arXiv

  10. [18]

    Reflection- tuning: Data recycling improves llm instruction-tuning

    Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Heng Huang, Jiuxiang Gu, and Tianyi Zhou. Reflection- tuning: Data recycling improves llm instruction-tuning. ArXiv, abs/2310.11716, 2023

    2023 arXiv

  11. [19]

    Superfiltering: Weak-to-strong data filtering for fast instruction-tuning, 2024

    Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning, 2024

    2024

  12. [20]

    From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning

    Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning. arXiv preprint arXiv:2308.12032, 2023

    2023 arXiv

  13. [21]

    Openeval: Benchmarking chinese llms across capability, alignment and safety

    Chuang Liu, Linhao Yu, Jiaxuan Li, Renren Jin, Yufei Huang, Ling Shi, Junhui Zhang, Xinmeng Ji, Tingting Cui, Tao Liu, et al. Openeval: Benchmarking chinese llms across capability, alignment and safety. arXiv preprint arXiv:2403.12316, 2024

    2024 arXiv

  14. [22]

    What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning

    Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312.15685, 2023

    2023 arXiv

  15. [23]

    Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V

    S. Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V . Le, Barret Zoph, Jason Wei, and Adam Roberts. The flan collection: Designing data and methods for effective instruction tuning. ArXiv, abs/2301.13688, 2023

    2023 arXiv

  16. [24]

    # instag: Instruction tagging for analyzing supervised fine-tuning of large language models

    Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Conference on Learning Representations, 2023

    2023

  17. [25]

    A comprehensive overview of large language models

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models. arXiv preprint arXiv:2307.06435, 2023

    2023 arXiv

  18. [26]

    Xgen-7b technical report

    Erik Nijkamp, Tian Xie, Hiroaki Hayashi, Bo Pang, Congying Xia, Chen Xing, Jesse Vig, Semih Yavuz, Philippe Laban, Ben Krause, et al. Xgen-7b technical report. arXiv preprint arXiv:2309.03450, 2023

    2023 arXiv

  19. [27]

    Large language model alignment: A survey

    Tianhao Shen, Renren Jin, Yufei Huang, Chuang Liu, Weilong Dong, Zishan Guo, Xinwei Wu, Yan Liu, and Deyi Xiong. Large language model alignment: A survey. arXiv preprint arXiv:2309.15025, 2023

    2023 arXiv

  20. [28]

    Roleeval: A bilingual role evaluation benchmark for large language models

    Tianhao Shen, Sun Li, Quan Tu, and Deyi Xiong. Roleeval: A bilingual role evaluation benchmark for large language models. arXiv preprint arXiv:2312.16132, 2023

    2023 arXiv

  21. [29]

    On the exploitability of instruction tuning, 2023

    Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tuning, 2023

    2023

  22. [30]

    Fuxitranyu: A multilingual large language model trained with balanced data

    Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, Menglong Cui, Jiangcun Dui, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, et al. Fuxitranyu: A multilingual large language model trained with balanced data. arXiv preprint arXiv:2408.06273, 2024. 11

    2024 arXiv

  23. [31]

    Rethinking optimization and architecture for tiny language models

    Yehui Tang, Fangcheng Liu, Yunsheng Ni, Yuchuan Tian, Zheyuan Bai, Yi-Qi Hu, Sichao Liu, Shangling Jui, Kai Han, and Yunhe Wang. Rethinking optimization and architecture for tiny language models. arXiv preprint arXiv:2402.02791, 2024

    2024 arXiv

  24. [32]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github. com/tatsu-lab/stanford_alpaca, 2023

    2023

  25. [33]

    Stanford alpaca: An instruction-following llama model, 2023

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-following llama model, 2023

    2023

  26. [34]

    Gemma: Open models based on gemini research and technology

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  27. [35]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  28. [36]

    Self-instruct: Aligning language models with self-generated instructions

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560, 2022

    2022 arXiv

  29. [37]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics...

    2023

  30. [38]

    Super- NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuzni...

    2022

  31. [39]

    Pangu- pi: Enhancing language model architectures via nonlinearity compensation

    Yunhe Wang, Hanting Chen, Yehui Tang, Tianyu Guo, Kai Han, Ying Nie, Xutao Wang, Hailin Hu, Zheyuan Bai, Yun Wang, et al. Pangu- pi: Enhancing language model architectures via nonlinearity compensation. arXiv preprint arXiv:2312.17276, 2023

    2023 arXiv

  32. [40]

    Dai, and Quoc V Le

    Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022

    2022

  33. [41]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  34. [42]

    Sheared llama: Accelerating language model pre-training via structured pruning

    Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. Sheared llama: Accelerating language model pre-training via structured pruning. arXiv preprint arXiv:2310.06694, 2023

    2023 arXiv

  35. [43]

    Less: Selecting influential data for targeted instruction tuning

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402.04333, 2024

    2024 arXiv

  36. [44]

    Wizardlm: Empowering large language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244, 2023

    2023 arXiv

  37. [45]

    Baize: An open-source chat model with parameter-efficient tuning on self-chat data

    Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196, 2023

    2023 arXiv

  38. [46]

    Rethinking the instruction quality: Lift is what you need, 2023

    Yang Xu, Yongqiang Yao, Yufan Huang, Mengnan Qi, Maoquan Wang, Bin Gu, and Neel Sundaresan. Rethinking the instruction quality: Lift is what you need, 2023

    2023

  39. [47]

    Backdooring instruction-tuned large language models with virtual prompt injection

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large language models with virtual prompt injection. In NeurIPS 2023 Workshop on Backdoors in Deep Learning-The Good, the Bad, and ...

    2023

  40. [48]

    Virtual prompt injection for instruction-tuned large language models, 2023

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Virtual prompt injection for instruction-tuned large language models, 2023

    2023

  41. [49]

    Harnessing the power of llms in practice: A survey on chatgpt and beyond, 2023

    Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu. Harnessing the power of llms in practice: A survey on chatgpt and beyond, 2023. 12

    2023

  42. [50]

    CrossFit: A few-shot learning challenge for cross-task generalization in NLP

    Qinyuan Ye, Bill Yuchen Lin, and Xiang Ren. CrossFit: A few-shot learning challenge for cross-task generalization in NLP. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7163–7189, Online and Punta Cana, Dominican Republic, Nove...

    2021

  43. [51]

    Glm-130b: An open bilingual pre-trained model

    Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414, 2022

    2022 arXiv

  44. [52]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  45. [53]

    A survey of large language models, 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong W...

    2023

  46. [54]

    [[A]]” if assistant A is the bset, “[[B]]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024. 13 A Appendix A.1 Promp...

    2024

  47. [55]

    It is a relatively simple and interpretable model that works well with linearly separable datasets

    Logistic regression: It is a statistical method that uses a logistic function to model a binary dependent variable. It is a relatively simple and interpretable model that works well with linearly separable datasets

  48. [56]

    SVMs can handle high-dimensional data and work well with datasets that are not linearly separable

    Support Vector Machines (SVMs): It is a popular classifier that is used for binary classification tasks. SVMs can handle high-dimensional data and work well with datasets that are not linearly separable

  49. [57]

    Random Forests: It is an ensemble learning method that works by constructing a multitude of decision trees at training time and outputs the class that is the mode of the classes output by the individual trees

  50. [58]

    Neural networks can handle non-linear relationships between variables and are capable of learning complex patterns in the data

    Neural networks: It is a type of machine learning model that is widely used for binary classification tasks. Neural networks can handle non-linear relationships between variables and are capable of learning complex patterns in the data. The choice of the model depends on the p...

  51. [59]

    It works by estimating the probability of the positive class and using it to make predictions

    Logistic Regression: This is a simple and effective model that can be used for binary classification problems. It works by estimating the probability of the positive class and using it to make predictions

  52. [60]

    They work by recursively splitting the data into subsets based on the values of the input features

    Decision Trees: Decision Trees are a popular machine learning model that can handle both categorical and numerical features. They work by recursively splitting the data into subsets based on the values of the input features

  53. [61]

    It works by creating a set of decision trees and then averaging their predictions

    Random Forest: Random Forest is an ensemble learning method that combines multiple decision trees to improve the accuracy of the model. It works by creating a set of decision trees and then averaging their predictions

  54. [62]

    It works by finding the hyperplane that maximally separates the classes in the feature space

    Support Vector Machines (SVM): SVM is a powerful model that can handle high-dimensional data. It works by finding the hyperplane that maximally separates the classes in the feature space

  55. [63]

    They can handle complex and nonlinear relationships between the input features and the output labels

    Neural Networks: Neural Networks are a set of algorithms that can learn and classify from data. They can handle complex and nonlinear relationships between the input features and the output labels. 1The choice of the model depends on various factors such as the size and comple...

  56. [64]

    It’s linear and can handle both categorical and continuous features

    Logistic Regression: A simple and interpretable model that works well for small to medium-sized datasets. It’s linear and can handle both categorical and continuous features

  57. [65]

    Random Forest is an ensemble method that combines multiple decision trees, reducing overfitting

    Decision Trees and Random Forest: These are easy to interpret and can handle both numerical and categorical data. Random Forest is an ensemble method that combines multiple decision trees, reducing overfitting

  58. [66]

    SVMs are robust to outliers and can be used for both linear and non-linear classification

    Support Vector Machines (SVM): Effective for high-dimensional data and can handle non-linear relationships through kernel functions. SVMs are robust to outliers and can be used for both linear and non-linear classification

  59. [67]

    It’s fast, easy to implement, and works well for text classification or when features are not highly correlated

    Naive Bayes: A probabilistic model that assumes features are independent. It’s fast, easy to implement, and works well for text classification or when features are not highly correlated

  60. [68]

    It’s non-parametric and can be useful for small datasets

    K-Nearest Neighbors (KNN): A simple model that classifies based on the majority class of its nearest neighbors. It’s non-parametric and can be useful for small datasets

  61. [69]

    They are powerful but require more data and computational resources

    Neural Networks (e.g., Feedforward, Convolutional, or Recurrent Neural Networks): Deep learning models that can capture complex patterns in large datasets. They are powerful but require more data and computational resources

  62. [70]

    They are often used for structured data and can handle high dimensionality

    Gradient Boosting Machines (GBM) or XGBoost: Ensemble methods that iteratively build weak learners (e.g., decision trees) to create a strong classifier. They are often used for structured data and can handle high dimensionality

  63. [71]

    It’s particularly useful when dealing with imbalanced datasets

    AdaBoost: Another boosting algorithm that combines weak learners to create a strong classifier. It’s particularly useful when dealing with imbalanced datasets

  64. [72]

    Neural Network Ensembles (e.g., stacking, blending): Combining predictions from multiple models can improve performance and robustness. To choose the best model, it’s essential to perform exploratory data analysis, split the data into training and testing sets, and evaluate th...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.