Pith. sign in

REVIEW 4 major objections 5 minor 41 references

AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read From ten seed examples, AIDE synthesizes thousands of training points that fine-tune models to outperform human-curated gold data in the paper's experiments.

desk verdict A genuinely novel data-synthesis recipe is undercut by a test-set seeding leak for BIG-Bench and TruthfulQA; the framework deserves a redo, not a citation yet. read the letter →

arxiv 2412.06136 v2 pith:XEZOFC52 submitted 2024-12-09 cs.CL

Jiayu Li , Xuan Zhu , Fang Liu , Yanjun Qi This is my paper · ORCID
classification cs.CL
keywords datasynthesismulti-hopexpansionknowledgetripletsattribute-guidedgenerationtask-specificfine-tuningLLMpersona-baseddiversityresidualconnections
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that task-specific fine-tuning data can be synthesized from as few as ten seed examples by repeatedly branching each example into new ones, guided by the topic and knowledge attributes of the seed and by a retrieved persona. The expansion runs for multiple hops, with each generated example becoming the seed for the next round, and a residual connection re-injects the original seed to keep deep hops on topic. In zero-shot evaluation, Mistral-7B fine-tuned on AIDE-generated data improves by over 6% relative to fine-tuning on human-curated gold data and by over 30% relative to existing data synthesis methods such as Evol-Instruct; Llama-3.1-8B and Llama-3.2-3B also match or beat gold-data fine-tuning. The paper concludes that with only a handful of labeled examples, practitioners can generate abundant, relevant, and diverse training data for fine-tuning. This matters because collecting task-specific data is often the main bottleneck in applying LLMs to new domains.

What carries the argument

The central object is the multi-hop synthesis tree with attribute-based control. For each data point $X_i^{(K)}$, an LLM extractor produces a representation $A_i^{(K)} = \{\langle t, r, a\rangle\}$ containing one topic and three related knowledge attributes. A synthesizer LLM then generates new data from a triplet plus a predefined operation (concretizing, adding constraints, adding reasoning), task demonstrations, and—for diversity—a persona retrieved by topic similarity from a persona hub. The process repeats for $K$ hops, and a residual connection feeds the original seed $X_i^{(0)}$ into the prompt for depths $d \le L$ to keep generated data on task. A self-reflection grader scores each generated point for relevance and diversity, filtering or rewriting low-scoring data.

What would settle it

Run AIDE with an open-source synthesizer whose pretraining corpus is documented to exclude MMLU, ARC-Challenge, TruthfulQA, MedQA, and BIG-Bench test sets, then compare fine-tuned accuracy against gold-data fine-tuning. If the reported average improvements (over 6% for Mistral-7B vs. gold, over 30% vs. evolution-based synthesis) shrink or vanish, the original result was partly an artifact of test-set leakage; if they persist, the method itself carries the gain.

Watch

Extended reading notes

Core claim

The paper's central claim is that structured multi-hop expansion—not the volume of seed data—is what unlocks data synthesis for task-specific fine-tuning. AIDE represents each data point as knowledge triplets $\langle t, r, a\rangle$ (topic, relation, attribute) and uses those triplets, together with task demonstrations, predefined operations, and retrieved personas, to synthesize new questions. Each new question seeds the next hop, so ten seeds grow into roughly three thousand training points. A residual connection from the original seed into prompts up to depth $L$ prevents the topic drift that otherwise appears at deeper hops. Fine-tuning Mistral-7B, Llama-3.1-8B, and Llama-3.2-3B on AIDE data yields average relative improvements of about 6%, 0.7%, and 1.5% over gold-data fine-tuning across the evaluated benchmarks, and on BIG-Bench zero-shot tasks AIDE-trained Mistral-7B substantially outperforms Evol-Instruct, DataTune, and Prompt2Model.

Load-bearing premise

The comparisons to gold data assume the LLM used to generate synthetic data has not memorized the public test questions of MMLU, ARC, TruthfulQA, MedQA, or BIG-Bench, because the seeds come from public datasets and no contamination check is reported.

Editorial extensions

If this is right

  • Fine-tuning from ten seeds can match or beat fine-tuning on full human-curated training sets, which would remove the need for large labeled collections in new task domains.
  • Because Evol-Instruct used 250K synthetic points and AIDE used about 3K per task while still winning on BIG-Bench zero-shot tasks, task-relevance control can matter more than raw synthesis volume.
  • Residual connections make deeper multi-hop expansion usable: increasing hop depth $K$ adds data and can improve accuracy, while residual depth $L$ tunes how tightly the generated data stays anchored to the original seeds.
  • Persona-guided synthesis injects diversity without sacrificing task relevance, addressing the diversity–relevance trade-off that prior methods struggle with.
  • Embedding analysis suggests AIDE's synthetic data covers the target task's distribution even for benchmarks without official training sets, implying the method could be applied to tasks that currently have no training data at all.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the result holds, the practical recipe for a new task becomes: gather roughly ten representative examples, let a capable synthesizer branch them under attribute and persona guidance, and fine-tune; specialist models could then be built without collecting large labeled datasets.
  • Because the synthesizer is a closed model trained on unknown web data and the seeds are public, a contamination check would settle how much of the advantage over gold data comes from the method rather than from test-set leakage; this is the natural next experiment.
  • The residual connection idea likely transfers to other recursive generation settings, such as iterative self-improvement or tree-of-thought data expansion, where maintaining a fixed anchor while branching is a general need.
  • The experiments use small test subsets for some tasks, so a larger-scale sweep with full benchmarks and multiple random seed sets would show whether the reported margins are stable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AIDE (Attribute-guided MultI-hop Data Expansion), a framework that synthesizes task-specific fine-tuning data from a small number of seed examples. AIDE extracts topic/attribute triplets from seeds, uses these to guide iterative multi-hop synthesis, injects personas for diversity, and applies residual connections to limit topical drift at larger hop depths. The empirical study fine-tunes Mistral-7B, Llama-3.1-8B, and Llama-3.2-3B on AIDE-generated data and compares against fine-tuning on human-curated gold data and against SOTA synthesis methods such as Evol-Instruct, Prompt2Model, and DataTune. The main claimed results are that AIDE beats gold-data fine-tuning by small margins on average, and beats SOTA synthesis methods by over 30% on BIG-Bench.

Significance. If the empirical claims are sound, AIDE would be a useful contribution: it is a clearly described, modular synthesis recipe with released code, and the residual-connection idea for preserving relevance at increasing hop depth is plausible and well-motivated. The paper also ships concrete prompts and an ablation study, which are valuable for reproducibility. However, the current experimental design does not yet establish the headline claims. The BIG-Bench and TruthfulQA evaluations appear to use test-set questions as synthesis seeds, the Evol-Instruct comparison is not a controlled same-budget comparison, and gold-data training sizes are not matched. These issues directly affect the main quantitative conclusions, so the paper needs substantial experimental revision before its central claims can be accepted.

major comments (4)
  1. [Appendix E / Appendix H / Tables 2 and 3] The evaluation setup appears to use test-set questions as seeds for BIG-Bench and TruthfulQA, making the comparison circular for those benchmarks. Appendix E states that "Since BIG-Bench lacks a training set, we sample 10 task-related seed data points per task from Hugging Face datasets to generate synthetic data," while Appendix H reports that the five selected BIG-Bench tasks consist only of test sets (60, 153, 492, 7,688, and 1,000 points). Appendix H similarly lists TruthfulQA as 817 test questions with no training split, yet Table 2 reports TruthfulQA results using 10 seeds and a "Gold training data" baseline of unexplained provenance. If the 10 seeds are drawn from the test sets, then the synthetic training data is generated from the evaluation distribution, and fine-tuning on near-duplicates of the test questions can inflate accuracy. This would directly inflate the BIG-Bench gains in Table 3 and the TruthfulQA gains in Table 2. The authors must either clarify the exact source of the seeds (including the specific Hugging Face dataset splits) or rerun the BIG-Bench and TruthfulQA evaluations with seeds drawn from a genuinely disjoint training source, and report an overlap check between synthetic data and test questions.
  2. [Table 3 / Section 4.1 / Appendix E] The comparison against Evol-Instruct is not a controlled experiment. Table 3 compares Mistral-7B fine-tuned with AIDE data against "Mistral-7B with Evol-Instruct," which, according to Section 4.1 and footnote 1, is the external WizardLM-2-7B model fine-tuned on 250K Evol-Instruct examples. This differs from AIDE in three confounding ways: the amount of training data (250K versus roughly 3K), the fine-tuning recipe, and possibly the base model version. As a result, the claimed "over 30%" improvement over SOTA synthesis methods conflates data quantity, training budget, and synthesis quality. The authors should run Evol-Instruct on the same 10 seeds, the same data budget, and the same LoRA fine-tuning setup as AIDE, and report the variance across multiple runs.
  3. [Table 2 / Section 4.2] The gold-data baselines in Table 2 are not matched in data quantity. For MedQA, Appendix H reports a 10,178-point training set, while AIDE synthesizes 2,173 points; for FinBen, the gold training set is 804 points versus 893 synthetic points; for MMLU, ARC-Challenge, and TruthfulQA, the paper does not report how many gold training examples were used per task. Only GSM8K is explicitly matched at 3.2K. The claim that AIDE "surpasses models fine-tuned on human-curated data" is therefore misleading if the gold models are trained on substantially larger or smaller datasets. The authors should either match the number of training examples across conditions or explicitly report the gold training sizes and interpret the comparison accordingly.
  4. [Section 4.4 / Appendix N] The relevance and diversity analyses in Section 4.4 are partially circular because Claude Sonnet 3.5 both generates the synthetic data and serves as the grader for relevance and diversity. Section 4.4 and Appendix N report scores assigned by Claude to its own outputs, which does not provide an independent quality check. The self-reflection filter in Appendix E also uses the same model to grade and rewrite. The authors should add either human evaluation, a second independent LLM, or a quantitative overlap metric on a held-out set to support RQ3. This concern is secondary to the leakage issue but should still be addressed before the quality claims are accepted.
minor comments (5)
  1. [Table 4] The sentence "We highlight the best performance and the base performance is in Table 3" is incomplete and unclear; please state explicitly which row is the base model and which number should be compared.
  2. [Figure 11 / Appendix G.2] The text says the synthetic data clusters are "largely disjoint" from real data in the Code task but then says the synthetic data "covers all real test data" from two tasks; these statements are contradictory and need clarification.
  3. [Figure 4] There is a typo in the x-axis label: "Mistra-7B" should be "Mistral-7B".
  4. [Section 4.3 / Figure 4] The residual-connection experiment on LegalBench is not listed in the main benchmark table or described in Appendix H; please add the dataset statistics and explain why this task is used only in the ablation.
  5. [Appendix G.1] The claim that AIDE achieves "comparable complexity" to Evol-Instruct is based on a visual histogram comparison; please provide numerical scores, sample sizes, and a statistical comparison.

Circularity Check

3 steps flagged · score 6.0 of 10

BIG-Bench and TruthfulQA evaluations are seeded from the same test questions on which accuracy is reported, making part of the headline comparison circular.

  1. other [Appendix E (Data Synthesis Setup); Appendix H (Benchmark Statistics); Table 3 (BIG-Bench results)]
    ""Since BIG-Bench lacks a training set, we sample 10 task-related seed data points per task from Hugging Face datasets to generate synthetic data." "The selected tasks without training sets include 60, 153, 492, 7.688k and 1k data points in their test sets, respectively.""

    Taken together, these statements show that the 10 seeds used to synthesize AIDE's BIG-Bench training data for Table 3 are sampled from the same test questions on which zero-shot accuracy is then reported. The synthetic training set therefore contains near-duplicates of the evaluation questions, so the reported gains over Evol-Instruct and the pretrained model on BIG-Bench are inflated by construction rather than measuring generalization to unseen data. No overlap check between synthesized data and test questions is reported.

  2. other [Appendix E (Data Synthesis Setup); Appendix H (TruthfulQA); Table 2 (TruthfulQA column)]
    ""For the remaining benchmarks, we similarly sample 10 seed data points per task from their respective training sets to produce synthetic data." "We select the multiple choice sets which contains 817 questions for testing.""

    TruthfulQA is described in Appendix H as test-only (817 questions) with no training split, yet Table 2 reports AIDE fine-tuned from 10 TruthfulQA seeds and a 'Gold training data' baseline. Since Appendix E says seeds for non-BIG-Bench benchmarks are drawn from training sets, and TruthfulQA has none, the 10 seeds used for synthesis must come from the test set or from an unexplained source. Either way, the >15% average improvement over gold on TruthfulQA is not an independent held-out measurement; it is partly circular because the synthesis input and evaluation set overlap.

1 more flagged steps
  1. other [Section 4.4 (Relevance and Diversity); Appendix E (Data Synthesis Setup); Figure 17]
    ""We employ Claude Sonnet 3.5 as the LLM generator, the LLM synthesizer, the LLM grader and the LLM annotator in AIDE." "Additionally, we employ Claude Sonnet 3.5 to assess the relevance of synthetic data to the seed data across the three tasks.""

    The relevance and diversity evidence for RQ3 is produced by the same Claude Sonnet 3.5 model that generated the synthetic data and that also serves as the grader in the self-reflection filter. These scores are therefore self-assessments rather than independent measurements; they do not validate the synthetic data against an external standard. This is a secondary issue and does not bear on the external accuracy comparisons.

full rationale

The core AIDE method is not a circular derivation: it is an empirical data-synthesis pipeline evaluated against external benchmarks, and there is no load-bearing self-citation chain or imported uniqueness theorem. Most benchmark families (MMLU, ARC-Challenge, GSM8K, MedQA) use proper train/test splits, so the central comparisons on those tasks retain independent content. However, the paper's own appendices establish a partial evaluation circularity for two benchmark families. Appendix E says BIG-Bench seeds are sampled from Hugging Face datasets, and Appendix H states those five tasks have only test sets; therefore the Table 3 BIG-Bench evaluation is seeded from the same test questions on which accuracy is reported. TruthfulQA is likewise test-only in Appendix H while Table 2 reports a 10-seed fine-tuning result, so its seeds and gold baseline are either drawn from the test set or unexplained. These two cases directly affect the headline claims of >30% improvement over SOTA synthesis methods and >15% improvement over gold data, so the central claim is partially circular. A secondary self-assessment issue is that Claude Sonnet 3.5 generates, grades, and filters the synthetic data in the RQ3 quality analysis, which weakens the relevance/diversity evidence but not the external accuracy results. Overall, the paper is not fully circular, but a substantial portion of the headline evaluation reduces to test-set-seeded synthesis.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method depends on experimental choices (K, L, threshold, P) and on unverified assumptions about seed representativeness, label correctness, and benchmark contamination. No mathematical derivation is attempted, so no standard-math axioms are needed.

free parameters (6)
  • hop_depth_K = 2 (main); 2-4 in Figure 3
    The number of expansion hops is chosen by the authors and controls synthetic data quantity and drift.
  • residual_depth_L = 2 when K=2 (implicit); 1-4 in Figure 4
    The residual connection depth is a hand-chosen hyperparameter that anchors deeper neighbors to the original seed.
  • top_P_personas = 5
    The number of retrieved personas per topic is a fixed design choice for diversity.
  • num_attributes = 1 topic + 3 attributes
    The extraction prompt requests one topic and three related attributes per data point.
  • synthesis_operations = 3 (concretizing, adding constraint, adding reasoning)
    The authors select three of Evol-Instruct's operations to drive complexity.
  • self_reflection_threshold = 5
    Synthetic data is kept if the LLM grader score is above 5; otherwise it is rewritten or discarded.
assumptions (4)
  • domain assumption The 10 seed questions are representative of each target task distribution.
    The entire expansion starts from 10 samples; if these are atypical, the synthetic data will be unrepresentative. See Section 4.1 and Appendix E.
  • domain assumption Claude Sonnet 3.5's pretraining does not leak the evaluation test sets into generated synthetic data.
    Claude is a closed model that may have trained on public benchmark data; no contamination check is provided. This is a serious threat to the 'surpasses gold' result.
  • domain assumption LLM-generated labels in the synthetic data are correct enough for fine-tuning.
    The annotator and self-reflection filter are both LLMs; no human verification of label correctness is reported.
  • domain assumption Embedding-similarity retrieval from Persona Hub and LLM scoring reflect real task relevance and diversity.
    The relevance and diversity analyses rely on Jina embeddings and Claude judgments, not human judgments. See Section 4.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning." pith.science (2026). https://pith.science/paper/XEZOFC52

@misc{pith2026241206136,
  author       = {Pith},
  title        = {Pith review of: AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XEZOFC52}},
  note         = {Machine review of arXiv:2412.06136}
}
read the original abstract

Fine-tuning large language models (LLMs) for specific tasks requires diverse, high-quality training data. However, obtaining sufficient relevant data remains a significant challenge. Existing data synthesis methods either depend on extensive seed datasets or struggle to balance task relevance and data diversity. To address these challenges, we propose Attribute-guided multI-hop Data Expansion (AIDE), a novel data synthesis framework that uses a multi-hop process to expand very few seed data points while ensuring data diversity and task relevance. AIDE extracts the main topic and key knowledge attributes from the seeds to guide the synthesis steps. The process repeats for K hops, using the generated data as seeds. To prevent irrelevant data generation as the hop depth increases, AIDE incorporates a residual connection mechanism. Our empirical results show that AIDE enables fine-tuning of Mistral-7B, Llama-3.1-8B and Llama-3.2-3B from 10 seeds, surpassing the models fine-tuned on human curated data. Furthermore, AIDE outperforms state-of-the-art data synthesis methods, such as Evol-Instruct, by over 30% in task-specific fine-tuning. Code is available at https://github.com/Code4Graph/AIDE.

Figures

Figures reproduced from arXiv: 2412.06136 by the authors.

Figure 1
Figure 1. Overview of the workflow of AIDE. X (0) i denotes the i-th task-related seed data point. AIDE includes four steps. (1) a LLM extractor extracts a topic t, knowledge attributes a1 and a2 with relationships r1 and r2 of a data point. (2) During the multi-hop synthesis at the depth of hop j, a LLM acts as a synthesizer with task demonstrations DT to generate data X (j) 1 , X (j) 2 and X (j) 3 along paths of synthesis w… view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Effect of data quantity with different number of K values in multi-hop synthesis based on the BIG-Bench. Mistra-7B Llama-3.1-8B Llama-3.2-3B Models fine-tuned with AIDE 46 48 50 52 54 A c c u r a c y (%) L = 1 L = 2 L = 3 L = 4 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: The effect of varying the depth of residual connec￾tions (L) when we fix the hop depth K as 4. K values due to the inherent ability gap between the pretrained model and the LLM synthesizer. Effect of Residual Connection. We use a con￾tract task from LegalBench (Guha et…
Figure 6
Figure 6. Figure 6: For exploring the relevance of synthetic data with the seed data, we compute the similarity between the [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: We assess the diversity of knowledge by randomly sampling 20 synthetic data points generated by [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Multi-hop synthesis with the depth of hop [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: An example of unfolded multi-hop synthesis when [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: The complexity of randomly sampling 500 synthetic data from AIDE based on different domains, including code, cause and effect and implicatures. We also compare the complexity of randomly sampling 500 synthetic data from the state-of-the-art data synthesis methods incl…
Figure 11
Figure 11. Figure 11: We observe that randomly sampling 600 synthetic data generated by AIDE using the seed data covers the all real test data from two tasks in the regions of embedding space, after projecting to two dimensions via t-SNE. and external knowledge beyond the text pro￾vided in…
Figure 12
Figure 12. Figure 12: Prompt for extracting a topic and knowledge attributes. [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Prompt for synthesis with a triplet and an operation [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Prompt for synthesis with persona and a constraint operation [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: An example of 10-hop synthesis without the residual connection. When the depth of hop K is large in multi-hop synthesis (i.e., K = 10), more irrelevant information can be introduced in the synthetic data [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: An example of 10-hop synthesis with the residual connection shown in [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: Prompt in the self-reflection can be used to evaluate the relevance score or diversity score of the synthetic [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 18
Figure 18. Figure 18: Prompt for self-reflection, which can be used to improve the relevance or diversity. [PITH_FULL_IMAGE:figures/full_fig_p018_18.png]
Figure 19
Figure 19. Figure 19: A LLM uses the prompt to judge the diversity of the synthetic data from the perspective of knowledge. [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: A LLM uses the prompt to judge the relevance of the synthetic data from the perspective of knowledge. [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 16 canonical work pages

  1. [1]

    BIG bench authors. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research

    work page 2023

  2. [2]

    Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, and Dong Yu. 2024. https://arxiv.org/abs/2406.20094 Scaling synthetic data creation with 1,000,000,000 personas . Preprint, arXiv:2406.20094

    arXiv 2024

  3. [3]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\

    work page 2023

  4. [4]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/1803.05457

    arXiv 2018

  5. [5]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  6. [6]

    Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, and Graham Neubig. 2024. https://arxiv.org/abs/2404.14361 Better synthetic data by retrieving and transforming existing datasets . Preprint, arXiv:2404.14361

    arXiv 2024

  7. [7]

    Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N

    Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Cho...

    arXiv 2023

  8. [8]

    Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. 2023. https://arxiv.org/abs/2310.19923 Jina embeddings 2: 8192-token general-purpose text embeddings for long documents . Preprint, arXiv:2310.19923

    arXiv 2023

Show all 41 references
  1. [9]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  2. [10]

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations

    2022

  3. [11]

    Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. 2023. Large language models on graphs: A comprehensive survey. arXiv preprint arXiv:2312.02783

    2023 arXiv

  4. [12]

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. arXiv preprint arXiv:2009.13081

    2020 arXiv

  5. [13]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/v1/2022.acl-long.229 T ruthful QA : Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

    2022 doi

  6. [14]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations

    2019

  7. [15]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: It...

    2023

  8. [16]

    Igor Melnyk, Pierre Dognin, and Payel Das. 2022. Knowledge graph generation from text. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1610--1622, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics

    2022

  9. [17]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, and et al. Agarwal. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, pages 27730--27744

    2022

  10. [18]

    Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2024. Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies. Transactions of the Association for Computational Linguistics, 12

    2024

  11. [19]

    Feifan Song, Bowen Yu, Hao Lang, Haiyang Yu, Fei Huang, Houfeng Wang, and Yongbin Li. 2024. https://arxiv.org/abs/2403.11124 Scaling data diversity for fine-tuning language models in human alignment . Preprint, arXiv:2403.11124

    2024 arXiv

  12. [20]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model

    2023

  13. [21]

    Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579--2605

    2008

  14. [22]

    Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, and Graham Neubig. 2023. https://doi.org/10.18653/v1/2023.emnlp-demo.38 P rompt2 M odel: Generating deployable models from natural language instructions . In Proceedings of the 2023 Conference on Empirical Method...

    2023 doi

  15. [23]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conferenc...

    2023

  16. [24]

    Zifeng Wang, Chun-Liang Li, Vincent Perot, Long Le, Jin Miao, Zizhao Zhang, Chen-Yu Lee, and Tomas Pfister. 2024. C odec LM : Aligning language models with tailored synthetic data. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico. A...

    2024

  17. [25]

    Dai, and Quoc V Le

    Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. Finetuned language models are zero-shot learners. In International Conference on Learning Representations

    2022

  18. [26]

    Yilin Wen, Zifeng Wang, and Jimeng Sun. 2024. Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

    2024

  19. [27]

    Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, Yijing Xu, Haoqiang Kang, Ziyan Kuang, Chenhan Yuan, Kailai Yang, Zheheng Luo, Tianlin Zhang, Zhiwei Liu, Guojun Xiong, Zhiyang Deng, Yuechen Jiang, Zhiy...

    2024 arXiv

  20. [28]

    Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023. https://arxiv.org/abs/2306.05443 Pixiu: A large language model, instruction data and evaluation benchmark for finance . Preprint, arXiv:2306.05443

    2023 arXiv

  21. [29]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. Wizard LM : Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations

    2024

  22. [30]

    Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Wang Yanggang, Haiyu Li, and Zhilin Yang. 2022. Z ero P rompt: Scaling prompt-based pretraining to 1,000 tasks improves zero-shot generalization. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4235--42...

    2022

  23. [31]

    Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong. 2022 a . https://doi.org/10.18653/v1/2022.emnlp-main.801 Z ero G en: Efficient zero-shot learning via dataset generation . In Proceedings of the 2022 Conference on Empirical Met...

    2022 doi

  24. [32]

    Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2022 b . https://doi.org/10.18653/v1/2022.findings-emnlp.269 P ro G en: Progressive zero-shot dataset generation via in-context feedback . In Findings of the Association for Computational Linguistic...

    2022 doi

  25. [33]

    Chenyang Zhao, Xueying Jia, Vijay Viswanathan, Graham Neubig, and Tongshuang Wu. 2024 a . https://openreview.net/forum?id=Dt6qXZsgaU Self-guide: Better task-specific instruction following via self-synthetic finetuning . In First Conference on Language Modeling

    2024

  26. [34]

    Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024 b . https://arxiv.org/abs/2405.01470 Wildchat: 1m chatgpt interaction logs in the wild . Preprint, arXiv:2405.01470

    2024 arXiv

  27. [35]

    Zhang, and Yongbin Li

    Yingxiu Zhao, Bowen Yu, Binyuan Hui, Haiyang Yu, Minghao Li, Fei Huang, Nevin L. Zhang, and Yongbin Li. 2024 c . Tree-instruct: A preliminary study of the intrinsic relationship between complexity and alignment. In Proceedings of the 2024 Joint International Conference on Comp...

    2024

  28. [36]

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. https://doi.org/10.1145/3209978.3210080 Texygen: A benchmarking platform for text generation models . In The 41st International ACM SIGIR Conference on Research & Development in Informatio...

    2018

  29. [37]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  31. [39]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  32. [40]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  33. [41]

    ( ) & 2 * Avg

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.