Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GraphGen claims that routing synthetic QA generation through a knowledge graph built from the source corpus improves supervised fine-tuning for closed-book knowledge-intensive question answering, beating five baseline synthesis methods on…

desk verdict Well-engineered KG-guided synthetic data pipeline with consistent QA gains, but the unmeasured fidelity of its extracted knowledge graph needs scrutiny before the mechanism is taken as established. read the letter →

arxiv 2505.20416 v1 pith:AHSLECIH submitted 2025-05-26 cs.CL cs.AI

Zihong Chen , Wanli Jiang , Jinzhe Li , Zhonghang Yuan , Huanjun Kong , Wanli Ouyang , Nanqing Dong This is my paper
classification cs.CLcs.AI
keywords knowledgegraphsyntheticdatagenerationsupervisedfine-tuningclosed-bookQAexpectedcalibrationerrorlong-tailmulti-hopreasoningsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GraphGen is a recipe for turning a raw text corpus into supervised fine-tuning (SFT) data for large language models when the goal is knowledge-intensive, closed-book question answering. The paper's claim is that building a fine-grained knowledge graph from the corpus first, then scoring each edge by how poorly the trainee model understands it, sampling k-hop subgraphs around high-loss edges, and converting those subgraphs into QA pairs, produces better training data than five existing synthetic-data pipelines. If this claim is right, the framework gives practitioners a more reliable way to generate SFT data without manual annotation, one that targets the long-tail knowledge a model actually lacks and can be tuned for atomic, aggregated, and multi-hop QA.

What carries the argument

The central object is the knowledge graph $G=(E,R)$ assembled in Step 1 from entities and relationships extracted from the source corpus, with each edge $R_i$ treated as a knowledge point. Each edge is scored by a comprehension loss $\mathrm{Loss}_C(R_i)$ derived from the trainee model's confidence on paraphrased true and false statements, following the expected calibration error principle. That loss drives a k-hop subgraph sampler with configurable depth, token-length, and edge-selection strategies, and the sampled subgraphs are converted by the synthesizer model into atomic, aggregated, or multi-hop QA pairs. The knowledge graph is what lets the pipeline preserve cross-document associations and target long-tail knowledge instead of generating from single text segments.

What would settle it

A concrete check: sample a set of edges from the Step 1 KG, verify each triple against the source corpus, and compute KG precision and recall; if either falls substantially, the downstream QA pairs inherit the errors and the claimed post-SFT gains over baselines would be expected to shrink or reverse.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a knowledge-graph intermediate representation changes what synthetic data contributes to fine-tuning. After SFT on GraphGen-generated data, Qwen2.5-7B-Instruct scores higher ROUGE-F than models trained on data from WRAP, Genie, LongForm, EntiGraph, or SELF-QA on all three evaluation sets, with margins of 1.08 points on SeedEval, 2.7 points on PQArefEval, and 4.73 points on HotpotEval over the best baseline. The paper also reports that comprehension loss over knowledge-graph edges is heavily skewed, meaning most generated knowledge is already known to the trainee, and that training on the high-loss tail yields larger gains, so the value of the data is concentrated in long-tail knowledge points. After training, comprehension loss drops, which the paper reads as evidence that GraphGen improved the model's grasp of the domain rather than merely its surface answer format.

Load-bearing premise

The load-bearing premise is that the knowledge graph extracted by the synthesizer LLM is factually accurate and complete enough that QA pairs generated from its triples faithfully represent the source corpus; the paper asserts low hallucination but never measures graph precision or recall.

Editorial extensions

If this is right

  • A corpus-to-KG pipeline can replace direct prompting on text chunks as the default way to synthesize SFT data for knowledge-intensive tasks, at least in closed-book QA.
  • Because comprehension loss is skewed and low-loss data adds little, training on a small high-loss subset, under 5% of GraphGen's data, may retain most of the fine-tuning benefit, reducing SFT cost.
  • Graph-structured answers preserve cross-document associations, preventing the post-SFT performance drop that single-segment baselines show on aggregated medical QA.
  • High-loss data is a usable selection signal: the paper's top-30% versus bottom-30% comparison shows that prioritizing long-tail knowledge points yields better ROUGE-F than training on already-mastered knowledge.
  • The improvement transfers across Qwen2.5-7B, LLaMA-3.1-8B, and MiniCPM3-4B trainees, so the effect is attributable to the data organization rather than one architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: because the paper never measures KG precision or recall, a direct audit of extracted triples against the source corpus would show whether the gains come from faithful graph organization or from errors that happen to be useful as training noise.
  • Inference: the ablation showing max-loss, min-loss, and random edge-selection strategies have nearly identical results suggests the ECE-based ordering may be less central than the graph structure itself; swapping the loss signal for random long-tail sampling would test this.
  • Inference: the skewed comprehension-loss distribution implies a practical data-mining recipe, build a large KG, rank by comprehension loss, and fine-tune on a small high-loss slice, that could cut synthetic-data SFT compute substantially, which the paper frames as a finding rather than a recommendation.
  • Inference: the Wikipedia entity-enrichment plug-in described in the appendix points toward combining GraphGen with external knowledge sources, but the paper does not test this; a version that refreshes the KG from Wikipedia before QA generation would be a direct extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GraphGen, a four-stage pipeline for knowledge-intensive synthetic data generation: (1) build a knowledge graph from raw source text, (2) score each KG edge by a 'comprehension loss' derived from the trainee model's confidence on paraphrased true/false statements, (3) sample subgraphs via k-hop traversal with loss-based edge selection, and (4) generate QA pairs in three scenarios (atomic, aggregated, multi-hop). The authors fine-tune Qwen2.5-7B-Instruct on the generated data and report ROUGE-F improvements over five synthetic-data baselines (WRAP, Genie, LongForm, EntiGraph, SELF-QA) across three adapted datasets (SeedEval, PQArefEval, HotpotEval), with additional experiments on two other trainee models and an analysis relating comprehension loss to downstream gains.

Significance. If the reported gains are robust, GraphGen offers a useful and generally applicable recipe: a KG-guided, loss-aware synthesis pipeline that outperforms open pipelines and produces more lexically diverse aggregated answers. The paper has concrete strengths: the code and data are released; the main result is reproduced on three trainee models of different families and scales; and the scaling-law and top/bottom-loss comparisons (Section 5.3, Appendix F.2) are falsifiable checks of the selection mechanism. The central weakness is that the fidelity of the extracted KG is assumed but never measured, and the evaluation statistics are single-run without variance or significance testing.

major comments (4)
  1. [Section 4, STEP 2; STEP 3; Section 5.5; Limitations] The pipeline assumes that every KG edge description R_i is 'unequivocally true' with P(R_i true)=1 (STEP 2, Eq. 3-4), and the max_loss selection strategy (Algorithm 1, Table 3) then prioritizes edges based on this assumption. However, the paper never measures KG precision or recall on any domain; it only cites Ibrahim et al. 2024 and Gillani et al. 2024 for a generally 'low rate of hallucination.' If the extracted triples contain false or unsupported statements, the comprehension loss targets the wrong knowledge points and the generated QA pairs become false supervision. The Limitations section lists computational cost and limited domain coverage but does not mention KG fidelity. Please report a quantitative estimate of KG precision/recall per dataset, e.g., by sampling edges and comparing against the source fragments with human or LLM judges, and either filter low-confidence triples or analyze how extraction errors propagate to downstream gains.
  2. [Abstract; Section 1; Section 4, STEP 2 (Eq. 3-4)] The method is presented as ECE-based ('using the expected calibration error metric'), but Eq. (3) defines an average confidence over true/false statements and Eq. (4) defines a cross-entropy loss; this is not the expected calibration error, which is computed over confidence bins and compares predicted confidence to empirical accuracy. Because the truth of R_i is assumed by construction, no accuracy statistic is computed. This is more than a terminology issue: the claimed principled identification of 'knowledge blind spots' via ECE is unsupported. Please either compute an actual ECE over binned confidence on a held-out set of statements with known labels, or rename the quantity (e.g., 'comprehension loss') and remove the ECE-based justification throughout.
  3. [Section 5.2, Figure 3; Appendix F.1; Table 8] All experimental results appear to be single training/evaluation runs. The headline improvements over the best baseline are 1.08, 2.7, and 4.73 ROUGE-F points (Section 5.2), while the ablation differences in Table 8 are as small as 0.04-0.3 points. Without standard deviations, confidence intervals, or significance tests, the reader cannot assess whether the reported advantages over the strongest baselines are reliable. Please provide multiple seeds or bootstrap confidence intervals, and a paired significance test (e.g., paired bootstrap over test instances) for the main comparisons and key ablations.
  4. [Section 5.1; Appendix E] The evaluation datasets are adapted from public benchmarks: PQArefEval and HotpotEval use the original references of PQAref and HotpotQA as the source corpus (Dsource), while the test sets are derived from the same benchmarks. The paper does not quantify the overlap between the generated QA pairs and the test questions, nor does it filter generated pairs that could be near-duplicates of test items. If the synthetic pipeline produces training examples that are close paraphrases of test questions, the post-SFT gains could reflect test-set memorization rather than generalizable knowledge acquisition. Please report n-gram or embedding-level overlap statistics between Dsynth and Deval, apply the same filtering or reporting consistently across all baselines, and discuss the implications for the knowledge-injection claim.
minor comments (5)
  1. [References] The references for Self-QA are duplicated: [Zhang and Yang, 2023a] and [Zhang and Yang, 2023b] are the same paper (arXiv:2305.11952); please consolidate.
  2. [Section 5.2] The sentence 'We note that at this stage, we only used knowledge-related data' should clarify that the results in Figure 3 are from the knowledge-only condition, while the mixed-instruction experiments appear only in Appendix F; this distinction should be explicit in the main text.
  3. [Figure 5] The y-axis label 'Average' is undefined; please state which datasets and metric are averaged, and define what 'proportions of training data' means (e.g., top x% of data sorted by comprehension loss).
  4. [Table 2] The caption does not define the reward model columns 'Ind' and 'Deb'; the definitions are only given in Appendix F.3, so the main-table caption should at least mention that they come from reward models.
  5. [Throughout] There are several typos: 'addressesing' in Section 4 STEP 1; 'We propose' with a capital W mid-sentence in Section 3; 'Selction' in Table 9. Please proofread.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: GraphGen's derivation is empirical and self-contained; minor self-benchmark and implementation references are not load-bearing.

full rationale

GraphGen's chain is pipeline-based: extract a knowledge graph from Dsource (Step 1), compute comprehension loss on KG statements (Step 2, Eqs. 3-4), sample subgraphs (Step 3), generate QA pairs (Step 4), fine-tune Mtrain, and evaluate on held-out QA test sets. No step defines its inputs in terms of its outputs, and no parameter is fitted to the test labels; the reported improvements are measured post-SFT on held-out questions. The comprehension-loss step assumes each extracted edge description R_i is 'unequivocally true' with P(R_i true)=1, but this is a factual-fidelity assumption about the KG, not a circular reduction: the loss is a self-supervised confidence measure used to prioritize data, and the downstream QA generation does not feed back into the ground truth of the evaluation. The unmeasured KG precision is a real validity risk, and the Limitations section omits it, but that concern is about correctness and completeness, not circularity. The only self-references are (i) SeedEval, adapted from SeedBench, which shares several authors with this paper, and (ii) the STEP 1 implementation credited to Kong 2025, a co-author's prior work. Both are tools or benchmarks, and the central claim does not reduce to them: PQArefEval and HotpotEval are derived from external sources, and the same performance trend appears across three trainee models. The minor self-benchmark overlap justifies a score of 2 rather than 0, but there is no load-bearing circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the reliability of the automatically constructed KG (never measured), on the validity of using softmax yes/no confidence as a knowledge-gap signal, and on the assumption that QA pairs generated from subgraphs preserve factual content. No invented entities are introduced.

free parameters (5)
  • pre_length (max_tokens) = 256 default in Table 3; 512 used in main Table 2 results
    Controls the maximum token length of subgraph premise descriptions. Ablation in Table 8 shows 256 gives best ROUGE-F on PQArefEval, yet the main results row in Table 2 corresponds to 512, an unexplained choice.
  • max_extra_edges = 5
    Maximum number of edges to expand the subgraph; set by hand in Table 3.
  • max_depth = 2
    Traversal depth in each direction; set by hand in Table 3.
  • edge_sampling = max_loss (default)
    Edge selection strategy in Step 3; ablation shows minimal impact, so the specific value does not strongly affect the central claim.
  • temperature = 0 for generation, 1 for rephrasing
    Sampling temperature for Msynth; set by hand in Appendix C.
assumptions (5)
  • domain assumption The LLM-extracted KG is factually accurate and complete with respect to the source corpus.
    STEP 1 uses Msynth to extract entities/relations and merge descriptions; the paper asserts low hallucination but does not measure KG precision or recall. Errors propagate to all generated QA data.
  • domain assumption The yes/no softmax confidence of Mtrain on paraphrased statements is a valid measure of knowledge, i.e., the comprehension loss in Eq. (4) reflects true knowledge gaps.
    STEP 2 defines LossC via averaged log-probabilities of 'yes' or 'no'; this assumes the model's token probabilities are meaningful confidence estimates for true or false statements.
  • domain assumption QA pairs rendered from subgraphs by Msynth preserve the facts of the KG and are suitable SFT targets.
    STEP 4 converts subgraphs into atomic, aggregated, or multi-hop QA pairs using LLM prompts; no verification that generated answers are faithful to the KG or source, beyond reward-model scores on aggregated QA.
  • domain assumption ROUGE-F on the adapted test sets (SeedEval, PQArefEval, HotpotEval) measures knowledge acquisition in closed-book QA.
    Section 5.1 and Appendix F.1 justify ROUGE-F as word-overlap F1; the paper uses no exact-match or human evaluation of factual content.
  • domain assumption The adapted evaluation datasets are unbiased and representative of the three QA scenarios.
    SeedEval is a subset of SeedBench (co-authored), PQArefEval is a subset of PQAref, and HotpotEval is the HotpotQA test set; dataset construction details are in Appendix E.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation." pith.science (2026). https://pith.science/paper/AHSLECIH

@misc{pith2026250520416,
  author       = {Pith},
  title        = {Pith review of: GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AHSLECIH}},
  note         = {Machine review of arXiv:2505.20416}
}
read the original abstract

Fine-tuning for large language models (LLMs) typically requires substantial amounts of high-quality supervised data, which is both costly and labor-intensive to acquire. While synthetic data generation has emerged as a promising solution, existing approaches frequently suffer from factual inaccuracies, insufficient long-tail coverage, simplistic knowledge structures, and homogenized outputs. To address these challenges, we introduce GraphGen, a knowledge graph-guided framework designed for three key question-answering (QA) scenarios: atomic QA, aggregated QA, and multi-hop QA. It begins by constructing a fine-grained knowledge graph from the source text. It then identifies knowledge gaps in LLMs using the expected calibration error metric, prioritizing the generation of QA pairs that target high-value, long-tail knowledge. Furthermore, GraphGen incorporates multi-hop neighborhood sampling to capture complex relational information and employs style-controlled generation to diversify the resulting QA data. Experimental results on knowledge-intensive tasks under closed-book settings demonstrate that GraphGen outperforms conventional synthetic data methods, offering a more reliable and comprehensive solution to the data scarcity challenge in supervised fine-tuning. The code and data are publicly available at https://github.com/open-sciencelab/GraphGen.

Figures

Figures reproduced from arXiv: 2505.20416 by the authors.

Figure 1
Figure 1. Pipeline of GraphGen. GraphGen optimizes LLM’s performance by effectively organizing knowledge [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Prompt for comprehension assessment. Through binary yes/no questions, we capture precise semantic information for confidence modeling. which measures the gap between the LLM’s current understanding and complete mastery of the knowl￾edge point. By assessing the comprehension loss of Mtrain, we can systematically evaluate whether fur￾ther training with these knowledge points is needed. STEP 3: Graph Organization Subgr… view at source ↗
Figure 3
Figure 3. Performance comparison on knowledge-intensive evaluation datasets. We use data generated through [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Distribution of comprehension loss for the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Performance comparison conducted with varying proportions of training data. The proportions are arranged in descending order based on loss. “Average” represents the mean score across three datasets. As the amount of training data increases, we observe a noticeable and …
Figure 6
Figure 6. Figure 6: Comprehension loss of the Trainee Model. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: User interface of GraphGen. “Input Configuration” is utilized to specify the data sources and the target [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Prompt for KG extraction [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Prompt for KG summarization [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Prompt for description rephrasing (opposite meaning). [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Prompt for description rephrasing (literal meaning). [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Prompt for atomic QA generation [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Prompt for aggregated answer rephrasing. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Prompt for question generation (aggregated QA). [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Prompt for multi-hop QA generation [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: An example of the extracted KG. Different colors represent different entity types. [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: Examples of the GraphGen data. The words indicating contrasts or clear logical relationships between [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: Performance comparison on knowledge-intensive evaluation datasets. The models are fine-tuned using [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]
Figure 19
Figure 19. Figure 19: Comparison of model performance trained on top 30% and bottom 30% data sorted by comprehension [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]
Figure 20
Figure 20. Figure 20: Character length distribution under the maximum loss sampling strategy. [PITH_FULL_IMAGE:figures/full_fig_p030_20.png]
Figure 21
Figure 21. Figure 21: Character length distribution under the minimum loss sampling strategy. [PITH_FULL_IMAGE:figures/full_fig_p030_21.png]
Figure 22
Figure 22. Figure 22: Character length distribution under the random selection strategy. [PITH_FULL_IMAGE:figures/full_fig_p031_22.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured Documents

    cs.CL 2025-07 reject novelty 4.0 of 10

    Easy Dataset packages existing document parsing, chunking, and persona-driven prompting into a GUI tool for synthesizing LLM fine-tuning data, but its single evaluation is compromised by overlap between training and t...

Reference graph

Works this paper leans on

43 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    Bojana Bašaragin, Adela Ljajić, Darija Medvecki, Lorenzo Cassano, Miloš Košprdić, and Nikola Milošević. 2024. How do you know that? teaching generative language models to reference answers to biomedical questions. arXiv preprint, arXiv:2407.05015

    work page Pith review arXiv 2024

  2. [2]

    Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. 2024. Instruction mining: Instruction data selection for tuning large language models. arXiv preprint, arXiv:2307.06290

    arXiv 2024

  3. [3]

    OpenCompass Contributors. 2023. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass. Accessed: 2025-02-13

    work page 2023

  4. [4]

    Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1342--1352

    work page 2017

  5. [5]

    Zichu Fei, Xin Zhou, Tao Gui, Qi Zhang, and Xuan-Jing Huang. 2022. Lfkqg: A controlled generation framework with local fine-tuning for question generation over knowledge bases. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6575--6585

    work page 2022

  6. [6]

    Khasa Gillani, Erik Novak, Klemen Kenda, and Dunja Mladeni \'c . 2024. Knowledge graph extraction from textual data using llm

    work page 2024

  7. [7]

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning, pages 1321--1330

    work page 2017

  8. [8]

    Shasha Guo, Lizi Liao, Jing Zhang, Yanling Wang, Cuiping Li, and Hong Chen. 2024 a . Sgsh: Stimulate large language models with skeleton heuristics for knowledge base question generation. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4613--4625

    work page 2024

Show all 43 references
  1. [9]

    Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. 2024 b . Lightrag: Simple and fast retrieval-augmented generation. arXiv preprint, arXiv:2410.05779

    2024 arXiv

  2. [10]

    Nourhan Ibrahim, Samar Aboulela, Ahmed Ibrahim, and Rasha Kashef. 2024. A survey on augmenting knowledge graphs (kgs) with large language models (llms): models, evaluation metrics, benchmarks, and challenges. Discover Artificial Intelligence, 4(1):76

    2024

  3. [11]

    Sathish Reddy Indurthi, Dinesh Raghu, Mitesh M Khapra, and Sachindra Joshi. 2017. Generating natural language question-answer pairs from a knowledge graph using a rnn based question generation model. In Proceedings of the 15th Conference of the European Chapter of the Associat...

    2017

  4. [12]

    Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023. Towards mitigating llm hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827--1843

    2023

  5. [13]

    Robin Jia and Percy Liang. 2016. Data recombination for neural semantic parsing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12--22

    2016

  6. [14]

    Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In International Conference on Machine Learning, pages 15696--15707. PMLR

    2023

  7. [15]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint, arXiv:2001.08361

    2020 arXiv

  8. [16]

    Huanjun Kong. 2025. Huixiangdou2: A graph-based augmented generation approach. https://github.com/tpoisonooo/HuixiangDou2. Accessed: 2025-02-13

    2025

  9. [17]

    Abdullatif Köksal, Timo Schick, Anna Korhonen, and Hinrich Schütze. 2024. Longform: Effective instruction tuning with reverse instructions. arXiv preprint, arXiv:2304.08460

    2024 arXiv

  10. [18]

    Dongyang Li, Junbing Yan, Taolin Zhang, Chengyu Wang, Xiaofeng He, Longtao Huang, Hui Xue, and Jun Huang. 2024 a . On the role of long-tail knowledge in retrieval augmented large language models. arXiv preprint, arXiv:2406.16367

    2024 arXiv

  11. [19]

    Huihan Li, Yuting Ning, Zeyi Liao, Siyuan Wang, Xiang Li, Ximing Lu, Wenting Zhao, Faeze Brahman, Yejin Choi, and Xiang Ren. 2024 b . In search of the long-tail: Systematic generation of long-tail inferential knowledge via logical rule guided search. In Proceedings of the 2024...

    2024

  12. [20]

    Yuanyuan Liang, Jianing Wang, Hanlun Zhu, Lei Wang, Weining Qian, and Yunshi Lan. 2023. Prompting large language models with chain-of-thought for few-shot knowledge base question generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process...

    2023

  13. [21]

    Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, et al. 2024. Best practices and lessons learned on synthetic data for language models. arXiv preprint, arXiv:2404.07503

    2024 arXiv

  14. [22]

    Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On llms-driven synthetic data generation, curation, and evaluation: A survey. arXiv preprint, arXiv:2406.15126

    2024 arXiv

  15. [23]

    Keer Lu, Keshi Zhao, Zheng Liang, Da Pan, Shusen Zhang, Xin Wu, Weipeng Chen, Zenan Zhou, Guosheng Dong, Bin Cui, et al. 2024. Versatune: Fine-tuning multi-ability llms efficiently. arXiv preprint, arXiv:2411.11266

    2024 arXiv

  16. [24]

    Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. 2024. Rephrasing the web: A recipe for compute and data-efficient language modeling. arXiv preprint, arXiv:2401.16380

    2024 arXiv

  17. [25]

    Philip M McCarthy and Scott Jarvis. 2010. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, 42(2):381--392

    2010

  18. [26]

    Nick Mecklenburg, Yiyou Lin, Xiaoxiao Li, Daniel Holstein, Leonardo Nunes, Sara Malvar, Bruno Silva, Ranveer Chandra, Vijay Aski, Pavan Kumar Reddy Yannam, et al. 2024. Injecting new knowledge into large language models via supervised fine-tuning. arXiv preprint, arXiv:2404.00213

    2024 arXiv

  19. [27]

    Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, and Arsalan Shahid. 2024. The ultimate guide to fine-tuning llms from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research challenges and opportunities. arXiv p...

    2024 arXiv

  20. [28]

    Dominic Seyler, Mohamed Yahya, and Klaus Berberich. 2017. Knowledge questions from knowledge graphs. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval, ICTIR ’17, page 11–18. ACM

    2017

  21. [29]

    Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. 2016. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 761--769

    2016

  22. [30]

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. Ai models collapse when trained on recursively generated data. Nature, 631(8022):755--759

    2024

  23. [31]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  24. [32]

    Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language models to commonsense models. In Proceedings of the 2022 Conference of the North American...

    2022

  25. [33]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2025. Qwen2.5 technical report. arXiv preprint, arXiv:2412.15115

    2025 arXiv

  26. [34]

    Cohen, Ruslan Salakhutdinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint, arXiv:1809.09600

    2018 arXiv

  27. [35]

    Zitong Yang, Neil Band, Shuangping Li, Emmanuel Candès, and Tatsunori Hashimoto. 2024. Synthetic continued pretraining. arXiv preprint, arXiv:2409.07431

    2024 arXiv

  28. [36]

    Asaf Yehudai, Boaz Carmeli, Yosi Mass, Ofir Arviv, Nathaniel Mills, Assaf Toledo, Eyal Shnarch, and Leshem Choshen. 2024. Genie: Achieving human parity in content-grounded datasets generation. arXiv preprint, arXiv:2401.14367

    2024 arXiv

  29. [37]

    Jie Ying, Zihong Chen, Zhefan Wang, Wanli Jiang, Chenyang Wang, Zhonghang Yuan, Haoyang Su, Huanjun Kong, Fan Yang, and Nanqing Dong. 2025. https://arxiv.org/abs/2505.13220 Seedbench: A multi-task benchmark for evaluating large language models in seed science . Preprint, arXiv...

    2025 arXiv

  30. [39]

    Xuanyu Zhang and Qing Yang. 2023 b . Self-qa: Unsupervised knowledge guided language model alignment. arXiv preprint arXiv:2305.11952

    2023 arXiv

  31. [40]

    Runhao Zhao, Jiuyang Tang, Weixin Zeng, Ziyang Chen, and Xiang Zhao. 2024 a . Zero-shot knowledge graph question generation via multi-agent llms and small models synthesis. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 3...

    2024

  32. [41]

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024 b . Marco-o1: Towards open reasoning models for open-ended solutions. arXiv preprint, arXiv:2411.14405

    2024 arXiv

  33. [42]

    Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022. Towards a unified multi-dimensional evaluator for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages ...

    2022

  34. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  35. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.