Pith. sign in

REVIEW 5 major objections 8 minor 1 cited by

Few-shot LLM Synthetic Data with Distribution Matching

T0 review · 5 major / 8 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SynAlign claims that LLM-generated synthetic text, when selected and reweighted to match the real data's embedding distribution, yields higher downstream classification accuracy and F1 than existing synthetic-data augmentation methods…

desk verdict SynAlign is a coherent integration of known components with credible industrial evidence, but the paper needs statistical rigor, a fixed Table 4, and a direct look at label noise before the marginal-matching story is convincing. read the letter →

arxiv 2502.08661 v2 pith:Z2NJ3VBQ submitted 2025-02-09 cs.CL cs.AI

Jiyuan Ren , Zhaocheng Du , Zhihao Wen , Qinglin Jia , Sunhao Dai , Chuhan Wu , Zhenhua Dong This is my paper · ORCID
classification cs.CLcs.AI
keywords syntheticdatagenerationdistributionmatchingmaximummeandiscrepancylargelanguagemodelsaugmentationGaussianprocesssamplingtextclassificationfew-shotlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that LLM-generated synthetic text can be made useful for training smaller models if its distribution is aligned with the real data, rather than simply added to it. SynAlign does this in three stages: a Gaussian Process uncertainty tracker selects diverse real demonstrations, the LLM reasons about latent linguistic attributes before generating, and Maximum Mean Discrepancy (MMD) is used to learn a sampling weight for every synthetic example so the final pool matches the real embedding distribution. Across SST-2, AGNEWS, and Amazon, the MMD-reweighted subset consistently reaches the highest accuracy and F1 among synthetic-data baselines, and the same pipeline lifted an online search-ad retriever's RPM by 2.86% and CPM by 2.31% in a week-long A/B test. A sympathetic reader would take the claim as: distribution matching is what separates synthetic data that helps from synthetic data that distorts.

What carries the argument

The load-bearing mechanism is the MMD sampling-weight objective of Eq. (10): synthetic and real texts are mapped to embeddings with Sentence-BERT, and a weight vector is learned by minimizing the squared distance between mean projections of the two sets under random linear projections, with Gram-Schmidt orthogonalization used to build the projection family. Before generation, a Gaussian Process with an RBF kernel tracks per-sample uncertainty so each round of demonstrations explores under-covered parts of the real embedding space, and a chain-of-thought step asks the LLM to summarize attributes of those demonstrations before writing new examples. Together these modules make the final resampled synthetic pool mimic the real data's marginal embedding distribution.

What would settle it

Run the MMD reweighting on a synthetic pool generated with random demonstrations and no attribute reasoning; if it matches SynAlign(mmd)'s accuracy and F1, then the GP exploration and latent-attribute modules are not load-bearing and the central claim about the full pipeline is falsified.

Watch

Extended reading notes

Core claim

The central claim is that synthetic data generated by an LLM can be post-aligned to real data by learning per-sample weights in an embedding space, and that this alignment, combined with exploration-aware demonstration selection and attribute-based generation, yields training data that improves downstream classifiers more than the full synthetic pool or random subsets. The paper argues that LLMs fail to cover all real linguistic attributes and over- or under-produce attribute proportions, and that MMD-based reweighting corrects the proportion mismatch after the fact. On three text classification tasks with BERT-base and DistilBERT, the method SynAlign(mmd) reports the best accuracy and macro-F1 in every configuration, and offline evaluation in a deployed system shows a 0.26% AUC gain while the online test shows RPM and CPM increases.

Load-bearing premise

The method assumes that matching the marginal distribution of Sentence-BERT embeddings between synthetic and real texts is enough to make the synthetic examples useful, without ever checking whether the LLM's assigned labels are correct.

Editorial extensions

If this is right

  • LLM-generated text can be safely mixed into real training sets only if the mix is selected to match the real distribution; using the whole generated pool or a random subset is consistently worse than using the MMD-weighted subset.
  • Exploration-aware demonstration selection makes the same token budget cover more of the real data's attribute space, so generation becomes more efficient as well as more accurate.
  • MMD reweighting is model-agnostic once an embedding space is chosen, so the alignment step transfers to any text-domain pair where Sentence-BERT-like embeddings are available.
  • The online A/B test indicates the method works outside academic benchmarks: a deployed search-ad retriever improved RPM by 2.86% and CPM by 2.31% over one week.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not explore per-class conditional MMD; a natural extension would be to match each class's embedding distribution separately, which could protect minority classes during reweighting.
  • The GP sampling and the MMD reweighting are separable components, so an ablation that replaces attribute reasoning with plain demonstrations would reveal which module actually drives the gains over random selection.
  • The same weighting scheme could be applied to other synthetic generation setups, such as code generation or vision-language data, whenever a fixed embedding model defines the target distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes SynAlign, a three-stage pipeline for augmenting few-shot text classification data with LLM-generated synthetic text. The first stage uses a Gaussian Process uncertainty tracker to select diverse demonstrations; the second asks the LLM to summarize latent linguistic attributes and generate new samples conditioned on them; the third reweights the synthetic samples by minimizing a random-projection MMD-like objective between Sentence-BERT embeddings of real and synthetic texts. The authors report accuracy and F1 improvements over Gold, SimPrompt, and AttrPrompt on SST-2, AGNEWS, and Amazon with BERT-base and DistilBERT, and they report an online A/B test in a search-advertising pre-ranking module. The central claim is that the combination of exploration-aware sampling and MMD-based reweighting yields synthetic data that better matches the real data distribution and thereby improves downstream classification.

Significance. If the empirical claims were fully supported, the paper would offer a practical, modular recipe for LLM-based data augmentation that avoids training a separate generative model and that has been tested in a deployed system. The inclusion of an online A/B test and the attempt to couple demonstration selection with post-hoc reweighting are valuable. However, the paper currently overstates its main result: the "consistently highest" claim is false in several Table 1 cells, the ablation table has swapped column values, and the MMD objective matches only text embeddings rather than the joint (x,y) distribution used in the paper's own definition of high-quality data. These issues must be resolved before the contribution can be assessed.

major comments (5)
  1. [Section 4.2, Table 1] The text states that "SynAlign(mmd) consistently achieves the highest Accuracy and F1 scores across datasets and models," but Table 1 contains counterexamples. On SST-2 with DistilBERT, SynAlign(mmd) has F1 0.9211 versus 0.9215 for AttrPrompt; on AGNEWS with DistilBERT, SynAlign(mmd) has F1 0.9433 versus 0.9442 for SynAlign(random) and 0.9451 for SynAlign(all); on Amazon with DistilBERT, SynAlign(mmd) has F1 0.8312 versus 0.8351 for SynAlign(all). The claim should be restricted to the cells where it actually holds, or the experiments should be rerun with a procedure that supports the stated conclusion.
  2. [Section 4.4.2, Table 4] The AGNEWS and Amazon columns of Table 4 appear to be swapped relative to Table 1. For example, the two-stage SynAlign(mmd) row gives 0.8381 for AGNEWS and 0.9475 for Amazon, whereas Table 1 reports 0.9475 for AGNEWS and 0.8381 for Amazon. Similar swaps occur for most other rows. Because Section 4.4.2 uses this table to conclude that the two-stage generation consistently outperforms the single-stage approach, the table must be corrected and the corresponding conclusions rechecked.
  3. [Section 3.4, Eq. (10)] The alignment objective matches only the marginal distribution of Sentence-BERT embeddings E_ori and E_gen; the labels y_ori and y'_gen do not appear in the loss. Since Section 1 defines high-quality synthetic data as data matching the distribution under which the model is evaluated, and the downstream tasks are classification problems, the relevant object is the joint distribution P(x,y), not P(x). Without a label-correctness analysis (e.g., agreement with real labels on a held-out subset) or a label-aware baseline, the paper has not established that the MMD reweighting improves performance by aligning the true evaluation distribution; the gains could come from generic filtering of low-quality text. I ask the authors to either extend Eq. (10) to a joint embedding (for instance, concatenating text and label embeddings) or provide evidence that the labels of the selected synthetic samples are reliable.
  4. [Section 3.4, Eq. (10) and implementation details] The objective is described as MMD but is implemented as a distance between averages of random linear projections theta of the embeddings. This is not the RKHS MMD of Gretton et al. unless those projections are random features for a specific kernel, which the paper does not state. The optimization over the sampling weights omega is also underspecified: Eq. (10) has no constraints (e.g., omega >= 0, sum(omega) = N or M), no regularization, and no discussion of how the linear equation set is solved by gradient descent, including learning rate, iterations, and initialization. Without these details the procedure is not reproducible, and the name "MMD" may be misleading. Please provide the exact objective, constraints, and optimization protocol, or rename the method to reflect the projection-based distance actually used.
  5. [Section 4.2 and Section 4.1, Table 1] All experimental results appear to be single runs; no standard deviations, number of seeds, or significance tests are reported. Several differences between methods are small (for example, SST-2 BERT-base accuracy is 0.9248 for Gold, 0.9260 for AttrPrompt, and 0.9330 for SynAlign(mmd); AGNEWS DistilBERT accuracy is 0.9441 for SynAlign(random) and 0.9464 for SynAlign(mmd)). The "consistently" language in Section 4.2 is stronger than the data support. The authors should report means and standard deviations over multiple seeds and perform paired significance tests for the main comparisons, or explicitly state that the reported differences are not statistically tested.
minor comments (8)
  1. [Section 3.2, Eq. (2)] The sentence before Eq. (2) says the RBF kernel ensures covariance between selected and unselected samples increases as text embedding similarity decreases, but the kernel exp(-||ei-ej||/(2 tau)) decreases with distance, so covariance increases with similarity, not with dissimilarity. Please correct the description.
  2. [Algorithm 1] The Output line of Algorithm 1 reads "Original Dataset D_gen"; the intended output appears to be the aligned synthetic dataset, so the name should be corrected to something like D'_gen.
  3. [Section 4.1, Table 1] The row "LLM zero-shot" in Table 1 is not described in Section 4.1. Please clarify whether this is the LLM's zero-shot classification accuracy on the test set and why it is included in a table of data-augmentation methods.
  4. [Abstract and Section 4.6] The abstract refers to an "online retriever," but Section 4.6 describes deployment in the pre-ranking module of a search advertising system. Please align the terminology.
  5. [Section 3.4, Eq. (10)] The notation in Eq. (10) uses theta both as a projection matrix and in the dot product theta * E; please specify whether theta is a vector in R^n or a matrix and define the dimensions explicitly.
  6. [Abstract and code availability] The abstract states "Our code is available here," but no URL is provided in the manuscript. Please include a working link or remove the sentence.
  7. [Section 4.1 and Table 6] The number of selected synthetic samples (#Sampled) is listed in Table 6 but the symbol zeta used in the baseline descriptions is never defined, nor is the selection rule for #Sampled explained. Please add a definition.
  8. [References] References [37] and [38] are duplicate entries for the same DistilBERT paper by Sanh; please consolidate them.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: held-out accuracy and online A/B test ground the central claims; auxiliary Wasserstein diagnostic is partly self-confirming but not load-bearing.

full rationale

The paper's central claims are downstream Accuracy/F1 on held-out SST-2, AGNEWS, and Amazon test sets plus an online A/B test on a deployed retriever; these are not used to fit SynAlign's sampling weights or hyperparameters, so the main evaluation is externally grounded. The MMD objective (Eq. 10) is exactly the proposed alignment mechanism, and the Wasserstein-distance numbers in Table 2 are an auxiliary diagnostic computed on the same embeddings that the reweighting was optimized to match, so they are partly self-confirming; however, the paper does not use Table 2 to derive the accuracy results, and the accuracy gains are measured independently. The only self-citation ([16], cited for the empirical claim that LLM synthetic data has disproportionate linguistic attributes) is minor and not load-bearing for the method or the evaluation. No equation reduces to its own input, no fitted parameter is renamed as a prediction, and no uniqueness or ansatz is imported from the authors' prior work. Hence no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or conceptual entities; its burden is in domain assumptions and tuned hyperparameters. The central assumptions are about embedding geometry, GP-based coverage, and the sufficiency of marginal embedding matching for task performance.

free parameters (5)
  • RBF kernel bandwidth tau = Dataset-specific optimum, e.g., 0.9 for SST-2
    Controls GP covariance smoothness in exploration-aware sampling; selected per dataset in Section 4.5.1.
  • Nearest-neighbor count k = Not fixed in text; varied with dataset, e.g., around 20 for SST-2
    Controls number of demonstrations per selection round; chosen as hyperparameter in Section 4.5.2.
  • GP uncertainty threshold sigma = Not reported
    Stopping criterion for demonstration selection in Algorithm 1; has direct effect on how much synthetic data is generated.
  • Number of projection matrices |Theta| = e.g., 50 for SST-2 and Amazon, 100 for AGNEWS
    Determines expressiveness of MMD alignment; tuned per dataset in Section 4.5.3.
  • MMD sampling weights omega = Learned per synthetic sample by gradient descent
    Core alignment parameters fitted by minimizing Eq. (10); constraints and regularization are not specified.
assumptions (4)
  • domain assumption Sentence-BERT embedding similarity reflects linguistic-attribute similarity in real and synthetic text.
    Used in Eq. (1)-(2) for the GP kernel and in Eq. (10) for MMD; if embeddings encode irrelevant geometry, both sampling and alignment lose validity.
  • domain assumption A Gaussian Process with zero prior mean, unit variances, and RBF kernel is a meaningful uncertainty tracker for coverage of the real-data distribution.
    Section 3.2: selection is based on GP posterior variance; there is no evidence that this variance maps to linguistic-diversity coverage.
  • domain assumption Matching the marginal distribution of text embeddings is sufficient for downstream classification improvement, and LLM-generated labels are reliable.
    Section 3.4 objective (Eq. 10) aligns only embeddings; label correctness is never checked or incorporated.
  • ad hoc to paper A finite set of random orthogonal linear projections approximates the MMD in RKHS well enough for distribution alignment.
    Eq. (10)-(11): the paper replaces kernel MMD with mean matching over random projection matrices without theoretical guarantee or kernel comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Few-shot LLM Synthetic Data with Distribution Matching." pith.science (2026). https://pith.science/paper/Z2NJ3VBQ

@misc{pith2026250208661,
  author       = {Pith},
  title        = {Pith review of: Few-shot LLM Synthetic Data with Distribution Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z2NJ3VBQ}},
  note         = {Machine review of arXiv:2502.08661}
}
read the original abstract

As large language models (LLMs) advance, their ability to perform in-context learning and few-shot language generation has improved significantly. This has spurred using LLMs to produce high-quality synthetic data to enhance the performance of smaller models like online retrievers or weak LLMs. However, LLM-generated synthetic data often differs from the real data in key language attributes (e.g., styles, tones, content proportions, etc.). As a result, mixing these synthetic data directly with real data may distort the original data distribution, potentially hindering performance improvements. To solve this, we introduce SynAlign: a synthetic data generation and filtering framework based on key attribute distribution matching. Before generation, SynAlign employs an uncertainty tracker surrogated by the Gaussian Process model to iteratively select data clusters distinct from selected ones as demonstrations for new data synthesis, facilitating the efficient exploration diversity of the real data. Then, a latent attribute reasoning method is employed: the LLM summarizes linguistic attributes of demonstrations and then synthesizes new data based on them. This approach facilitates synthesizing diverse data with linguistic attributes that appear in real data.After generation, the Maximum Mean Discrepancy is used as the objective function to learn the sampling weight of each synthetic data, ensuring distribution matching with the real data. Our experiments on multiple text prediction tasks show significant performance improvements. We also conducted an online A/B test on an online retriever to demonstrate SynAlign's effectiveness.

Figures

Figures reproduced from arXiv: 2502.08661 by the authors.

Figure 1
Figure 1. A case on how LLM synthetic samples misalign with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. SynAlign comprises three modules for aligning the distribution of synthetic samples. First, the Exploration-aware [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. t-SNE visualization of sentence embeddings from [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (a) Convex hull coverage example for SST-2. (b) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Coverage rate as a function of RBF kernel length scale [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Prompt used in Stage 1 (Attribute Summarization). [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Prompt used in Stage 2 (Synthetic Data Generation). [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction

    cs.HC 2026-02 conditional novelty 6.0 of 10

    Fine-tuned Qwen2.5-0.5B classifies leader-follower roles with 86.66% accuracy in single-turn interactions, but accuracy falls to chance in one-shot multi-turn interactions.

Reference graph

Works this paper leans on

65 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Kushal Arora, Layla El Asri, Hareesh Bahuleyan, and Jackie Chi Kit Cheung

    work page

  2. [2]

    John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. InPro- ceedings of the 45th annual meeting of the association of computational linguistics . 440–447. Few-shot LLM Synthetic Data with Distribution Matching WWW Companion ’25, April 28-May 2, 2025, Sydney,...

    work page 2007

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  4. [4]

    Derek Chen, Celine Lee, Yunan Lu, Domenic Rosati, and Zhou Yu. 2023. Mixture of Soft Prompts for Controllable Data Generation. arXiv:2303.01580 [cs.CL] https://arxiv.org/abs/2303.01580

    arXiv 2023

  5. [5]

    Sunhao Dai, Weihao Liu, Yuqi Zhou, Liang Pang, Rongju Ruan, Gang Wang, Zhenhua Dong, Jun Xu, and Ji-Rong Wen. 2024. Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration. Findings of the Association for Computational Linguistics: ACL 2024 (2024)

    2024

  6. [6]

    Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongx- iang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. In Proceedings of the 17th ACM Conference on Recom- mender Systems. 1126–1132

    2023

  7. [7]

    Sunhao Dai, Ninglu Shao, Jieming Zhu, Xiao Zhang, Zhenhua Dong, Jun Xu, Quanyu Dai, and Ji-Rong Wen. 2024. Modeling user attention in music recom- mendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 761–774

    2024

  8. [8]

    Sunhao Dai, Yuqi Zhou, Jun Xu, and Ji-Rong Wen. 2023. Dually Enhanced Delayed Feedback Modeling for Streaming Conversion Rate Prediction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Manage- ment. 390–399

    work page 2023

Show all 65 references
  1. [9]

    Yi Dai, Hao Lang, Yinhe Zheng, Fei Huang, and Yongbin Li. 2023. Long-tailed question answering in an open world. arXiv preprint arXiv:2305.06557 (2023)

    2023 arXiv

  2. [10]

    Florian Daniel, Pavel Kucherbaev, Cinzia Cappiello, Boualem Benatallah, and Mohammad Allahbakhsh. 2018. Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions.ACM Computing Surveys (CSUR) 51, 1 (2018), 1–40

    2018

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

    2019 arXiv

  4. [12]

    Zhaocheng Du, Junhao Chen, Qinglin Jia, Chuhan Wu, Jieming Zhu, Zhenhua Dong, and Ruiming Tang. 2024. LightCS: Selecting Quadratic Feature Crosses in Linear Complexity. In Companion Proceedings of the ACM on Web Conference

    2024

  5. [13]

    Zhaocheng Du, Chuhan Wu, Qinglin Jia, Jieming Zhu, and Xu Chen. 2024. A Tutorial on Feature Interpretation in Recommender Systems. In Proceedings of the 18th ACM Conference on Recommender Systems . 1281–1282

    2024

  6. [14]

    Ronen Eldan and Yuanzhi Li. 2023. TinyStories: How Small Can Language Models Be and Still Speak Coherent English? arXiv:2305.07759 [cs.CL] https: //arxiv.org/abs/2305.07759

    2023 arXiv

  7. [15]

    Steven Y Feng, Varun Gangal, Dongyeop Kang, Teruko Mitamura, and Eduard Hovy. 2020. Genaug: Data augmentation for finetuning text generators. arXiv preprint arXiv:2010.01794 (2020)

    2020 arXiv

  8. [16]

    Jingtong Gao, Zhaocheng Du, Xiaopeng Li, Xiangyu Zhao, Yichao Wang, Xi- angyang Li, Huifeng Guo, and Ruiming Tang. 2025. SampleLLM: Optimizing Tabular Data Synthesis in Recommendations. arXiv preprint arXiv:2501.16125 (2025)

    2025 arXiv

  9. [17]

    Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Weizhong Zhang, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. 2023. Self-Guided Noise- Free Data Generation for Efficient Zero-Shot Learning. arXiv:2205.12679 [cs.CL] https://arxiv.org/abs/2205.12679

    2023 arXiv

  10. [18]

    Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences of the United States of America 120 (2023). https://api.semanticscholar. org/CorpusID:257766307

    2023

  11. [19]

    Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. The Journal of Machine Learning Research 13, 1 (2012), 723–773

    2012

  12. [20]

    Xingwei He, Zhenghao Lin, Yeyun Gong, Alex Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, Weizhu Chen, et al . 2023. Annollm: Making large language models to be better crowdsourced annotators. arXiv preprint arXiv:2303.16854 (2023)

    2023 arXiv

  13. [21]

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. arXiv preprint a...

    2023 arXiv

  14. [22]

    Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2023. Large Language Models Can Self-improve. https: //openreview.net/forum?id=NiEtU7blzN

    2023

  15. [23]

    Pengyue Jia, Yejing Wang, Zhaocheng Du, Xiangyu Zhao, Yichao Wang, Bo Chen, Wanyu Wang, Huifeng Guo, and Ruiming Tang. 2024. Erase: Benchmarking feature selection methods for deep recommender systems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and ...

    2024

  16. [24]

    Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 39–48

    2020

  17. [25]

    Steven J Leon, Åke Björck, and Walter Gander. 2013. Gram-Schmidt orthogonal- ization: 100 years and more. Numerical Linear Algebra with Applications 20, 3 (2013), 492–532

    2013

  18. [26]

    Minzhi Li, Taiwei Shi, Caleb Ziems, Min-Yen Kan, Nancy F Chen, Zhengyuan Liu, and Diyi Yang. 2023. Coannotating: Uncertainty-guided work allocation between human and large language models for data annotation. arXiv preprint arXiv:2310.15638 (2023)

    2023 arXiv

  19. [27]

    Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. 2023. Synthetic data generation with large language models for text classification: Potential and limitations. arXiv preprint arXiv:2310.07849 (2023)

    2023 arXiv

  20. [28]

    Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, et al. 2024. Best practices and lessons learned on synthetic data. In First Conference on Language Modeling

    2024

  21. [29]

    Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey. arXiv:2406.15126 [cs.CL] https://arxiv.org/abs/2406.15126

    2024 arXiv

  22. [30]

    Yan Lyu, Sunhao Dai, Peng Wu, Quanyu Dai, Yuhao Deng, Wenjie Hu, Zhenhua Dong, Jun Xu, Shengyu Zhu, and Xiao-Hua Zhou. 2022. A Semi-Synthetic Dataset Generation Framework for Causal Inference in Recommender Systems. arXiv preprint arXiv:2202.11351 (2022)

    2022 arXiv

  23. [31]

    Abdelzaher, and Jiawei Han

    Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, Tarek F. Abdelzaher, and Jiawei Han. 2022. Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning. In International Conference on Ma- chine Learning. https://api.semanticscholar.org/Corp...

    2022

  24. [32]

    Kevin P Murphy. 2012. Machine learning: a probabilistic perspective . MIT press

    2012

  25. [33]

    Weili Nie, Nina Narodytska, and Ankit B. Patel. 2019. RelGAN: Relational Gener- ative Adversarial Networks for Text Generation. In International Conference on Learning Representations. https://api.semanticscholar.org/CorpusID:68160504

    2019

  26. [34]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  27. [35]

    Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer. 2021. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations

    2021

  28. [36]

    N Reimers. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  29. [38]

    V Sanh. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    2019 arXiv

  30. [39]

    Nabeel Seedat, Nicolas Huynh, Boris van Breugel, and Mihaela van der Schaar

  31. [40]

    Mohamed Ashik Shahul Hameed, Asifa Mehmood Qureshi, and Abhishek Kaushik. 2024. Bias Mitigation via Synthetic Data Generation: A Review. Elec- tronics (2079-9292) 13, 19 (2024)

    2024

  32. [41]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  33. [42]

    Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W White, Longqi Yang, et al. 2024. Tnt- llm: Text mining at scale with large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowled...

    2024

  34. [43]

    Chaojun Wang and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural machine translation. arXiv preprint arXiv:2005.03642 (2020)

    2020 arXiv

  35. [44]

    Ruida Wang, Wangchunshu Zhou, and Mrinmaya Sachan. 2023. Let’s Synthe- size Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models. arXiv preprint arXiv:2310.13671 (2023)

    2023 arXiv

  36. [45]

    Yejing Wang, Zhaocheng Du, Xiangyu Zhao, Bo Chen, Huifeng Guo, Ruiming Tang, and Zhenhua Dong. 2023. Single-shot feature selection for multi-task recommendations. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval...

    2023

  37. [46]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  38. [47]

    Christopher Williams and Carl Rasmussen. 1995. Gaussian processes for regres- sion. Advances in neural information processing systems 8 (1995)

    1995

  39. [48]

    Fei Wu, Raphael Hoffmann, and Daniel S Weld. 2008. Information extraction from Wikipedia: Moving down the long tail. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining . 731–739

    2008

  40. [49]

    Ruixuan Xiao, Yiwen Dong, Junbo Zhao, Runze Wu, Minmin Lin, Gang Chen, and Haobo Wang. 2023. Freeal: Towards human-free active learning in the era of WWW Companion ’25, April 28-May 2, 2025, Sydney, NSW, Australia Ren and Du et al. large language models. arXiv preprint arXiv:2...

    2023 arXiv

  41. [50]

    Jiacheng Ye, Jiahui Gao, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong

  42. [51]

    Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyeong Park

  43. [52]

    Ratner, Ranjay Kr- ishna, Jiaming Shen, and Chao Zhang

    Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander J. Ratner, Ranjay Kr- ishna, Jiaming Shen, and Chao Zhang. 2023. Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias. ArXiv abs/2306.15895 (2023). https://api.semanticscholar.org/Corpus...

    2023 arXiv

  44. [53]

    arXiv preprint arXiv:2210.12329 (2022)

    Progen: Progressive zero-shot dataset generation via in-context feedback. arXiv preprint arXiv:2210.12329 (2022)

    2022 arXiv

  45. [54]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. NeurIPS 28 (2015)

    2015

  46. [55]

    Bo Zhao and Hakan Bilen. 2023. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 6514–6523

    2023

  47. [56]

    Yuang Zhao, Zhaocheng Du, Qinglin Jia, Linxuan Zhang, Zhenhua Dong, and Ruiming Tang. 2024. Retrievable Domain-Sensitive Feature Memory for Multi- Domain Recommendation. arXiv preprint arXiv:2405.12892 (2024). Few-shot LLM Synthetic Data with Distribution Matching WWW Companio...

    2024 arXiv

  48. [57]

    Yue Yu, Yuchen Zhuang, Rongzhi Zhang, Yu Meng, Jiaming Shen, and Chao Zhang. 2023. Regen: Zero-shot text classification via training data generation with progressive dense retrieval. arXiv preprint arXiv:2305.10703 (2023)

    2023 arXiv

  49. [61]

    Summarize the following characteristics: <List of attributions>

  50. [62]

    Now, begin summarizing: Figure 6: Prompt used in Stage 1 (Attribute Summarization)

    Provide the summary in the following JSON format. Now, begin summarizing: Figure 6: Prompt used in Stage 1 (Attribute Summarization). The LLM is instructed to analyze a dataset example and ex- tract key linguistic and semantic attributes, such as topics, language habits, and w...

  51. [63]

    Each review should describe a product in the same category as the one in the Product Info, but for a different product with similar characteristics

  52. [64]

    Follow the attributes described in the review_info (e.g., usage experience, writing style, subtopics)

  53. [65]

    Select only 1-2 subtopics from the provided list for each review

  54. [66]

    Now, begin summarizing: Figure 7: Prompt used in Stage 2 (Synthetic Data Generation)

    Ensure each review is unique, concise, and distinct in language. Now, begin summarizing: Figure 7: Prompt used in Stage 2 (Synthetic Data Generation). The LLM generates multiple unique samples based on the attributes summarized in Stage 1. For example, in the Ama- zon dataset,...

  55. [2021]

    In Conference on Empirical Methods in Natural Language Processing

    GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation. In Conference on Empirical Methods in Natural Language Processing . https: //api.semanticscholar.org/CorpusID:233296100

  56. [2022]

    arXiv preprint arXiv:2204.01171 (2022)

    Why exposure bias matters: An imitation learning perspective of error accumulation in language generation. arXiv preprint arXiv:2204.01171 (2022)

    2022 arXiv

  57. [2023]

    arXiv preprint arXiv:2312.12112 (2023)

    Curated llm: Synergy of llms and data curation for tabular augmentation in ultra low-data regimes. arXiv preprint arXiv:2312.12112 (2023)

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.