Pith. sign in

REVIEW 11 cited by

Large Language Models for Data Annotation and Synthesis: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.13446 v3 pith:O2OKSZJQ submitted 2024-02-21 cs.CL

Zhen Tan , Dawei Li , Song Wang , Alimohammad Beigi , Bohan Jiang , Amrita Bhattacharjee , Mansooreh Karami , Jundong Li
show 2 more authors
Lu Cheng Huan Liu
This is my paper
classification cs.CL
keywords dataannotationllmsmodelssurveysynthesisannotationsllm-generated
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Data annotation and synthesis generally refers to the labeling or generating of raw data with relevant information, which could be used for improving the efficacy of machine learning models. The process, however, is labor-intensive and costly. The emergence of advanced Large Language Models (LLMs), exemplified by GPT-4, presents an unprecedented opportunity to automate the complicated process of data annotation and synthesis. While existing surveys have extensively covered LLM architecture, training, and general applications, we uniquely focus on their specific utility for data annotation. This survey contributes to three core aspects: LLM-Based Annotation Generation, LLM-Generated Annotations Assessment, and LLM-Generated Annotations Utilization. Furthermore, this survey includes an in-depth taxonomy of data types that LLMs can annotate, a comprehensive review of learning strategies for models utilizing LLM-generated annotations, and a detailed discussion of the primary challenges and limitations associated with using LLMs for data annotation and synthesis. Serving as a key guide, this survey aims to assist researchers and practitioners in exploring the potential of the latest LLMs for data annotation, thereby fostering future advancements in this critical field.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FunnelAL: Retrieve-then-Rank Active Learning for Single-Class Discovery

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A precision-triggered retrieve-then-rank active-learning pipeline achieves better label efficiency and F1 than recent single-class discovery baselines on three image benchmarks.

  2. CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams

    cs.LG 2025-08 reject novelty 6.0 of 10

    CALM uses an LLM-as-a-Judge to curate anomalies for continuous fine-tuning of a time-series foundation model, improving anomaly detection on held-out stream segments.

  3. Evaluating Large Language Models as Expert Annotators

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    Material Fingerprinting recovers the form and parameters of hyperelastic material models by nearest-neighbor matching of test data against a simulated fingerprint database: exact at zero noise, degrading under 5% noise.

  4. What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Moderately diverse LLM-generated data can improve fine-tuned model performance in low-data settings when distribution shift is minimal, while high diversity or large distribution shift hurts.

  5. MELT: Towards Automated Multimodal Emotion Data Annotation by Leveraging LLM Embedded Knowledge

    cs.AI 2025-05 conditional novelty 6.0 of 10

    GPT-4o text-only prompts can re-annotate a Friends-based emotion dataset, and models trained on those labels outperform models trained on the original human labels in cross-corpus speech emotion recognition.

  6. Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement

    cs.CL 2025-05 conditional novelty 6.0 of 10

    SiDyP improves classifiers trained on LLM-generated noisy labels by retrieving likely true labels from embedding-space neighbors and iteratively refining them with a simplex diffusion model, reporting average gains of...

  7. Generating Privacy Stories From Software Documentation

    cs.SE 2025-06 conditional novelty 5.0 of 10

    LLMs can extract privacy behaviors from software documents and draft privacy stories, but the best overall F1 is 0.766, not the abstract's 0.8+.

  8. Enterprise Large Language Model Evaluation Benchmark

    cs.AI 2025-06 reject novelty 5.0 of 10

    A 14-task enterprise LLM benchmark built mostly from GPT-4o-generated labels and scored by GPT-4o-as-judge shows open-source models closing the reasoning gap, but the dataset is not public and the evaluation is partly...

  9. StackingNet: Collective Inference Across Independent AI Foundation Models

    cs.AI 2026-02 conditional novelty 4.0 of 10

    A lightweight weighted-average 'meta-model' over black-box LLM/VLM outputs improves accuracy, reduces bias, and ranks/prunes unreliable models across regression and classification tasks.

  10. Evaluating LLMs Without Oracle Feedback: Agentic Annotation Evaluation Through Unsupervised Consistency Signals

    cs.CL 2025-09 conditional novelty 4.0 of 10

    The ratio of agreement to disagreement between a small student model and an LLM correlates with the LLM's annotation accuracy across ten datasets and can heuristically select better models.

  11. Generative Models for Synthetic Data: Transforming Data Mining in the GenAI Era

    cs.LG 2025-08 unverdicted novelty 1.0 of 10

    A tutorial proposal outlining how generative models can synthesize data across modalities for data mining, with no new research results.

Pith tools