Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

This survey argues that evaluating LLM-generated data should shift from downstream task performance to intrinsic quality and trustworthiness metrics, organized under a unified cross-modal framework.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:11 UTC pith:MSDRLNLH

load-bearing objection A genuinely useful cross-modal metric catalog, but the gap analysis that drives the headline claim is not independently auditable — it needs a real methodology before the deficiency findings can be trusted. the 3 major comments →

arxiv 2601.17717 v3 pith:MSDRLNLH submitted 2026-01-25 cs.AI cs.LG

A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

Kaituo Zhang , Mingzhi Hu , Hoang Anh Duy Le , Fariha Kabir Torsha , Zhimeng Jiang , Minh Khai Bui , Chia-Yuan Chang , Yu-Neng Chuang
show 4 more authors
Zhen Xiong Ying Lin Guanchu Wang Na Zou
This is my paper
classification cs.AI cs.LG
keywords LLM-generated datasynthetic data evaluationdata quality metricstrustworthinessintrinsic evaluationcross-modal surveyevaluation gap analysisLLM Data Auditor
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes the LLM Data Auditor framework, which organizes the evaluation of synthetic data around two intrinsic pillars—quality (validity, fidelity, diversity, utility) and trustworthiness (safety, faithfulness, privacy, fairness, robustness, provenance)—rather than extrinsic downstream task performance. It applies this taxonomy uniformly across six data modalities: text, symbolic and logical reasoning, tabular, semi-structured (graph/JSON/log), vision–language, and agent data. By auditing the experimental evaluations of representative generation methods for each modality, the paper identifies systematic gaps: safety reporting is largely absent, fairness is mostly missing, and diversity, provenance, and privacy are under-measured. The authors conclude that the literature lacks a unified framework to audit synthetic data before it enters the training loop, and they offer concrete recommendations to make intrinsic evaluation a standard reporting component.

Core claim

The paper's central claim is that synthetic data from LLMs should be audited intrinsically—on its own properties—before being used for training or evaluation, and that a unified metric taxonomy can reveal where current practice falls short. Applying the LLM Data Auditor framework, the authors analyze representative generation methods across six modalities and find that validity and fidelity are relatively well covered, while safety, fairness, privacy, diversity, and provenance are frequently neglected. The discovered deficiencies are presented as empirical facts about the literature: for example, text generation methods rarely report standardized toxicity metrics, tabular methods largely omi

What carries the argument

The central object is the LLM Data Auditor framework itself: a two-pillar taxonomy of intrinsic evaluation metrics (quality and trustworthiness) with fine-grained subcategories, instantiated for each of six modalities. The framework is operationalized through tables (Tables 2–6) that rate representative generation methods with ✓/△/× marks indicating whether each dimension is explicitly evaluated, partially covered, or absent. These tables carry the argument: the claimed deficiencies in safety, fairness, privacy, diversity, provenance, and robustness are read directly off the rating patterns.

Load-bearing premise

The entire deficiency analysis rests on the manual ratings of a hand-picked set of 'representative' generation methods using ✓/△/× marks, with no disclosed selection criteria, rating rubric, or inter-annotator agreement; if those ratings are wrong or unrepresentative, the claimed gaps in safety, fairness, and privacy evaluation collapse.

What would settle it

A re-audit of the same representative methods (plus a broader random sample of papers from each modality) using a pre-registered, rubric-based coding scheme with multiple independent annotators and reported agreement—showing that safety, fairness, privacy, and provenance are in fact regularly evaluated—would directly refute the paper's central empirical claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the framework is adopted, future papers on LLM-based data generation would be expected to report intrinsic quality and trustworthiness metrics as standard practice, not just downstream task performance.
  • Safety evaluation for synthetic text would likely become a standard reporting component, using protocols like toxicity probability and expected maximum toxicity, enabling systematic comparison across methods.
  • Tabular generation studies would add fairness metrics (e.g., statistical parity difference, equalized odds) to the train-on-synthetic/test-on-real protocol, reducing the risk of deploying biased models trained on synthetic data.
  • Vision–language and video generation papers would be encouraged to include provenance metrics such as C2PA credential validation and watermark robustness, making synthetic content traceable.
  • A cross-modal consensus on metrics like validity, fidelity, and diversity would allow researchers to compare data quality across different data types within a single evaluation language.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The gap analysis likely overstates the absence of safety, fairness, and privacy evaluation because the paper's hand-picked method selection and ✓/△/× rubric are not externally validated; researchers who actually report such metrics may be underrepresented.
  • The framework could be operationalized as a living benchmark or auto-audit tool that scans generation papers for metric coverage, making the gap analysis reproducible and testable rather than a one-time manual rating.
  • If the identified gaps are real, they suggest that the field is currently training and evaluating models on synthetic data whose failure modes—toxicity, unfairness, privacy leakage, hallucination—are systematically undetected, which would amplify risks in real-world deployments.
  • The emphasis on intrinsic evaluation may encourage a shift in research incentives: instead of merely maximizing downstream benchmarks, generation pipelines would be optimized for a balanced profile of validity, fidelity, diversity, and trustworthiness, potentially changing how synthetic data is curated.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the LLM Data Auditor framework for evaluating LLM-generated synthetic data across six modalities: text, symbolic/logical reasoning, tabular, semi-structured (graph/JSON/log), vision-language, and agent data. It organizes evaluation into two pillars—quality (validity, fidelity, diversity, utility) and trustworthiness (safety, faithfulness, privacy, fairness, robustness, provenance)—and provides formula-level definitions for representative metrics, drawing on external benchmarks such as CoLA, RealToxicityPrompts, TabSyn, RARR, and MM-SafetyBench. It then audits a set of 'representative' generation methods for each modality in Tables 2–6 and reports substantial deficiencies, e.g., safety reporting is largely absent in text, fairness is mostly missing in tabular synthesis, and privacy is rare in semi-structured data. The paper concludes with recommendations for evaluation practice and with examples of how synthetic data are used for training, alignment, evaluation, and risk management.

Significance. If its gap analysis were fully auditable, this survey would be a valuable organizing reference: it spans six modalities, gives a broad metric taxonomy with concrete formulas and citations, and translates that taxonomy into practical recommendations. The metric restatements appear faithful to their sources, and the GitHub repository is a useful pointer for the community. The main significance is as a structured map of intrinsic evaluation methods and their current usage. That significance is currently limited by the fact that the paper's central empirical claim—that current evaluation practice is 'substantially deficient' in several trustworthiness dimensions—rests on manual ratings in Tables 2–6 that are not reproducible as presented.

major comments (3)
  1. [§2.4, Tables 2–6 (also §4.4, §5.4)] The paper's headline findings—e.g., 'Safety reporting is largely absent' (§2.4) and 'fairness evaluation is mostly missing' (§4.4)—are derived entirely from the ✓/△/× ratings in Tables 2–6. The paper gives no inclusion criteria for choosing the 'representative' methods, no operational rubric for distinguishing △ from ×, no inter-annotator agreement, and no released rating data. The table legend collapses 'not reported' and 'not applicable' into the same × symbol, making the deficiency counts ambiguous. Concretely, Table 5 lists PrivGraph, RedactOR, and Aghili even though these are not presented as representative generation methods in §5.1, which suggests post-hoc method selection. Please publish the rating rubric, per-method evidence, and justifications, or explicitly reframe the tables as illustrative examples rather than as a literature-level audit.
  2. [§1 vs §4.2] The paper repeatedly contrasts its approach with 'extrinsic evaluation, which relies on downstream task performance' and claims to focus on 'inherent properties of the data itself' (abstract, §1). However, the quality pillar includes Utility, and §4.2 defines Utility via the Train-on-Synthetic Test-on-Real protocol, reporting AUC and RMSE—standard downstream task metrics. This is an internal inconsistency in the framework's core distinction. Either Utility should be removed from the 'intrinsic' metric set, or the paper should clarify that the LLM Data Auditor is a data-side audit that may still include downstream performance as one component. As written, the abstract's central framing is contradicted by the taxonomy.
  3. [§2.3–§7.3] The claimed 'unified cross-modal coverage' is under-specified because the trustworthiness subdimensions are not applied uniformly across modalities. Text includes faithfulness and safety but not privacy, fairness, or robustness; tabular includes privacy and fairness but not safety or robustness; vision-language includes safety and provenance but not faithfulness, privacy, or fairness. The paper never states the criterion by which a dimension is included or excluded for a modality. This makes the framework appear assembled per-modality rather than derived from a shared definition, weakening the 'unified framework' contribution. Please add a modality × dimension applicability matrix with explicit rules (e.g., risk exposure, data type, downstream use).
minor comments (4)
  1. [§3.4] The sentence 'Collectively, thesegapssuggestthatevaluationsfocusedoncorrectness...' appears to be missing spaces; please fix typography.
  2. [§5.3.1] In the paragraph after DL2, 'we emphasize degree-based deviations provide useful although not sufficient auxiliary signals' has an awkward construction; consider 'provide useful, though not sufficient, auxiliary signals.'
  3. [Table 1] The note says that for modalities with multiple data types, metrics for the first data type are shown and the rest 'follow the same structure.' This is underspecified for readers; a pointer to the exact sections is helpful, but a complete mapping in the table would be clearer.
  4. [§2.2] The definition of GAR uses a threshold τ that is said to be 'pre-specified' but no guidance is given for choosing it. Consider adding a short sentence on sensitivity analysis or standard choices.

Circularity Check

0 steps flagged

No significant circularity: the survey's metric definitions are imported from external benchmarks and the gap analysis, while auditability-limited, is not self-referential by construction.

full rationale

This paper is a survey and taxonomy paper rather than a derivation chain with fitted parameters or predictive claims. The LLM Data Auditor framework organizes intrinsic evaluation metrics (validity, fidelity, diversity, utility, safety, faithfulness, privacy, fairness, robustness, provenance), and for each metric it cites or defines concrete protocols drawn from external, independently established work (e.g., CoLA for grammatical acceptability, Perspective API for toxicity, TabSyn protocols for tabular fidelity, RARR attribution metrics, MM-SafetyBench for multimodal safety). These definitions do not depend on the paper's own conclusions. The central evaluative contribution is the gap analysis in Tables 2–6, where representative methods are marked as explicitly evaluated, partially covered, or not reported/not applicable. One might criticize these tables for lacking published selection criteria, a rating rubric, inter-annotator agreement, or released rating data, and for collapsing 'not reported' with 'not applicable' into the same symbol. However, that is a transparency and correctness concern, not circularity: the gap findings are empirical claims about the surveyed papers' reporting practices, not consequences of the metric definitions by construction. No equation in the paper reduces a predicted quantity to an input fit, and no load-bearing self-citation chain is used to justify the framework. The paper does not manufacture a result from its own assumptions; it applies an externally grounded taxonomy to summarize existing practice. Therefore, the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 1 invented entities

The survey introduces no fitted parameters; protocol thresholds such as τ=0.5 are inherited from cited benchmarks. Its load-bearing premises are: intrinsic metrics are the right rubric; the six-modality split allows metric transfer; the manual gap ratings fairly represent the literature; judge-based proxies are acceptable; and inherited benchmark protocols properly operationalize the constructs. The only invented entity is the framework itself.

axioms (5)
  • domain assumption Intrinsic (quality + trustworthiness) evaluation is the correct primary lens; focus should shift from extrinsic downstream-task performance.
    The entire framework and gap analysis presuppose intrinsic metrics are the right rubric (Section 1, Figures 1–2). If extrinsic utility were primary, the 'deficiencies' identified would look very different.
  • domain assumption The same metric families transfer meaningfully across all six modalities (text, reasoning, tabular, graph/JSON/log, vision-language, agent data).
    The survey assumes a unified cross-modal structure (Sections 2.2–7.2); fidelity defined via text embeddings is applied by analogy to graphs and agent trajectories without evidence that the constructs are commensurable.
  • ad hoc to paper The 'representative' methods selected in Tables 2–6 fairly represent each modality, and the ✓/△/× ratings are correct.
    Selection criteria and the rating rubric are never stated; every deficiency claim (Sections 2.4–6.4) rests on these hand-assigned judgments. This is a methodological choice specific to this paper.
  • domain assumption LLM-as-judge scoring (e.g., GPT-4 judge agreement ρ_LLM-human) is an acceptable fidelity proxy when executable verification is unavailable.
    Section 3.2 treats benchmark-validated model judges as adequate stand-ins for human judgment; the survey's recommendations inherit any bias of judge-based evaluation.
  • domain assumption Inherited measurement protocols (CoLA classifier for GAR, SentenceTransformer for EDS, Perspective API for toxicity, Inception Score for diversity) are valid operationalizations of the named constructs.
    Sections 2.2, 2.3, and 6.2.1 adopt these protocols wholesale; any failure of these operationalizations transfers to the survey's metric recommendations.
invented entities (1)
  • LLM Data Auditor framework (quality/trustworthiness metric taxonomy) no independent evidence
    purpose: Unify intrinsic evaluation of LLM-generated data across six modalities and define the rubric used for the survey's gap analysis.
    A conceptual taxonomy rather than a physical entity; its only evidence is internal consistency and the prior validation of the cited metrics. It makes no falsifiable prediction outside the paper — adoption would be its validation.

pith-pipeline@v1.3.0-alltime-deepseek · 44106 in / 14932 out tokens · 163474 ms · 2026-08-03T08:11:02.204964+00:00 · methodology

0 comments
read the original abstract

Large Language Models (LLMs) have emerged as powerful tools for generating data across various modalities. By transforming data from a scarce resource into a controllable asset, LLMs mitigate the bottlenecks imposed by the acquisition costs of real-world data for model training, evaluation, and system iteration. However, ensuring the high quality of LLM-generated synthetic data remains a critical challenge. Existing research primarily focuses on generation methodologies, with limited direct attention to the quality of the resulting data. Furthermore, most studies are restricted to single modalities, lacking a unified perspective across different data types. To bridge this gap, we propose the \textbf{LLM Data Auditor framework}. In this framework, we first describe how LLMs are utilized to generate data across six distinct modalities. More importantly, we systematically categorize intrinsic metrics for evaluating synthetic data from two dimensions: quality and trustworthiness. This approach shifts the focus from extrinsic evaluation, which relies on downstream task performance, to the inherent properties of the data itself. Using this evaluation system, we analyze the experimental evaluations of representative generation methods for each modality and identify substantial deficiencies in current evaluation practices. Based on these findings, we offer concrete recommendations for the community to improve the evaluation of data generation. Finally, the framework outlines methodologies for the practical application of synthetic data across different modalities.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. LLM-Driven Performance-Space Augmentation for Meta-Learning-Based Algorithm Selection

    cs.LG 2026-05 unverdicted novelty 7.0

    LLM-generated synthetic datasets steered uniformly across a 2D performance space defined by two landmark algorithms improve meta-learner performance on algorithm selection for regression tasks.

Reference graph

Works this paper leans on

283 extracted references · 10 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    LLM -powered synthetic environments for self-driving scenarios

    Oluwanifemi Adebayo Moses Adekanye. LLM -powered synthetic environments for self-driving scenarios. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 23721--23723, 2024. doi:10.1609/aaai.v38i21.30540. URL https://ojs.aaai.org/index.php/AAAI/article/view/30540

    work page doi:10.1609/aaai.v38i21.30540 2024

  3. [3]

    Madisetti

    Harshvardhan Aditya, Siddansh Chawla, Gunika Dhingra, Parijat Rai, Saumil Sood, Tanmay Singh, Zeba Mohsin Wase, Arshdeep Bahga, and Vijay K. Madisetti. Evaluating privacy leakage and memorization attacks on large language models (llms) in generative ai applications. Journal of Software Engineering and Applications, 17 0 (5): 0 421--447, 2024. doi:10.4236/...

    arXiv 2024

  4. [4]

    Think inside the json: Reinforcement strategy for strict llm schema adherence, 2025 a

    Bhavik Agarwal, Ishan Joshi, and Viktoria Rojkova. Think inside the json: Reinforcement strategy for strict llm schema adherence, 2025 a . URL https://arxiv.org/abs/2502.14905

    Pith/arXiv arXiv 2025

  5. [5]

    L3m+p: Lifelong planning with large language models, 2025 b

    Krish Agarwal, Yuqian Jiang, Jiaheng Hu, Bo Liu, and Peter Stone. L3m+p: Lifelong planning with large language models, 2025 b . URL https://arxiv.org/abs/2508.01917

    Pith/arXiv arXiv 2025

  6. [6]

    Protecting privacy in software logs: What should be anonymized?, 2025

    Roozbeh Aghili, Heng Li, and Foutse Khomh. Protecting privacy in software logs: What should be anonymized?, 2025. URL https://arxiv.org/abs/2409.11313

    Pith/arXiv arXiv 2025

  7. [7]

    Opencodeinstruct: A large-scale instruction tuning dataset for code llms, 2025

    Wasi Uddin Ahmad, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Vahid Noroozi, Somshubra Majumdar, and Boris Ginsburg. Opencodeinstruct: A large-scale instruction tuning dataset for code llms, 2025. URL https://arxiv.org/abs/2504.04030

    Pith/arXiv arXiv 2025

  8. [8]

    Do as i can and not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kua...

    Pith/arXiv arXiv 2022

  9. [9]

    Synthetically generated reasoning dataset (gsm8k-inspired) with enhanced diversity using gretel navigator and meta-llama/meta-llama-3.1-405b, 9 2024

    Gretel AI. Synthetically generated reasoning dataset (gsm8k-inspired) with enhanced diversity using gretel navigator and meta-llama/meta-llama-3.1-405b, 9 2024. URL https://huggingface.co/datasets/gretelai/gretel-math-gsm8k-v1

    2024

  10. [10]

    Alaa, Boris Van Breugel, Evgeny S

    Ahmed M. Alaa, Boris Van Breugel, Evgeny S. Saveliev, and Mihaela van der Schaar. How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pp.\ 290--306. PMLR, 2022. URL htt...

    2022

  11. [11]

    A survey on data selection for language models, 2024

    Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, Colin Raffel, Shiyu Chang, Tatsunori Hashimoto, and William Yang Wang. A survey on data selection for language models, 2024. URL https://arxiv.org/abs/2402.16827

    Pith/arXiv arXiv 2024

  12. [12]

    Enhancing model safety through pretraining data filtering, 2025

    Anthropic . Enhancing model safety through pretraining data filtering, 2025. URL https://alignment.anthropic.com/2025/pretraining-data-filtering/

    2025

  13. [13]

    Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, ...

    Pith/arXiv arXiv 2022

  14. [14]

    Fairness and Machine Learning: Limitations and Opportunities

    Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning: Limitations and Opportunities. MIT Press, 2023. URL https://fairmlbook.org/

    2023

  15. [15]

    Barr, Joshua Quan, Eddie Guo, and Emre Sezgin

    Austin A. Barr, Joshua Quan, Eddie Guo, and Emre Sezgin. Large language models generating synthetic clinical datasets: A feasibility and comparative analysis with real-world perioperative data. Frontiers in Artificial Intelligence, 8: 0 1533508, 2025. doi:10.3389/frai.2025.1533508. URL https://www.frontiersin.org/articles/10.3389/frai.2025.1533508

    arXiv 2025

  16. [16]

    Guiding llms the right way: Fast, non-invasive constrained generation, 2024

    Luca Beurer-Kellner, Marc Fischer, and Martin Vechev. Guiding llms the right way: Fast, non-invasive constrained generation, 2024. URL https://arxiv.org/abs/2403.06988

    Pith/arXiv arXiv 2024

  17. [17]

    Exploring digital twin-based fault monitoring: Challenges and opportunities

    Jherson Bofill, Mideth Abisado, Jocelyn Villaverde, and Gabriel Avelino Sampedro. Exploring digital twin-based fault monitoring: Challenges and opportunities. Sensors, 23 0 (16), 2023. ISSN 1424-8220. doi:10.3390/s23167087. URL https://www.mdpi.com/1424-8220/23/16/7087

    work page doi:10.3390/s23167087 2023

  18. [18]

    Language models are realistic tabular data generators, 2023

    Vadim Borisov, Kathrin Seßler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Language models are realistic tabular data generators, 2023. URL https://arxiv.org/abs/2210.06280

    Pith/arXiv arXiv 2023

  19. [19]

    Kgvalidator: A framework for automatic validation of knowledge graph construction, 2024

    Jack Boylan, Shashank Mangla, Dominic Thorn, Demian Gholipour Ghalandari, Parsa Ghaffari, and Chris Hokamp. Kgvalidator: A framework for automatic validation of knowledge graph construction, 2024. URL https://arxiv.org/abs/2404.15923

    Pith/arXiv arXiv 2024

  20. [20]

    The JavaScript Object Notation (JSON) Data Interchange Format

    Tim Bray. The JavaScript Object Notation (JSON) Data Interchange Format . RFC 8259, December 2017. URL https://www.rfc-editor.org/info/rfc8259

    2017

  21. [21]

    Text2scenario: Text-driven scenario generation for autonomous driving test

    Xuan Cai, Xuesong Bai, Zhiyong Cui, Danmu Xie, Daocheng Fu, Haiyang Yu, and Yilong Ren. Text2scenario: Text-driven scenario generation for autonomous driving test. arXiv preprint arXiv:2503.02911, 2025. URL https://arxiv.org/abs/2503.02911

    Pith/arXiv arXiv 2025

  22. [22]

    Evaluation criteria for the leaderboard 2.0, 2025 a

    CARLA Autonomous Driving Leaderboard . Evaluation criteria for the leaderboard 2.0, 2025 a . URL https://leaderboard.carla.org/evaluation_v2_0/

    2025

  23. [23]

    Carla autonomous driving leaderboard (leaderboard table), 2025 b

    CARLA Autonomous Driving Leaderboard . Carla autonomous driving leaderboard (leaderboard table), 2025 b . URL https://leaderboard.carla.org/leaderboard/

    2025

  24. [24]

    Extracting training data from large language models

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert - Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, \" U lfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium , pp.\ 2633--2650, 2021. URL https://www.usenix.org/confer...

    2021

  25. [25]

    Turner, Eric Undersander, and Tsung-Yen Yang

    Matthew Chang, Gunjan Chhablani, Alexander Clegg, Mikael Dallaire Cote, Ruta Desai, Michal Hlavac, Vladimir Karashchuk, Jacob Krantz, Roozbeh Mottaghi, Priyam Parashar, Siddharth Patki, Ishita Prasad, Xavier Puig, Akshara Rai, Ram Ramrakhya, Daniel Tran, Joanne Truong, John M. Turner, Eric Undersander, and Tsung-Yen Yang. Partnr: A benchmark for planning ...

    Pith/arXiv arXiv 2024

  26. [26]

    Code alpaca: An instruction-following llama model for code generation, 2023

    Sahil Chaudhary. Code alpaca: An instruction-following llama model for code generation, 2023. URL https://github.com/sahil280114/codealpaca

    2023

  27. [27]

    Understanding multi-turn toxic behaviors in open-domain chatbots

    Bocheng Chen, Guangjing Wang, Hanqing Guo, Yuanda Wang, and Qiben Yan. Understanding multi-turn toxic behaviors in open-domain chatbots. In Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses, RAID 2023, pp.\ 282–296. ACM, October 2023 a . doi:10.1145/3607199.3607237. URL http://dx.doi.org/10.1145/3607199.3607237

    arXiv 2023

  28. [28]

    Integrating large language model and digital twins in the context of industry 5.0: Framework, challenges and opportunities

    Chong Chen, Kuanhong Zhao, Jiewu Leng, Chao Liu, Junming Fan, and Pai Zheng. Integrating large language model and digital twins in the context of industry 5.0: Framework, challenges and opportunities. Robotics and Computer-Integrated Manufacturing, 94: 0 102982, 2025 a . ISSN 0736-5845. doi:https://doi.org/10.1016/j.rcim.2025.102982. URL https://www.scien...

    arXiv 2025

  29. [29]

    Sharegpt4v: Improving large multi-modal models with better captions, 2023 b

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions, 2023 b . URL https://arxiv.org/abs/2311.12793

    Pith/arXiv arXiv 2023

  30. [30]

    Sharegpt4video: Improving video understanding and generation with better captions, 2024 a

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, Li Yuan, Yu Qiao, Dahua Lin, Feng Zhao, and Jiaqi Wang. Sharegpt4video: Improving video understanding and generation with better captions, 2024 a . URL https://arxiv.org/abs/2406.04325

    Pith/arXiv arXiv 2024

  31. [31]

    Comm: A coherent interleaved image-text dataset for multimodal understanding and generation

    Wei Chen, Lin Li, Yongqi Yang, Bin Wen, Fan Yang, Tingting Gao, Yu Wu, and Long Chen. Comm: A coherent interleaved image-text dataset for multimodal understanding and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025 b . URL https://openaccess.thecvf.com/content/CVPR2025/papers/Chen_CoMM_A_Cohere...

    2025

  32. [32]

    Self-play fine-tuning converts weak language models to strong language models, 2024 b

    Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models, 2024 b . URL https://arxiv.org/abs/2401.01335

    Pith/arXiv arXiv 2024

  33. [33]

    Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation, 2024

    Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation, 2024. URL https://arxiv.org/abs/2407.06135

    Pith/arXiv arXiv 2024

  34. [34]

    Content credentials: C2PA technical specification, version 2.2

    Coalition for Content Provenance and Authenticity . Content credentials: C2PA technical specification, version 2.2. Specification, May 2025. URL https://c2pa.org/specifications/specifications/2.2/specs/C2PA_Specification.html

    2025

  35. [35]

    Training verifiers to solve math word problems, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL https://arxiv.org/abs/2110.14168

    Pith/arXiv arXiv 2021

  36. [36]

    Embodiment Collaboration, Abby O'Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Andrey Kolobov, Anikait Singh, Animesh Garg, Aniruddha...

    Pith/arXiv arXiv 2025

  37. [37]

    Ultrafeedback: boosting language models with scaled ai feedback

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: boosting language models with scaled ai feedback. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024. URL https://dl.acm.org/doi/abs/10.5555/3692070.3692454

    arXiv 2024

  38. [38]

    Automated generation of test scenarios for autonomous driving using llms

    Aaron Agyapong Danso and Ulrich Büker. Automated generation of test scenarios for autonomous driving using llms. Electronics, 14 0 (16), 2025. ISSN 2079-9292. doi:10.3390/electronics14163177. URL https://www.mdpi.com/2079-9292/14/16/3177

    work page doi:10.3390/electronics14163177 2025

  39. [39]

    Detection: Single table, 2024

    DataCebo . Detection: Single table, 2024. URL https://docs.sdv.dev/sdmetrics/data-metrics/metrics-in-beta/detection-single-table

    2024

  40. [40]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    DeepSeek-AI . Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. doi:10.48550/arXiv.2501.12948. URL https://arxiv.org/abs/2501.12948

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.12948 2025

  41. [41]

    Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian L...

    Pith/arXiv arXiv 2024

  42. [42]

    A systematic review of a digital twin city: A new pattern of urban governance toward smart cities

    Tianhu Deng, Keren Zhang, and Zuo-Jun (Max) Shen. A systematic review of a digital twin city: A new pattern of urban governance toward smart cities. Journal of Management Science and Engineering, 6 0 (2): 0 125--134, 2021. ISSN 2096-2320. doi:https://doi.org/10.1016/j.jmse.2021.03.003. URL https://www.sciencedirect.com/science/article/pii/S2096232021000238

    doi:10.1016/j.jmse.2021.03.003 2021

  43. [43]

    Implicit chain of thought reasoning via knowledge distillation, 2023

    Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. Implicit chain of thought reasoning via knowledge distillation, 2023. URL https://arxiv.org/abs/2311.01460

    Pith/arXiv arXiv 2023

  44. [44]

    Chain-of-verification reduces hallucination in large language models

    Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. Chain-of-verification reduces hallucination in large language models. In Findings of the Association for Computational Linguistics: ACL 2024 , pp.\ 3563--3578, Bangkok, Thailand, 2024. Association for Computational Linguistics. URL https://aclanth...

    2024

  45. [45]

    Synthdoc: Bilingual documents synthesis for visual document understanding, 2024

    Chuanghao Ding, Xuejing Liu, Wei Tang, Juan Li, Xiaoliang Wang, Rui Zhao, Cam-Tu Nguyen, and Fei Tan. Synthdoc: Bilingual documents synthesis for visual document understanding, 2024. URL https://arxiv.org/abs/2408.14764

    Pith/arXiv arXiv 2024

  46. [46]

    A tale of tails: Model collapse as a change of scaling laws

    Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe. A tale of tails: Model collapse as a change of scaling laws. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp.\ 11165--11197. PMLR, 2024. URL https://proceedings.mlr.press/v235/dohmatob24b.html

    2024

  47. [47]

    Carla: An open urban driving simulator, 2017

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator, 2017. URL https://arxiv.org/abs/1711.03938

    Pith/arXiv arXiv 2017

  48. [48]

    outlines-core , 2025

    dottxt-ai . outlines-core , 2025. URL https://github.com/dottxt-ai/outlines-core

    2025

  49. [49]

    Graphmaster: Automated graph synthesis via llm agents in data-limited environments, 2025

    Enjun Du, Xunkai Li, Tian Jin, Zhihan Zhang, Rong-Hua Li, and Guoren Wang. Graphmaster: Automated graph synthesis via llm agents in data-limited environments, 2025. URL https://arxiv.org/abs/2504.00711

    Pith/arXiv arXiv 2025

  50. [50]

    Blenderllm: Training large language models for computer-aided design with self-improvement, 2024

    Yuhao Du, Shunian Chen, Wenbo Zan, Peizhao Li, Mingxuan Wang, Dingjie Song, Bo Li, Yan Hu, and Benyou Wang. Blenderllm: Training large language models for computer-aided design with self-improvement, 2024. URL https://arxiv.org/abs/2412.14203

    Pith/arXiv arXiv 2024

  51. [51]

    Duchi, Michael I

    John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. Local privacy, data processing inequalities, and statistical minimax rates, 2014. URL https://arxiv.org/abs/1302.3203

    Pith/arXiv arXiv 2014

  52. [52]

    The algorithmic foundations of differential privacy

    Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., 9 0 (3–4): 0 211–407, August 2014. ISSN 1551-305X. doi:10.1561/0400000042. URL https://doi.org/10.1561/0400000042

    doi:10.1561/0400000042 2014

  53. [53]

    Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, and Philip S. Yu. Tabgen-icl: Residual-aware in-context example selection for tabular data generation, 2025. URL https://arxiv.org/abs/2502.16414

    Pith/arXiv arXiv 2025

  54. [54]

    Large language models(llms) on tabular data: Prediction, generation, and understanding -- a survey, 2024

    Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Scott Nickleach, Diego Socolinsky, Srinivasan Sengamedu, and Christos Faloutsos. Large language models(llms) on tabular data: Prediction, generation, and understanding -- a survey, 2024. URL https://arxiv.org/abs/2402.17944

    Pith/arXiv arXiv 2024

  55. [55]

    Certifying and removing disparate impact, 2015

    Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and removing disparate impact, 2015. URL https://arxiv.org/abs/1412.3756

    Pith/arXiv arXiv 2015

  56. [56]

    Ontology-grounded automatic knowledge graph construction by llm under wikidata schema

    Xiaohan Feng, Xixin Wu, and Helen Meng. Ontology-grounded automatic knowledge graph construction by llm under wikidata schema. In Proceedings of the HI-AI@KDD Workshop, pp.\ 117--135. CEUR Workshop Proceedings, 2024. URL https://arxiv.org/abs/2412.20942

    Pith/arXiv arXiv 2024

  57. [57]

    A high-quality dataset and reliable evaluation for interleaved image-text generation, 2025 a

    Yukang Feng, Jianwen Sun, Chuanhao Li, Zizhen Li, Jiaxin Ai, Fanrui Zhang, Yifan Chang, Sizhuo Zhou, Shenglin Zhang, Yu Dai, and Kaipeng Zhang. A high-quality dataset and reliable evaluation for interleaved image-text generation, 2025 a . URL https://arxiv.org/abs/2506.09427

    arXiv 2025

  58. [58]

    Democratizing large language model-based graph data augmentation via latent knowledge graphs, 2025 b

    Yushi Feng, Tsai Hor Chan, Guosheng Yin, and Lequan Yu. Democratizing large language model-based graph data augmentation via latent knowledge graphs, 2025 b . URL https://arxiv.org/abs/2502.13555

    Pith/arXiv arXiv 2025

  59. [59]

    Anwar Hossain, Chunsheng Yang, and Abdulmotaleb El Saddik

    Rahatara Ferdousi, M. Anwar Hossain, Chunsheng Yang, and Abdulmotaleb El Saddik. Defecttwin: When llm meets digital twin for railway defect inspection, 2024. URL https://arxiv.org/abs/2409.06725

    Pith/arXiv arXiv 2024

  60. [60]

    Angelopoulos, Jiantao Jiao, Banghua Zhu, Joseph E

    Evan Frick, Tianle Li, Connor Chen, Wei-Lin Chiang, Anastasios N. Angelopoulos, Jiantao Jiao, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. How to evaluate reward models for rlhf, 2024. URL https://arxiv.org/abs/2410.14872

    Pith/arXiv arXiv 2024

  61. [61]

    Ocrbench v2: An improved benchmark for evaluating large multimodal models on visual text localization and reasoning, 2025

    Ling Fu, Zhebin Kuang, Jiajun Song, Mingxin Huang, Biao Yang, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, Zhang Li, Guozhi Tang, Bin Shan, Chunhui Lin, Qi Liu, Binghong Wu, Hao Feng, Hao Liu, Can Huang, Jingqun Tang, Wei Chen, Lianwen Jin, Yuliang Liu, and Xiang Bai. Ocrbench v2: An improved benchmark for evaluating large multimodal models on vis...

    Pith/arXiv arXiv 2025

  62. [62]

    Tall: Temporal activity localization via language query, 2017

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query, 2017. URL https://arxiv.org/abs/1705.02101

    Pith/arXiv arXiv 2017

  63. [63]

    RARR : Researching and revising what language models say, using language models

    Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. RARR : Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\...

    doi:10.18653/v1/2023.acl-long.910 2023

  64. [64]

    Pal: Program-aided language models, 2023 b

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models, 2023 b . URL https://arxiv.org/abs/2211.10435

    Pith/arXiv arXiv 2023

  65. [65]

    From words to collisions: LLM -guided evaluation and adversarial generation of safety-critical driving scenarios

    Yuan Gao, Mattia Piccinini, Korbinian M \" o ller, Amr Alanwar, and Johannes Betz. From words to collisions: LLM -guided evaluation and adversarial generation of safety-critical driving scenarios. In IEEE Intelligent Transportation Systems Conference (ITSC), 2025. doi:10.48550/arXiv.2502.02145. URL https://arxiv.org/abs/2502.02145

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.02145 2025

  66. [66]

    LM Format Enforcer , 2025

    Noam Gat. LM Format Enforcer , 2025. URL https://github.com/noamgat/lm-format-enforcer

    2025

  67. [67]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. Realtoxicityprompts: Evaluating neural toxic degeneration in language models, 2020. URL https://arxiv.org/abs/2009.11462

    Pith/arXiv arXiv 2020

  68. [68]

    Jsonschemabench: A rigorous benchmark of structured outputs for language models, 2025

    Saibo Geng, Hudson Cooper, Michał Moskal, Samuel Jenkins, Julian Berman, Nathan Ranchin, Robert West, Eric Horvitz, and Harsha Nori. Jsonschemabench: A rigorous benchmark of structured outputs for language models, 2025. URL https://arxiv.org/abs/2501.10868

    Pith/arXiv arXiv 2025

  69. [69]

    Llm trainer: Automated robotic data generating via demonstration augmentation using llms, 2025

    Abraham George and Amir Barati Farimani. Llm trainer: Automated robotic data generating via demonstration augmentation using llms, 2025. URL https://arxiv.org/abs/2509.20070

    Pith/arXiv arXiv 2025

  70. [70]

    Ai-enhanced creation of digital twins from iphone lidar for immersive xr experiences in nvidia omniverse

    Kamran Gholizadeh HamlAbadi, Monica Vahdati, Haiwei Dong, and Abdulmotaleb El Saddik. Ai-enhanced creation of digital twins from iphone lidar for immersive xr experiences in nvidia omniverse. In Proceedings of the International Workshop on Intelligent Immersification in the Metaverse: AI-Driven Immersive Multimedia, I2M-MM '25, pp.\ 25–33, New York, NY, U...

    arXiv 2025

  71. [71]

    Synthid, 2025

    Google DeepMind . Synthid, 2025. URL https://deepmind.google/models/synthid

    2025

  72. [72]

    Sven Gowal, Rudy Bunel, Florian Stimberg, David Stutz, Guillermo Ortiz-Jimenez, Christina Kouridi, Mel Vecerik, Jamie Hayes, Sylvestre-Alvise Rebuffi, Paul Bernard, Chris Gamble, Miklós Z. Horváth, Fabian Kaczmarczyck, Alex Kaskasoli, Aleksandar Petrov, Ilia Shumailov, Meghana Thotakuri, Olivia Wiles, Jessica Yung, Zahra Ahmed, Victor Martin, Simon Rosen,...

    arXiv 2025

  73. [73]

    Mandeep Goyal and Qusay H. Mahmoud. A systematic review of synthetic data generation techniques using generative AI . Electronics, 13 0 (17): 0 3509, 2024. doi:10.3390/electronics13173509. URL https://www.mdpi.com/2079-9292/13/17/3509

    doi:10.3390/electronics13173509 2024

  74. [74]

    A survey on llm-as-a-judge, 2025

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on llm-as-a-judge, 2025. URL https://arxiv.org/abs/2411.15594

    Pith/arXiv arXiv 2025

  75. [75]

    Kolkin, Yuchen Liu, Richard Zhang, and Tobias Hinz

    Cusuh Ham, Matthew Fisher, James Hays, Nicholas I. Kolkin, Yuchen Liu, Richard Zhang, and Tobias Hinz. Personalized residuals for concept-driven text-to-image generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 8186--8195. IEEE, 2024. doi:10.1109/CVPR52733.2024.00782. URL https://doi.org/10.1109/CVPR52733.2024.00782

    arXiv 2024

  76. [76]

    a m \"a l \

    Perttu H \"a m \"a l \"a inen, Mikke Tavast, and Anton Kunnari. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). ACM, 2023. doi:10.1145/3544548.3580688. URL https://dl.acm.org/doi/10.1145/3544548.3580688

    arXiv 2023

  77. [77]

    Equality of opportunity in supervised learning, 2016

    Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning, 2016. URL https://arxiv.org/abs/1610.02413

    Pith/arXiv arXiv 2016

  78. [78]

    Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. Drain: An online log parsing approach with fixed depth tree. In 2017 IEEE International Conference on Web Services (ICWS), pp.\ 33--40, 2017. doi:10.1109/ICWS.2017.13. URL https://ieeexplore.ieee.org/document/8029742

    arXiv 2017

  79. [79]

    Clipscore: A reference-free evaluation metric for image captioning, 2022

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning, 2022. URL https://arxiv.org/abs/2104.08718

    Pith/arXiv arXiv 2022

  80. [80]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018. URL https://arxiv.org/abs/1706.08500

    Pith/arXiv arXiv 2018

Showing first 80 references.