Pith. sign in

REVIEW 4 major objections 6 minor 300 references

A Survey of LLM $\times$ DATA

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey claims the first comprehensive, lifecycle-based map of the two-way relationship between LLMs and data management.

desk verdict A useful but under-documented survey map; the IaaS lens is a framing device rather than a validated instrument, yet the paper deserves serious referee time. read the letter →

arxiv 2505.18458 v3 pith:T6QNMILN submitted 2025-05-24 cs.DB cs.AIcs.CLcs.IRcs.LG

Xuanhe Zhou , Junxuan He , Wei Zhou , Haodong Chen , Zirui Tang , Haoyu Zhao , Xin Tong , Guoliang Li
show 9 more authors
Youmin Chen Jun Zhou Zhaojun Sun Binyuan Hui Shuo Wang Conghui He Zhiyuan Liu Jingren Zhou Fan Wu
This is my paper
classification cs.DBcs.AIcs.CLcs.IRcs.LG
keywords largelanguagemodelsdatamanagementdata-centricAIqualityprocessingpipelinesretrieval-augmentedgenerationLLMagentsIaaSconcept
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that LLMs and data management are one bidirectionally connected field and to give that field a shared map. It organizes more than 400 papers into two directions: DATA4LLM, where data processing, storage, and serving supply models across pre-training, fine-tuning, reinforcement learning, RAG, agents, and evaluation; and LLM4DATA, where LLMs serve as general-purpose engines for data cleaning, analysis, and system optimization. To make data quality discussable, the authors introduce the 'IaaS' lens, which says good LLM data should be inclusive, abundant, articulated, and sanitized. A sympathetic reader would care because the map turns scattered techniques into a common vocabulary and exposes which stage-technique combinations are still empty.

What carries the argument

The carrying mechanism is the IaaS taxonomy, a four-part definition of what makes LLM data good: inclusiveness (broad, diverse coverage across domains, tasks, sources, languages, styles, and modalities), abundance (sufficient volume with balanced composition), articulation (well-formatted, clean, instructive, step-by-step data), and sanitization (privacy-compliant, toxicity-free, ethically consistent, risk-mitigated content). Around this lens the survey builds a lifecycle-based taxonomy that maps data processing, storage, and serving techniques onto LLM stages, from pre-training through evaluation, RAG, and agents, in a stage-task table, and pairs it with an LLM4DATA taxonomy for data manipulation, analysis, and system optimization. The taxonomy does the argumentative work: it turns scattered papers into comparable cells, exposes where techniques exist or are missing, and gives the field a shared reference structure.

What would settle it

Take 100 recent papers on LLM data management from an independent, broad literature search and try to place each one into the survey's stage-task and direction taxonomies; if more than about 10 percent fall outside the named categories or are described in ways their own abstracts contradict, the survey's comprehensiveness and accuracy claims would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the intersection of LLMs and data management is genuinely bidirectional and should be studied as one field, not two. On the DATA4LLM side, the paper identifies three families of techniques—data processing (acquisition, deduplication, filtering, selection, mixing, synthesis, and end-to-end pipelines), data storage (formats, distribution, organization, movement, fault tolerance, and KV caches), and data serving (shuffling, compression, packing, and provenance)—and shows how each applies differently across pre-training, continual pre-training, SFT, RL, RAG, agents, and evaluation. On the LLM4DATA side, it argues that LLMs are becoming general-purpose engines for data manipulation, analysis over structured, semi-structured, and unstructured data, and system optimization such as configuration tuning, query rewriting, and anomaly diagnosis. The authors position the survey as the first to cover this full lifecycle, and they anchor the data-quality discussion in the IaaS concept: good LLM data should be inclusive, abundant, articulated, and sanitized.

Load-bearing premise

The load-bearing premise is that the 400-plus papers examined, selected without a documented protocol, accurately represent the LLM-data literature and are each characterized correctly enough to support the taxonomy.

Editorial extensions

If this is right

  • Dataset quality can now be assessed along four named axes, so a data pipeline can be audited for what it lacks rather than only for what it filters.
  • Every LLM stage has its own data profile; what works for pre-training is not what works for SFT, RAG, or agents, and the survey makes those differences explicit.
  • Data management should be treated as a first-class component of LLM development, with storage, movement, and serving as important as model architecture.
  • LLMs can take over classical data tasks such as cleaning, schema matching, and query tuning, shifting the bottleneck from handcrafted rules to prompt design and retrieval-augmented reasoning.
  • The survey's stage-task table marks combinations with no reported work, giving researchers a direct list of open technique-stage gaps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One natural next step is to turn the IaaS dimensions into a quantitative scoring rubric and test whether datasets that score higher on inclusiveness, abundance, articulation, and sanitization consistently produce better downstream model performance.
  • The same taxonomy could be reused as a living map: with explicit inclusion criteria and versioned updates, later surveys could track how quickly the empty cells in the stage-task table fill in.
  • If LLMs really are general-purpose data engines, then data-management products may converge on natural-language interfaces where users describe a cleaning or analysis task and the system composes the operations, an outcome the survey's LLM4DATA section points toward but does not itself predict.
  • Because the survey's literature selection is not documented, an immediate extension is to test whether the taxonomy is stable under a different, independently chosen corpus of LLM-data papers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This survey reviews the bidirectional relationship between large language models and data management, organizing the literature into DATA4LLM (data processing, storage, and serving across pre-training, continual pre-training, SFT, RL, RAG, agents, and evaluation) and LLM4DATA (data manipulation, data analysis, and system optimization). The authors introduce an "IaaS" concept with four quality dimensions (inclusiveness, abundance, articulation, sanitization), claim to be the first lifecycle-based bidirectional survey, and base the taxonomy on an investigation of over 400 papers. The paper covers a wide range of techniques and systems, with a companion GitHub repository of cited works.

Significance. If the IaaS lens and the claimed comprehensiveness hold, this survey would provide a useful shared vocabulary and map for the rapidly growing LLM-data intersection, spanning both training-side data management and LLM-driven data management. The breadth is substantial, and the companion GitHub repository is a valuable community resource. However, the paper's central contributions are empirical and organizational rather than formal: there are no machine-checked proofs or parameter-free derivations, and the value rests on whether the taxonomy is a genuine, non-redundant decomposition of the literature and whether the 400+ paper sample is representative. These conditions are asserted rather than demonstrated, so the significance is conditional on methodological support that the manuscript does not currently provide.

major comments (4)
  1. [Section 2.1] The derivation of the IaaS concept is not supported. The text says "Based on our investigation of over 400 papers, we introduce the IaaS concept," but it does not describe how the four dimensions were selected, what the candidate space was, how overlap among dimensions was resolved (e.g., articulation vs. filtering, abundance vs. data mixing), or how exhaustiveness was assessed. Since Figures 1 and 4 and Tables 1-7 are organized around IaaS, this is load-bearing. The authors should add a methodology paragraph describing the coding procedure, or at minimum a falsifiable consistency test, such as reporting inter-annotator agreement on a random sample of cited papers classified by the four dimensions.
  2. [Section 1.3 and Section 2.1] The comprehensiveness claim is not verifiable from the manuscript. The statement "over 400 papers" is supported only by a GitHub link; there is no search strategy, inclusion/exclusion criteria, deduplication procedure, or validation against a second literature sample. Because the survey distinguishes itself from prior surveys on the basis of comprehensiveness and lifecycle coverage, this is a load-bearing methodological gap. I recommend adding a short survey-methodology subsection or, failing that, explicitly tempering the "first comprehensive" claim to "covering the selected 400+ papers."
  3. [Table 1 and Section 2.5.1] There is an internal inconsistency in the lifecycle taxonomy. Table 1 marks RAG-stage filtering as "×" (no relevant work reported), yet Section 2.5.1 explicitly discusses RAG knowledge filtering and re-ranking with citations [280], [114], [87], [128], [12], [318], [47]. This contradiction undermines the reliability of the table as a summary of the field. The table should either correct the RAG filtering entry or explain what kind of filtering "×" refers to, and it should include a legend distinguishing "×" from "N/A."
  4. [Section 2.3.7] The subsection numbering is erroneous: the text labels pipeline subsections as 2.2.7.1, 2.2.7.2, and 2.2.7.3 when they appear under Section 2.3.7. This is an editorial defect that makes the internal references in the introduction (e.g., Section 2.3.7) and the table of contents inconsistent, and it should be fixed before publication.
minor comments (6)
  1. [Section 2.3.5] "Slimpajama" should be "SlimPajama" in the description of [347].
  2. [Section 2.3.4] In the description of AutoDS, the second prompt question is rendered as "It it educationally valuable"; this should be "Is it educationally valuable."
  3. [Section 2.3.7.1] The word "strctured" should be "structured" in the description of the data processing framework from [368].
  4. [Section 2.4] The opening sentence says the storage techniques are "categorized accroding to the tasks they address"; "accroding" should be "according."
  5. [Section 2.4.4] The sentence "if a data isn't cached locally" is ungrammatical; it should be "if data is not cached locally."
  6. [Section 2.2] The statement that RAG data "must be strictly reviewed to ensure authenticity and validity" is not connected to a specific processing step or table entry; consider adding a cross-reference to Section 2.5.1 or clarifying how authenticity is achieved in the RAG data pipeline.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey organizes literature via an asserted taxonomy (IaaS) and claims scope/novelty, but nothing in the paper reduces by construction to its own inputs.

full rationale

This is a survey, so the circularity patterns that apply to derivational papers (fitted parameters renamed as predictions, uniqueness theorems, ansatz smuggling) have no purchase. The paper's central constructs—the IaaS data-quality lens (Section 2.1) and the DATA4LLM/LLM4DATA lifecycle taxonomy (Sections 2-3)—are organizational: they classify the surveyed literature rather than deriving quantitative claims from inputs. Section 2.1 states 'Based on our investigation of over 400 papers, we introduce the IaaS concept for evaluating the quality of LLM datasets'; this is a literature-grounded taxonomy presented as an evaluative lens, and no later section uses IaaS to prove a result by definition. The survey's strongest claim (Section 1.3, 'Different from existing LLM and data management surveys, our survey offers a comprehensive and detailed overview') is a scope/novelty assertion, not a derivation. Self-citations such as Data-Juicer [90] and Data-Juicer Sandbox [91] (works from the authors' own institutions) appear as descriptive entries among other frameworks; nothing load-bearing reduces to them. The genuinely weak step is empirical, not circular: the 'over 400 papers' corpus is asserted without a documented search/inclusion protocol, so representativeness and comprehensiveness are unverified. That is a validation/transparency risk, not a by-construction equivalence. No circular step can be exhibited, so the score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

As a survey, the paper introduces no free parameters or invented physical or mathematical entities. Its claims rest on the accuracy of the cited literature and on the usefulness of its own IaaS taxonomy.

assumptions (2)
  • domain assumption The 400+ cited works accurately support the survey's characterizations.
    The survey's content is a synthesis of cited papers; any mischaracterization would break the accuracy of the taxonomy and the claimed comprehensiveness.
  • ad hoc to paper The proposed IaaS quality dimensions are a meaningful and non-redundant decomposition of LLM dataset quality.
    The IaaS acronym is introduced in Section 2.1 as the paper's own framing, and it is not derived from an external theory or benchmark. Its usefulness is assumed rather than demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey of LLM $\times$ DATA." pith.science (2026). https://pith.science/paper/T6QNMILN

@misc{pith2026250518458,
  author       = {Pith},
  title        = {Pith review of: A Survey of LLM $\times$ DATA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T6QNMILN}},
  note         = {Machine review of arXiv:2505.18458}
}
read the original abstract

The integration of large language model (LLM) and data management (DATA) is rapidly redefining both domains. In this survey, we comprehensively review the bidirectional relationships. On the one hand, DATA4LLM, spanning large-scale data processing, storage, and serving, feeds LLMs with high quality, diversity, and timeliness of data required for stages like pre-training, post-training, retrieval-augmented generation, and agentic workflows: (i) Data processing for LLMs includes scalable acquisition, deduplication, filtering, selection, domain mixing, and synthetic augmentation; (ii) Data Storage for LLMs focuses on efficient data and model formats, distributed and heterogeneous storage hierarchies, KV-cache management, and fault-tolerant checkpointing; (iii) Data serving for LLMs tackles challenges in RAG (e.g., knowledge post-processing), LLM inference (e.g., prompt compression, data provenance), and training strategies (e.g., data packing and shuffling). On the other hand, in LLM4DATA, LLMs are emerging as general-purpose engines for data management. We review recent advances in (i) data manipulation, including automatic data cleaning, integration, discovery; (ii) data analysis, covering reasoning over structured, semi-structured, and unstructured data, and (iii) system optimization (e.g., configuration tuning, query rewriting, anomaly diagnosis), powered by LLM techniques like retrieval-augmented prompting, task-specialized fine-tuning, and multi-agent collaboration.

Figures

Figures reproduced from arXiv: 2505.18458 by the authors.

Figure 1
Figure 1. Overview of LLM × DATA (with “IaaS” Concept). 1 INTRODUCTION L arge language models (LLMs1 ) have made remarkable progress in both general domain applications (e.g., open￾domain question answering [332], cross-modal video summa￾rization [175], general-purpose code generation [191]) and specific domain applications (e.g., biomedical literature anal- • ¶ Co-first authors with equal contributions. 1. We use LLMs to ref… view at source ↗
Figure 2
Figure 2. Example Data Characteristics across LLM Stages - (a) Pretraining data [109], [224], (b) Continual pre-training [111], (c) SFT [447], (d) Reinforcement learning [429], [162], [253], (e) RAG [415], (f) Agent [396], [351]. cluding LLM-generated and hybrid retrieval strategies [369]; (3) Enhancing optimization performance through task￾specific training [53], [196], [438]. • Anomaly Diagnosis. This task involves identify… view at source ↗
Figure 3
Figure 3. Example LLM Data Distributions - (a) General Domain (SFT)[110], (b) General Domain (Eval) [244], (c) Law (SFT)[447], (d) Law (Eval)[115], (e) Code (SFT) [294], (f) Code (Eval)[208]. on tasks pertinent to these fields. For example, DISC-Law￾SFT [447] is a legal SFT dataset containing 295k data en￾tries from various legal scenarios, such as legal information extraction (32k), legal judgment prediction (16k), legal eve… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Overview of DATA4LLM Techniques. million free ebooks in Open Library [28], and film-aigned book descriptions in BookCorpus [497]). • Code repositories (e.g., GitHub [14], GitLab [20], Bit￾bucket [7]) offer abundant programming data that can fa￾cilitate code search and …
Figure 5
Figure 5. Figure 5: Example Data Filtering Workflows [238], [45], [264]. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Typical data processing pipelines for LLMs. on paragraph, line, and gram level [328]. Besides, it con￾ducts fuzzy deduplication using individual MinHash dedu￾plication for each CommonCrawl snapshot, as this approach matches RefinedWeb’s performance, whereas global dedu…
Figure 7
Figure 7. Figure 7: The storage architecture of 3FS [15]. 2.4.2 Data Distribution With the development of LLMs, the scale of LLM training datasets and the number of parameters of LLMs themselves are growing rapidly (e.g., 9.5 PB data form Common Crawl [183], DeepSeek-R1 [162] has 617B par…
Figure 8
Figure 8. Figure 8: Overview of LLM4DATA Techniques. Traditional data cleaning methods depend on rigid rules and constraints (e.g., zip code validation), demanding sub￾stantial manual effort and domain expertise (e.g., schema knowledge in financial data) [237], [432]. Additionally, they o…
Figure 9
Figure 9. Figure 9: Overview of LLM for Data Analysis. 3.2.1.1 Relational Data Analysis LLM for Natural Language Interfaces. Basic analysis jobs for relational data are typically characterized by well￾defined operations. These include basic calculations (e.g., summation, averaging, counti…
Figure 10
Figure 10. Figure 10: General Workflows - (a) Multi-Step Relational Data QA. (b) End-to-End Relational Data QA. emphasize enhancing relational data analysis through gener￾ating Python code (e.g., Pandas, NumPy), which includes a vast number of library APIs characterized by high variability…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

300 extracted references · 10 canonical work pages

  1. [280]

    Y. Ma, Y. Cao, Y. Hong, and A. Sun. Large language model is not a good few-shot information extractor, but a good reranker for hard samples! InFindings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics, 2023

    2023

  2. [114]

    J. Cui, Z. Li, Y. Yan, B. Chen, and L. Yuan. Chatlaw: Open- source legal large language model with integrated external knowledge bases.CoRR, abs/2306.16092, 2023

    arXiv 2023

  3. [87]

    Chang, Z

    C.-Y. Chang, Z. Jiang, V. Rakesh, M. Pan, et al. Main-rag: Multi-agent filtering retrieval-augmented generation, 2024

    2024

  4. [128]

    ARAGOG: Advanced RAG Output Grading

    M. Eibich, S. Nagpal, and A. Fred-Ojala. Aragog: Advanced rag output grading.arXiv preprint arXiv:2404.01037, 2024

    work page Pith review arXiv 2024

  5. [12]

    https://docs.cohere.com

  6. [47]

    Abdallah, J

    A. Abdallah, J. Mozafari, B. Piryani, and A. Jatowt. Asrank: Zero-shot re-ranking with answer scent for document retrieval. arXiv preprint arXiv:2501.15245, 2025

    arXiv 2025

  7. [1]

    https://arangodb.com/

  8. [3]

    https://aws.amazon.com/cn/ai/generative- ai/nova/understanding/

Show all 300 references
  1. [4]

    https://aws.amazon.com/s3

  2. [5]

    https://bailian.console.aliyun.com/xiyan

  3. [6]

    https://beautiful-soup-4.readthedocs.io/en/latest/

  4. [7]

    https://bitbucket.org/product/

  5. [8]

    https://blazegraph.com/

  6. [9]

    https://cachelib.org/

  7. [10]

    https://cocodataset.org/

  8. [11]

    https://commoncrawl.org/

  9. [13]

    https://docs.python.org/3/library/pickle.html

  10. [15]

    https://github.com/deepseek-ai/3fs

  11. [16]

    https://github.com/juicedata/juicefs

  12. [17]

    https://github.com/neo4j/neo4j

  13. [18]

    https://github.com/paddlepaddle/paddleocr

  14. [19]

    https://github.com/seleniumhq/selenium

  15. [21]

    https://graphdb.ontotext.com/

  16. [22]

    https://huggingface.co/

  17. [23]

    https://huggingface.co/ckiplab/bert-tiny-chinese

  18. [24]

    https://huggingface.co/infgrad/stella-large-zh-v2

  19. [28]

    https://openlibrary.org/

  20. [29]

    https://paddlenlp.readthedocs.io

  21. [30]

    https://playwright.dev/

  22. [32]

    https://pytorch.org/

  23. [35]

    https://www.dbdoctor.cn/

  24. [36]

    https://www.elastic.co/elasticsearch

  25. [37]

    https://www.eyelevel.ai/post/do-vector-databases-lose- accuracy-at-scale

  26. [38]

    https://www.gutenberg.org/

  27. [39]

    https://www.llamaindex.ai/

  28. [40]

    https://www.mindspore.cn/

  29. [41]

    https://www.pawsql.com/

  30. [42]

    https://www.tensorflow.org

  31. [43]

    https://www.tensorflow.org/guide/data

  32. [44]

    https://www.tensorflow.org/tutorials/load data/tfrecord

  33. [45]

    Abbas, E

    A. Abbas, E. Rusak, K. Tirumala, W. Brendel, K. Chaudhuri, and A. S. Morcos. Effective pruning of web-scale datasets based on complexity of concept clusters.arXiv preprint arXiv:2401.04578, 2024

    2024 arXiv

  34. [46]

    Abbas, K

    A. Abbas, K. Tirumala, D. Simig, S. Ganguli, and A. S. Morcos. Semdedup: Data-efficient learning at web-scale through seman- tic deduplication.arXiv preprint arXiv:2303.09540, 2023

    2023 arXiv

  35. [48]

    Abiteboul

    S. Abiteboul. Querying semi-structured data. InDatabase The- ory—ICDT’97: 6th International Conference Delphi, Greece, January 8–10, 1997 Proceedings 6, pages 1–18. Springer, 1997

    1997

  36. [49]

    Aggarwal, A

    K. Aggarwal, A. Khandelwal, K. Tanmay, O. M. Khan, Q. Liu, M. Choudhury, H. H. Chauhan, S. Som, V. Chaudhary, and S. Tiwary. Dublin: Visual document understanding by language-image network, 2023

    2023

  37. [50]

    Aguerrebere, I

    C. Aguerrebere, I. Bhati, M. Hildebrand, M. Tepper, and T. Willke. Similarity search in the blink of an eye with com- pressed indices, 2023

    2023

  38. [51]

    Ahmed, K

    T. Ahmed, K. S. Pai, P. Devanbu, and E. Barr. Automatic semantic augmentation of language model prompts (for code summarization). InProceedings of the IEEE/ACM 46th Inter- national Conference on Software Engineering, ICSE ’24, New York, NY, USA, 2024. Association for Computing...

    2024

  39. [52]

    Akbik, T

    A. Akbik, T. Bergmann, D. Blythe, K. Rasul, S. Schweter, and R. Vollgraf. Flair: An easy-to-use framework for state- of-the-art nlp. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics (demonstrations), pages 54–5...

    2019

  40. [53]

    Akioyamen, Z

    P. Akioyamen, Z. Yi, and R. Marcus. The unreasonable effec- tiveness of llms for query optimization.CoRR, abs/2411.02862, 2024

    2024 arXiv

  41. [54]

    M. M. Alam and W. Wang. A comprehensive survey on data provenance: State-of-the-art approaches and their deployments for iot security enforcement.J. Comput. Secur., 29(4):423–446, 2021

    2021

  42. [55]

    Albalak, Y

    A. Albalak, Y. Elazar, S. M. Xie, S. Longpre, N. Lambert, X. Wang, N. Muennighoff, B. Hou, L. Pan, H. Jeong, et al. A survey on data selection for language models.arXiv preprint arXiv:2402.16827, 2024

    2024 arXiv

  43. [56]

    Albalak, L

    A. Albalak, L. Pan, C. Raffel, and W. Y. Wang. Efficient online data mixing for language model pre-training. InR0- FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023

    2023

  44. [57]

    K. An, F. Yang, L. Li, J. Lu, S. Cheng, S. Si, L. Wang, P. Zhao, L. Cao, Q. Lin, et al. Thread: A logic-based data organization paradigm for how-to question answering with retrieval aug- mented generation.arXiv preprint arXiv:2406.13372, 2024

    2024

  45. [58]

    Q. An, C. Ying, Y. Zhu, Y. Xu, M. Zhang, and J. Wang. LEDD: large language model-empowered data discovery in data lakes. CoRR, abs/2502.15182, 2025

    2025 arXiv

  46. [59]

    R. Angles. A comparison of current graph database models. In 2012 IEEE 28th International Conference on Data Engineering Workshops, pages 171–177, 2012

    2012

  47. [60]

    Angles and C

    R. Angles and C. Gutierrez. Survey of graph database models. ACM Comput. Surv., 40(1), Feb. 2008

    2008

  48. [61]

    Ankner, C

    Z. Ankner, C. Blakeney, K. Sreenivasan, M. Marion, M. L. Leavitt, and M. Paul. Perplexed by perplexity: Perplexity- based data pruning with small reference models.arXiv preprint arXiv:2405.20541, 2024

    2024 arXiv

  49. [62]

    Appalaraju, P

    S. Appalaraju, P. Tang, Q. Dong, N. Sankaran, Y. Zhou, and R. Manmatha. Docformerv2: Local features for document understanding, 2023

    2023

  50. [63]

    Arora, B

    S. Arora, B. Yang, S. Eyuboglu, A. Narayan, A. Hojel, I. Trum- mer, and C. R´ e. Language models enable simple systems for generating structured views of heterogeneous data lakes.Proc. VLDB Endow., 17(2):92–105, 2023

    2023

  51. [64]

    Artetxe, S

    M. Artetxe, S. Bhosale, N. Goyal, T. Mihaylov, M. Ott, S. Shleifer, X. V. Lin, J. Du, S. Iyer, R. Pasunuru, et al. Efficient large scale language modeling with mixtures of experts.arXiv preprint arXiv:2112.10684, 2021

    2021 arXiv

  52. [65]

    G. A. Atemezing. Empirical evaluation of a cloud-based graph database: the case of neptune. InKnowledge Graphs and Se- mantic Web: Third Iberoamerican Conference and Second Indo- American Conference, KGSWC 2021, Kingsville, Texas, USA, November 22–24, 2021, Proceedings 3, page...

    2021

  53. [66]

    Attendu and J.-P

    J.-M. Attendu and J.-P. Corbeil. Nlu on data diets: Dynamic data subset selection for nlp classification tasks.arXiv preprint arXiv:2306.03208, 2023

    2023 arXiv

  54. [67]

    Audibert, Y

    A. Audibert, Y. Chen, D. Graur, A. Klimovic, J. ˇSimˇ sa, and C. A. Thekkath. tf. data service: A case for disaggregating ml input data processing. InProceedings of the 2023 ACM Symposium on Cloud Computing, pages 358–375, 2023

    2023

  55. [68]

    Ayoola, S

    T. Ayoola, S. Tyagi, J. Fisher, C. Christodoulopoulos, and A. Pierleoni. Refined: An efficient zero-shot-capable approach to end-to-end entity linking, 2022

    2022

  56. [69]

    J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  57. [70]

    S. Bai, K. Chen, X. Liu, et al. Qwen2.5-vl technical report. CoRR, abs/2502.13923, 2025

    2025 arXiv

  58. [71]

    Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    2022 arXiv

  59. [72]

    M. I. L. Balaka, D. Alexander, Q. Wang, Y. Gong, A. Krisnadhi, and R. C. Fernandez. Pneuma: Leveraging llms for tabular data representation and retrieval in an end-to-end system.arXiv preprint arXiv:2504.09207, 2025

    2025 arXiv

  60. [73]

    Barbaresi

    A. Barbaresi. Trafilatura: A web scraping library and command-line tool for text discovery and extraction. In H. Ji, J. C. Park, and R. Xia, editors,Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confere...

    2021

  61. [74]

    Bendinelli, A

    T. Bendinelli, A. Dox, and C. Holz. Exploring llm agents for cleaning tabular machine learning datasets. InICLR 2025 Workshop on Foundation Models in the Wild, 2025. arXiv:2503.06664

    2025 arXiv

  62. [75]

    Berglund and B

    M. Berglund and B. van der Merwe. Formalizing bpe tokeniza- tion.arXiv preprint arXiv:2309.08715, 2023

    2023 arXiv

  63. [76]

    Bevendorff, S

    J. Bevendorff, S. Gupta, J. Kiesel, and B. Stein. An em- pirical comparison of web content extraction algorithms. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, page 2594–2603, New York, NY, USA, 2...

    2023

  64. [77]

    Biderman, U

    S. Biderman, U. Prashanth, L. Sutawika, H. Schoelkopf, Q. An- thony, S. Purohit, and E. Raff. Emergent and predictable memorization in large language models.Advances in Neural Information Processing Systems, 36, 2024

    2024

  65. [78]

    Biester, M

    F. Biester, M. Abdelaal, and D. D. Gaudio. Llmclean: Context- aware tabular data cleaning via llm-generated ofds. InADBIS (Short Papers), volume 2186 ofCommunications in Computer and Information Science, pages 68–78. Springer, 2024

    2024

  66. [79]

    Borthakur et al

    D. Borthakur et al. Hdfs architecture guide.Hadoop apache project, 53(1-13):2, 2008

    2008

  67. [80]

    Brandfonbrener, H

    D. Brandfonbrener, H. Zhang, A. Kirsch, J. R. Schwarz, and S. Kakade. Color-filter: Conditional loss reduction filtering for targeted language model pre-training.arXiv preprint arXiv:2406.10670, 2024

    2024 arXiv

  68. [81]

    A. Z. Broder. On the resemblance and containment of docu- ments. InProceedings. Compression and Complexity of SE- QUENCES 1997 (Cat. No. 97TB100171), pages 21–29. IEEE, 1997

    1997

  69. [82]

    L. Cao. Tablemaster: A recipe to advance table understanding with language models, 2025

    2025

  70. [83]

    Q. Cao, M. Najibi, and S. Mehta. Ctrlsynth: Controllable image text synthesis for data-efficient multimodal learning, 2024

    2024

  71. [84]

    Y. Cao, Y. Kang, C. Wang, and L. Sun. Instruction mining: Instruction data selection for tuning large language models. arXiv preprint arXiv:2307.06290, 2023

    2023 arXiv

  72. [85]

    Casey, K

    B. Casey, K. Damian, A. Cotaj, and J. C. S. Santos. An empirical study of safetensors’ usage trends and developers’ perceptions, 2025

    2025

  73. [86]

    C. Chai, J. Wang, Y. Luo, Z. Niu, and G. Li. Data management for machine learning: A survey.IEEE Trans. Knowl. Data Eng., 35(5):4646–4667, 2023

    2023

  74. [88]

    M. S. Charikar. Similarity estimation techniques from rounding algorithms. InProceedings of the thiry-fourth annual ACM symposium on Theory of computing, pages 380–388, 2002

    2002

  75. [89]

    Che, X.-L

    T.-Y. Che, X.-L. Mao, T. Lan, and H. Huang. A hierarchical context augmentation method to improve retrieval-augmented llms on scientific papers. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Min- ing, pages 243–254, 2024

    2024

  76. [90]

    D. Chen, Y. Huang, Z. Ma, H. Chen, X. Pan, C. Ge, D. Gao, Y. Xie, Z. Liu, J. Gao, et al. Data-juicer: A one-stop data processing system for large language models. InCompanion of the 2024 International Conference on Management of Data, pages 120–134, 2024

    2024

  77. [91]

    D. Chen, H. Wang, Y. Huang, C. Ge, Y. Li, B. Ding, and J. Zhou. Data-juicer sandbox: A feedback-driven suite for multimodal data-model co-development.arXiv preprint arXiv:2407.11784, 2024

    2024 arXiv

  78. [92]

    H. Chen, A. Waheed, X. Li, Y. Wang, J. Wang, B. Raj, and M. I. Abdin. On the diversity of synthetic data and its impact on training large language models.arXiv preprint arXiv:2410.15226, 2024

    2024 arXiv

  79. [93]

    J. Chen, Z. Chen, J. Wang, K. Zhou, Y. Zhu, J. Jiang, Y. Min, W. X. Zhao, Z. Dou, J. Mao, et al. Towards effective and efficient continual pre-training of large language models.arXiv preprint arXiv:2407.18743, 2024

    2024 arXiv

  80. [94]

    J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu. M3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge dis- tillation.ACL, 2024

    2024

  81. [95]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021. 48

    2021 arXiv

  82. [96]

    S. Chen, J. Fan, B. Wu, N. Tang, C. Deng, P. Wang, Y. Li, J. Tan, F. Li, J. Zhou, and X. Du. Automatic database config- uration debugging using retrieval-augmented language models. CoRR, abs/2412.07548, 2024

    2024 arXiv

  83. [97]

    T. Chen, H. Wang, S. Chen, W. Yu, K. Ma, X. Zhao, H. Zhang, and D. Yu. Dense x retrieval: What retrieval granularity should we use?arXiv preprint arXiv:2312.06648, 2023

    2023 arXiv

  84. [98]

    Z. Chen, T. Liu, M. Tian, W. Luo, Z. Liu, et al. Advancing mathematical reasoning in language models: The impact of problem-solving data, data synthesis methods, and training stages. InThe Thirteenth International Conference on Learn- ing Representations, 2025

    2025

  85. [99]

    Cheng, Y

    D. Cheng, Y. Gu, S. Huang, J. Bi, M. Huang, and F. Wei. Instruction pre-training: Language models are supervised mul- titask learners.arXiv preprint arXiv:2406.14491, 2024

    2024 arXiv

  86. [100]

    Cheng, Z

    S. Cheng, Z. Zhuang, Y. Xu, F. Yang, C. Zhang, X. Qin, X. Huang, L. Chen, Q. Lin, D. Zhang, S. Rajmohan, and Q. Zhang. Call me when necessary: Llms can efficiently and faithfully reason over structured environments, 2024

    2024

  87. [101]

    Cheng, X

    X. Cheng, X. Wang, X. Zhang, T. Ge, S.-Q. Chen, F. Wei, H. Zhang, and D. Zhao. xrag: Extreme context compression for retrieval-augmented generation with one token.arXiv preprint arXiv:2405.13792, 2024

    2024 arXiv

  88. [102]

    Chevalier, A

    A. Chevalier, A. Wettig, A. Ajith, and D. Chen. Adapt- ing language models to compress contexts.arXiv preprint arXiv:2305.14788, 2023

    2023 arXiv

  89. [103]

    J. Choi, J. Yun, K. Jin, and Y. Kim. Multi-news+: Cost- efficient dataset cleansing via llm-based data annotation. In EMNLP, pages 15–29. Association for Computational Linguis- tics, 2024

    2024

  90. [104]

    Chowdhery, S

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Au...

    2022

  91. [105]

    Christ, S

    M. Christ, S. Gunn, and O. Zamir. Undetectable watermarks for language models. InThe Thirty Seventh Annual Conference on Learning Theory, pages 1125–1139. PMLR, 2024

    2024

  92. [106]

    Cobbe, V

    K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  93. [107]

    E. F. Codd. A relational model of data for large shared data banks.Commun. ACM, 13(6):377–387, June 1970

    1970

  94. [108]

    Colson, P

    B. Colson, P. Marcotte, and G. Savard. An overview of bilevel optimization.Annals of operations research, 153:235–256, 2007

    2007

  95. [109]

    Computer

    T. Computer. Redpajama: An open source recipe to reproduce llama training dataset, 2023

    2023

  96. [110]

    Conover, M

    M. Conover, M. Hayes, A. Mathur, J. Xie, J. Wan, S. Shah, A. Ghodsi, P. Wendell, M. Zaharia, and R. Xin. Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023

    2023

  97. [111]

    Cossu, A

    A. Cossu, A. Carta, L. Passaro, V. Lomonaco, T. Tuytelaars, and D. Bacciu. Continual pre-training mitigates forgetting in language and vision.Neural Networks, 179:106492, 2024

    2024

  98. [112]

    A. I. Cowen-Rivers, W. Lyu, Z. Wang, R. Tutunov, J. Hao, J. Wang, and H. Bou-Ammar. HEBO: heteroscedastic evolu- tionary bayesian optimisation.CoRR, abs/2012.03826, 2020

    2012 arXiv

  99. [113]

    G. Cui, L. Yuan, N. Ding, G. Yao, B. He, W. Zhu, Y. Ni, G. Xie, R. Xie, Y. Lin, et al. Ultrafeedback: Boosting language models with scaled ai feedback. InForty-first International Conference on Machine Learning, 2024

    2024

  100. [115]

    Y. Dai, D. Feng, J. Huang, H. Jia, Q. Xie, Y. Zhang, W. Han, W. Tian, and H. Wang. Laiw: A chinese legal large language models benchmark, 2024

    2024

  101. [116]

    H. Ding, Z. Wang, G. Paolini, V. Kumar, A. Deoras, D. Roth, and S. Soatto. Fewer truncations improve language modeling. arXiv preprint arXiv:2404.10830, 2024

    2024 arXiv

  102. [117]

    N. Ding, Y. Chen, B. Xu, Y. Qin, Z. Zheng, S. Hu, Z. Liu, M. Sun, and B. Zhou. Enhancing chat language models by scaling high-quality instructional conversations.arXiv preprint arXiv:2305.14233, 2023

    2023 arXiv

  103. [118]

    Y. Ding, Z. Wang, W. Ahmad, M. K. Ramanathan, R. Nal- lapati, P. Bhatia, D. Roth, and B. Xiang. CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context. In N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, editors,Proceedings of the 2024 Jo...

    2024

  104. [119]

    Y. Ding, Q. Zeng, and T. Weninger. Chatel: Entity linking with chatbots, 2024

    2024

  105. [120]

    Dodge, M

    J. Dodge, M. Sap, A. Marasovi´ c, W. Agnew, G. Ilharco, D. Groeneveld, M. Mitchell, and M. Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus.arXiv preprint arXiv:2104.08758, 2021

    2021 arXiv

  106. [121]

    F. Dong, Y. He, Y. Liang, Z. Liu, Y. Wu, P. Chen, and T. Yang. Simisketch: Efficiently estimating similarity of streaming mul- tisets.arXiv preprint arXiv:2405.19711, 2024

    2024 arXiv

  107. [122]

    G. Dong, D. Pan, Y. Sun, S. Zhang, Z. Liang, X. Wu, Y. Shen, F. Yang, H. Sun, T. Li, et al. Baichuanseed: Sharing the potential of extensive data collection and deduplication by introducing a competitive large language model baseline.arXiv preprint arXiv:2408.15079, 2024

    2024 arXiv

  108. [123]

    G. Dong, H. Yuan, K. Lu, C. Li, M. Xue, D. Liu, W. Wang, Z. Yuan, C. Zhou, and J. Zhou. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv preprint arXiv:2310.05492, 2023

    2023 arXiv

  109. [124]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    2021

  110. [125]

    Douze, A

    M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazar´ e, M. Lomeli, L. Hosseini, and H. J´ egou. The faiss library. arXiv preprint arXiv:2401.08281, 2024

    2024 arXiv

  111. [126]

    Q. Du, C. Zong, and J. Zhang. Mods: Model-oriented data selection for instruction tuning.arXiv preprint arXiv:2311.15653, 2023

    2023 arXiv

  112. [127]

    D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson. From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130, 2024

    2024 arXiv

  113. [129]

    M. Y. Eltabakh, Z. A. Naeem, M. S. Ahmad, M. Ouzzani, and N. Tang. Retclean: Retrieval-based tabular data cleaning using llms and data lakes.Proc. VLDB Endow., 17(12):4421–4424, 2024

    2024

  114. [130]

    Engstrom, A

    L. Engstrom, A. Feldmann, and A. Madry. Dsdm: Model- aware dataset selection with datamodels.arXiv preprint arXiv:2401.12926, 2024

    2024 arXiv

  115. [131]

    Fairoze, S

    J. Fairoze, S. Garg, S. Jha, S. Mahloujifar, M. Mahmoody, and M. Wang. Publicly detectable watermarking for language models.arXiv preprint arXiv:2310.18491, 2023

    2023 arXiv

  116. [132]

    C. Fan, Z. Pan, W. Sun, C. Yang, and W. Chen. Latuner: An llm-enhanced database tuning system based on adaptive sur- rogate model. InECML/PKDD (5), volume 14945 ofLecture Notes in Computer Science, pages 372–388. Springer, 2024

    2024

  117. [133]

    L. Fan, D. Krishnan, P. Isola, D. Katabi, and Y. Tian. Improv- ing clip training with language rewrites, 2023

    2023

  118. [134]

    M. Fan, X. Han, J. Fan, C. Chai, N. Tang, G. Li, and X. Du. Cost-effective in-context learning for entity resolution: A design space exploration. InICDE, pages 3696–3709. IEEE, 2024

    2024

  119. [135]

    S. Fan, M. Pagliardini, and M. Jaggi. Doge: Domain reweighting with generalization estimation.arXiv preprint arXiv:2310.15393, 2023

    2023 arXiv

  120. [136]

    T. Fan, J. Wang, X. Ren, and C. Huang. Minirag: Towards ex- tremely simple retrieval-augmented generation.arXiv preprint arXiv:2501.06713, 2025

    2025 arXiv

  121. [137]

    Fayyaz, E

    M. Fayyaz, E. Aghazadeh, A. Modarressi, M. T. Pilehvar, Y. Yaghoobzadeh, and S. E. Kahou. Bert on a data diet: Finding important examples by gradient-based pruning.arXiv preprint arXiv:2211.05610, 2022. 49

    2022 arXiv

  122. [138]

    H. Feng, Q. Liu, H. Liu, J. Tang, W. Zhou, H. Li, and C. Huang. Docpedia: Unleashing the power of large multimodal model in the frequency domain for versatile document understanding, 2024

    2024

  123. [139]

    S. Feng, S. Prabhumoye, K. Kong, D. Su, M. Patwary, M. Shoeybi, and B. Catanzaro. Maximize your data’s potential: Enhancing llm accuracy with two-phase pretraining.arXiv preprint arXiv:2412.15285, 2024

    2024 arXiv

  124. [140]

    W. Feng, C. Hao, Y. Zhang, Y. Han, and H. Wang. Mixture-of- loras: An efficient multitask tuning for large language models. arXiv preprint arXiv:2403.03432, 2024

    2024 arXiv

  125. [141]

    Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou. Codebert: A pre-trained model for programming and natural languages, 2020

    2020

  126. [142]

    S. Ferr´ e. First steps of an approach to the arc challenge based on descriptive grid models and the minimum description length principle.arXiv preprint arXiv:2112.00848, 2021

    2021 arXiv

  127. [143]

    Feuer, Y

    B. Feuer, Y. Liu, C. Hegde, and J. Freire. Archetype: A novel framework for open-source column type annotation using large language models.Proc. VLDB Endow., 17(9):2279–2292, 2024

    2024

  128. [144]

    A. Finn, N. Kushmerick, and B. Smyth. Fact or fiction: Content classification for digital libraries. InDELOS Workshops / Conferences, 2001

    2001

  129. [145]

    Fuest, P

    M. Fuest, P. Ma, M. Gui, J. Schusterbauer, V. T. Hu, and B. Ommer. Diffusion models and representation learning: A survey, 2024

    2024

  130. [146]

    S. Y. Gadre, G. Ilharco, A. Fang, J. Hayase, G. Smyrnis, T. Nguyen, R. Marten, M. Wortsman, D. Ghosh, J. Zhang, et al. Datacomp: In search of the next generation of multimodal datasets.Advances in Neural Information Processing Systems, 36:27092–27112, 2023

    2023

  131. [147]

    Gandhi, M

    S. Gandhi, M. Zhao, A. Skiadopoulos, and C. Kozyrakis. Recy- cle: Resilient training of large dnns using pipeline adaptation. InProceedings of the ACM SIGOPS 30th Symposium on Oper- ating Systems Principles, pages 211–228, 2024

    2024

  132. [148]

    B. Gao, Z. He, P. Sharma, Q. Kang, D. Jevdjic, J. Deng, X. Yang, Z. Yu, and P. Zuo.{Cost-Efficient}large language model serving for multi-turn conversations with {CachedAttention}. In2024 USENIX Annual Technical Con- ference (USENIX ATC 24), pages 111–126, 2024

    2024

  133. [149]

    L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling.arXiv preprint arXiv:2101.00027, 2020

    2020 arXiv

  134. [150]

    S. Gao, Y. Chen, and J. Shu. Fast state restoration in llm serving with hcache, 2024

    2024

  135. [151]

    S. Gao, C. Gao, Y. He, J. Zeng, L. Nie, X. Xia, and M. Lyu. Code structure–guided transformer for source code summa- rization.ACM Transactions on Software Engineering and Methodology, 32(1):1–32, Jan. 2023

    2023

  136. [152]

    C. Ge, Z. Ma, D. Chen, Y. Li, and B. Ding. Bimix: A bivariate data mixing law for language model pretraining, 2025

    2025

  137. [153]

    Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun. Yolox: Exceeding yolo series in 2021.arXiv preprint arXiv:2107.08430, 2021

    2021 arXiv

  138. [154]

    M. Geng, S. Wang, D. Dong, H. Wang, G. Li, Z. Jin, X. Mao, and X. Liao. Large language models are few-shot summarizers: Multi-intent comment generation via in-context learning, 2023

    2023

  139. [155]

    Giannakouris and I

    V. Giannakouris and I. Trummer. DBG-PT: A large language model assisted query performance regression debugger.Proc. VLDB Endow., 17(12):4337–4340, 2024

    2024

  140. [156]

    Giannankouris and I

    V. Giannankouris and I. Trummer.λ-tune: Harnessing large language models for automated database system tuning. CoRR, abs/2411.03500, 2024

    2024 arXiv

  141. [157]

    Goyal, P

    S. Goyal, P. Maini, Z. C. Lipton, A. Raghunathan, and J. Z. Kolter. Scaling laws for data filtering–data curation cannot be compute agnostic. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22702–22711, 2024

    2024

  142. [158]

    Graur, D

    D. Graur, D. Aymon, D. Kluser, T. Albrici, C. A. Thekkath, and A. Klimovic. Cachew: Machine learning input data pro- cessing as a service. In2022 usenix annual technical conference (usenix atc 22), pages 689–706, 2022

    2022

  143. [159]

    Graur, O

    D. Graur, O. Mraz, M. Li, S. Pourghannad, C. A. Thekkath, and A. Klimovic. Pecan:{Cost-Efficient}{ML}data prepro- cessing with automatic transformation ordering and hybrid placement. In2024 USENIX Annual Technical Conference (USENIX ATC 24), pages 649–665, 2024

    2024

  144. [160]

    J. Gu, Z. Yang, C. Ding, R. Zhao, and F. Tan. Cmr scaling law: Predicting critical mixture ratios for continual pre-training of language models.arXiv preprint arXiv:2407.17467, 2024

    2024 arXiv

  145. [161]

    R. Gu, K. Zhang, Z. Xu, Y. Che, B. Fan, H. Hou, H. Dai, L. Yi, Y. Ding, G. Chen, et al. Fluid: Dataset abstraction and elastic acceleration for cloud-native deep learning training jobs. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2182–2195. IEEE, 2022

    2022

  146. [162]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  147. [163]

    Y. Guo, Z. Hu, Y. Mao, B. Zheng, Y. Gao, and M. Zhou. Birdie: Natural language-driven table discovery using differentiable search index, 2025

    2025

  148. [164]

    Z. GUO, L. Xia, Y. Yu, T. Ao, and C. Huang. LightRAG: Simple and fast retrieval-augmented generation, 2024

    2024

  149. [165]

    Gupta, P

    V. Gupta, P. Kandoi, M. B. Vora, S. Zhang, Y. He, R. Reinanda, and V. Srikumar. Temptabqa: Temporal question answering for semi-structured tables, 2023

    2023

  150. [166]

    W. L. Hamilton, R. Ying, and J. Leskovec. Inductive repre- sentation learning on large graphs. InProceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 1025–1035, Red Hook, NY, USA, 2017. Curran Associates Inc

    2017

  151. [167]

    N. He, W. Xiong, H. Liu, Y. Liao, L. Ding, K. Zhang, G. Tang, X. Han, and W. Yang. Softdedup: an efficient data reweighting method for speeding up language model pre-training.arXiv preprint arXiv:2407.06654, 2024

    2024 arXiv

  152. [168]

    Y. He, Z. Wang, Z. Shen, G. Sun, Y. Dai, Y. Wu, H. Wang, and A. Li. Shed: Shapley-based automated dataset refinement for instruction fine-tuning.arXiv preprint arXiv:2405.00705, 2024

    2024 arXiv

  153. [169]

    Hernandez, T

    D. Hernandez, T. Brown, T. Conerly, N. DasSarma, D. Drain, S. El-Showk, N. Elhage, Z. Hatfield-Dodds, T. Henighan, T. Hume, et al. Scaling laws and interpretability of learning from repeated data.arXiv preprint arXiv:2205.10487, 2022

    2022 arXiv

  154. [170]

    Hoffmann, S

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, O. Vinyals, J. W. Rae, and L. Sifr...

    2022

  155. [171]

    S. Hong, Y. Lin, B. Liu, B. Liu, B. Wu, C. Zhang, C. Wei, D. Li, J. Chen, J. Zhang, J. Wang, L. Zhang, L. Zhang, M. Yang, M. Zhuge, T. Guo, T. Zhou, W. Tao, X. Tang, X. Lu, X. Zheng, X. Liang, Y. Fei, Y. Cheng, Z. Gou, Z. Xu, and C. Wu. Data interpreter: An llm agent for data ...

    2024

  156. [172]

    M. J. Hosseini, Y. Gao, T. Baumg¨ artner, A. Fabrikant, and R. K. Amplayo. Scalable and domain-general abstractive proposition segmentation.arXiv preprint arXiv:2406.19803, 2024

    2024 arXiv

  157. [173]

    Z. Hou, X. Lv, R. Lu, J. Zhang, Y. Li, Z. Yao, J. Li, J. Tang, and Y. Dong. Advancing language model reasoning through reinforcement learning and inference scaling.arXiv preprint arXiv:2501.11651, 2025

    2025 arXiv

  158. [174]

    A. Hu, H. Xu, J. Ye, M. Yan, L. Zhang, B. Zhang, C. Li, J. Zhang, Q. Jin, F. Huang, and J. Zhou. mplug-docowl 1.5: Unified structure learning for ocr-free document understand- ing, 2024

    2024

  159. [175]

    H. Hua, Y. Tang, C. Xu, and J. Luo. V2xum-llm: Cross- modal video summarization with temporal prompt instruction tuning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3599–3607, 2025

    2025

  160. [176]

    Huang, D

    J. Huang, D. Guo, C. Wang, J. Gu, S. Lu, J. P. Inala, C. Yan, J. Gao, N. Duan, and M. R. Lyu. Contextualized data- wrangling code generation in computational notebooks. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE ’24, pag...

    2024

  161. [177]

    Huang, H

    X. Huang, H. Li, J. Zhang, X. Zhao, Z. Yao, Y. Li, Z. Yu, T. Zhang, H. Chen, and C. Li. E2etune: End-to-end knob tuning via fine-tuned generative language model.CoRR, abs/2404.11581, 2025. 50

    2025 arXiv

  162. [178]

    Huang, X

    Y. Huang, X. Lin, Z. Liu, Q. Cao, H. Xin, H. Wang, Z. Li, L. Song, and X. Liang. Mustard: Mastering uniform synthesis of theorem and proof data.arXiv preprint arXiv:2402.08957, 2024

    2024 arXiv

  163. [179]

    Huang, X

    Y. Huang, X. Liu, Y. Gong, Z. Gou, Y. Shen, N. Duan, and W. Chen. Key-point-driven data synthesis with its enhancement on mathematical reasoning.arXiv preprint arXiv:2403.02333, 2024

    2024 arXiv

  164. [180]

    Huang, T

    Y. Huang, T. Lv, L. Cui, Y. Lu, and F. Wei. Layoutlmv3: Pre- training for document ai with unified text and image masking, 2022

    2022

  165. [181]

    Husain, H.-H

    H. Husain, H.-H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt. CodeSearchNet challenge: Evaluat- ing the state of semantic code search.arXiv preprint arXiv:1909.09436, 2019

    1909 arXiv

  166. [182]

    Ilharco, M

    G. Ilharco, M. Wortsman, N. Carlini, R. Taori, A. Dave, V. Shankar, H. Namkoong, J. Miller, H. Hajishirzi, A. Farhadi, and L. Schmidt. Openclip, July 2021

    2021

  167. [183]

    Ilyankou, M

    I. Ilyankou, M. Wang, S. Cavazzi, and J. Haworth. Cc-gpx: Ex- tracting high-quality annotated geospatial data from common crawl, 2024

    2024

  168. [184]

    Ilyas, S

    A. Ilyas, S. M. Park, L. Engstrom, G. Leclerc, and A. Madry. Datamodels: Predicting predictions from training data.arXiv preprint arXiv:2202.00622, 2022

    2022 arXiv

  169. [185]

    Islam, M

    K. Islam, M. Z. Zaheer, A. Mahmood, and K. Nandakumar. Diffusemix: Label-preserving data augmentation with diffusion models, 2024

    2024

  170. [186]

    I. Jang, Z. Yang, Z. Zhang, X. Jin, and M. Chowdhury. Oobleck: Resilient distributed training of large models using pipeline templates. InProceedings of the 29th Symposium on Operating Systems Principles, pages 382–395, 2023

    2023

  171. [187]

    Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung. Survey of hallucination in natural language generation.ACM Computing Surveys, 55(12):1–38, 2023

    2023

  172. [188]

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed. Mistral 7b, 2023

    2023

  173. [189]

    Jiang, Q

    H. Jiang, Q. Wu, C.-Y. Lin, Y. Yang, and L. Qiu. Llmlin- gua: Compressing prompts for accelerated inference of large language models.arXiv preprint arXiv:2310.05736, 2023

    2023 arXiv

  174. [190]

    Jiang, Q

    H. Jiang, Q. Wu, X. Luo, D. Li, C.-Y. Lin, Y. Yang, and L. Qiu. Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression.arXiv preprint arXiv:2310.06839, 2023

    2023 arXiv

  175. [191]

    Jiang, F

    J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim. A survey on large language models for code generation.arXiv preprint arXiv:2406.00515, 2024

    2024 arXiv

  176. [192]

    Jiang, K

    J. Jiang, K. Zhou, Z. Dong, K. Ye, W. X. Zhao, and J.-R. Wen. Structgpt: A general framework for large language model to reason over structured data, 2023

    2023

  177. [193]

    Jiang, K

    J. Jiang, K. Zhou, W. X. Zhao, and J.-R. Wen. Unikgqa: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph, 2023

    2023

  178. [194]

    Jiang, H

    Z. Jiang, H. Lin, Y. Zhong, Q. Huang, Y. Chen, Z. Zhang, Y. Peng, X. Li, C. Xie, S. Nong, et al.{MegaScale}: Scaling large language model training to more than 10,000{GPUs}. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 745–760, 2024

    2024

  179. [195]

    X. Jiao, Y. Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu. Tinybert: Distilling bert for natural language understanding.arXiv preprint arXiv:1909.10351, 2019

    1909 arXiv

  180. [196]

    R. L. J. X. Y. C. P. H. C. H. M. D. Z. Jie Tan, Kangfei Zhao and Y. Rong. Can large language models be query optimizer for relational databases?CoRR, abs/2502.05562, 2025

    2025 arXiv

  181. [197]

    C. Jin, Z. Zhang, X. Jiang, F. Liu, X. Liu, X. Liu, and X. Jin. Ragcache: Efficient knowledge caching for retrieval-augmented generation.arXiv preprint arXiv:2404.12457, 2024

    2024 arXiv

  182. [198]

    D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021

    2021

  183. [199]

    Joulin, E

    A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov. Bag of tricks for efficient text classification.arXiv preprint arXiv:1607.01759, 2016

    2016 arXiv

  184. [200]

    D. Jung, Q. Liu, T. Huang, B. Zhou, and M. Chen. Familiarity- aware evidence compression for retrieval augmented genera- tion.arXiv preprint arXiv:2409.12468, 2024

    2024

  185. [201]

    Kaplan, S

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  186. [202]

    A. Kay. Tesseract: an open-source optical character recognition engine.Linux J., 2007(159):2, July 2007

    2007

  187. [203]

    Kayali, A

    M. Kayali, A. Lykov, I. Fountalis, N. Vasiloglou, D. Olteanu, and D. Suciu. CHORUS: foundation models for unified data discovery and exploration.Proc. VLDB Endow., 17(8):2104– 2114, 2024

    2024

  188. [204]

    Kayali, F

    M. Kayali, F. Wenz, N. Tatbul, and C ¸ . Demiralp. Mind the data gap: Bridging llms to enterprise data integration. In Proceedings of the 2025 Conference on Innovative Data Systems Research (CIDR), Chaminade, California, 2025. CIDR 2025

    2025

  189. [205]

    G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu. Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing sys- tems, 30, 2017

    2017

  190. [206]

    J. D. M.-W. C. Kenton and L. K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of naacL-HLT, volume 1, page 2. Minneapolis, Minnesota, 2019

    2019

  191. [207]

    A. Khan, R. Underwood, C. Siebenschuh, Y. Babuji, A. Ajith, K. Hippe, O. Gokdemir, A. Brace, K. Chard, and I. Foster. Lshbloom: Memory-efficient, extreme-scale document dedupli- cation.arXiv preprint arXiv:2411.04257, 2024

    2024

  192. [208]

    M. A. M. Khan, M. S. Bari, X. L. Do, W. Wang, M. R. Parvez, and S. Joty. xcodeeval: A large scale multilingual multitask benchmark for code understanding, generation, translation and retrieval, 2023

    2023

  193. [209]

    D. Kim, B. Kim, D. Han, and M. Eibich. Autorag: Automated framework for optimization of retrieval augmented generation pipeline, 2024

    2024

  194. [210]

    Kim and J

    J. Kim and J. Lee. Strategic data ordering: Enhancing large lan- guage model performance through curriculum learning.arXiv preprint arXiv:2405.07490, 2024

    2024 arXiv

  195. [211]

    T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. InProceedings of the 5th In- ternational Conference on Learning Representations (ICLR),

  196. [212]

    Kirchenbauer, J

    J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, and T. Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061– 17084. PMLR, 2023

    2023

  197. [213]

    Kirkpatrick, C

    S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi. Optimization by simulated annealing.science, 220(4598):671–680, 1983

    1983

  198. [214]

    Kocetkov, R

    D. Kocetkov, R. Li, L. Ben Allal, J. Li, C. Mou, C. Mu˜ noz Fer- randis, Y. Jernite, M. Mitchell, S. Hughes, T. Wolf, D. Bah- danau, L. von Werra, and H. de Vries. The stack: 3 tb of permissively licensed source code.Preprint, 2022

    2022

  199. [215]

    Kolluru, M

    K. Kolluru, M. Mohammed, S. Mittal, S. Chakrabarti, et al. Alignment-augmented consistent translation for multilingual open information extraction. InProceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2502–2...

    2022

  200. [216]

    W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  201. [217]

    Korini and C

    K. Korini and C. Bizer. Evaluating knowledge generation and self-refinement strategies for llm-based column type annota- tion.CoRR, abs/2503.02718, 2025

    2025 arXiv

  202. [218]

    M. M. Krell, M. Kosec, S. P. Perez, and A. Fitzgibbon. Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance.arXiv preprint arXiv:2107.02027, 2021

    2021 arXiv

  203. [219]

    A. V. Kumar and M. Sivathanu. Quiver: An informed storage cache for deep learning. In18th USENIX Conference on File and Storage Technologies (FAST 20), pages 283–296, 2020

    2020

  204. [220]

    W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. Efficient memory man- agement for large language model serving with pagedattention, 2023

    2023

  205. [221]

    J. Lai, W. Gan, J. Wu, Z. Qi, and P. S. Yu. Large language models in law: A survey.AI Open, 2024. 51

    2024

  206. [222]

    Z. Lai, H. Zhang, B. Zhang, W. Wu, H. Bai, A. Timofeev, X. Du, Z. Gan, J. Shan, C.-N. Chuah, Y. Yang, and M. Cao. Veclip: Improving clip training via visual-enriched captions, 2024

    2024

  207. [223]

    J. Lao, Y. Wang, Y. Li, J. Wang, Y. Zhang, Z. Cheng, W. Chen, M. Tang, and J. Wang. Gptuner: A manual-reading database tuning system via gpt-guided bayesian optimization.Proc. VLDB Endow., 17(8):1939–1952, 2024

    1939

  208. [224]

    Lauren¸ con, L

    H. Lauren¸ con, L. Saulnier, L. Tronchon, S. Bekman, A. Singh, A. Lozhkov, T. Wang, S. Karamcheti, A. Rush, D. Kiela, et al. Obelics: An open web-scale filtered dataset of interleaved image-text documents.Advances in Neural Information Pro- cessing Systems, 36, 2024

    2024

  209. [225]

    K. Lee, M. Joshi, I. Turc, H. Hu, F. Liu, J. Eisenschlos, U. Khandelwal, P. Shaw, M.-W. Chang, and K. Toutanova. Pix2struct: Screenshot parsing as pretraining for visual lan- guage understanding, 2023

    2023

  210. [226]

    F. Lei, X. Li, Y. Wei, S. He, Y. Huang, J. Zhao, and K. Liu. S3HQA: A three-stage approach for multi-hop text-table hy- brid question answering. In A. Rogers, J. Boyd-Graber, and N. Okazaki, editors,Proceedings of the 61st Annual Meeting of the Association for Computational Li...

    2023

  211. [227]

    Leviathan, M

    Y. Leviathan, M. Kalman, and Y. Matias. Fast inference from transformers via speculative decoding. InICML, volume 202 of Proceedings of Machine Learning Research, pages 19274–19286. PMLR, 2023

    2023

  212. [228]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    2020

  213. [229]

    B. Li, Y. Luo, C. Chai, G. Li, and N. Tang. The dawn of natural language to sql: Are we fully ready?Proceedings of the VLDB Endowment, 17(11):3318–3331, July 2024

    2024

  214. [230]

    D. Li, S. Cao, T. Griggs, S. Liu, X. Mo, S. G. Patil, M. Zaharia, J. E. Gonzalez, and I. Stoica. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374, 2025

    2025 arXiv

  215. [231]

    G. Li, X. Zhou, S. Li, and B. Gao. Qtune: A query-aware database tuning system with deep reinforcement learning. Proc. VLDB Endow., 12(12):2118–2130, 2019

    2019

  216. [232]

    H. Li, Y. Chen, Q. Ai, Y. Wu, R. Zhang, and Y. Liu. Lexeval: A comprehensive chinese legal benchmark for evaluating large language models.arXiv preprint arXiv:2409.20288, 2024

    2024 arXiv

  217. [233]

    H. Li, Q. Dong, Z. Tang, C. Wang, X. Zhang, H. Huang, S. Huang, X. Huang, Z. Huang, D. Zhang, et al. Synthetic data (almost) from scratch: Generalized instruction tuning for language models.arXiv preprint arXiv:2402.13064, 2024

    2024 arXiv

  218. [234]

    H. Li, J. Zhang, H. Liu, J. Fan, X. Zhang, J. Zhu, R. Wei, H. Pan, C. Li, and H. Chen. Codes: Towards building open- source language models for text-to-sql, 2024

    2024

  219. [235]

    J. LI, E. Beeching, L. Tunstall, B. Lipkin, R. Soletskyi, S. C. Huang, K. Rasul, L. Yu, A. Jiang, Z. Shen, Z. Qin, B. Dong, L. Zhou, Y. Fleureau, G. Lample, and S. Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath-CoT](https: //github.com/project-numina/aimo-progress-...

    2024

  220. [236]

    J. Li, A. Fang, G. Smyrnis, M. Ivgi, M. Jordan, S. Gadre, H. Bansal, E. Guha, S. Keh, K. Arora, et al. Datacomp-lm: In search of the next generation of training sets for language models.arXiv preprint arXiv:2406.11794, 2024

    2024 arXiv

  221. [237]

    L. Li, L. Fang, and V. I. Torvik. Autodcworkflow: Llm- based data cleaning workflow auto-generation and benchmark. CoRR, abs/2412.06724, 2024

    2024 arXiv

  222. [238]

    M. Li, Y. Zhang, S. He, Z. Li, H. Zhao, J. Wang, N. Cheng, and T. Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning.arXiv preprint arXiv:2402.00530, 2024

    2024 arXiv

  223. [239]

    M. Li, Y. Zhang, Z. Li, J. Chen, L. Chen, N. Cheng, J. Wang, T. Zhou, and J. Xiao. From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning.arXiv preprint arXiv:2308.12032, 2023

    2023 arXiv

  224. [240]

    P. Li, Y. He, D. Yashar, W. Cui, S. Ge, H. Zhang, D. R. Fainman, D. Zhang, and S. Chaudhuri. Table-gpt: Table-tuned gpt for diverse table tasks, 2023

    2023

  225. [241]

    S. Li, X. Ning, L. Wang, T. Liu, X. Shi, S. Yan, G. Dai, H. Yang, and Y. Wang. Evaluating quantized large language models. In ICML. OpenReview.net, 2024

    2024

  226. [242]

    X. Li, Z. Wu, J. Wu, H. Cui, J. Jia, R.-H. Li, and G. Wang. Graph learning in the era of llms: A survey from the perspective of data, models, and tasks, 2024

    2024

  227. [243]

    Y. Li, H. Li, P. Zhao, J. Zhang, X. Zhang, T. Ji, L. Sun, C. Li, and H. Chen. Is large language model good at database knob tuning? A comprehensive experimental evaluation.CoRR, abs/2408.02213, 2024

    2024 arXiv

  228. [244]

    Y. LI, G. Zhang, X. Qu, J. Li, Z. Li, Z. Wang, H. Li, R. Yuan, Y. Ma, K. Zhang, W. Zhou, Y. Liang, L. Zhang, L. Ma, J. Zhang, Z. Li, S. W. Huang, C. Lin, and J. Fu. Cif-bench: A chinese instruction-following benchmark for evaluating the generalizability of large language models, 2024

    2024

  229. [245]

    Z. Li, Y. Du, M. Zheng, and M. Song. Mimotable: A multi- scale spreadsheet benchmark with meta operations for table reasoning, 2024

    2024

  230. [246]

    Z. Li, S. Fan, Y. Gu, X. Li, Z. Duan, B. Dong, N. Liu, and J. Wang. Flexkbqa: A flexible llm-powered framework for few- shot knowledge base question answering, 2024

    2024

  231. [247]

    Z. Li, X. Wang, J. Zhao, S. Yang, G. Du, X. Hu, B. Zhang, Y. Ye, Z. Li, R. Zhao, and H. Mao. Pet-sql: A prompt-enhanced two-round refinement of text-to-sql with cross-consistency, June 2024

    2024

  232. [248]

    Z. Li, H. Yuan, H. Wang, G. Cong, and L. Bing. LLM-R2: A large language model enhanced rule-based rewrite system for boosting query efficiency.Proc. VLDB Endow., 18(1):53–65, 2024

    2024

  233. [249]

    Z. Li, X. Zhang, Y. Zhang, D. Long, P. Xie, and M. Zhang. Towards general text embeddings with multi-stage contrastive learning.arXiv preprint arXiv:2308.03281, 2023

    2023 arXiv

  234. [250]

    J. Lian, X. Liu, Y. Shao, et al. Chatbi: Towards natu- ral language to complex business intelligence SQL.CoRR, abs/2405.00527, 2024

    2024 arXiv

  235. [251]

    Liang, K

    H. Liang, K. Zhao, Y. Yang, B. Cui, G. Dong, Z. Zhou, and W. Zhang. Data proportion detection for optimized data management for large language models.arXiv preprint arXiv:2409.17527, 2024

    2024 arXiv

  236. [252]

    Liang, T

    Y. Liang, T. Xie, G. Peng, Z. Huang, Y. Lan, and W. Qian. Nat- nl2gql: A novel multi-agent framework for translating natural language to graph query language, 2024

    2024

  237. [253]

    Lightman, V

    H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    2023 arXiv

  238. [254]

    X. Lin, W. Wang, Y. Li, S. Yang, F. Feng, Y. Wei, and T.- S. Chua. Data-efficient fine-tuning for llm-based recommen- dation. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Re- trieval, pages 365–374, 2024

    2024

  239. [255]

    A. Liu, J. Liu, Z. Pan, Y. He, R. Haffari, and B. Zhuang. Minicache: Kv cache compression in depth dimension for large language models.Advances in Neural Information Processing Systems, 37:139997–140031, 2024

    2024

  240. [256]

    A. Liu, L. Pan, X. Hu, S. Li, L. Wen, I. King, and S. Y. Philip. An unforgeable publicly verifiable watermark for large language models. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  241. [257]

    C. Liu, H. Wei, J. Chen, L. Kong, Z. Ge, Z. Zhu, L. Zhao, J. Sun, C. Han, and X. Zhang. Focus anywhere for fine-grained multi- page document understanding, 2024

    2024

  242. [258]

    H. Liu, C. Li, Y. Li, and Y. J. Lee. Improved baselines with visual instruction tuning, 2024

    2024

  243. [259]

    H. Liu, Q. Peng, Q. Yang, K. Liu, and H. Xu. Bucket pre- training is all you need.arXiv preprint arXiv:2407.07495, 2024

    2024 arXiv

  244. [260]

    H. Liu, Y. Zhang, Y. Luo, and A. C.-C. Yao. Augmenting math word problems via iterative question composing.arXiv preprint arXiv:2401.09003, 2024

    2024 arXiv

  245. [261]

    Liu and B

    J. Liu and B. Mozafari. Query rewriting via large language models.CoRR, abs/2403.09060, 2024

    2024

  246. [262]

    J. Liu, K. Wang, Y. Chen, X. Peng, Z. Chen, L. Zhang, and Y. Lou. Large language model-based agents for software engi- neering: A survey, 2024

    2024

  247. [263]

    Q. Liu, X. Zheng, N. Muennighoff, G. Zeng, L. Dou, T. Pang, J. Jiang, and M. Lin. Regmix: Data mixture as regression for language model pre-training.arXiv preprint arXiv:2407.01492, 2024

    2024 arXiv

  248. [264]

    W. Liu, W. Zeng, K. He, Y. Jiang, and J. He. What makes good data for alignment? a comprehensive study of auto- matic data selection in instruction tuning.arXiv preprint arXiv:2312.15685, 2023. 52

    2023 arXiv

  249. [265]

    Y. Liu, H. Li, Y. Cheng, S. Ray, Y. Huang, Q. Zhang, K. Du, J. Yao, S. Lu, G. Ananthanarayanan, et al. Cachegen: Kv cache compression and streaming for fast large language model serv- ing. InProceedings of the ACM SIGCOMM 2024 Conference, pages 38–56, 2024

    2024

  250. [266]

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A ro- bustly optimized bert pretraining approach, 2019

    2019

  251. [267]

    Y. Liu, E. Pe˜ na, A. S. R. Santos, E. Wu, and J. Freire. Mag- neto: Combining small and large language models for schema matching.CoRR, abs/2412.08194, 2024

    2024 arXiv

  252. [268]

    Z. Liu, Y. Huang, X. Yu, L. Zhang, Z. Wu, C. Cao, H. Dai, L. Zhao, Y. Li, P. Shu, et al. Deid-gpt: Zero-shot medical text de-identification by gpt-4.arXiv preprint arXiv:2303.11032, 2023

    2023

  253. [269]

    Z. Liu, A. Karbasi, and T. Rekatsinas. Tsds: Data se- lection for task-specific model finetuning.arXiv preprint arXiv:2410.11303, 2024

    2024 arXiv

  254. [270]

    Z. Liu, Q. Liao, W. Gu, and C. Gao. Software vulnerability detection with gpt and in-context learning. In2023 8th In- ternational Conference on Data Science in Cyberspace (DSC), pages 229–236, 2023

    2023

  255. [271]

    Z. Liu, Z. Tang, J. Zhang, X. Xia, and X. Yang. Pre-training by predicting program dependencies for vulnerability analysis tasks, 2024

    2024

  256. [272]

    L. Long, R. Wang, R. Xiao, J. Zhao, X. Ding, G. Chen, and H. Wang. On llms-driven synthetic data generation, curation, and evaluation: A survey.arXiv preprint arXiv:2406.15126, 2024

    2024 arXiv

  257. [273]

    D. Lu, H. Wu, J. Liang, Y. Xu, Q. He, Y. Geng, M. Han, Y. Xin, and Y. Xiao. Bbt-fin: Comprehensive construction of chinese financial domain pre-trained language model, corpus and benchmark.arXiv preprint arXiv:2302.09432, 2023

    2023 arXiv

  258. [274]

    W. Lu, J. Zhang, J. Fan, Z. Fu, Y. Chen, and X. Du. Large language model for table processing: A survey.Frontiers of Computer Science, 19(2):192350, 2025

    2025

  259. [275]

    Lukas, A

    N. Lukas, A. Salem, R. Sim, S. Tople, L. Wutschitz, and S. Zanella-B´ eguelin. Analyzing leakage of personally identifi- able information in language models. In2023 IEEE Symposium on Security and Privacy (SP), pages 346–363. IEEE, 2023

    2023

  260. [276]

    Z. Luo, X. Zhang, X. Liu, H. Li, Y. Gong, C. Qi, and P. Cheng. Velocitune: A velocity-based dynamic domain reweighting method for continual pre-training.arXiv preprint arXiv:2411.14318, 2024

    2024 arXiv

  261. [277]

    C. Ma, S. Chakrabarti, A. Khan, and B. Moln´ ar. Knowledge graph-based retrieval-augmented generation for schema match- ing.CoRR, abs/2501.08686, 2025

    2025 arXiv

  262. [278]

    G. Ma, Y. Ma, X. Wu, Z. Su, M. Zhou, and S. Hu. Task-level distributionally robust optimization for large language model- based dense retrieval.arXiv preprint arXiv:2408.10613, 2024

    2024 arXiv

  263. [279]

    L. Ma, N. Thakurdesai, J. Chen, J. Xu, E. K¨ orpeoglu, S. Ku- mar, and K. Achan. Llms with user-defined prompts as generic data operators for reliable data processing. InIEEE Big Data, pages 3144–3148. IEEE, 2023

    2023

  264. [281]

    Z. Ma, B. Zhang, J. Zhang, J. Yu, X. Zhang, X. Zhang, S. Luo, X. Wang, and J. Tang. Spreadsheetbench: Towards challenging real world spreadsheet manipulation, 2024

    2024

  265. [282]

    Maini, S

    P. Maini, S. Seto, H. Bai, D. Grangier, Y. Zhang, and N. Jaitly. Rephrasing the web: A recipe for compute and data-efficient language modeling.arXiv preprint arXiv:2401.16380, 2024

    2024 arXiv

  266. [283]

    Manber and G

    U. Manber and G. Myers. Suffix arrays: a new method for on- line string searches.siam Journal on Computing, 22(5):935– 948, 1993

    1993

  267. [284]

    Y. Mao, X. Li, W. Li, X. Wang, and L. Xie. Scla: Automated smart contract summarization via llms and semantic augmen- tation, 2024

    2024

  268. [285]

    Marion, A

    M. Marion, A. ¨Ust¨ un, L. Pozzobon, A. Wang, M. Fadaee, and S. Hooker. When less is more: Investigating data pruning for pretraining llms at scale.arXiv preprint arXiv:2309.04564, 2023

    2023 arXiv

  269. [286]

    J. L. McClelland, B. L. McNaughton, and R. C. O’Reilly. Why there are complementary learning systems in the hippocam- pus and neocortex: insights from the successes and failures of connectionist models of learning and memory.Psychological review, 102(3):419, 1995

    1995

  270. [287]

    McCloskey and N

    M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109–

  271. [288]

    Mekala, A

    D. Mekala, A. Nguyen, and J. Shang. Smaller language models are capable of selecting instruction-tuning training data for larger language models.arXiv preprint arXiv:2402.10430, 2024

    2024 arXiv

  272. [289]

    Minaee, T

    S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao. Large language models: A survey. arXiv preprint arXiv:2402.06196, 2024

    2024 arXiv

  273. [290]

    Mitra, L

    A. Mitra, L. Del Corro, G. Zheng, S. Mahajan, D. Rouhana, A. Codas, Y. Lu, W.-g. Chen, O. Vrousgos, C. Rosset, et al. Agentinstruct: Toward generative teaching with agentic flows. arXiv preprint arXiv:2407.03502, 2024

    2024 arXiv

  274. [291]

    Mohan, A

    J. Mohan, A. Phanishayee, and V. Chidambaram. CheckFreq: Frequent, Fine-Grained DNN checkpointing. In19th USENIX Conference on File and Storage Technologies (FAST 21), pages 203–216. USENIX Association, Feb. 2021

    2021

  275. [292]

    Monteiro, F

    J. Monteiro, F. S´ a, and J. Bernardino. Graph databases as- sessment: Janusgraph, neo4j, and tigergraph. InPerspectives and Trends in Education and Technology: Selected Papers from ICITED 2022, pages 655–665. Springer, 2023

    2022

  276. [293]

    J. Mu, X. Li, and N. Goodman. Learning to compress prompts with gist tokens.Advances in Neural Information Processing Systems, 36, 2024

    2024

  277. [294]

    Muennighoff, Q

    N. Muennighoff, Q. Liu, A. Zebaze, Q. Zheng, B. Hui, T. Y. Zhuo, S. Singh, X. Tang, L. von Werra, and S. Longpre. Oc- topack: Instruction tuning code large language models, 2024

    2024

  278. [295]

    C. Na, I. Magnusson, A. H. Jha, T. Sherborne, E. Strubell, J. Dodge, and P. Dasigi. Scalable data ablation approximations for language models through modular training and merging. arXiv preprint arXiv:2410.15661, 2024

    2024 arXiv

  279. [296]

    Navigli, S

    R. Navigli, S. Conia, and B. Ross. Biases in large language models: origins, inventory, and discussion.ACM Journal of Data and Information Quality, 15(2):1–21, 2023

    2023

  280. [297]

    Nguyen, C

    T. Nguyen, C. V. Nguyen, V. D. Lai, H. Man, N. T. Ngo, F. Dernoncourt, R. A. Rossi, and T. H. Nguyen. CulturaX: A cleaned, enormous, and multilingual dataset for large language models in 167 languages. In N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, edito...

    2024

  281. [298]

    Nunes, M

    I. Nunes, M. Heddes, P. Verg´ es, D. Abraham, A. Veidenbaum, A. Nicolau, and T. Givargis. Dothash: Estimating set similarity metrics for link prediction and document deduplication. InPro- ceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages ...

    2023

  282. [299]

    Nystrom, C

    A. Nystrom, C. Zhang, C. Callison-Burch, D. Ippolito, D. Eck, K. Lee, and N. Carlini. Deduplicating training data makes language models better. 2022

    2022

  283. [300]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion.arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  284. [301]

    Ouyang, J

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    2022

  285. [302]

    R. Pan, J. Zhang, X. Pan, R. Pi, X. Wang, and T. Zhang. Scalebio: Scalable bilevel optimization for llm data reweighting. arXiv preprint arXiv:2406.19976, 2024

    2024 arXiv

  286. [303]

    Z. Pan, Q. Wu, H. Jiang, M. Xia, X. Luo, J. Zhang, Q. Lin, V. R¨ uhle, Y. Yang, C.-Y. Lin, et al. Llmlingua-2: Data distilla- tion for efficient and faithful task-agnostic prompt compression. arXiv preprint arXiv:2403.12968, 2024

    2024 arXiv

  287. [304]

    Parciak, B

    M. Parciak, B. Vandevoort, F. Neven, L. M. Peeters, and S. Vansummeren. Schema matching with large language mod- els: an experimental study. InVLDB Workshops. VLDB.org, 2024

    2024

  288. [305]

    H. Park, S. Lee, G. Gim, Y. Kim, D. Kim, and C. Park. Dataverse: Open-source etl (extract, transform, load) pipeline for large language models.arXiv preprint arXiv:2403.19340, 2024. 53

    2024 arXiv

  289. [306]

    Patnaik, H

    S. Patnaik, H. Changwal, M. Aggarwal, S. Bhatia, Y. Kumar, and B. Krishnamurthy. Cabinet: Content relevance based noise reduction for table question answering, 2024

    2024

  290. [307]

    D. A. Patterson, G. Gibson, and R. H. Katz. A case for redun- dant arrays of inexpensive disks (raid). InProceedings of the 1988 ACM SIGMOD international conference on Management of data, pages 109–116, 1988

    1988

  291. [308]

    Peeters, A

    R. Peeters, A. Steiner, and C. Bizer. Entity matching using large language models. InEDBT, pages 529–541. OpenPro- ceedings.org, 2025

    2025

  292. [2017]

    Published as a conference paper at ICLR 2017

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.