Pith. sign in

REVIEW 3 major objections 5 minor 84 references

Large Language Models in the Data Science Lifecycle: A Systematic Mapping Study

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper is a systematic mapping study that claims to be the first to map LLM applications across all five stages of the data science lifecycle, with the finding that research is concentrated in data analysis and model building while depl

desk verdict Useful first full-lifecycle map of LLMs in data science, but the exact stage counts and RQ5 synthesis are shaky enough that I'd treat the numbers as directional rather than precise. read the letter →

arxiv 2508.11698 v1 pith:CTI6FMEP submitted 2025-08-12 cs.CY

Sai Sanjna Chintakunta , Nathalia Nascimento , Everton Guimaraes This is my paper · ORCID
classification cs.CY
keywords largelanguagemodelsdatasciencesystematicmappingstudylifecycleLLMevaluationresearchgapsdeploymentanalysisautomation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a systematic mapping study of how large language models are being applied across the data science lifecycle. It claims to be the first to cover all five stages—problem definition, data preparation, exploration and analysis, model building, and deployment—rather than a single stage. Its central result is a quantitative map: research clusters heavily in data analysis (41 papers) and model building (33), with far less on data preparation (23), almost none on problem definition (3), and only one paper on deployment and maintenance. If this map is right, the field has a clear frontier: the later stages of the lifecycle are where LLM-based data science work is most needed and least studied. The paper also synthesizes how these systems are evaluated and what benefits and risks the literature reports.

What carries the argument

The central object is the five-stage data science lifecycle (Problem Definition; Data Collection and Preparation; Data Exploration and Analysis; Model Building and Evaluation; Deployment and Maintenance), used as the classification grid for every reviewed study. The method is the systematic mapping study protocol: goal-question-metric-derived research questions, a search string run in two bibliographic databases, screening by abstract and introduction, forward and backward snowballing, and structured data extraction. The lifecycle grid is what turns a scattered set of papers into a quantified map with visible gaps.

What would settle it

Re-run the search with terms like 'MLOps', 'model monitoring', and 'LLM agent' in additional scholarly databases, and count how many empirical studies at the deployment/maintenance stage appear; if the count moves well beyond 1, the paper's central gap claim would not hold. Alternatively, recover the RQ5 keyword list and show that a paper classified as reporting negative impacts uses none of the listed keywords, which would break the thematic synthesis.

Watch

Extended reading notes

Core claim

The paper's discovery is a stage-by-stage map of the literature: 41 studies address data exploration and analysis, 33 address model building and evaluation, 23 address data collection and preparation, 3 address problem definition, and 1 addresses deployment and maintenance. It reports that 87.7% of the included studies are empirical, that accuracy is the dominant evaluation metric, that the GPT family is the most studied model line, and that the reported benefits (productivity, accuracy, accessibility, code quality) are balanced by concerns about reproducibility, privacy, and handling complexity. The authors frame this as the first systematic mapping study to take the whole lifecycle as its

Load-bearing premise

The map's stage frequencies are only as reliable as the assumption that one search string across two databases, built on one five-stage lifecycle framework, captured the relevant literature.

Editorial extensions

If this is right

  • The research frontier is at the ends of the lifecycle: problem definition and, especially, deployment and maintenance.
  • Evaluation practice in this literature rests mainly on accuracy, so cross-study comparison would benefit from shared benchmarks per stage.
  • The reported evidence positions LLMs as supplements to human analysts, not standalone replacements, for real-world data science.
  • Because most included studies are empirical, the field is ready for more standardized, reproducible evaluation protocols.
  • Domain-specific applications in health, geospatial, and finance are emerging alongside general-purpose ones, so stage gaps may vary by domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the search was restricted to two databases and one lifecycle taxonomy, the near-zero deployment count is likely an undercount; work published under labels like 'MLOps' or 'model monitoring' may exist outside the retrieved set.
  • Beyond the paper: the coarse five-stage grid means a paper assigned to 'model building' may be primarily about code generation or visualization, so stage frequencies should be read as approximate rather than exact.
  • Beyond the paper: the recurring non-determinism finding points to a concrete reporting norm—seeds, sampling, and archived outputs—that would make future results comparable.
  • Beyond the paper: the same mapping method could be applied periodically to track whether the deployment gap closes as tooling matures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This systematic mapping study (SMS) claims to be the first to map applications of large language models across all five stages of a data science lifecycle: problem definition, data collection and preparation, data exploration and analysis, model building and evaluation, and deployment and maintenance. The authors executed the search string in §4.2 on Scopus and IEEE Xplore, screened 172 initial records against inclusion/exclusion criteria, applied forward and backward snowballing, and analyzed a final corpus reported as 66 papers. Six research questions are addressed: stage/task coverage, evaluation techniques and datasets, LLM families, application domains, reported positive/negative impacts, and research gaps. The central quantitative map (Fig. 3) reports 41 papers on data analysis, 33 on model building, 23 on data preparation, 3 on problem definition, and 1 on deployment, which drives the paper's main gap claims.

Significance. If the underlying corpus is reliable, this is a useful and timely contribution: it provides a structured, quantified overview of a fast-moving area, ships a reproducible search string, and makes the full coding of the 66 papers available in supplementary material. The authors are also transparent about several validity threats (§7). However, the significance is conditional: the stage-frequency map and the gap analysis rest entirely on the search string's term coverage and on the accuracy of the keyword-based categorization, and both have concrete weaknesses documented below. The paper would be a solid reference point for the community once those load-bearing points are addressed.

major comments (3)
  1. [§4.2, Fig. 3] The search string's third disjunct lists visualization, code generation, data preparation, data analysis, and deployment terms, but omits the core vocabulary of the Model Building and Evaluation stage: 'feature engineering', 'model training', 'model evaluation', 'AutoML', 'machine learning', and 'predictive modeling'. Since Fig. 3's stage-frequency map (model building: 33) and the resulting gap analysis are the paper's central quantitative output, these counts may be artifacts of the chosen query rather than a property of the literature. The general caveat in §7.2.1 is not sufficient; please rerun or report a sensitivity analysis with an expanded search string and quantify the impact on Fig. 3.
  2. [§4.2, §4] Corpus construction is not fully reproducible. The TITLE-ABS-KEY query requires 'Generative AI', 'Large Language Models', or 'LLM', and omits common model names such as ChatGPT, GPT-4, Llama, and Gemini, so papers framed by model name are excluded unless an abstract happens to contain a generic term. The report gives no counts after each screening step from 172 to 66, and snowballing is mentioned but not quantified. Because all RQ counts depend on this corpus, please provide a PRISMA-style flow, the number of papers added by snowballing, and a discussion of known missed papers, especially those that use model names in title/abstract.
  3. [§5.5, Figs. 10–11] RQ5's quantitative synthesis is not verifiable as reported. The text states that the analysis was conducted on a corpus of 66 papers, while the captions of Figs. 10 and 11 give denominators of 62. The 'keyword-based categorization' is not accompanied by the keyword list or by any inter-rater reliability statistic, yet it produces the reported frequencies (49 positive, 42 negative, 45 gaps). Please correct the denominator and supply the coding instrument, a complete keyword list, and inter-rater agreement measures, or explicitly report that the categorization was a single-reviewer judgment.
minor comments (5)
  1. [§5.5] The bullet 'Generation of High-Quality Code and Datasets' repeats the preceding bullet 'Improved Task Accuracy and Performance' almost verbatim (the Jadoon 93%, Guo 100%, and Pido 95% examples appear in both). Replace with distinct evidence for code/data generation.
  2. [§4.3] Inclusion criterion (iii) allows 'preprints in established repositories' while the same criterion requires 'peer-reviewed' sources. Please clarify how preprints are treated.
  3. [§5.1.1, §5.1.5] Reference [86] is cited for both Problem Definition and Deployment; since the text later notes that this study covers all lifecycle phases, state this explicitly to avoid apparent double-counting.
  4. [§2.1] The lifecycle taxonomy is taken from [32], a commercial blog. Since the search string is built directly on this taxonomy, justify the choice relative to established frameworks such as CRISP-DM or KDD.
  5. [§5.2] The claim that 87.7% of reviewed papers conducted an empirical study should state the denominator (e.g., 58 of 66) so the percentage can be checked.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a descriptive systematic mapping study with no fitted parameters or derived predictions; its acknowledged search-string limitations are validity threats, not circular reasoning.

full rationale

This paper performs a systematic mapping study: it constructs a corpus via a documented search string, manually screens and classifies papers, and reports descriptive frequencies (e.g., Fig. 3). There is no derivation chain in which an output is defined by an input, no fitted parameter renamed as a prediction, and no uniqueness theorem imported from the authors' prior work. The five-stage lifecycle adopted from [32] is an explicit analytical lens, not a result claimed to be derived from the corpus; the central map is an empirical categorization, not an equation. The authors include their own prior work ([48], [49]) as cited context and as one corpus item, but these citations do not define the stage counts or the gap analysis; nothing in the conclusions reduces to those self-citations. The search string's coverage limitations are expressly acknowledged in Section 7.2.1 ('we acknowledge that some relevant studies may still have been missed'), which is an honest validity threat about corpus completeness rather than a circular step. The stray editorial sentence in Section 3 ('The paper addresses software engineering programming capabilities...') is an internal inconsistency in related-work description, but it does not function as a load-bearing argument. Accordingly, no circular step can be exhibited, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities apply; this is a literature synthesis. The three axioms above are the load-bearing assumptions on which the map's accuracy rests.

assumptions (3)
  • domain assumption The five-stage data science lifecycle model (Problem Definition; Data Collection and Preparation; Data Exploration and Analysis; Model Building and Evaluation; Deployment and Maintenance) is the correct organizing taxonomy.
    Adopted from commercial blog [32]; all paper classification is relative to this model, but no justification is given for choosing it over KDD, CRISP-DM, or Microsoft's nine-stage workflow (Section 2.1).
  • domain assumption The search string executed in Scopus and IEEE Xplore returns a representative sample of the relevant literature.
    Section 4.2 defines the string; the paper acknowledges in Section 7.2.1 that relevant studies were likely missed, which undercuts the 'comprehensive' claim.
  • ad hoc to paper Paper screening and keyword-based categorization of impacts and gaps are accurate despite no reported inter-rater reliability or defined keyword lists.
    Section 5.5 states a keyword-based categorization was used but does not list the keywords; Figure 2 shows the screening flow but no agreement statistics are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Models in the Data Science Lifecycle: A Systematic Mapping Study." pith.science (2026). https://pith.science/paper/CTI6FMEP

@misc{pith2026250811698,
  author       = {Pith},
  title        = {Pith review of: Large Language Models in the Data Science Lifecycle: A Systematic Mapping Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CTI6FMEP}},
  note         = {Machine review of arXiv:2508.11698}
}
read the original abstract

In recent years, Large Language Models (LLMs) have emerged as transformative tools across numerous domains, impacting how professionals approach complex analytical tasks. This systematic mapping study comprehensively examines the application of LLMs throughout the Data Science lifecycle. By analyzing relevant papers from Scopus and IEEE databases, we identify and categorize the types of LLMs being applied, the specific stages and tasks of the data science process they address, and the methodological approaches used for their evaluation. Our analysis includes a detailed examination of evaluation metrics employed across studies and systematically documents both positive contributions and limitations of LLMs when applied to data science workflows. This mapping provides researchers and practitioners with a structured understanding of the current landscape, highlighting trends, gaps, and opportunities for future research in this rapidly evolving intersection of LLMs and data science.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

84 extracted references · 63 canonical work pages

  1. [1]

    Computers in Industry 161, 104128 (2024)

    Abumalloh, R.A., Nilashi, M., Ooi, K.B., Tan, G.W.H., Chan, H.K.: Impact of gen- erative artificial intelligence models on the performance of citizen data scientists in retail firms. Computers in Industry 161, 104128 (2024)

    2024

  2. [2]

    Translational and Clini- cal Pharmacology 32(2), 73–82 (2024)

    Ahn, S.: Data science through nat- ural language with chatgpt’s code interpreter. Translational and Clini- cal Pharmacology 32(2), 73–82 (2024). https://doi.org/10.12793/tcp.2024.32.e8

    work page doi:10.12793/tcp.2024.32.e8 2024

  3. [3]

    In: 2019 IEEE/ACM 41st International Confer- ence on Software Engineering: Software Engi- neering in Practice (ICSE-SEIP)

    Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., Zimmermann, T.: Software engineering for machine learning: A case study. In: 2019 IEEE/ACM 41st International Confer- ence on Software Engineering: Software Engi- neering in Practice (ICSE-SEIP). pp. 291– 300 (2019). https://doi.org/10.1109/ICSE- SEIP.2019.00042

    arXiv 2019

  4. [4]

    encyclo- pedia of software engineering-2 volume set

    Basili, V.R., Caldiera, G., Rombach, H.D.: The goal question metric approach. encyclo- pedia of software engineering-2 volume set. Copyright by John Wiley & Sons, Inc pp. 528–532 (1994)

    1994

  5. [5]

    In: Proceedings of the HILDA Workshop co-located with SIGMOD (2024)

    Beasley, C., Abouzied, A.: Pipe(line) dreams: Fully automated end-to-end analysis and visualization. In: Proceedings of the HILDA Workshop co-located with SIGMOD (2024). https://doi.org/10.1145/3665939.3665962

    work page arXiv 2024

  6. [6]

    In: Proceedings of the 3rd Eclipse Security, AI, Architecture and Mod- elling Conference on Cloud to Edge Continuum

    Bedn´ ar, P., Sarnovsky, M., Vanko, J.I.: Cognitive architecture for process indus- tries. In: Proceedings of the 3rd Eclipse Security, AI, Architecture and Mod- elling Conference on Cloud to Edge Continuum. pp. 15–20. eSAAM ’23, Ludwigsburg, Germany (Oct 2023). https://doi.org/10.1145/3624486.3624489

    work page arXiv 2023

  7. [7]

    In: Proceedings of the 44th International Confer- ence on Software Engineering

    Biswas, S., Wardat, M., Rajan, H.: The art and practice of data science pipelines: A com- prehensive study of data science pipelines in theory, in-the-small, and in-the-large. In: Proceedings of the 44th International Confer- ence on Software Engineering. pp. 2091–2103 (2022)

  8. [8]

    Journal of Pre- ventive Medicine and Hygiene 65(2), E285– E289 (2024)

    Causio, F.A., et al.: Perspectives on ai use in medicine: views of the italian society of arti- ficial intelligence in medicine. Journal of Pre- ventive Medicine and Hygiene 65(2), E285– E289 (2024). https://doi.org/10.15167/2421- 4248/jpmh2024.65.2.3261

    work page doi:10.15167/2421- 2024

Show all 84 references
  1. [9]

    IEEE Trans- actions on Visualization and Computer Graphics 31(1), 1301–1311 (Jan 2025)

    Chen, N., et al.: Viseval: A bench- mark for data visualization in the era of large language models. IEEE Trans- actions on Visualization and Computer Graphics 31(1), 1301–1311 (Jan 2025). https://doi.org/10.1109/TVCG.2024.3456320

    2025

  2. [10]

    Chintakunta, S.S., Nascimento, N., Guimaraes, E.: Llms in data science lifecycle: Selected papers (Aug 2025), https://doi.org/10.5281/zenodo.16816445

    2025 doi

  3. [11]

    In: 2023 International Conference on Artificial Intel- ligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD)

    Chugh, T., Tyagi, K., Seth, R., Srinivasan, P.: Intelligent agents driven data analyt- ics using large language models. In: 2023 International Conference on Artificial Intel- ligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD). pp. 152–157. IEEE (2023)

    2023

  4. [12]

    Ding, B., Qin, C., Zhao, R., Luo, T., Li, X., Chen, G., Xia, W., Hu, J., Luu, A.T., Joty, S.: Data augmentation using large lan- guage models: Data perspectives, learning paradigms and challenges (2024), https:// arxiv.org/abs/2403.02990

    2024 arXiv

  5. [14]

    In: Casta˜ neda, A., Enea, C., Gupta, N

    Essabri, M.A., Rebii, J., Erradi, M.: Tool augmented llms for big data analysis. In: Casta˜ neda, A., Enea, C., Gupta, N. (eds.) Networked Systems. pp. 118–123. Springer Nature Switzerland, Cham (2024)

    2024

  6. [15]

    Fang, X., Xu, W., Tan, F.A., Zhang, J., Hu, Z., Qi, Y., Nickleach, S., Socolinsky, D., Sen- gamedu, S., Faloutsos, C.: Large language models(llms) on tabular data: Prediction, generation, and understanding – a survey (2024)

    2024

  7. [16]

    Com- mun

    Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: The kdd process for extracting useful knowledge from volumes of data. Com- mun. ACM 39(11), 27–34 (Nov 1996). 15 https://doi.org/10.1145/240455.240464, https://doi.org/10.1145/240455.240464

    1996

  8. [17]

    Neuro-Oncology Advances 7(1), vdae230 (Dec 2024)

    Foltyn-Dumitru, M., Rastogi, A., Cho, J., Schell, M., Mahmutoglu, M.A., Kessler, T., Sahm, F., Wick, W., Bendszus, M., Brug- nara, G., Vollmuth, P.: The potential of gpt-4 advanced data analysis for radiomics-based machine learning models. Neuro-Oncology Advances 7(1), vdae230...

    2024 doi

  9. [18]

    IEEE Transactions on Visualization and Com- puter Graphics 31(1), 514–524 (Jan 2025)

    Gao, L., et al.: Fine-tuned large language model for visualization system: A study on self-regulated learning in education. IEEE Transactions on Visualization and Com- puter Graphics 31(1), 514–524 (Jan 2025). https://doi.org/10.1109/TVCG.2024.3456145

    2025

  10. [19]

    In: 2024 IEEE International Conference on Web Services (ICWS)

    Ghazzai, S., Grigori, D., Benatallah, B., Rebai, R.: Harnessing gpt for data transfor- mation tasks. In: 2024 IEEE International Conference on Web Services (ICWS). pp. 1329–1334. IEEE (2024)

    2024

  11. [20]

    In: Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowl- edge Discovery

    Gramacki, P., Martins, B., Szyma´ nski, P.: Evaluation of code llms on geospa- tial code generation. In: Proceedings of the 7th ACM SIGSPATIAL International Workshop on AI for Geographic Knowl- edge Discovery. p. 54–62. ACM (Oct 2024). https://doi.org/10.1145/3687123.3698286

    2024

  12. [21]

    In: Proceedings of the Conference on Human Factors in Computing Systems (CHI) (2024)

    Gu, K., Grunde-McLaughlin, M., McNutt, A.M., Heer, J., Althoff, T.: How do data analysts respond to ai assistance? a wizard-of-oz study. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI) (2024). https://doi.org/10.1145/3613904.3641891

    2024

  13. [22]

    Guo, S., Deng, C., Wen, Y., Chen, H., Chang, Y., Wang, J.: Ds-agent: Automated data sci- ence by empowering large language models with case-based reasoning (2024), https:// arxiv.org/abs/2402.17453

    2024 arXiv

  14. [23]

    Halevy, A., Choi, Y., Floratou, A., Franklin, M.J., Noy, N., Wang, H.: Will llms reshape, supercharge, or kill data sci- ence? (vldb 2023 panel). Proc. VLDB Endow. 16(12), 4114–4115 (Aug 2023). https://doi.org/10.14778/3611540.3611634

    2023

  15. [24]

    Proceedings of the VLDB Endowment 16(12), 4114–4115 (2023)

    Halevy, A., Choi, Y., Floratou, A., Franklin, M.J., Noy, N., Wang, H.: Will llms reshape, supercharge, or kill data science?(vldb 2023 panel). Proceedings of the VLDB Endowment 16(12), 4114–4115 (2023)

    2023

  16. [25]

    Big Data and Cognitive Computing 7(2), 62 (2023)

    Hassani, H., Silva, E.S.: The role of chatgpt in data science: How ai- assisted conversational interfaces are revolutionizing the field. Big Data and Cognitive Computing 7(2), 62 (2023). https://doi.org/10.3390/bdcc7020062

    2023 doi

  17. [26]

    In: 16th International Conference on Spa- tial Information Theory (COSIT 2024)

    Hojati, M., Feick, R.: Large language mod- els: Testing their capabilities to understand and explain spatial concepts (short paper). In: 16th International Conference on Spa- tial Information Theory (COSIT 2024). pp. 31–1. Schloss Dagstuhl–Leibniz-Zentrum f¨ ur Informatik (2024)

    2024

  18. [27]

    In: Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS)

    Hollmann, N., M¨ uller, S., Hutter, F.: Large language models for automated data science: Introducing caafe for context-aware auto- mated feature engineering. In: Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS). pp. 44753– 44775 (2023)

    2023

  19. [28]

    ACM Transactions on Software Engineering and Methodology pp

    Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., Wang, H.: Large language models for software engi- neering: A systematic literature review. ACM Transactions on Software Engineering and Methodology pp. 1–79 (2024)

    2024

  20. [29]

    In: Proceedings of Machine Learning Research

    Hu, X., et al.: Infiagent-dabench: Eval- uating agents on data analysis tasks. In: Proceedings of Machine Learning Research. vol. 235, pp. 19544–19572 (2024), https://www.scopus.com/inward/record. uri?eid=2-s2.0-85201943666, available via Scopus

    2024

  21. [30]

    In: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering

    Huang, J., Guo, D., Wang, C., Gu, J., Lu, S., Inala, J.P., Yan, C., Gao, J., Duan, N., Lyu, M.R.: Contextualized data- wrangling code generation in computa- tional notebooks. In: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. p. 12...

    2024

  22. [31]

    Jour- nal of Global Health 14, 04070 (Mar 2024)

    Huang, Y., Wu, R., He, J., Xiang, Y.: Evaluating chatgpt-4.0’s data analytic pro- ficiency in epidemiological studies: A com- parative analysis with sas, spss, and r. Jour- nal of Global Health 14, 04070 (Mar 2024). https://doi.org/10.7189/jogh.14.04070 16

    2024 doi

  23. [32]

    https://www.institutedata.com/us/blog/ 5-steps-in-data-science-lifecycle/ (2025), accessed: 2025-07-08

    Institute of Data: What are the 5 steps in data science lifecycle. https://www.institutedata.com/us/blog/ 5-steps-in-data-science-lifecycle/ (2025), accessed: 2025-07-08

    2025

  24. [33]

    CCF Transactions on Perva- sive Computing and Interaction 6, 199–227 (2024)

    Jadoon, A.K., Yu, C., Shi, Y.: Contextmate: a context-aware smart agent for efficient data analysis. CCF Transactions on Perva- sive Computing and Interaction 6, 199–227 (2024). https://doi.org/10.1007/s42486-023- 00144-7

    2024 doi

  25. [34]

    https://doi.org/10.1007/s10994-022-06259-9

    Jaimovitch-L´ opez, G., Ferri, C., Hern´ andez- Orallo, J., Mart ´ ınez Plumed, F., Ram ´ ırez-Quintana, M.J.: Can language models automate data wrangling? Machine Learning 112(6), 2053–2082 (2023). https://doi.org/10.1007/s10994-022-06259-9

    doi

  26. [35]

    ISPRS Inter- national Journal of Geo-Information 13(1) (2024)

    Jiang, Y., Yang, C.: Is chatgpt a good geospa- tial data analyst? exploring the integration of natural language into structured query lan- guage within a spatial database. ISPRS Inter- national Journal of Geo-Information 13(1) (2024). https://doi.org/10.3390/ijgi13010026

    2024 doi

  27. [37]

    Khadka, S., Zhang, L.: Scaling data-driven building energy modelling using large lan- guage models (2024), https://arxiv.org/abs/ 2407.03469

    2024 arXiv

  28. [38]

    In: 2024 2nd International Conference on Self Sustainable Artificial Intelligence Sys- tems (ICSSAS)

    Korkanti, S.: Enhancing financial fraud detec- tion using llms and advanced data analytics. In: 2024 2nd International Conference on Self Sustainable Artificial Intelligence Sys- tems (ICSSAS). pp. 1328–1334. IEEE (2024)

    2024

  29. [39]

    In: International Conference on Machine Learning

    Lai, Y., Li, C., Wang, Y., Zhang, T., Zhong, R., Zettlemoyer, L., Yih, W.t., Fried, D., Wang, S., Yu, T.: Ds-1000: A natural and reli- able benchmark for data science code genera- tion. In: International Conference on Machine Learning. pp. 18319–18345. PMLR (2023)

    2023

  30. [40]

    In: 2024 IEEE Conference on Artificial Intel- ligence (CAI)

    Lanfermann, F., Rios, T., Menzel, S.: Large language model-assisted clustering and con- cept identification of engineering design data. In: 2024 IEEE Conference on Artificial Intel- ligence (CAI). pp. 788–795. IEEE (2024)

    2024

  31. [41]

    IT Professional 20(1), 45–52 (2018)

    Lin, Y.T., Huang, S.J.: The design of a soft- ware engineering lifecycle process for big data projects. IT Professional 20(1), 45–52 (2018)

    2018

  32. [42]

    In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP)

    Liu, S., Wang, S., Chang, T., Lin, W., Hsiung, C., Hsieh, Y., Cheng, Y., Luo, S., Zhang, J.: JarviX: A LLM no code platform for tabular data analysis and optimization. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track (E...

    2023 doi

  33. [43]

    IEEE Access 11, 45181–45193 (2023)

    Maddigan, P., Susnjak, T.: Chat2vis: Gener- ating data visualizations via natural language using chatgpt, codex and gpt-3 large language models. IEEE Access 11, 45181–45193 (2023)

    2023

  34. [44]

    In: 2024 IEEE 3rd International Conference on AI in Cyberse- curity (ICAIC)

    Mishra, M., Braham, A., Marsom, C., Chung, B., Griffin, G., Sidnerlikar, D., Sarin, C., Rajaram, A.: Dataagent: evaluating large lan- guage models’ ability to answer zero-shot, natural language queries. In: 2024 IEEE 3rd International Conference on AI in Cyberse- curity (ICAIC...

    2024

  35. [45]

    In: 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE)

    Mohammed, M., Talburt, J.R., Syed, H., et al.: Metadata: An integral component of the modern data strategy. In: 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE). pp. 1628–

    2023

  36. [46]

    In: 2024 11th IEEE Swiss Conference on Data Science (SDS)

    Mondal, M., Audiffren, J., Dolamic, L., Bovet, G., Cudr´ e-Mauroux, P.: Cleaning semi-structured errors in open data using large language models. In: 2024 11th IEEE Swiss Conference on Data Science (SDS). pp. 258–261. IEEE (2024)

    2024

  37. [47]

    In: World Conference on Infor- mation Systems and Technologies

    Musazade, N., Mezei, J., Wang, X.: Explor- ing the performance of large language models for data analysis tasks through the crisp-dm framework. In: World Conference on Infor- mation Systems and Technologies. pp. 56–65. Springer (2024)

    2024

  38. [48]

    In: 2025 IEEE/ACM 22nd Inter- national Conference on Mining Software 17 Repositories (MSR)

    Nascimento, N., Guimaraes, E., Chintakunta, S.S., Boominathan, S.A.: How effective are llms for data science coding? a controlled experiment. In: 2025 IEEE/ACM 22nd Inter- national Conference on Mining Software 17 Repositories (MSR). pp. 211–222. IEEE (2025)

    2025

  39. [49]

    In: 2023 IEEE international conference on big data (BigData)

    Nascimento, N., Tavares, C., Alencar, P., Cowan, D.: Gpt in data science: A practi- cal exploration of model selection. In: 2023 IEEE international conference on big data (BigData). pp. 4325–4334. IEEE (2023)

    2023

  40. [50]

    In: International Conference on Human-Computer Interaction

    Nasseri, M., Brandtner, P., Zimmermann, R., Falatouri, T., Darbanian, F., Obinwanne, T.: Applications of large language models (llms) in business analytics–exemplary use cases in data preparation tasks. In: International Conference on Human-Computer Interaction. pp. 182–198. S...

    2023

  41. [51]

    Journal of Soft- ware: Evolution and Process 37(1) (2025)

    Nejjar, M., Zacharias, L., Stiehle, F., Weber, I.: Llms for science: Usage for code gen- eration and data analysis. Journal of Soft- ware: Evolution and Process 37(1) (2025). https://doi.org/10.1002/smr.2723

    2025 doi

  42. [52]

    In: 2024 Interna- tional Research Conference on Smart Com- puting and Systems Engineering (SCSE)

    Nirusanan, T., Prasanth, S., Banujan, K., Kumara, S.: Refining large language mod- els for tabular data analysis in business domain by laymen text. In: 2024 Interna- tional Research Conference on Smart Com- puting and Systems Engineering (SCSE). vol. 7, pp. 1–5. IEEE (2024)

    2024

  43. [53]

    In: 2024 IEEE International Conference on Big Data (BigData)

    Nissen, D., Yu, T., Babtista, R., Shang, Y.: Utilizing large language models (llms) in data analysis pipeline for digital phenotyping: Description, prediction, and visualization. In: 2024 IEEE International Conference on Big Data (BigData). pp. 7513–7520. IEEE (2024)

    2024

  44. [54]

    International Journal of Intelligent Systems and Applications in Engineering 12(17s), 700–705 (Feb 2024), https://ijisae.org/index

    Ojha, P.K., Gautam, A., Agrahari, A., Singh, P.: Sft for improved text-to-sql translation. International Journal of Intelligent Systems and Applications in Engineering 12(17s), 700–705 (Feb 2024), https://ijisae.org/index. php/IJISAE/article/view/4938

    2024

  45. [55]

    arxiv 2303.08774

    OpenAI, R.: Gpt-4 technical report. arxiv 2303.08774. View in Article p. 1 (2023)

    2023 arXiv

  46. [56]

    In: Arai, K

    Patel, P., Shooshtarian, S.: The role of chat- bots in data analytics: An evaluation of func- tional abilities. In: Arai, K. (ed.) Intelligent Computing. pp. 148–168. Springer Nature Switzerland, Cham (2024)

    2024

  47. [57]

    IEEE Access 11, 45972–45988 (2023)

    Pid´ o, S., Pinoli, P., Crovari, P., Ieva, F., Gar- zotto, F., Ceri, S.: Ask your data—supporting data science processes by combining automl and conversational interfaces. IEEE Access 11, 45972–45988 (2023)

    2023

  48. [58]

    In: Tekli, J., Gamper, J., Chbeir, R., Manolopoulos, Y., Sassi, S., Ivanovic, M., Vargas-Solar, G., Zumpano, E

    Pons, G., Dimic, M., Bilalli, B.: Captur- ing analytical intents from text. In: Tekli, J., Gamper, J., Chbeir, R., Manolopoulos, Y., Sassi, S., Ivanovic, M., Vargas-Solar, G., Zumpano, E. (eds.) New Trends in Database and Information Systems. pp. 81–94. Springer Nature Switzer...

    2025

  49. [59]

    In: 2023 IEEE EMBS International Conference on Biomedical and Health Infor- matics (BHI)

    Sadeghi, M., Egger, B., Agahi, R., Richer, R., Capito, K., Rupp, L.H., Schindler-Gmelch, L., Berking, M., Eskofier, B.M.: Exploring the capabilities of a language model-only approach for depression detection in text data. In: 2023 IEEE EMBS International Conference on Biomedic...

    2023

  50. [60]

    In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Eval- uation (LREC-COLING 2024)

    Schuster, S., Ansar, A., Agarwal, O., Dem- berg, V.: SpreadNaLa: A naturalistic code generation evaluation dataset of spreadsheet formulas. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Eval- uation (LREC-COLING...

    2024

  51. [61]

    SPE East- ern Regional Meeting (2024), https://api

    Seow, M.J., Qian, L.: Knowledge augmented intelligence using large language models for advanced data analytics. SPE East- ern Regional Meeting (2024), https://api. semanticscholar.org/CorpusID:273271048

    2024

  52. [62]

    In: Proceedings of the 26th Australasian Computing Education Conference

    Sheese, B., Liffiton, M., Savelka, J., Denny, P.: Patterns of student help- seeking when using a large language model-powered programming assistant. In: Proceedings of the 26th Australasian Computing Education Conference. p. 49–57. ACE 2024, ACM (Jan 2024). https://doi.org/10....

    2024

  53. [63]

    Advances in Neural Information Processing Systems 36, 38154–38180 (2023)

    Shen, Y., Song, K., Tan, X., Li, D., Lu, W., Zhuang, Y.: Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems 36, 38154–38180 (2023)

    2023

  54. [64]

    In: ICT for Intel- ligent Systems

    Shinde, S., Pande, A., Patil, P., Chaudhari, A., More, K., Shaikh, A.S.: Llama2 and django web application approach for simpli- fying table parsing: Review. In: ICT for Intel- ligent Systems. pp. 67–78. Springer Nature Singapore, Singapore (2024)

    2024

  55. [65]

    Stodden, V.: The data science life cycle: a dis- ciplined approach to advancing data science 18 as a science. Commun. ACM p. 58–66 (2020). https://doi.org/10.1145/3360646

    2020 doi

  56. [66]

    In: 2024 4th International Conference on Pervasive Computing and Social Networking (ICPCSN)

    Sv, S., Sunil, S., AS, P.A., Satish, G.: Democ- ratizing data science: using language models for intuitive data insights and visualizations. In: 2024 4th International Conference on Pervasive Computing and Social Networking (ICPCSN). pp. 1065–1069. IEEE (2024)

    2024

  57. [67]

    In: Al-Onaizan, Y., Bansal, M., Chen, Y.N

    Tan, Z., Li, D., Wang, S., Beigi, A., Jiang, B., Bhattacharjee, A., Karami, M., Li, J., Cheng, L., Liu, H.: Large language models for data annotation and synthesis: A survey. In: Al-Onaizan, Y., Bansal, M., Chen, Y.N. (eds.) In Proc. of the Conf. on Empirical Methods in Natura...

    2024 doi

  58. [68]

    Nature Communications 15(1), 1603 (2024)

    Tayebi Arasteh, S., Han, T., Lotfinia, M., Kuhl, C., Kather, J.N., Truhn, D., Nebelung, S.: Large language models streamline auto- mated machine learning for clinical studies. Nature Communications 15(1), 1603 (2024)

    2024

  59. [69]

    arXiv preprint arXiv:2312.11805 (2023)

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., Millican, K., et al.: Gemini: a family of highly capable multimodal mod- els. arXiv preprint arXiv:2312.11805 (2023)

    2023 arXiv

  60. [70]

    arXiv preprint arXiv:2307.09288 (2023)

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  61. [71]

    https://doi.org/10.3389/feduc.2024.1418006

    Valverde-Rebaza, J., Gonz´ alez, A., Navarro- Hinojosa, O., Noguez, J.: Advanced large language models and visualization tools for data analytics learning V olume 9 (2024). https://doi.org/10.3389/feduc.2024.1418006

    2024

  62. [72]

    In: 2024 11th IEEE Swiss Conference on Data Science (SDS)

    Varma, S., Shivam, S., Ray, B., Biswas, S.: Reimagining enterprise data management using generative artificial intelligence. In: 2024 11th IEEE Swiss Conference on Data Science (SDS). pp. 107–114. IEEE (2024)

    2024

  63. [73]

    In: CEUR Workshop Proceedings

    Whitehead, R., Nguyen, A., J¨ arv¨ el¨ a, S.: Generative multimodal analysis (gma) for learning process data analytics. In: CEUR Workshop Proceedings. vol. 3667, pp. 214–218 (2024), https://www.scopus.com/ inward/record.uri?eid=2-s2.0-85191987888, available via Scopus

    2024

  64. [74]

    In: Proceedings of the 4th international confer- ence on the practical applications of knowl- edge discovery and data mining

    Wirth, R., Hipp, J.: Crisp-dm: Towards a standard process model for data mining. In: Proceedings of the 4th international confer- ence on the practical applications of knowl- edge discovery and data mining. vol. 1, pp. 29–39. Manchester (2000)

    2000

  65. [75]

    Wohlin, C., Runeson, P., H¨ ost, M., Ohlsson, M.C., Regnell, B., Wessl´ en, A., et al.: Exper- imentation in software engineering, vol. 236. Springer (2012)

    2012

  66. [76]

    In: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (UIST)

    Xie, L., Zheng, C., Xia, H., Qu, H., Zhu-Tian, C.: Waitgpt: Monitoring and steering conversational llm agent in data analysis with on-the-fly code visualization. In: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology (UIST). pp. 1–14 (2024)....

    2024

  67. [77]

    IEEE Internet of Things Magazine 8(1), 26– 33 (2024)

    Yuan, Z., Liu, K., Peng, R., Li, S., Leybourne, D., Musa, N., Huang, H., Yang, P.: Pest- gpt: Leveraging large language models and iot for timely and customized recommendation generation in sustainable pest management. IEEE Internet of Things Magazine 8(1), 26– 33 (2024)

    2024

  68. [78]

    Zhang, W., Wang, Y., Song, Y., Wei, V.J., Tian, Y., Qi, Y., Chan, J.H., Wong, R.C.W., Yang, H.: Natural language interfaces for tabular data querying and visualization: A survey (2024), https://arxiv.org/abs/2310. 17894

    2024

  69. [79]

    In: Proceed- ings of the 33rd ACM International Confer- ence on Information and Knowledge Man- agement (CIKM)

    Zhang, Y., Li, N., Gan, Q., Zhang, W., Wipf, D., Wang, M.: Elf-gym: Evaluat- ing large language models generated fea- tures for tabular prediction. In: Proceed- ings of the 33rd ACM International Confer- ence on Information and Knowledge Man- agement (CIKM). pp. 5420–5424 (202...

    2024

  70. [80]

    In: Ku, L.W., Martins, A., Srikumar, V

    Zhang, Y., Jiang, Q., XingyuHan, X., Chen, N., Yang, Y., Ren, K.: Benchmark- ing data science agents. In: Ku, L.W., Martins, A., Srikumar, V. (eds.) Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). pp. 5677–57...

    2024 doi

  71. [81]

    In: Proceedings of the European Chapter of the Association for Computational Linguistics (EACL) (2024), https://aclanthology.org/2024.eacl-long.179/

    Zhang, Z., et al.: Mlcopilot: Unleashing the power of large language models in solving machine learning tasks. In: Proceedings of the European Chapter of the Association for Computational Linguistics (EACL) (2024), https://aclanthology.org/2024.eacl-long.179/

    2024

  72. [82]

    In: Proceedings of the IEEE Interna- tional Conference on Information Reuse and Integration (IRI)

    Zhao, F., Yu, F., Shang, Y.: A new method supporting qualitative data analysis through prompt generation for inductive coding. In: Proceedings of the IEEE Interna- tional Conference on Information Reuse and Integration (IRI). pp. 164–169 (2024). https://doi.org/10.1109/IRI6220...

    2024

  73. [83]

    In: Proceedings of the VLDB Endow- ment

    Zhao, X., Zhou, X., Li, G.: Chat2data: An interactive data analysis system with rag, vector databases and llms. In: Proceedings of the VLDB Endow- ment. vol. 17, pp. 4481–4484 (2024). https://doi.org/10.14778/3685800.3685905

    2024

  74. [84]

    IEEE Transactions on Visualization and Computer Graphics (2024)

    Zhao, Y., Wang, J., Xiang, L., Zhang, X., Guo, Z., Turkay, C., Zhang, Y., Chen, S.: Lightva: Lightweight visual analytics with llm agent-based task planning and execu- tion. IEEE Transactions on Visualization and Computer Graphics (2024)

    2024

  75. [85]

    IEEE Transactions on Visu- alization and Computer Graphics 31(3), 1830–1847 (2024)

    Zhao, Y., Zhang, Y., Zhang, Y., Zhao, X., Wang, J., Shao, Z., Turkay, C., Chen, S.: Leva: Using large language models to enhance visual analytics. IEEE Transactions on Visu- alization and Computer Graphics 31(3), 1830–1847 (2024)

    2024

  76. [86]

    Journal of Information Systems Education 35, 138–143 (01 2024)

    Zhong, C., Kim, J.: Teaching case: Teaching business students logistic regression in r with the aid of chatgpt. Journal of Information Systems Education 35, 138–143 (01 2024). https://doi.org/10.62273/DYLI2468 20

    2024 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.