Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies

Alfredo Nazabal; Angus Williams; Camila Rangel Smith; Christopher K.I. Williams; Giovanni Colavizza

Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.12929 v1 pith:3ROKOVLF submitted 2020-04-27 cs.DB

Alfredo Nazabal , Christopher K.I. Williams , Giovanni Colavizza , Camila Rangel Smith , Angus Williams This is my paper · ORCID

classification cs.DB

keywords dataengineeringanalystclassificationhelptasksacquiringanalyses

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

Consider the situation where a data analyst wishes to carry out an analysis on a given dataset. It is widely recognized that most of the analyst's time will be taken up with \emph{data engineering} tasks such as acquiring, understanding, cleaning and preparing the data. In this paper we provide a description and classification of such tasks into high-levels groups, namely data organization, data quality and feature engineering. We also make available four datasets and example analyses that exhibit a wide variety of these problems, to help encourage the development of tools and techniques to help reduce this burden and push forward research towards the automation or semi-automation of the data engineering process.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

FlowETL: An Autonomous Example-Driven Pipeline for Data Engineering
cs.SE 2025-07 conditional novelty 5.0 of 10

FlowETL uses LLMs and a small target dataset to automatically infer and apply data-cleaning transformations, reporting high data-quality scores across 14 datasets.

Pith tools