Pith. sign in

REVIEW 4 major objections 5 minor 32 references

Variability-Aware Machine Learning Model Selection: Feature Modeling, Instantiation, and Experimental Case Study

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that encoding ML model selection as feature diagrams with logical constraints, instantiated from Scikit-Learn's flowchart, makes selection explicit and interpretable and can recommend algorithms that beat published…

desk verdict Plausible variability-modeling approach for ML model selection; the case study is too weak to support the outperformance claim. read the letter →

arxiv 2501.00532 v1 pith:23NRCTZC submitted 2024-12-31 cs.SE

Cristina Tavares , Nathalia Nascimento , Paulo Alencar , Donald Cowan This is my paper · ORCID
classification cs.SE
keywords variabilitymodelingfeaturediagramsmachinelearningmodelselectionalgorithmheuristicsScikit-LearnexperimentalcasestudysoftwareengineeringforML
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a variability-aware design approach for machine learning model selection. The authors claim that selection factors—sample size, data type, prediction type, labeling, and performance—can be captured as feature diagrams and logical constraints, and that instantiating these diagrams from an existing heuristic such as the Scikit-Learn flowchart turns an ad hoc choice into an explicit, adaptive, and interpretable one. In a case study on a 299-patient heart-failure dataset, the approach recommends Linear SVC, which reaches $F_1=0.780$ against 0.746 for the Random Forest reported in the comparison study. The point of the work is that model selection procedures can be treated as design artifacts: represented, compared, reused, and reconfigured.

What carries the argument

The central object is the feature diagram, a tree-like variability model in which mandatory, optional, or, and alternative features capture selection factors, and cross-tree logical constraints capture dependencies between them. The paper instantiates generic feature diagrams for ML modeling technique selection and modeling assumptions into Scikit-Learn-specific diagrams, then encodes the flowchart's rules as constraints C1-C6, covering prediction type, modeling category, regression, dimensionality reduction, classification, and clustering. This machinery turns a heuristic into a formal, reconfigurable specification: given dataset attributes, the constraints fire in order to restrict the candidate algorithm set and produce a recommendation queue.

What would settle it

A decisive check is to replay every decision path of the Scikit-Learn flowchart through constraints C1-C6; any path where the constraints fire a different algorithm than the flowchart, such as C1's equivalence failing for a dataset that predicts both a category and a quantity, would show the instantiation is unfaithful. Separately, re-running the heart-failure experiment with repeated stratified cross-validation would show whether Linear SVC's $F_1=0.780$ advantage over Random Forest's 0.746 replicates or is an artifact of one split.

Watch

Extended reading notes

Core claim

The central claim is that the commonalities and variations in ML model selection can be formally represented as feature models, and that an existing selection procedure can be instantiated in those models. The paper builds two feature diagrams from the Scikit-Learn 1.0.1 flowchart—one for modeling techniques and one for modeling assumptions—and expresses the flowchart rules as a set of logical constraints (C1-C6). Applying the instantiated model to the heart-failure survival dataset identifies the problem as classification and recommends Linear SVC, KNeighbors Classifier, SVC, and ensemble classifiers, with Linear SVC first. Evaluated with the same 80/20 stratified split as the comparison study, Linear SVC achieves $F_1=0.780$, balanced accuracy 0.848, Matthews correlation 0.672, sensitivity 0.854, and specificity 0.842, outperforming the Random Forest result ($F_1=0.746$) reported in the baseline paper.

Load-bearing premise

The approach works only if the encoded heuristics correctly capture how a good model should be chosen, and the paper does not formally verify that the Scikit-Learn flowchart rules or their logical encoding are right.

Editorial extensions

If this is right

  • A model selection heuristic encoded as a feature diagram can be inspected, compared with other heuristics, and reused across projects instead of being reinvented ad hoc.
  • Changes in dataset attributes, such as the sample size crossing 100K or the prediction target switching from category to quantity, automatically trigger a different algorithm recommendation by re-instantiating the diagram.
  • The encoded constraints can be used to prune the algorithm search space before running experiments, reducing the cost of approaches that train many models in parallel.
  • The same experimental template can be applied to other published studies: extract the dataset assumptions, run the recommended algorithms, and compare metrics against the paper's reported models.
  • The approach provides an explicit rationale for a selection, in the form of the constraints that fired, supporting accountability and explainability for non-expert users.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the encoding is faithful, this design pattern extends beyond Scikit-Learn: heuristics from other platforms could be captured in the same formalism and compared systematically, which the paper lists as future work.
  • The failure of a recommended model could be fed back as the 'not working' condition, turning the static flowchart into a runtime adaptation loop driven by monitored performance.
  • The logical constraint layer could itself be checked mechanically against the source flowchart, and inconsistencies in the encoding would be caught as part of the instantiation step rather than left implicit.
  • The F1 comparison rests on a single split; a natural strengthening would be repeated stratified cross-validation to confirm that the recommended algorithm's advantage is not split-specific.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a variability-aware approach to ML model selection in which contextual factors (sample size, prediction type, labeled data, text data, etc.) are captured as feature diagrams with cross-tree logical constraints. The approach is structured into five phases: identifying and modeling variabilities, instantiating a feature diagram for a specific heuristic (here, the scikit-learn flowchart), designing an experimental case study, selecting modeling techniques, and evaluating results. The authors instantiate the scikit-learn selection heuristics as constraints C1–C6, then apply the approach to a heart-failure survival prediction dataset used in prior studies by Chicco and Jurman and by Leenings et al. Their concrete empirical claim is that the recommended model, LinearSVC, outperforms the baselines in those papers, reporting F1 = 0.780 versus 0.746 for Leenings et al.'s Random Forest. The paper positions this as a step toward explicit, adaptive, transparent, and interpretable model selection.

Significance. If the approach is validated, it would offer a useful software-engineering contribution: a formal, reusable notation for encoding model-selection heuristics, with the potential to make selection decisions explainable and automatable. The paper has several strengths: it grounds the feature model in an external, public heuristic (the scikit-learn flowchart), it separates the modeling phases clearly, and it candidly acknowledges threats to validity, including the lack of formal evaluation of the underlying heuristics. However, the empirical demonstration is currently too weak to support the outperformance claim, and several logical errors in the constraint formalization undermine the claim of faithful representation. The central design idea is defensible, but the case-study evidence and the constraint semantics need substantial rework before the paper's claims can be accepted.

major comments (4)
  1. [Section IV, Phase E] The claimed outperformance of LinearSVC (F1 = 0.780) over Leenings et al. (F1 = 0.746) is not statistically grounded. The evaluation uses a single stratified 80/20 split (60 test instances, roughly 19 positive cases), so a difference of 0.034 in F1 can correspond to one or two individual predictions. No repeated splits, random seeds, confidence intervals, or effect sizes are reported. Moreover, the 'not working' threshold of 0.77 was chosen because the baseline F1 was 0.76, which means the evaluation rule already encodes the expectation of beating the baseline. The conclusion in Phase E should be weakened to a demonstration of the workflow, or the experiment should be redone with repeated resampling and a pre-specified threshold.
  2. [Section III.B, constraints C1 and C6.4] The logical encoding of the scikit-learn flowchart contains substantive errors. C1, 'Category ⇔ ¬(Quantity ∧ Structure)', does not exclude the quantity case: if Quantity = true and Structure = false, the right-hand side is true and Category can also be true. An exclusive-or among Category, Quantity, and Structure requires a different formula. C6.4, 'Clustering ∧ ¬Knowncategories ⇒ Samplesize < 10K', is not entailed by the scikit-learn flowchart and would incorrectly constrain the sample size for clustering problems without known categories. Because the approach's recommendations inherit the semantics of these constraints, the errors must be corrected and the corrected constraints checked against the flowchart.
  3. [Section IV, Phase E and Table I] The baseline numbers are internally inconsistent. The prose states that Chicco and Jurman 'achieved a lower F1 score of 0.547', while Table I reports an F1 of 0.714 for their Logistic Regression estimator. The text also gives a somewhat different description of which estimators Leenings et al. tested. These inconsistencies must be reconciled before the claimed 0.780 versus 0.746 comparison can be assessed, since the entire case study rests on comparing against those baselines.
  4. [Section V.B] The paper acknowledges that the scikit-learn heuristics 'have not been formally evaluated in terms of the parameters they use and their correctness'. This is a significant limitation for a method whose central claim is to provide a faithful, explicit representation of model-selection procedures: if the encoded constraints are wrong or incomplete, every recommended algorithm inherits that error. The threats-to-validity discussion should be expanded to state what evidence would validate the encoding, for example, a comparison against expert-selected algorithms or against exhaustive search on a range of datasets.
minor comments (5)
  1. [Section I, Paper Structure] The paper-structure paragraph says 'Section IV presents our variability-aware ML model selection approach' and then 'Section IV presents an experimental case study'; the approach is actually in Section III and the case study in Section IV, so the paragraph should be corrected.
  2. [Throughout] There are several typographical and naming inconsistencies, including 'Futher' in the abstract, 'variabiliy' in Section II, 'Enseble Classifiers' in Section IV, 'mattews corr' in the Phase E output, and inconsistent spelling of 'Leenings' as 'Leening'.
  3. [Section III.B] The constraints C1–C6 use inconsistent numbering (C1 has one item, C2 has four, C5 has six); giving each constraint a unique identifier would make the formalization easier to reference and verify.
  4. [Section III.B and Phase E] The 'not working' criterion is introduced as a feature in the instantiated diagram, but its operationalization as an F1 threshold appears only in Phase E; the threshold choice should be discussed where the feature diagram is defined, since it is part of the encoded heuristic.
  5. [Section IV] The case study would benefit from a data and code availability statement; the dataset is public and the tools are standard, so sharing the exact split, grid-search configuration, and evaluation script would substantially improve reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

The approach's model-suggestion chain is non-circular (it encodes the external Scikit-Learn flowchart), but the case study's headline outperformance claim is partially constructed: the 'not working' threshold is set just above the baseline that the selected model is then said to beat.

  1. fitted input called prediction [Section IV, Phase E ('Evaluate Selected Techniques')]
    "We assumed the F1-score below a specific value represented the ”not working” criteria. The F1-score goal was set as 0.77, taking into consideration the baseline of 0.76, which was the highest F1-score found in Leening et al.’s work [14]. ... According to our evaluation, the Linear SVC algorithm outperforms the results presented in [14]."

    The 'not working' threshold is calibrated to the same baseline used as the comparison target: 0.77 = 0.76 + 0.01. Under the paper's decision rule, a suggested algorithm is accepted only when its F1-score is at least 0.77, so any accepted 'working' model is guaranteed to have F1 above the 0.76 baseline. The conclusion that Linear SVC 'outperforms' [14] is therefore entailed by the stopping rule, not demonstrated by an independent comparison. The measured 0.780 is a real number, but the qualitative claim of superiority is built into the threshold choice, making this part of the evaluation circular by construction.

full rationale

The central derivation is not circular. The feature diagrams and constraints in Phase B are explicitly built from the external Scikit-Learn 1.0.1 flowchart, and the Phase D suggestions (LinearSVC, KNeighborsClassifier, SVC, EnsembleClassifiers) follow from those constraints; the self-citations [2,3] are background only and are not load-bearing. The one concrete reduction I can exhibit is in Phase E: the 'not working' F1 goal of 0.77 was chosen because the baseline was 0.76, so the rule that selects the reported model already requires it to beat the baseline. This makes the empirical outperformance claim partly constructed, although the feature-model formalism itself remains an independent contribution. The paper's own statement that the heuristics 'have not been formally evaluated' is a correctness/validity limitation, not a circular step, and I do not count it as one.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The approach rests on the authority of the scikit-learn heuristics and on the adequacy of feature models as a representation language. The only fitted quantity is the 'not working' F1 threshold. No new physical or conceptual entities are postulated beyond the modeling constructs themselves.

free parameters (1)
  • F1-score not-working threshold = 0.77
    Set as the stopping criterion for the Scikit-Learn 'not working' branch; chosen based on the 0.76 F1 baseline from Leenings et al., so it is fitted to the comparison target.
assumptions (3)
  • domain assumption Scikit-Learn flowchart heuristics are a valid basis for model selection
    The entire approach encodes these heuristics; the paper acknowledges in Section V.B that 'these heuristics have not been formally evaluated in terms of the parameters they use and their correctness'.
  • ad hoc to paper Feature diagrams and logical constraints can faithfully represent ML model selection procedures
    This is the core modeling assumption of the approach; no formal semantics or verification of the mapping from flowchart to constraints is provided.
  • ad hoc to paper The constraints C1-C6 correctly encode the Scikit-Learn flowchart
    The constraints are manually extracted and contain logical errors, e.g., C1 and C6.4, so this assumption is questionable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Variability-Aware Machine Learning Model Selection: Feature Modeling, Instantiation, and Experimental Case Study." pith.science (2026). https://pith.science/paper/23NRCTZC

@misc{pith2026250100532,
  author       = {Pith},
  title        = {Pith review of: Variability-Aware Machine Learning Model Selection: Feature Modeling, Instantiation, and Experimental Case Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23NRCTZC}},
  note         = {Machine review of arXiv:2501.00532}
}
read the original abstract

The emergence of machine learning (ML) has led to a transformative shift in software techniques and guidelines for building software applications that support data analysis process activities such as data ingestion, modeling, and deployment. Specifically, this shift is impacting ML model selection, which is one of the key phases in this process. There have been several advances in model selection from the standpoint of core ML methods, including basic probability measures and resampling methods. However, from a software engineering perspective, this selection is still an ad hoc and informal process, is not supported by a design approach and representation formalism that explicitly captures the selection process and can not support the specification of existing model selection procedures. The selection adapts to a variety of contextual factors that affect the model selection, such as data characteristics, number of features, prediction type, and their intricate dependencies. Further, it does not provide an explanation for selecting a model and does not consider the contextual factors and their interdependencies when selecting a technique. Although the current literature provides a wide variety of ML techniques and algorithms, there is a lack of design approaches to support algorithm selection. In this paper, we present a variability-aware ML algorithm selection approach that considers the commonalities and variations in the model selection process. The approach's applicability is illustrated by an experimental case study based on the Scikit-Learn heuristics, in which existing model selections presented in the literature are compared with selections suggested by the approach. The proposed approach can be seen as a step towards providing a more explicit, adaptive, transparent, interpretable, and automated basis for model selection.

Figures

Figures reproduced from arXiv: 2501.00532 by the authors.

Figure 1
Figure 1. Variability-aware ML algorithm selection approach. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Feature diagram for ML modeling technique selection. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Feature diagram for ML modeling assumptions. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Scikit-Learn flowchart for selecting a machine learning algorithm. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Feature diagram of the instance of a ML algorithm selection from Scikit-Learn. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Feature diagram of the instance of a ML algorithm assumptions from Scikit-Learn. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Feature model representing an adaptation triggered by changes in the application dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Feature model representing an adaptation triggered by changes related to design decisions. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 29 canonical work pages

  1. [1]

    Current approaches for executing big data science projects—a systematic literature review,

    J. S. Saltz and I. Krasteva, “Current approaches for executing big data science projects—a systematic literature review,” PeerJ Computer Science, vol. 8, p. e862, 2022

    work page 2022

  2. [2]

    Towards automated variability-aware machine-learning-based modeling analysis,

    C. Tavares, N. Nascimento, P. Alencar, and D. Cowan, “Towards automated variability-aware machine-learning-based modeling analysis,” in 2021 IEEE International Conference on Big Data (Big Data) . IEEE, 2021, pp. 3890–3896

    work page 2021

  3. [3]

    Adaptive method for machine learning model selection in data science projects,

    ——, “Adaptive method for machine learning model selection in data science projects,” in 2022 IEEE International Conference on Big Data (Big Data). IEEE, 2022, pp. 2682–2688

    work page 2022

  4. [4]

    Variability in software systems—a systematic literature review,

    M. Galster, D. Weyns, D. Tofan, B. Michalik, and P. Avgeriou, “Variability in software systems—a systematic literature review,” IEEE Transactions on Software Engineering, vol. 40, no. 3, pp. 282–306, 2013

    work page 2013

  5. [5]

    A taxonomy of variability realization techniques,

    M. Svahnberg, J. Van Gurp, and J. Bosch, “A taxonomy of variability realization techniques,” Software: Practice and experience , vol. 35, no. 8, pp. 705–754, 2005

    work page 2005

  6. [6]

    Ruva: A runtime software variability algorithm,

    A. Valdezate, R. Capilla, J. Crespo, and R. Barber, “Ruva: A runtime software variability algorithm,” IEEE Access, 2022

    work page 2022

  7. [7]

    B ¨uhne and K

    S. B ¨uhne and K. Pohl, Domain Requirements Engineering . Berlin, Heidelberg: Springer Berlin Heidelberg, 2005, pp. 193–216. [Online]. Available: https://doi.org/10.1007/3-540-28901-110

    work page doi:10.1007/3-540-28901-110 2005

  8. [8]

    A variability model for query optimizers,

    A. Caplinskas et al. , “A variability model for query optimizers,” in Databases and Information Systems VII: Selected Papers from the Tenth International Baltic Conference, DB & IS 2012 , vol. 249. IOS Press, 2013, p. 15

    work page 2012

Show all 32 references
  1. [9]

    Spl driven approach for vari- ability in database design,

    S. Bouarar, S. Jean, and N. Siegmund, “Spl driven approach for vari- ability in database design,” in Model and Data Engineering . Springer, 2015, pp. 332–342

    2015

  2. [10]

    Kang, Kyo C.and Lee, Variability Modeling

    H. Kang, Kyo C.and Lee, Variability Modeling. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 25–42. [Online]. Available: https://doi.org/10.1007/978-3-642-36583-6

    2013 doi

  3. [11]

    Crisp-dm 1.0: Step-by-step data mining guide,

    P. Chapman, J. Clinton, R. Kerber, T. Khabaza, T. Reinartz, C. Shearer, R. Wirth et al., “Crisp-dm 1.0: Step-by-step data mining guide,” SPSS inc, vol. 9, no. 13, pp. 1–73, 2000

    2000

  4. [12]

    An overview of the machine learning applied in smart cities,

    R. P. Franc ¸a, A. C. B. Monteiro, R. Arthur, and Y . Iano, “An overview of the machine learning applied in smart cities,” Smart cities: A data analytics perspective, pp. 91–111, 2021

    2021

  5. [13]

    Taking human out of learning applications: A survey on automated machine learning,

    Q. Yao, M. Wang, Y . Chen, W. Dai, Y .-F. Li, W.-W. Tu, Q. Yang, and Y . Yu, “Taking human out of learning applications: A survey on automated machine learning,” arXiv preprint arXiv:1810.13306 , 2018

    2018 arXiv

  6. [14]

    Photonai—a python api for rapid machine learning model development,

    R. Leenings, N. R. Winter, L. Plagwitz, V . Holstein, J. Ernsting, K. Sarink, L. Fisch, J. Steenweg, L. Kleine-Vennekate, J. Gebker et al. , “Photonai—a python api for rapid machine learning model development,” Plos one, vol. 16, no. 7, p. e0254062, 2021

    2021

  7. [15]

    Scikit-learn: Machine learning in python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg et al. , “Scikit-learn: Machine learning in python,”Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011

    2011

  8. [16]

    Machine learning algorithm cheat sheet for azure machine learning designer,

    Microsoft, “Machine learning algorithm cheat sheet for azure machine learning designer,” https://docs.microsoft.com/en-us/azure/machine- learning/algorithm-cheat-sheet, June 2022

    2022

  9. [17]

    What is a feature? a qualitative study of features in industrial software product lines,

    T. Berger, D. Lettner, J. Rubin, P. Gr ¨unbacher, A. Silva, M. Becker, M. Chechik, and K. Czarnecki, “What is a feature? a qualitative study of features in industrial software product lines,” in Proceedings of the 19th International Conference on Software Product Line , 2015, ...

    2015

  10. [18]

    Mlops- definitions, tools and challenges,

    G. Symeonidis, E. Nerantzis, A. Kazakis, and G. A. Papakostas, “Mlops- definitions, tools and challenges,” in 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC) . IEEE, 2022, pp. 0453–0460

    2022

  11. [19]

    Scalable architecture for automating machine learning model monitoring,

    J. R. Mart ´ınez, “Scalable architecture for automating machine learning model monitoring,” Ph.D. dissertation, KTH Royal Institute of Technol- ogy, Stockholm, Sweden, 2020

    2020

  12. [20]

    Context-aware data ana- lytics variability in iot neural network-based systems,

    N. Nascimento, P. Alencar, and D. Cowan, “Context-aware data ana- lytics variability in iot neural network-based systems,” in 2021 IEEE International Conference on Big Data (Big Data) . IEEE, 2021, pp. 3595–3600

    2021

  13. [21]

    Project, EDISON Data Science Framework (EDSF)

    E. Project, EDISON Data Science Framework (EDSF) . Http://Edison- Project.Eu/Edison/Edison-Data-Science-Framework-Edsf, 2017

    2017

  14. [22]

    Machine learning algorithms and their relationship with modern technologies,

    P. Kashyap, “Machine learning algorithms and their relationship with modern technologies,” in Machine Learning for Decision Makers . Berkeley, CA: Apress, 2023, pp. 165–245

    2023

  15. [23]

    Explainability of a machine learning granting scoring model in peer- to-peer lending,

    M. J. Ariza-Garz ´on, J. Arroyo, A. Caparrini, and M.-J. Segovia-Vargas, “Explainability of a machine learning granting scoring model in peer- to-peer lending,” Ieee Access, vol. 8, pp. 64 873–64 890, 2020

    2020

  16. [24]

    Opening the black box of artificial intelligence for clinical decision support: A study predicting stroke outcome,

    E. Zihni, V . I. Madai, M. Livne, I. Galinovic, A. A. Khalil, J. B. Fiebach, and D. Frey, “Opening the black box of artificial intelligence for clinical decision support: A study predicting stroke outcome,” Plos one, vol. 15, no. 4, p. e0231166, 2020

    2020

  17. [25]

    Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone,

    D. Chicco and G. Jurman, “Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone,” BMC medical informatics and decision making , vol. 20, no. 1, pp. 1–16, 2020

    2020

  18. [26]

    Deequ-data quality validation for machine learning pipelines,

    S. Schelter, S. Grafberger, P. Schmidt, T. Rukat, M. Kiessling, A. Tap- tunov, F. Biessmann, and D. Lange, “Deequ-data quality validation for machine learning pipelines,” in Machine Learning Systems Workshop at the Conference on Neural Information Processing Systems (NeurIPS) , 2018

    2018

  19. [27]

    Modelops: Cloud-based lifecycle management for reliable and trusted ai,

    W. Hummer, V . Muthusamy, T. Rausch, P. Dube, K. El Maghraoui, A. Murthi, and P. Oum, “Modelops: Cloud-based lifecycle management for reliable and trusted ai,” in 2019 IEEE International Conference on Cloud Engineering (IC2E) . IEEE, 2019, pp. 113–120

    2019

  20. [28]

    Learning under concept drift: A review,

    J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang, “Learning under concept drift: A review,” IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 12, pp. 2346–2363, 2018

    2018

  21. [29]

    Automl to date and beyond: Challenges and opportunities,

    S. K. Karmaker, M. M. Hassan, M. J. Smith, L. Xu, C. Zhai, and K. Veeramachaneni, “Automl to date and beyond: Challenges and opportunities,” ACM Computing Surveys (CSUR) , vol. 54, no. 8, pp. 1–36, 2021

    2021

  22. [30]

    Software engineering for machine learning: A case study,

    S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Na- gappan, B. Nushi, and T. Zimmermann, “Software engineering for machine learning: A case study,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SE...

    2019

  23. [31]

    Mon- itoring and explainability of models in production,

    J. Klaise, A. Van Looveren, C. Cox, G. Vacanti, and A. Coca, “Mon- itoring and explainability of models in production,” arXiv preprint arXiv:2007.06299, 2020

    2007 arXiv

  24. [32]

    Ensemble classifiers: Api reference,

    Scikit, “Ensemble classifiers: Api reference,” https://scikit- learn.org/stable/modules/classes.htmlmodule-sklearn.ensemble, May 2022

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.