Pith. sign in

REVIEW 4 major objections 8 minor 38 references

Tabular Numeric Stretch Transformation

T0 review · 4 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that numeric tabular features should be preprocessed to make the target function smoother, and that the resulting supervised stretch transformation consistently outperforms all baselines across 38 benchmark datasets.

desk verdict A useful empirical study and a clean geometric observation, but the Dirichlet-energy theory is really a lower-bound surrogate in disguise and the abstract's 'consistently outperforms' claim overshoots the data. read the letter →

arxiv 2608.09162 v1 pith:NVR7ZIVX submitted 2026-08-10 cs.LG cs.AI

Zihao Ye , Juyong Kim , Johnna Sundberg , Burak Varici , Pradeep Ravikumar This is my paper · ORCID
classification cs.LGcs.AI
keywords tabulardatafeaturetransformationDirichletenergysupervisedpreprocessingpiecewiselinearencodingtargetspectralbiasregression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most numeric preprocessing for tabular data is unsupervised: it reshapes feature distributions without looking at the target. This paper argues that the right goal is instead to make the target function easier for a neural network to learn, and that this reduces to allocating transformed-space width across bins of each feature. Its supervised stretch method minimizes a discrete Dirichlet energy of the target in the transformed space, giving a closed-form width per bin proportional to the bin's total target variation. The paper reports that on 38 TALENT datasets, across five architectures, supervised stretch consistently outperforms all seven baselines, with the largest gains in regression. If true, this establishes target-function smoothness as a design principle for tabular preprocessing rather than an incidental byproduct of distribution matching.

What carries the argument

The central object is the stretch transformation, a strictly increasing, piecewise-linear map s(x)=c_{t-1}+(x-b_{t-1})/(b_t-b_{t-1}) w_t that sends each numeric feature into [0,1] while preserving order; its design problem is the width vector {w_t}. The argument is carried by the discrete Dirichlet energy E_disc = sum_i ||Δf_i||$_2^{2}$ / Δy_i, which measures how much the target changes per unit of transformed space. Because substituting the piecewise-linear map into this energy is numerically unstable, the paper replaces each bin's within-bin contribution by the Cauchy-Schwarz lower bound $S_t^{2}$/w_t, turning the width optimization into the convex problem min sum $S_t^{2}$/w_t with sum w_t=1, whose closed-form solution is w_t* = S_t / sum_u S_u. This identity, width proportional to bin-wise total target variation, is what makes the method tractable and is the mechanism behind all theoretical connections and empirical gains.

What would settle it

On a real dataset, compute the exact discrete Dirichlet energy E_disc(w) = sum_t (b_t-b_{t-1})/w_t * sum_{i in I_t} ||Δf_i||$_2^{2}$/Δx_i under the piecewise-linear stretch map for a grid of width vectors w, and compare its minimizer to the surrogate-optimal w_t* = S_t/sum S_u. If for many features the exact minimizer differs substantially from w_t* while the surrogate energy is lower at w_t*, then the lower-bound surrogate is not faithfully optimizing the paper's own smoothness objective, even if the empirical gains remain.

Watch

Extended reading notes

Core claim

The central claim is that the learnability of a numeric feature for a neural network is governed by the smoothness of the target function in the transformed feature space, and that smoothness can be optimized directly by a monotone piecewise-linear 'stretch' map. For a feature partitioned into T quantile bins, the map assigns each bin a width w_t. Supervised stretch chooses widths to minimize the discrete Dirichlet energy of the target's conditional expectation; because the exact piecewise-linear objective is numerically unstable, the paper minimizes a Cauchy-Schwarz lower bound and obtains w_t* = S_t / sum_u S_u, where S_t is the total variation of the estimated target within bin t. This concentrates resolution where the target varies rapidly, equalizing slope magnitudes across bins. In the unsupervised case, minimax reasoning over bounded-variation targets yields uniform widths, recovering a piecewise-linear approximation of the empirical CDF and, by arc-length parameterization, an equivalent of Piecewise Linear Encoding. The paper further shows that in the fine-binning limit supervised stretch approaches target encoding, giving a theoretical explanation for why target-based transformations work.

Load-bearing premise

The load-bearing premise is that the Cauchy-Schwarz lower bound is a faithful stand-in for the true Dirichlet energy of the piecewise-linear transformation, even though the true objective is numerically unstable and the surrogate ignores the within-bin constant-slope constraint.

Editorial extensions

If this is right

  • If supervised stretch is right, target-aware preprocessing is a repeatable performance lever: across 38 datasets and five architectures it beats all seven baselines, and on regression tasks the margin is decisive.
  • Unsupervised stretch gives a target-free method that matches or exceeds piecewise linear encoding while using O(1) memory per feature and no dimensional expansion, so the benefit of density reshaping does not require PLE's computational cost.
  • The framework provides explanations for existing empirical findings: uniform stretch approximates empirical CDF transformation as T grows, explaining why CDF preprocessing reduces high-frequency content; supervised stretch approaches target encoding in the fine-binning limit, explaining why target-based transforms help.
  • The bin count T acts as a regularization parameter: small T preserves distributional structure, large T approaches full CDF (unsupervised) or target-encoding-like behavior (supervised), giving a controlled interpolation between extremes.
  • Because bin widths are proportional to bin-wise total target variation, supervised stretch tends to equalize the target function's slope magnitudes across bins, which is the concrete sense in which the learned function becomes smoother.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Testable extension: learn the widths w_t jointly with the network parameters instead of fixing them before training; the smoothness argument suggests differentiable width optimization could improve on the closed-form allocation, since the current method optimizes preprocessing independently of the learned function.
  • The spectral-bias explanation implies an interaction with network scale: supervised stretch's gains should be larger for small or undertrained networks that have difficulty with high-frequency components, and shrink as capacity and training time grow; this is measurable with the paper's protocol.
  • The method is marginal, so a joint version that allocates width per feature while accounting for other features' target variation could address cases where the target changes mostly through interactions; the paper explicitly leaves interaction-awareness out of scope.
  • The Dirichlet-energy objective applies to any continuous target, so supervised stretch should transfer to multi-output regression, survival targets, or quantile targets without changing the formula; this is a direct corollary of the vector-valued target treatment noted in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper introduces "stretch transformations" for numeric tabular features. Unsupervised stretch is a piecewise-linear monotone map with uniform bin widths, derived from a minimax worst-case Dirichlet energy argument, and is claimed to be a memory-efficient alternative to PLE and an approximation to the empirical CDF. Supervised stretch allocates bin widths proportionally to the bin-wise total variation of an out-of-fold estimate of the target function, motivated by minimizing a lower bound on the target's discrete Dirichlet energy in the transformed space. The authors prove formal connections to PLE, CDF transformation, and target encoding in limiting cases, and report experiments on 38 TALENT datasets with five neural architectures, concluding that supervised stretch consistently outperforms all baselines, especially in regression.

Significance. The empirical study is extensive (38 datasets, 5 models, 100 Optuna trials, 15 seeds) and the out-of-fold estimation protocol is a thoughtful attempt to avoid label leakage. If the empirical results survive correction of the overclaim in the abstract, supervised stretch would be a useful addition to the tabular preprocessing toolbox. The theoretical connections to PLE and CDF (unsupervised) are elegant and largely correct. However, the central mechanistic claim—that supervised stretch minimizes the Dirichlet energy of the target function—is not established, because the method optimizes a surrogate lower bound that can deviate from the exact objective. The paper's contribution is therefore better described as an empirically driven heuristic with a plausible smoothness rationale, rather than a method derived from its stated optimization problem.

major comments (4)
  1. [Section 3.4 / Appendix A.3] The paper claims to minimize the discrete Dirichlet energy (Eq. 7) under the piecewise-linear stretch map, but the width allocation w_t ∝ S_t (Eq. 15) is derived from the Cauchy-Schwarz lower bound (Eq. 14), not from the exact objective. Under the piecewise-linear map, the exact within-bin energy is E_t = ((b_t-b_{t-1})/w_t) Σ_{i∈I_t} ||Δf_i||^2/Δx_i (Eq. 29), and minimizing Σ_t E_t over w_t gives w_t ∝ sqrt((b_t-b_{t-1}) Σ_{i∈I_t} ||Δf_i||^2/Δx_i). These two objectives can disagree: two bins with identical S_t but different within-bin sample densities receive equal widths under Eq. 15, while the exact optimizer gives more width to the denser bin. Since Appendix A.3 explicitly calls Eq. (14) a "robust lower-bound surrogate," the abstract and Section 4.2 (Finding 1) overstate the theoretical contribution when they attribute the empirical gains to "minimizing the Dirichlet energy." The paper should either provide empirical evidence that the surrogate tracks the exact energy on representative data, or reframe the theoretical claim as a heuristic width allocation.
  2. [Abstract / Section 4.2 / Table 1] The claim that supervised stretch "consistently outperforms all baselines" is not supported by Table 1. For MLP, the Sup. Overall Score is 0.6343, below PLE (0.6835), RS-SC (0.6653), and Standard (0.6423); for MLP-PLR, Sup. (0.6238) is below Unsup. (0.7347) and Standard (0.6443); and for ResNet Avg. R^2, Sup. (0.651) is below Minmax (0.655). The paper should replace "consistently outperforms" with a precise, qualified statement (e.g., strongest aggregate performance in regression, top-tier in classification) or provide a formal meta-analysis across the 190 dataset-model combinations that quantifies how often supervised stretch wins.
  3. [Sections 1, 3.3, 4.2] The claim that unsupervised stretch requires only O(1) memory per feature instead of O(T) is incorrect: a piecewise-linear map with T bins stores T+1 boundary values and T widths (or equivalently T slopes), which is O(T) memory per feature. The O(1) quantity is the output dimensionality (a scalar) compared to PLE's T-dimensional encoding, which is a different statement. This should be corrected in the contribution list and in Section 4.2, Finding 2.
  4. [Section 3.1 / Section 4.1] The transformation definition uses the dataset range [x_min, x_max] and quantile binning to set boundaries, and the evaluation protocol says the test split is used only after configuration selection. It should be stated explicitly that all transformation parameters (bin boundaries, bin widths, and the out-of-fold regression estimates used for supervised stretch) are fit on the training split only and then applied unchanged to validation and test splits. Without this clarification, it is difficult to rule out feature/target leakage in the reported results.
minor comments (8)
  1. [Abstract] The phrase "thestretch transformation framework" is a typo; it should read "the stretch transformation framework."
  2. [Section 2] The sentence beginning "A feature transformations" should be singular ("A feature transformation"), and the following sentence beginning "but assumes" should start with a capital letter.
  3. [Section 3.3] The minimax derivation introduces the assumption ||Δf_i||^2 ≤ C without discussing how C is chosen or why the resulting allocation is independent of C; a sentence stating that C cancels out would clarify the argument.
  4. [Section 3.4] The connection to target encoding via Eq. (16) is informal; the "remarkably similar" relationship to min-max scaled target encoding should be made precise, since for non-monotonic target functions the mapped coordinates are cumulative target variation, not the target values themselves.
  5. [Appendix A.1] The integration-by-parts step in Eq. (18) assumes Neumann boundary conditions g'(0)=g'(1)=0; these boundary conditions should be stated in the main text where the Dirichlet energy is introduced.
  6. [Table 4] Some dataset names appear to contain typos (e.g., "E-CommereShippingData", "Biascorrectionr", "polreg"), which should be checked against the source benchmark.
  7. [Appendix B.2] The PLE fallback scheme replaces PLE with standardization on some dataset-model combinations; the paper should list which combinations are affected, because those cells no longer evaluate PLE.
  8. [Figure 2] The caption and text refer to "the adaptive threshold used in Table 1" but the figure sweeps fixed thresholds; the relationship would be clearer if the adaptive threshold value were marked on the plot.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the width allocation is an explicitly acknowledged lower-bound surrogate of the Dirichlet-energy objective, and the empirical claims are evaluated on held-out TALENT test partitions; no prediction reduces to its input by construction.

full rationale

Walking the derivation chain with the paper's own equations: (1) The kernel-smoothness-to-Dirichlet-energy reduction (Eqs. 4-6) is standard external mathematics (Varadhan [31], Molchanov [24]); the discrete energy (Eq. 7) follows from the piecewise-linear interpolant by construction, and the unsupervised minimax argument (Appendix A.2) is self-contained, yielding uniform widths. (2) The supervised widths w_t = S_t / Sum S_u (Eq. 15) are exact minimizers of the Cauchy-Schwarz lower-bound surrogate (Eqs. 13-14), which the paper explicitly frames as a lower bound ('we instead optimize a robust lower bound on the energy', Section 3.4) and admits in Appendix A.3 is 'a robust bin-level surrogate rather than the exact unrestricted sample-spacing optimum'. The gap between this surrogate and the exact piecewise-linear objective (Eq. 29) is a correctness/fidelity concern, not circularity: no fitted parameter is relabeled as a prediction, and target information enters only through out-of-fold estimates (Appendix B.1), with the transformation fixed before test evaluation. (3) The connections to CDF, PLE, and target encoding (Eqs. 10-12, 16) are identities derived from the construction: the PLE arc-length relation is a mathematical equivalence, and the target-encoding limit is explicitly acknowledged as 'remarkably similar to applying min-max scaling to target encoding', making them explanatory rather than predictions forced by definition. (4) There are no self-citations; load-bearing external supports (TALENT [23,35], spectral bias [28,34], heat kernel [24,31]) are independent benchmarks or standard results. (5) The empirical claim is self-contained: 38 TALENT datasets, official splits, 15-seed evaluation, Optuna tuning on validation only, and a symmetric significance filter (Eq. 37). Non-circularity caveats for the reader: the abstract's claim that supervised stretch 'consistently outperforms all baselines' overstates Table 1 (PLE exceeds Sup. in the MLP panel, and Unsup. exceeds Sup. in the MLP-PLR panel), and the 'theoretical justification' of target encoding is conditional on the paper's own smoothness-helps assumption, which the surrogate analysis does not independently validate.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central method depends on the number of bins T and on the out-of-fold kernel estimates, which bring parameters k and K. The theoretical framework rests on the spectral-bias assumption, the heat-kernel expansion, and an ad hoc bounded-variation constraint. No new entities are introduced.

free parameters (3)
  • Number of bins T = Tuned per dataset/model by Optuna from {1,2,4,8,16,32,64,128,256,512,100000} for stretch; int[2,256] for PLE
    T controls the granularity of the piecewise linear map and acts as the regularization parameter for supervised stretch. It is chosen by validation, not derived.
  • k (neighbors for adaptive Nadaraya-Watson bandwidth) = Not specified in the main text, likely tuned or fixed in implementation
    The out-of-fold target estimates bf(x_i) that determine the supervised bin widths depend on the kernel bandwidth, which is set by k. Unspecified in the paper.
  • K (number of folds in out-of-fold estimation) = K=10, or fewer if a class is too small (classification); standard K-fold for regression
    The out-of-fold protocol depends on K, affecting the bias-variance tradeoff of the target estimates.
assumptions (5)
  • domain assumption ReLU networks learn low-frequency components before high-frequency components (spectral bias).
    This motivates the smoothness objective. Cited from [28,34] and treated as established.
  • standard math Heat kernel expansion T_sigma = e^{sigma^2/2 d^2/dy^2} for small bandwidth sigma.
    Used in Section A.1 to connect kernel smoothing to Dirichlet energy. Standard result from [31,24].
  • domain assumption Neumann boundary conditions g'(0)=g'(1)=0 for integration by parts.
    Required for the integration-by-parts step in Eq. (18). The target function's derivative is not guaranteed to vanish at the boundaries.
  • ad hoc to paper Bounded local variation: ||Delta f_i||_2^2 <= C for all adjacent sample pairs in the minimax derivation.
    This is a worst-case assumption introduced to make the unsupervised objective tractable; it is not derived from data or prior theory.
  • domain assumption Quantile binning yields approximately equal sample counts per bin, so T bins each contain about n/T samples.
    Used to justify the uniform allocation w_t=1/T in the unsupervised derivation; approximate for finite n and requires distinct feature values.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tabular Numeric Stretch Transformation." pith.science (2026). https://pith.science/paper/NVR7ZIVX

@misc{pith2026260809162,
  author       = {Pith},
  title        = {Pith review of: Tabular Numeric Stretch Transformation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NVR7ZIVX}},
  note         = {Machine review of arXiv:2608.09162}
}
read the original abstract

Tabular data presents unique challenges for deep learning due to its heterogeneous nature, where numeric features exhibit diverse distributions, scales, and statistical properties. Although recent advances have improved how models learn from tabular data, how numeric data are transformed into model-friendly representations remains comparatively underexplored. We introduce the stretch transformation framework, which formulates numeric feature preprocessing as an optimization problem to make the target function smoother and thus more learnable. Our framework has two variants: (1) unsupervised stretch, which uniformly redistributes feature density via minimax optimization, and (2) supervised stretch, which optimizes target-aware numeric feature transformations from the perspective of target-function smoothness by minimizing the target function's Dirichlet energy in the transformed space. Our theoretical analysis further connects this framework to several popular transformations: unsupervised stretch is closely related to Piecewise Linear Encoding through a shared piecewise-linear geometry and approaches the empirical CDF transformation as the number of bins grows, while supervised stretch becomes closely related to target encoding in the fine-binning limit. Comprehensive experiments on 38 datasets from the TALENT benchmark demonstrate that supervised stretch consistently outperforms all baselines. These results show that explicitly optimizing for target function smoothness is a powerful and underexplored strategy for tabular deep learning.

Figures

Figures reproduced from arXiv: 2608.09162 by the authors.

Figure 1
Figure 1. Overview: Stretch Transformation Framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Sensitivity analysis of significance thresholds. Average normalized score is plotted against [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Pairwise win-rates between transformations, grouped by task. Each cell [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Stretch Supervised vs PLE [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Stretch Supervised vs PLE-T [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Stretch Supervised vs Standardization 18 [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Stretch Unsupervised vs PLE [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Stretch Unsupervised vs PLE-T [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Stretch Unsupervised vs Standardization 19 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: shows the aggregated mean rank as a function of noise level; Figures 11a–12b show per-dataset ranks. Supervised Stretch does not collapse under noise. The mean rank of Supervised Stretch (red) stays in a competitive range over the full noise sweep and does not show th…
Figure 11
Figure 11. Figure 11: Detailed Mean Rank Trajectories (1/2). Mean rank across 5 models for NHANES-age and Contaminant datasets under varying noise levels. (a) ibm-employee-performance (Classifica￾tion) (b) yeast (Classification) [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: Detailed Mean Rank Trajectories (2/2). Mean rank across 5 models for IBM-Employee and Yeast datasets under varying noise levels. H Additional Metrics For completeness, we report aggregated results across seven evaluation metrics: classification (Ac￾curacy, Average Rec…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 20 canonical work pages

  1. [1]

    Optuna: A next-generation hyperparameter optimization framework, 2019

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework, 2019. URLhttps://arxiv. org/abs/1907.10902

    arXiv 2019

  2. [2]

    Tabnet: Attentive interpretable tabular learning

    Sercan ¨O Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. InProceed- ings of the AAAI conference on artificial intelligence, volume 35, pages 6679–6687, 2021

    work page 2021

  3. [3]

    An inductive bias for tabular deep learning

    Ege Beyazit, Jonathan Kozaczuk, Bo Li, Vanessa Wallace, and Bilal H Fadlallah. An inductive bias for tabular deep learning. InThirty-seventh Conference on Neural Information Processing Systems, 2023. URLhttps://openreview.net/forum?id=XEUc1JegGt

    work page 2023

  4. [4]

    Deep neural networks and tabular data: A survey.IEEE Transactions on Neural Networks and Learning Systems, 35(6):7499–7519, June 2024

    Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey.IEEE Transactions on Neural Networks and Learning Systems, 35(6):7499–7519, June 2024. ISSN 2162-2388. doi: 10.1109/tnnls.2022.3229161. URLhttp://dx.doi.org/10.1109/TNNLS.2022. 3229161

    arXiv 2024

  5. [5]

    G. E. P. Box and D. R. Cox. An analysis of transformations.Journal of the Royal Statistical Society. Series B (Methodological), 26(2):211–252, 1964. ISSN 00359246. URLhttp: //www.jstor.org/stable/2984418

    work page arXiv 1964

  6. [6]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 785–794, New York, NY , USA, 2016. Association for Computing Machinery. ISBN 9781450342322. doi: 10.1145/2939672.2939785. URLhttps://doi.org/10. 1145/2939672.2939785

    arXiv 2016

  7. [7]

    Approximation by superpositions of a sigmoidal function.Mathematics of control, signals and systems, 2(4):303–314, 1989

    George Cybenko. Approximation by superpositions of a sigmoidal function.Mathematics of control, signals and systems, 2(4):303–314, 1989

    1989

  8. [8]

    Tabarena: A living benchmark for machine learning on tabular data, 2025

    Nick Erickson, Lennart Purucker, Andrej Tschalzev, David Holzm¨uller, Prateek Mutalik Desai, David Salinas, and Frank Hutter. Tabarena: A living benchmark for machine learning on tabular data, 2025. URLhttps://arxiv.org/abs/2506.16791

    arXiv 2025

Show all 38 references
  1. [9]

    Revisiting deep learning models for tabular data

    Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. InNeurIPS, 2021

    2021

  2. [10]

    On embeddings for numerical features in tabular deep learning

    Yury Gorishniy, Ivan Rubachev, and Artem Babenko. On embeddings for numerical features in tabular deep learning. InNeurIPS, 2022

    2022

  3. [11]

    Tabm: Advancing tabular deep learning with parameter-efficient ensembling

    Yury Gorishniy, Akim Kotelnikov, and Artem Babenko. Tabm: Advancing tabular deep learning with parameter-efficient ensembling. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id= Sd4wYYOhmY

    2025

  4. [12]

    Why do tree-based models still out- perform deep learning on typical tabular data?Advances in neural information processing systems, 35:507–520, 2022

    L ´eo Grinsztajn, Edouard Oyallon, and Ga ¨el Varoquaux. Why do tree-based models still out- perform deep learning on typical tabular data?Advances in neural information processing systems, 35:507–520, 2022. 10

    2022

  5. [13]

    Survey on categorical data for neural networks.Jour- nal of Big Data, 7, 04 2020

    John Hancock and Taghi Khoshgoftaar. Survey on categorical data for neural networks.Jour- nal of Big Data, 7, 04 2020. doi: 10.1186/s40537-020-00305-w

    2020 doi

  6. [14]

    Deep residual learning for image recognition, 2015

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. URLhttps://arxiv.org/abs/1512.03385

    2015 arXiv

  7. [15]

    Tabpfn: A trans- former that solves small tabular classification problems in a second

    Noah Hollmann, Samuel M ¨uller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A trans- former that solves small tabular classification problems in a second. InInternational Confer- ence on Learning Representations 2023, 2023

    2023

  8. [16]

    Accurate predictions on small data with a tabular foundation model.Nature, 01 2025

    Noah Hollmann, Samuel M ¨uller, Lennart Purucker, Arjun Krishnakumar, Max K¨orfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model.Nature, 01 2025. doi: 10.1038/s41586-024-08328-6. URLhttps: //www.nat...

    2025 doi

  9. [17]

    Better by default: Strong pre-tuned MLPs and boosted trees on tabular data

    David Holzm ¨uller, Leo Grinsztajn, and Ingo Steinwart. Better by default: Strong pre-tuned MLPs and boosted trees on tabular data. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id= 3BNPUDvqMt

    2024

  10. [18]

    Approximation capabilities of multilayer feedforward networks.Neural net- works, 4(2):251–257, 1991

    Kurt Hornik. Approximation capabilities of multilayer feedforward networks.Neural net- works, 4(2):251–257, 1991

    1991

  11. [19]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei, editors,Proceedings of the 32nd International Conference on Machine Learning, volume 37 ofProceedings of Machine Le...

    2015

  12. [20]

    Carte: pretraining and transfer for tabular learning.arXiv preprint arXiv:2402.16785, 2024

    Myung Jun Kim, L ´eo Grinsztajn, and Ga ¨el Varoquaux. Carte: pretraining and transfer for tabular learning.arXiv preprint arXiv:2402.16785, 2024

    2024 arXiv

  13. [21]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  14. [22]

    MNIST handwritten digit database

    Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URLhttp: //yann.lecun.com/exdb/mnist/

    2010

  15. [23]

    Talent: A tabular analytics and learning toolbox.arXiv preprint arXiv:2407.04057, 2024

    Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, and Han-Jia Ye. Talent: A tabular analytics and learning toolbox.arXiv preprint arXiv:2407.04057, 2024

    2024 arXiv

  16. [24]

    S. A. Molchanov. Diffusion Processes and Riemannian Geometry.Russian Mathematical Surveys, 30(1):1–63, February 1975. doi: 10.1070/RM1975v030n01ABEH001400

    1975 doi

  17. [25]

    Nadaraya

    E. Nadaraya. On estimating regression.Theory of Probability and Its Applications, 9:141–142,

  18. [26]

    Scikit- learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    Fabian Pedregosa, Ga ¨el Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    2011

  19. [27]

    TabICL: A tabular foundation model for in-context learning on large data

    Jingang Qu, David Holzm ¨uller, Ga¨el Varoquaux, and Marine Le Morvan. TabICL: A tabular foundation model for in-context learning on large data. InForty-second International Con- ference on Machine Learning, 2025. URLhttps://openreview.net/forum?id= 0VvD1PmNzM

    2025

  20. [28]

    On the spectral bias of neural networks

    Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. InInternational conference on machine learning, pages 5301–5310. PMLR, 2019

  21. [29]

    Tabular data: Deep learning is not all you need, 2021

    Ravid Shwartz-Ziv and Amitai Armon. Tabular data: Deep learning is not all you need, 2021. URLhttps://arxiv.org/abs/2106.03253. 11

    2021 arXiv

  22. [30]

    Why tabular foundation models should be a research priority, 2024

    Boris van Breugel and Mihaela van der Schaar. Why tabular foundation models should be a research priority, 2024. URLhttps://arxiv.org/abs/2405.01147

    2024 arXiv

  23. [31]

    S. R. S. Varadhan. On the behavior of the fundamental solution of the heat equation with variable coefficients.Communications on Pure and Applied Mathematics, 20(2):431–455,

  24. [32]

    Geoffrey S. Watson. Smooth regression analysis.Sankhy ¯a: The Indian Journal of Statistics, Series A (1961-2002), 26(4):359–372, 1964. ISSN 0581572X. URLhttp://www.jstor. org/stable/25049340

    1961

  25. [33]

    Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms, 2017

    Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms, 2017

    2017

  26. [34]

    Frequency principle: Fourier analysis sheds light on deep neural networks.arXiv preprint arXiv:1901.06523, 2019

    Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks.arXiv preprint arXiv:1901.06523, 2019

    1901 arXiv

  27. [35]

    A closer look at deep learning on tabular data.arXiv preprint arXiv:2407.00956, 2024

    Han-Jia Ye, Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, and De-Chuan Zhan. A closer look at deep learning on tabular data.arXiv preprint arXiv:2407.00956, 2024

    2024

  28. [36]

    optional

    In-Kwon Yeo and Richard A. Johnson. A new family of power transformations to improve normality or symmetry.Biometrika, 87(4):954–959, 12 2000. ISSN 0006-3444. doi: 10.1093/ biomet/87.4.954. URLhttps://doi.org/10.1093/biomet/87.4.954. 12 A Theoretical Foundations and Derivation...

  29. [1964]

    URLhttps://api.semanticscholar.org/CorpusID:120067924

  30. [1967]

    URLhttps://onlinelibrary

    doi: https://doi.org/10.1002/cpa.3160200210. URLhttps://onlinelibrary. wiley.com/doi/abs/10.1002/cpa.3160200210

    doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.