REVIEW 6 cited by
Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Later
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
The widespread enthusiasm for deep learning has recently expanded into the domain of tabular data. Recognizing that the advancement in deep tabular methods is often inspired by classical methods, e.g., integration of nearest neighbors into neural networks, we investigate whether these classical methods can be revitalized with modern techniques. We revisit a differentiable version of $K$-nearest neighbors (KNN) -- Neighbourhood Components Analysis (NCA) -- originally designed to learn a linear projection to capture semantic similarities between instances, and seek to gradually add modern deep learning techniques on top. Surprisingly, our implementation of NCA using SGD and without dimensionality reduction already achieves decent performance on tabular data, in contrast to the results of using existing toolboxes like scikit-learn. Further equipping NCA with deep representations and additional training stochasticity significantly enhances its capability, being on par with the leading tree-based method CatBoost and outperforming existing deep tabular models in both classification and regression tasks on 300 datasets. We conclude our paper by analyzing the factors behind these improvements, including loss functions, prediction strategies, and deep architectures. The code is available at https://github.com/qile2000/LAMDA-TALENT.
Forward citations
Cited by 6 Pith papers
-
TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning
TabPack packs MLPs with diverse sampled hyperparameters into one vectorized model, selects ensemble members online during training, and matches tuned baselines at a fraction of the compute cost.
-
On Finetuning Tabular Foundation Models
Full finetuning of TabPFNv2 outperforms in-context learning and partial finetuning on medium tabular datasets, and its gains come from sharper query-key attention that better reflects target similarity.
-
MultiTab: A Comprehensive Benchmark Suite for Multi-Dimensional Evaluation in Tabular Domains
A regime-stratified benchmark of 196 tabular datasets shows that model rankings depend strongly on dataset characteristics such as sample size, feature correlation, and label imbalance.
-
TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data
Shared-backbone low-rank ensemble adapters let neural tabular models match much of full-ensemble accuracy on large data without linear parameter growth or frequent OOMs.
-
Realistic Evaluation of TabPFN v2 in Open Environments
TabPFN v2 underperforms tree-based models on most open-environment tabular tasks and is only preferable on small, covariate-shifted, class-balanced data.
-
Random at First, Fast at Last: NTK-Guided Fourier Pre-Processing for Tabular DL
Fixed random Fourier projections on tabular inputs are claimed to bound the NTK, speed up gradient descent, and improve accuracy across four architectures and eight benchmarks.
Discussion (0). Sign in to comment.