{"id":"a2aaba69-4c41-4f89-a844-e1ffbeec38d8","arxiv_id":"2412.10871","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"FTAT is a fully test-time adaptation method for tabular classification that re-weights predictions by an estimated shifted label distribution, trusts locally consistent test points, and ensembles multiple learning-rate copies of the model.","lead":"This paper introduces FTAT, a method that lets a pre-trained tabular classifier keep adapting while in use, using only the new test data. It estimates how the class mix shifted, checks each prediction against nearby test points, and blends several model copies; on six shifted tabular benchmarks it beats six existing adaptation methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CDO's unbiased label-distribution estimator is not derived: Eq. 4 conditions bCt on predicted class, so bCt^{-1} ePt is not generally the true label distribution, and Eq. 5's update has a minus sign that would move bPt away from the targeted estimate.","rationale":"The reader's weakest_assumption identifies exactly this load-bearing premise: the CDO's label-distribution estimator is asserted to be unbiased without derivation, and the update rule in Eq. 5 is problematic. My analysis of Eq. 4 confirms that the matrix bCt is conditioned on predicted class, not true class, so the standard unbiased correction cannot be applied as written. This is the single most load-bearing concern because the ablation shows CDO contributes a large share of FTAT's F1 gains, especially on DIABETE. If the estimator is biased, the mechanism behind the strongest improvements is unsupported, even though the average accuracy numbers might still hold empirically. The reader's verdict of CONDITIONAL already reflects the need to fix this derivation; my read does not move the verdict to REJECT because the reported experiments could still be valid if the implementation differs from the paper's equations or if the heuristic reweighting helps for other reasons. A direct labeled-batch diagnostic would settle the unbiasedness question. No independent fatal flaw beyond the reader's identified issues emerged from the full text; per-dataset failures and missing significance tests are secondary and already covered by the conditional verdict.","tokens_in":18429,"tokens_out":9828,"duration_ms":93588,"concrete_test":"On a labeled held-out split of DIABETE (or any TableShift test set with ground-truth labels), simulate 50 random batches of size 512 from the test stream. For each batch, compute ePt and bCt exactly as in Eqs. 3–4 using the pretrained MLP, then compare bCt^{-1} ePt with the empirical label distribution of that batch using total variation or KL divergence. If the average bias is not small, or if components fall outside [0,1] in any batch, the 'unbiased' claim in Eq. 4 is empirically false and the CDO reweighting is unjustified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on the Confident Distribution Optimizer, which the ablation (Table 5) shows is responsible for the large F1 increase on DIABETE (FTAT 59.27 vs FTAT w/o CDO 51.26). The module's correctness depends on Eq. 4 producing an unbiased estimate of the shifted label distribution via bCt^{-1} ePt. As written, bCt's k-th row is the average of bfθt(xi) over samples whose argmax prediction is k, i.e., rows are conditional on the model's predicted class. For the standard BBSE-style correction C^{-1}q to recover the true label distribution, C must be the confusion matrix conditional on true labels (C_{pred,true}), which cannot be estimated from unlabeled test batches without labels. Consequently, bCt^{-1} ePt is not generally the true label distribution; in simple two-class cases it can produce negative components or values outside [0,1]. Equation 5 compounds the problem: bPt = Norm(bPt−1 − α·bCt^{-1} ePt) subtracts the new estimate from the old one rather than moving toward it, and Softmax is not a projection onto the probability simplex. These issues are not merely cosmetic: if the estimator is biased, the prediction reweighting in Eq. 2 mis-calibrates the model, and the large F1 gains that support the headline claim become an artifact of a misspecified update rather than a validated mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FTAT, a fully test-time adaptation method for tabular classification, consisting of three modules: a Confident Distribution Optimizer, a Local Consistent Weighter, and a Dynamic Model Ensembler. The authors motivate the method by three observations about tabular distribution shifts, ineffective augmentations, and sensitivity to learning rates, and they evaluate on six TableShift datasets with three backbone models, reporting average accuracy/balanced accuracy/F1 improvements over non-adaptation and existing FTTA baselines such as TENT, CoTTA, LAME, and ODS. The claimed contribution is a tuning-light, augmentation-free FTTA method specifically designed for tabular data.","tokens_in":18613,"tokens_out":6329,"duration_ms":58258,"significance":"If the empirical claims were supported, FTAT would be a useful contribution to test-time adaptation for tabular data, an important but underexplored area. The paper provides a broad comparison of existing FTTA methods on tabular benchmarks, releases code and datasets, and includes ablations that isolate the effect of each module. However, the validity of the central Confident Distribution Optimizer rests on an asserted but unproven unbiasedness claim, and the reported headlining F1 gains depend on that module. As written, the contribution is not established at the level required for a journal publication.","major_comments":[{"comment":"The claim that the quantity \\hat C_t^{-1} \\hat P_t is an unbiased estimate of the shifted label distribution is asserted without derivation and is not generally correct. The matrix in Eq. (4) has rows indexed by the model's predicted class, so it represents a conditional distribution given predicted labels, while a BBSE-style label-shift correction requires the confusion matrix conditional on true labels. In a fully test-time setting with no labels, the required true-label confusion matrix is not estimable from Eq. (3). This is load-bearing because the ablation in Table 5 shows that removing CDO drops DIABETE F1 from 59.27 to 51.26, meaning the headline F1 gains are carried by this unsubstantiated estimator.","section":"§3.1, Eq. (4)"},{"comment":"The update rule \\hat P_t = Norm(\\hat P_{t-1} - α \\hat C_t^{-1} \\hat P_t) is not a plausible tracking rule: it subtracts the newly estimated distribution from the running estimate rather than moving toward it, and Softmax is not a projection onto the probability simplex. The paper provides no derivation or justification for this particular form, and the minus sign is especially concerning because it does not match an exponential moving average. The authors need to provide a correct update or derive the rule from an explicit objective.","section":"§3.1, Eq. (5)"},{"comment":"The claim that FTAT 'consistently outperforms non-adaptation baseline' is overstated. On ANES with MLP, FTAT's F1 (83.42) is lower than the non-adaptation baseline (84.24), and several other dataset-metric combinations show differences within one standard deviation. The average improvements in Table 3 are driven by large F1 gains on a subset of datasets, notably those where CDO is responsible for most of the gain. The authors should report per-dataset statistical significance or confidence intervals and temper the 'consistent outperformance' language.","section":"§4.2, Table 4"},{"comment":"The paper's own limitation statement concedes that the method 'lacks deep theoretical understanding,' but the method section simultaneously makes a specific mathematical assertion of unbiasedness for the CDO estimator. Since the estimator is neither derived nor an obvious consequence of standard label-shift theory, the empirical results do not compensate for the missing correctness argument. The authors should either supply a rigorous derivation or clearly frame CDO as a heuristic and validate it with additional controlled experiments, such as label-shift-only and covariate-shift-only synthetic settings.","section":"§6, Limitation"}],"minor_comments":[{"comment":"The term 'covariate matrix' is used for what appears to be a confusion matrix; please clarify the terminology and define the matrix entry indices explicitly.","section":"§3.1"},{"comment":"The symbol ∘ is not defined; if it denotes elementwise multiplication or a composition operation, please state the intended meaning.","section":"§3.1, Eq. (2)"},{"comment":"The initialization of \\hat P_0 is not specified; the recurrence requires a starting distribution, presumably the source label distribution P0, but this should be stated.","section":"§3.1, Eq. (5)"},{"comment":"The paper states that AdapTable and TabLog require training data, which undercuts the comparison scope; this claim should be substantiated with specific details or softened, as AdapTable is presented as a test-time adaptation method in the cited work.","section":"§5"},{"comment":"Figure 2 shows the estimated label distribution without error bars or repeated-run information, making it hard to assess the reliability of the low-entropy estimation claim.","section":"§2.2, Fig. 2"},{"comment":"The hyperparameter robustness plot shows F1 variation of roughly 0.07–0.08 across the tested α values; the text claims that 'minor perturbations' do not significantly affect performance, but this magnitude is not negligible and should be acknowledged.","section":"§4.3, Fig. 6"}],"recommendation":"reject","confidential_remarks":"The Confident Distribution Optimizer is closely related to the authors' own ODS method (Zhou et al., 2023), which the paper presents as a failing baseline; the novelty boundary between CDO and ODS should be examined carefully. Additionally, the dismissal of AdapTable and TabLog as requiring training data appears inaccurate at least for AdapTable, and the editor may want to verify this point because it shapes the claimed novelty of the fully test-time setting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is the first paper I have seen that treats fully test-time adaptation specifically for tabular data, and on the six TableShift benchmarks it reports real gains—MLP F1 from 60.59 to 72.00, FT-Transformer F1 from 54.29 to 69.56—with ablations showing both main modules contribute. The problem analysis is clear, the benchmark choice is sensible, and they test three backbones. They also ship code and data, which counts for something. That part is solid.\n\nSecond thing: the load-bearing math does not hold up as written. The Confident Distribution Optimizer's 'unbiased' label-distribution estimate is asserted, not derived. Eq. 4 conditions C_t on the model's argmax prediction, so C_t^{-1} ePt is not the BBSE-style correction you need—that requires a confusion matrix conditional on true labels, which unlabeled test batches do not provide. In simple two-class cases the estimate can go negative or outside [0,1]. Eq. 5 is worse: bPt = Norm(bPt−1 − α·C_t^{-1} ePt) subtracts the new estimate from the old one, which moves away from the distribution you are tracking, and it is not temporal ensembling as the text claims. Softmax is also not a projection onto the probability simplex. The stress-test note is right on all of this.\n\nThe ablation makes the problem concrete: removing CDO drops DIABETE F1 from 59.27 to 51.26, so the gains ride on exactly the module whose theory is unsupported. The paper's own conclusion concedes the design 'lacks deep theoretical understanding'—honest, but it means the central claim is empirical all the way down. The CDO descends from the authors' own ODS, which is properly cited, so the lineage is transparent; the issue is the math, not the citation.\n\nLesser issues: per-dataset results are mixed (ANES degrades on some metrics), three seeds with no significance tests, hyperparameters partly tuned on the same datasets, and the manuscript has visible artifacts—Chinese placeholder text in the Figure 4 caption, a garbled Eq. 4, and scattered typos.\n\nNet: the contribution is real as a first benchmark in an underexplored setting, and the empirical pattern is consistent across backbones, but the paper is not credible as written on mechanism. Send it to review, but require a derived or corrected estimator, a fixed update rule, and a clean manuscript. If the CDO theory can be repaired, this becomes a useful paper; if not, the empirical gains may still stand, but readers deserve a believable mechanism.","headline":"A real first benchmark for tabular fully test-time adaptation with consistent reported gains, but the core label-distribution estimator is asserted rather than derived and the update rule as written moves the wrong way.","tokens_in":19335,"tokens_out":8334,"would_cite":true,"duration_ms":61850,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FTAT, a fully test-time adaptation method for tabular data, reweights predictions to match the shifted label distribution and consistently outperforms non-adaptation and existing test-time adaptation baselines on six benchmarks.","keywords":["fully test-time adaptation","tabular data","distribution shift","label distribution shift","entropy minimization","local consistency","model ensembling","TableShift"],"falsifier":"Run FTAT on a synthetic tabular stream with a known label distribution shift and compare the Confident Distribution Optimizer's estimate to the true distribution at every batch using KL divergence; if the corrected estimate stays far from the truth while low-entropy predictions are selected, the central mechanism fails.","tokens_in":18027,"feed_emoji":"📊","tokens_out":8236,"duration_ms":62975,"temperature":0.7,"pith_summary":"Fully test-time adaptation (FTTA) adjusts a pretrained model using only unlabeled test data, but existing methods were built for images and rely on augmentation. The paper argues that tabular data has its own obstacles—simultaneous label and covariate shifts, weak augmentation, and sensitivity to learning-rate choices—and that these make existing FTTA methods underperform the no-adaptation baseline. It proposes FTAT, which recalibrates predictions to the estimated shifted label distribution, weights test points by local neighborhood consistency instead of augmentation, and ensembles models trained with different learning rates. On six TableShift datasets with three backbones, FTAT reports the best accuracy, balanced accuracy, and F1 among all compared methods, with its largest gains on F1.","feed_headline":"FTAT adapts tabular models at test time, beating six baselines","feed_subtitle":"It beats all tested baselines on six real-world tabular benchmarks, with the largest gains in F1.","key_machinery":"The Confident Distribution Optimizer is the load-bearing mechanism: it estimates the shifted label distribution $\\hat{P}_t$ from predictions whose entropy is below a threshold, corrects that estimate with the inverse of the model's per-batch confusion matrix $\\hat{C}_t^{-1}$, and updates it with a temporal-ensemble step. The corrected estimate is used to reweight model predictions as $\\hat{f}(x) = f(x) \\circ \\hat{P}_t / P_0$ (elementwise ratio), aligning outputs to the current label distribution. The Local Consistent Weighter and Dynamic Model Ensembler support it by suppressing unstable samples and averaging models across learning rates.","core_discovery":"The central claim is that a fully test-time adaptation method can work for tabular data if it does three things at once: track the shifted label distribution from confident predictions, replace augmentation-based consistency with local nearest-neighbor consistency, and remove learning-rate sensitivity by ensembling several adapted models. The Confident Distribution Optimizer estimates the current label distribution from low-entropy predictions, corrects the estimate using the model's batch-level confusion matrix, and smooths it over time; predictions are then reweighted by the ratio of estimated shift to the original label distribution. The authors report that FTAT consistently outperforms the non-adaptation baseline and all six compared FTTA methods across MLP, TabTransformer, and FT-Transformer backbones on six benchmark datasets, and that the ablation without the distribution optimizer loses almost all of the F1 gain on DIABETE.","pith_inferences":["The inverse-confusion correction is a generic way to debias low-entropy label estimates; it could be transferred to any test-time method that tracks label shift, including non-tabular settings.","One testable extension would apply the Local Consistent Weighter to time-series or graph data, where augmentation is also hard and nearest-neighbor consistency is well defined.","Because FTAT adapts without source data, it could be combined with models that output calibrated probabilities to push the label-distribution estimate closer to unbiased."],"forward_implications":["FTAT outperforms the non-adaptation baseline and all compared test-time adaptation methods on six TableShift datasets across all three backbones.","On the DIABETE dataset, removing the Confident Distribution Optimizer drops F1 from 59.27 to 51.26, so the label-shift correction is the main source of the gain.","The Dynamic Model Ensembler matches or beats the best single learning rate without needing to know that rate in advance."],"supporting_citations":[{"why":"Supplies the six TableShift benchmark datasets and the evaluation protocol used in all main experiments.","marker":"Gardner, Popovic, and Schmidt 2023"},{"why":"The ODS baseline that also addresses covariate and label shifts; FTAT is designed to fix its failure on tabular data.","marker":"Zhou et al. 2023"},{"why":"The TENT baseline that optimizes parameters by entropy minimization; FTAT's reweighting is motivated by TENT's degradation on tabular data.","marker":"Wang et al. 2021"},{"why":"The CoTTA baseline that relies on augmentation; its poor tabular performance motivates the local-consistency alternative.","marker":"Wang et al. 2022"},{"why":"The LAME baseline that optimizes predictions rather than parameters; FTAT compares against it and analyzes its label-distribution estimation.","marker":"Boudiaf et al. 2022"},{"why":"Supplies the temporal-ensembling update used in Eq. (5) for smooth label-distribution tracking.","marker":"Laine and Aila 2017"},{"why":"The nearest-neighbor tabular study that inspires the Local Consistent Weighter's neighborhood consistency.","marker":"Gorishniy et al. 2024"},{"why":"Supplies the online ensemble paradigm used by the Dynamic Model Ensembler.","marker":"Bai et al. 2022"}],"fun_headline_variants":["FTAT: test-time adaptation for tabular data beats six baselines","Adapting tabular models on the fly: FTAT tops six benchmarks","Fully test-time adaptation that actually works for tabular data","FTAT handles label and covariate shifts without retraining","New tabular FTTA method FTAT wins on all six datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains depend on the assumption that the label-distribution estimate built from confident predictions and corrected by the batch-level confusion matrix is unbiased; if it is biased, the prediction re-weighting mis-calibrates the model and the reported gains shrink or reverse.","fun_headline_variants_meta":{"raw":{"variants":["FTAT: test-time adaptation for tabular data beats six baselines","Adapting tabular models on the fly: FTAT tops six benchmarks","Fully test-time adaptation that actually works for tabular data","FTAT handles label and covariate shifts without retraining","New tabular FTTA method FTAT wins on all six datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000376,"raw_usage":{"total_tokens":1985,"prompt_tokens":909,"completion_tokens":1076,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":986}},"tokens_in":525,"tokens_out":1076,"duration_ms":9222,"temperature":1.0,"reasoning_tokens":986,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:33:33.477674+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FTAT on a synthetic tabular stream with a known label distribution shift and compare the Confident Distribution Optimizer's estimate to the true distribution at every batch using KL divergence; if the corrected estimate stays far from the truth while low-entropy predictions are selected, the central mechanism fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the six TableShift benchmark datasets and the evaluation protocol used in all main experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The ODS baseline that also addresses covariate and label shifts; FTAT is designed to fix its failure on tabular data."},{"cited_title":"A.; and Darrell, T","cited_arxiv_id":null,"evidence_quote":"The TENT baseline that optimizes parameters by entropy minimization; FTAT's reweighting is motivated by TENT's degradation on tabular data."},{"cited_title":"V.; and Dai, D","cited_arxiv_id":null,"evidence_quote":"The CoTTA baseline that relies on augmentation; its poor tabular performance motivates the local-consistency alternative."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The LAME baseline that optimizes predictions rather than parameters; FTAT compares against it and analyzes its label-distribution estimation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the temporal-ensembling update used in Eq. (5) for smooth label-distribution tracking."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The nearest-neighbor tabular study that inspires the Local Consistent Weighter's neighborhood consistency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the online ensemble paradigm used by the Dynamic Model Ensembler."}],"review_version":1}