{"id":"a1e4e288-ce1f-4834-80d4-47cb8f34435c","arxiv_id":"2608.06004","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":1,"one_line_summary":"All evaluated tabular foundation models violate both marginalization and factorization consistency, so their autoregressive joint distributions are order-dependent and cannot arise from any single joint distribution.","lead":"Tabular AI models that predict one column at a time give contradictory answers when their predictions are combined into multi-column forecasts. This paper shows every current tabular foundation model fails two basic self-consistency checks on every dataset tested.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regression-side violations may be quadrature artifacts: the paper's lower-bound argument compares against the K-atom approximation, so nonzero TV does not rigorously imply inconsistency of the continuous densities.","rationale":"The reader identified the regression quadrature as the weakest assumption but concluded it does not break the central claim, relying on the paper's 'correctly-directed lower bound' argument. My analysis shows that argument is not logically sufficient: the computed TV is a lower bound for the distance to the approximated mixture, not to the true continuous mixture. Therefore the regression experiments provide only suggestive, not conclusive, evidence of violation. This is load-bearing because the abstract and conclusion explicitly state 'for both classification and regression.' The classification results are exact (finite sums over the categorical heads), so the classification half of the claim is solid. But the regression half, as currently presented, could be an artifact of the discretization pipeline. A concrete validation using an oracle model with known-consistent predictions would settle this: if the pipeline reports nonzero TV for an exact model, the pipeline is biased; if it reports zero, the concern is resolved. In the absence of this test (or a formal error bound), the appropriate verdict is CONDITIONAL: accept the paper's contributions, but require the authors to either provide the missing bound/validation or soften the regression claim. This matches the stress-test goal of identifying the single most load-bearing concern without manufacturing objections.","tokens_in":22325,"tokens_out":7106,"duration_ms":72083,"concrete_test":"Construct an oracle TFM whose univariate predictive densities are derived exactly from a known joint distribution (e.g., a multivariate Gaussian or a Bayesian network), so that (C1) and (C2) hold identically. Run the paper's regression pipeline (K=1,000 equal-mass atoms, S=20 grid, and the 128x128 factorization grid) on this oracle. If the pipeline returns TV substantially above machine precision (e.g., >1e-6) for either check, the discretization introduces false positives and the paper's regression violations are not evidence of inconsistency. If it returns TV≈0, the pipeline is validated and the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim extends to regression, but the regression pipeline does not establish a violation of the continuous consistency conditions. In the marginalization check, the paper computes TV between the direct marginal and the finite-K atom approximation of the mixture, then argues that coarsening makes the result a lower bound for the continuous TV. This is logically incomplete: the coarse-grid TV is a lower bound for TV between the direct marginal and the approximated mixture, not necessarily for TV between the direct marginal and the true mixture. Without a bound on the quadrature error TV(q, q_K), a nonzero observed TV can arise even if the model's true continuous predictions satisfy (C1). The same issue affects the factorization check, where 128x128 cell representatives approximate the continuous conditionals; the two discretized joints could differ even if the continuous joints coincide. The paper provides no formal error bound and no validation that the discretization preserves zero-TV for a known-consistent model. Since the abstract claims violations for both classification and regression, and classification is exact while regression is not, the universal claim is not yet rigorously supported on the regression side.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether the univariate predictive distributions issued by tabular foundation models (TFMs) can be the marginals and conditionals of any single joint distribution over the columns of a table. It formalizes two necessary conditions: marginalization consistency (C1), which requires the model's direct marginal for a target to equal the mixture of its conditionals over an intermediate target, and factorization consistency (C2), which requires the two autoregressive chain-rule orders to produce the same joint. The paper proves that (C2) implies (C1), gives a counterexample showing the converse fails, and then evaluates six TFMs on ten classification and seven regression datasets. Classification checks are exact finite sums; regression checks discretize continuous predictive distributions via quantile heads, using K=1,000 equal-mass atoms and a 20-cell or 128x128 grid. The reported result is that every evaluated TFM violates both conditions on every dataset, for both classification and regression, and that the two violation measures correlate strongly.","tokens_in":22505,"tokens_out":6369,"duration_ms":64462,"significance":"If the claims were fully established, the paper would make a valuable contribution: it provides label-free, ground-truth-free consistency diagnostics for TFMs, it draws a clean logical connection between the two forms of inconsistency, and the classification-side experiments are exact finite-sum checks that give mathematically certain violations. The proposed metrics could become a useful additional evaluation axis alongside accuracy and calibration. The paper is also careful in places: the proof of Proposition 1 is a direct calculation, the counterexample in Appendix A correctly separates the two properties, and the classification methodology leaves no room for sampling noise. The main weakness is that the regression-side evidence, which is needed for the abstract's universal claim, relies on discretization and quadrature steps whose error is not bounded and whose preservability of zero-TV for known-consistent models is not validated.","major_comments":[{"comment":"The inference from a nonzero discretized TV to a violation of the continuous marginalization condition (C1) is not justified. The experiment computes TV between the S-cell coarsening of the direct marginal and the S-cell coarsening of the K-atom mixture. Coarsening is non-expansive, so TV(q_dir^S, q_mixK^S) is at most TV(q_dir, q_mixK), and the triangle inequality gives TV(q_dir, q_mixK) ≤ TV(q_dir, q_mix) + TV(q_mix, q_mixK). The observed quantity is therefore not a lower bound on TV(q_dir, q_mix); a large value can be produced entirely by quadrature error TV(q_mix, q_mixK). The paper asserts the contrary in §4.2 ('The resulting value for TV is therefore a lower bound ... in particular for the TV between the underlying continuous distributions'), but that assertion holds only for the K-atom approximation, not for the true mixture. A formal bound on the quadrature error, or a controlled experiment with a model known to be continuous-consistent, is needed before the regression violations can be claimed.","section":"§4.2 and Appendix C"},{"comment":"In the factorization check, the two discretized joints are computed by evaluating conditionals at cell representatives (midpoint quadrature in probability space, Appendix C). For a continuous model that is exactly factorization-consistent, the true continuous joints coincide, but the two discretized tables need not coincide, because each cell mass in order a→b uses p(a|b*_l,x) evaluated at a single representative rather than averaged over the cell. Hence a nonzero table TV does not establish a violation of (C2). This is not a minor technicality: the regression factorization violations in Table 2 are the main evidence for the regression part of the abstract's universal claim. The authors should either bound the discretization error, demonstrate that a known-consistent model yields zero table TV on these datasets, or present the numbers as violations of the discretized conditions only.","section":"§4.3 and Appendix C"},{"comment":"The central conclusion as stated in the abstract and §4.4 ('Every TFM that we evaluate violates both of these requirements for both classification and regression across all datasets') is too strong relative to the evidence. The classification side is exact, but the regression side is not yet rigorously connected to the continuous conditions on (C1) and (C2). Since this is the paper's headline claim, the manuscript needs a major revision either to supply the missing error analysis or validation, or to restrict the claim to the classification results and to the discretized regression checks.","section":"Abstract and §4.4"}],"minor_comments":[{"comment":"The phrase 'for both classification and regression' is imprecise because TabICLv1 and TabFM lack probabilistic regression heads; the claim should specify that it applies to the models that provide probabilistic regression.","section":"Abstract and §4.1"},{"comment":"The decision to drop instances with NLL below 0.05 is a post hoc filtering step that can affect the reported per-row analysis; it should be described in the main text and accompanied by a sensitivity check over the threshold.","section":"Appendix D"},{"comment":"A code or data release link would aid reproducibility of the exact finite-sum classification checks and the deterministic regression pipeline described in Appendix C.","section":"General"},{"comment":"The column header 'Feat. num./cat.' is ambiguous; splitting it into two separate columns for numerical and categorical feature counts would make the table easier to read.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the classification diagnostics are exact and reproducible in principle. My reservation is focused: the regression pipeline's unquantified approximation directly undercuts the universal claim. This is fixable with additional analysis or a more careful claim, so I recommend major revision rather than reject. The reader's report appears to accept the lower-bound argument; I would push back on that specific point because coarsening and quadrature error do not compose in the direction the paper needs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this paper has a genuinely useful idea—checking whether a univariate TFM's predictions could come from any joint distribution by testing marginalization and factorization consistency—and the classification experiments deliver a clean, exact, universal violation. The regression experiments are the soft spot: the claim that their discretized TV is a lower bound on the continuous TV does not survive contact with the triangle inequality.\n\nWhat's new and good: the two conditions are clearly stated, the proof that factorization consistency implies marginalization consistency is correct and useful, and the counterexample separating them is neat. On classification, because targets are finite, the checks are closed-form finite sums; every model violates both conditions on every dataset, with no sampling noise. That alone is a solid negative result for the TFM subfield. The discussion of TabDPT, trained on real data with swapped target/feature roles yet still inconsistent, is interesting and well handled.\n\nThe soft spot is the regression pipeline. The authors approximate the mixture integral by K=1,000 quantile atoms, obtaining q_K, then compare the direct marginal p to q_K on a 20-cell grid. They argue that coarsening makes the observed TV a lower bound on the continuous TV. But coarsening only bounds TV(p, q_K), not TV(p, q); without a bound on the quadrature error TV(q, q_K), a nonzero observed TV can be an artifact of the approximation. The same issue hits the 128x128 factorization check. The paper gives no error bound and no positive control, such as a known-consistent synthetic model that should yield TV=0 in their pipeline. So the regression-side universal claim is not yet rigorously supported. The gaps are large enough that the conclusion is probably right, but 'probably' is not what the abstract claims.\n\nAlso worth noting: no code shipped, which matters for a metrics paper, and the self-citation of Yalavarthi et al. is fine but not essential.\n\nWho it's for: people building or benchmarking TFMs, and anyone using autoregressive synthetic data generation from these models. It deserves a serious referee; the classification result alone justifies it. I'd send it to review with the instruction that the regression claim needs either a rigorous bound or a downgrade to 'strongly suggests.'\n\nReading group: yes.","headline":"Crisp negative result on TFM consistency; the classification half is exact and convincing, but the regression half overclaims the rigor of its discretization argument.","tokens_in":23018,"tokens_out":3766,"would_cite":true,"duration_ms":36284,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Every tabular foundation model tested fails basic probability checks","keywords":["tabular foundation models","marginalization consistency","factorization consistency","autoregressive sampling","total variation distance","Bayesian posterior predictive","coherent predictive systems","in-context learning"],"falsifier":"Run the paper's two consistency checks on a model whose predictive conditionals are computed exactly from a single joint distribution (e.g., a hand-built graphical model); if the total-variation distance is not exactly zero, the measurement procedure rather than the model is producing the violation. Alternatively, any current TFM that scores exactly zero on both checks for a dataset with two dependent targets would falsify the paper's universal claim.","tokens_in":22144,"feed_emoji":"📊","tokens_out":5869,"duration_ms":49131,"temperature":0.7,"pith_summary":"This paper asks whether tabular foundation models (TFMs) — transformers that predict one column of a table given the others — behave as if their predictions come from any single joint distribution over the table's columns. Since the true posterior is unknown, the authors check two necessary properties that any coherent joint must obey: marginalizing a conditional over an intermediate target must recover the directly predicted marginal, and the two chain-rule orderings must build the same joint. They prove that the second property implies the first, and then measure violations with total variation distance using only each model's own predictions. Across six classifiers and four regressors, on every dataset tested, every model violates both conditions; the authors conclude that the univariate outputs of current TFMs do not arise from any well-defined joint, so autoregressive constructions from them are order-dependent and incoherent. This matters because practitioners use these models to generate multivariate records and to reason about dependence between targets.","feed_headline":"Tabular foundation models fail basic probability checks on every dataset","feed_subtitle":"Their univariate predictions cannot come from any joint distribution, so autoregressive sampling order changes the result.","key_machinery":"The load-bearing objects are two consistency conditions stated for a pair of targets $A$ and $B$. Marginalization consistency (C1) requires the directly predicted marginal of $A$ to equal the mixture of the conditional of $A$ given $B$ weighted by the marginal of $B$, i.e. the law of total probability. Factorization consistency (C2) requires the two chain-rule products, $A$-then-$B$ and $B$-then-$A$, to be equal, i.e. compatibility of conditionals and marginals. The argument runs on these identities: if a model satisfied (C2), integrating over $B$ would reduce the mixture to the direct marginal, yielding (C1) by normalization; the contrapositive makes the cheaper univariate check a certificate of order-dependence. Violations are quantified by total-variation distance between the two sides of each identity, computed exactly for classification and via deterministic quantile-based discretization for regression; because total-variation distance is non-increasing under coarsening, the regression gaps are lower bounds.","core_discovery":"The paper's central discovery is a universal negative result: every tabular foundation model evaluated — TabPFNv2/v3, TabICLv1/v2, TabDPTv1.2, and TabFM — violates both marginalization consistency (C1) and factorization consistency (C2) on all ten classification and seven regression datasets. The violations are systematic and measurable from the models' own outputs: for classification, the checks are exact finite sums; for regression, they are evaluated on discretized grids with total-variation distance, giving lower bounds. As a corollary, the autoregressive joint constructed by sampling one target and conditioning on it is order-dependent: choosing which target to sample first changes the predicted distribution. The paper also proves that factorization consistency implies marginalization consistency, so each observed marginalization failure already certifies that the two factorization orders disagree.","pith_inferences":["If the result holds across more models and datasets, consistency should become a standard evaluation axis alongside accuracy and calibration for any TFM used as a joint density estimator.","The two consistency conditions suggest a concrete training target: penalize the marginalization or factorization gap directly, or derive all univariate heads from a shared latent joint so coherence holds by construction.","The same label-free TV-measurement lens transfers to any conditionally specified predictor, including large language models, whose autoregressive token probabilities have been separately suspected of violating probability laws.","Once a TFM emits classification and regression from shared weights, the mixed categorical-continuous test the paper outlines would probe coherence across the two heads — a boundary current models cannot cross."],"forward_implications":["Autoregressive data generation with current TFMs produces order-dependent joints: the distribution over a generated row changes with the sampling order of the columns.","Marginalization consistency is a cheap, label-free diagnostic (it compares only univariate heads) and, by the paper's contrapositive, a detected violation certifies that factorization also fails.","Benchmarks that rank TFMs by accuracy or RMSE do not capture this axis; the paper finds no systematic correlation between predictive quality and consistency, so a state-of-the-art model can be among the most inconsistent.","Training a TFM with a self-supervised objective that swaps target and feature roles (as TabDPT does) does not in itself yield factorization-consistent conditionals.","The two consistency conditions extend from a pair of targets to arbitrary subsets, so future models with native multivariate heads would still need to align the directly predicted joint with the joint induced by one-target-at-a-time factorization."],"supporting_citations":[{"why":"Establishes the PFN blueprint that TFMs use to approximate the Bayesian posterior predictive, the promise being tested.","marker":"Müller et al. 2022"},{"why":"Defines the autoregressive construction that turns univariate TFM heads into multivariate joints, whose coherence this paper checks.","marker":"Vetter et al. 2025"},{"why":"Martingale property of coherent predictive systems; underlies marginalization consistency as the law of total probability.","marker":"Berti, Pratelli, and Rigo 2004"},{"why":"Martingale posterior framework; shows why factorization consistency implies marginalization consistency via normalization.","marker":"Fong, Holmes, and Walker 2023"},{"why":"Compatibility condition for conditionals; motivates factorization consistency.","marker":"Besag 1974"},{"why":"Characterizes when a family of conditionals and marginals arises from one joint; basis for (C2).","marker":"Arnold and Press 1989"},{"why":"Total-variation distance is non-increasing under coarsening, making the regression grid comparisons lower bounds.","marker":"Csiszár and Shields 2004"},{"why":"Guaranteed deterministic bounds on total-variation distance between univariate mixtures; supports the discretization argument.","marker":"Nielsen and Sun 2018"},{"why":"Supplies the TabPFNv2/v3 models evaluated, including their bin-wise density heads used in the regression checks.","marker":"Hollmann et al. 2025"},{"why":"Supplies TabDPT, a model trained with a self-supervised column-swapping objective; its observed inconsistency is a key discussion point.","marker":"Ma et al. 2025"}],"fun_headline_variants":["Tabular foundation models don't even agree with themselves","Tabular models fail every probability consistency check","Tabular AI predictions change with autoregressive sampling order","No tabular model passes any consistency test on any dataset","Tabular models violate probability rules on all datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For regression, the conclusion assumes that the $K=1{,}000$ equal-mass atom quadrature and the 20-cell or $128\\times128$ discretization faithfully represent the continuous predictive distributions; the paper argues that coarsening can only hide discrepancies, but no formal error bound for the quadrature is given.","fun_headline_variants_meta":{"raw":{"variants":["Tabular foundation models don't even agree with themselves","Tabular models fail every probability consistency check","Tabular AI predictions change with autoregressive sampling order","No tabular model passes any consistency test on any dataset","Tabular models violate probability rules on all datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000596,"raw_usage":{"total_tokens":2757,"prompt_tokens":881,"completion_tokens":1876,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":1802}},"tokens_in":497,"tokens_out":1876,"duration_ms":13704,"temperature":1.0,"reasoning_tokens":1802,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T18:18:51.717514+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's two consistency checks on a model whose predictive conditionals are computed exactly from a single joint distribution (e.g., a hand-built graphical model); if the total-variation distance is not exactly zero, the measurement procedure rather than the model is producing the violation. Alternatively, any current TFM that scores exactly zero on both checks for a dataset with two dependent targets would falsify the paper's universal claim.","supporting_citations":[{"cited_title":"Effortless,","cited_arxiv_id":null,"evidence_quote":"Defines the autoregressive construction that turns univariate TFM heads into multivariate joints, whose coherence this paper checks."},{"cited_title":"Guaranteed Deterministic Bounds on the","cited_arxiv_id":null,"evidence_quote":"Guaranteed deterministic bounds on total-variation distance between univariate mixtures; supports the discretization argument."}],"review_version":1}