{"id":"1df8a40c-e45d-4877-bc7d-5e0ab60412fc","arxiv_id":"2505.03393","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A regularization framework trains trees, LASSO, and ensembles to avoid relying on features that are missing at test time, cutting missingness reliance sharply with small AUROC losses.","lead":"Missingness-avoiding machine learning adds a penalty to the training objective that discourages models from using features that are often missing at test time. The authors build this penalty for decision trees, sparse linear models, and ensembles, and show on six real-world datasets that the resulting models rarely need missing values while keeping prediction accuracy close to standard models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported ρ reductions may partly reflect an asymmetric accuracy–reliance trade-off: MA models receive an explicit penalty α, while baselines are not given a matched budget to trade AUROC for lower ρ; a fair comparison at matched ρ is needed.","rationale":"The reader's weakest_assumption focuses on the structural requirement that missingness be predictable from observed variables (ODDC rules), which the paper explicitly acknowledges and analyzes in Sec. 5.2. My concern is different and more directly tied to the headline empirical numbers: the comparison between MA models and unregularized baselines may not hold the accuracy–reliance trade-off fixed. The paper's own selection rule gives MA models a dedicated mechanism to reduce ρ, and if baselines are selected without that mechanism, the reported reductions are not a fair estimate of the contribution of the MA penalty. This is a concrete, testable threat to the central claim, not a disagreement with the paper's theoretical framing. The reader did flag baseline hyperparameter ambiguity as one of three reasons for a conditional verdict, so there is partial overlap, but the reader did not make it the load-bearing concern. I would keep the verdict CONDITIONAL: the concern is addressable by re-running the comparison under a matched selection rule, and the theoretical framework plus the code release provide independent support worth preserving. I do not see grounds for rejection, because the paper's core idea (regularizing the learning objective to reduce missingness reliance) is coherent and the experiments, even if re-analyzed, may well still show an advantage; the issue is that the current presentation does not yet establish that advantage cleanly.","tokens_in":27735,"tokens_out":5927,"duration_ms":65454,"concrete_test":"Reproduce Table 1 with the identical selection rule for all baselines: for LR, DT, RF, and XGBoost, select hyperparameters that minimize ρ among those with CV AUROC at least 95% of the maximum, exactly as is done for α*. Additionally, match test-time ρ directly: for each dataset, increase LR's L1 regularization strength (or reduce DT/RF max_depth) until each baseline reaches the ρ reported for its MA counterpart, then compare test AUROC at that matched ρ. If MA models no longer dominate the AUROC–ρ frontier, the central empirical claim is unsupported; if they still do, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that MA models reduce missingness reliance ρ while keeping AUROC competitive (Table 1). The model-selection protocol (Sec. 6; App. C.3) chooses, for each MA model, the value α* minimizing ρ among candidates with AUROC at least 95% of the maximum. If the same criterion is applied to baseline hyperparameters, baselines also get the opportunity to trade accuracy for lower ρ (e.g., stronger L1 penalty for LR, shallower trees for DT/RF). The paper does not state explicitly whether baseline hyperparameters were selected under this same rule, and several baseline ρ values (e.g., LR at 100% on NHANES) suggest they were not, or that the baseline grids cannot exploit the trade-off. If baselines are selected purely for AUROC, the comparison conflates the effect of the MA penalty with a more favorable selection rule, and the headline reductions in ρ (66.0, 4.4, 70.2 percentage points) are not a controlled estimate of the method's benefit. This concern is independent of the acknowledged ODDC/MCAR limitation in Sec. 5.2: even when missingness is structured, the reported advantage could be an artifact of unequal selection budgets. Because the MA penalty is the paper's core contribution, its empirical demonstration must isolate the penalty from the selection rule.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces missingness-avoiding (MA) machine learning, a framework that penalizes a model's expected reliance on missing feature values at test time. The reliance measure ρ(h) is defined as the probability that computing h(x) requires the value of at least one feature that is missing in x. The authors derive model-specific regularizers for decision trees (Eq. 4), linear models (Eq. 5), and tree ensembles (Section 4.3), and provide implementations for MA-DT, MA-LASSO, MA-RF, and MA-GBT. The theory (Section 5) gives sufficient conditions for the existence of zero-reliance optimal models under observed deterministic data collection (ODDC) rules (Prop. 1, Cor. 1) and a lower bound on reliance when features are missing completely at random (Prop. 2). Experiments on six real-world datasets report AUROC and ρ for the MA models against several baselines, finding overlapping AUROC confidence intervals in most settings and substantially lower ρ for the MA models.","tokens_in":28000,"tokens_out":10428,"duration_ms":93070,"significance":"The framework addresses a real practical problem: models that at deployment require features that are currently missing, forcing imputation or compromising interpretability. The paper's contribution is primarily algorithmic and empirical. The theory is acknowledged by the authors to be largely definitional, but it provides a useful vocabulary for when the approach can work. The experiments are carefully designed with bootstrap confidence intervals, multiple datasets, and both zero and MICE imputation, and the code is provided. If the empirical comparison can be made fully controlled (see major comments), the paper would offer a valuable tool for practitioners who need models that make themselves robust to missingness at test time.","major_comments":[{"comment":"The model-selection protocol in Section 6 states that hyperparameters, including α, are chosen by selecting the candidate with the lowest ρ̂ among those with AUROC at least 95% of the maximum. The paper does not state whether the baseline models (LR, DT, RF, XGBoost, M-GAM, NeuMiss) were selected under this same rule or by standard AUROC-only selection. If baselines were selected for AUROC only, the reported reductions in ρ (e.g., 66.0 and 70.2 percentage points for MA-LASSO vs. LR and MA-RF vs. RF in Table 1) are not a controlled estimate of the MA penalty's benefit, because baselines were not given the same opportunity to trade accuracy for lower reliance. The authors should either confirm that the same rule was applied to all models and report the selected baseline hyperparameters, or add a matched comparison, such as reporting the AUROC-ρ Pareto frontier for every model class.","section":"Section 6, Table 1"},{"comment":"In Table 3, MA-GBT is restricted to 10 estimators and MA-RF to 50, while the XGBoost search space includes 100–500 estimators and the RF baseline uses the default ensemble size (100). Since the ensemble reliance in Eq. (6) is defined as the maximum over constituent trees, the number of estimators directly bounds the achievable ρ: a smaller ensemble has fewer opportunities to rely on missing features. The reported ρ comparisons between MA-GBT and XGBoost, and between MA-RF and RF, are therefore not controlled for model capacity. The authors should either match the number of estimators across methods or show how ρ and AUROC vary with ensemble size for both the MA and baseline models.","section":"Section 4.3, Table 3"}],"minor_comments":[{"comment":"The notation 'σi,j ∈ 0, 1' is missing set braces; it should be 'σ_{i,j} ∈ {0,1}'.","section":"Section 4.1, Eq. (4)"},{"comment":"The phrase 'encourages sparsities in linear models' should read 'encourages sparsity in linear models'.","section":"Section 4.2"},{"comment":"The sentence 'with an average decrease of 66.0 (MA-LASSO vs. LR), 4.4 (MA-DT vs. DT), and 70.2 (MA-RF vs. RF)' should specify that these are percentage-point decreases in ρ, not relative percentages.","section":"Section 6.1"},{"comment":"The main-text statement of Corollary 1 omits the caveat noted in Appendix A that the Bayes-optimal model h* is assumed to belong to the hypothesis class H; for restricted classes such as linear models this is not automatic, and the statement should be qualified accordingly.","section":"Section 5.1, Corollary 1"},{"comment":"The caption contains the typo 'MA-LASSSO'; it should be 'MA-LASSO'.","section":"Figure 4c caption"},{"comment":"The sentence 'adding models to an ensemble can only increase the average reliance on a variable' is imprecise because ρ is defined as the maximum over trees, not an average; consider rewording to 'can only increase the missingness reliance ρ(e,x)'.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope. The main risk is the empirical confounding described in the major comments; the authors should be asked to provide a controlled comparison, for instance by applying the same selection rule to baselines or reporting full AUROC-ρ frontiers. The theoretical results are modest but presented honestly, and the code release is a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Lena et al. extend the MINTY idea—training models to avoid relying on missing features at test time—from generalized linear rule models to the model classes people actually use: decision trees, LASSO, random forests, and gradient boosting. That is a useful step, and they do it carefully. The per-sample σ-weight update in boosting is a nice touch: it lets later trees in the ensemble reuse features already paid for by earlier trees, so the missingness penalty does not double-count. The ODDC-rule analysis is formal but modest, and the authors say so themselves; Corollary 1 follows by definition. The experiments cover six real datasets, report bootstrap CIs, and ship code. That is real evidence.\n\nThe main soft spot is the comparison protocol. The stated selection rule picks the model with lowest ρ among those within 95% of the max AUROC. The paper applies this to α for MA models, but it is not explicit that baseline hyperparameters (regularization strength, tree depth, etc.) were selected under the same rule. If baselines were chosen for AUROC only, then the headline ρ reductions (66, 4.4, 70.2 points) conflate the MA penalty with a more favorable selection budget. The stress-test note gets this right, and the baseline ρ values—LR at 100% on NHANES—suggest the grids were not given a real chance to trade accuracy for lower ρ. This is fixable: report matched-budget curves (AUROC vs ρ across regularization strengths) or explicitly apply the 95% rule to baselines. The within-MA comparison (α=0 vs α=α*) already shows the penalty does what it claims, so the central mechanism is not in question.\n\nMinor points: the interpretability benefit is asserted rather than measured; MA-LASSO loses AUROC on FICO with non-overlapping CIs; and the theory, while correct, is light. None of these are fatal.\n\nThis paper deserves a serious referee. The problem is practical, the method is clearly explained, and the code is available. I would send it to review, with the baseline-selection ambiguity as the main request for clarification.","headline":"A practical extension of the MINTY idea to trees, LASSO, and ensembles, with a solid empirical study but an ambiguous baseline-selection protocol that should be clarified before the headline ρ reductions are taken at face value.","tokens_in":28555,"tokens_out":3045,"would_cite":true,"duration_ms":28809,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training a model with a missingness-reliance penalty makes it avoid requiring missing feature values at test time, with AUROC essentially unchanged.","keywords":["missing values","missingness reliance","test-time missingness","regularized decision trees","sparse linear models","tree ensembles","interpretable prediction","observed deterministic data collection rules"],"falsifier":"Run the NHANES benchmark again but alter the test-time missingness mask, for instance by dropping the survey-year feature so the deterministic collection rules no longer hold for a random subset of test rows, and measure $\\rho$ and AUROC of a fixed MA-DT. If reliance rises back toward the unregularized tree's level while AUROC falls with it, the claimed trade-off holds only under exactly matched missingness, contradicting the paper's implicit deployment claim.","tokens_in":27528,"feed_emoji":"🧩","tokens_out":12575,"duration_ms":102359,"temperature":0.7,"pith_summary":"When a deployed predictor asks for a feature that is missing, the user must impute, guess, or accept an incomplete result, each option adding bias or hiding what the model actually used. This paper proposes missingness-avoiding (MA) machine learning: add a term $\\alpha \\rho(h)$ to the learning objective that penalizes any prediction which, for a particular example, requires a feature whose value is missing. The authors implement this penalty for decision trees (MA-DT), sparse linear models (MA-LASSO), random forests (MA-RF), and gradient-boosted trees (MA-GBT), and show on six real datasets that these models cut the share of test predictions relying on a missing value, in the extreme case from 100% to 0%, while AUROC confidence intervals overlap with unregularized baselines in nearly every comparison. The reason to care: a model that avoids missing values preserves interpretability where it matters, because the clinician or analyst never has to supply data the model was designed not to need.","feed_headline":"A missingness penalty cuts reliance on missing data to zero","feed_subtitle":"Tuned models keep matching unregularized AUROC while rarely needing a missing feature at test time.","key_machinery":"The load-bearing object is the missingness-reliance penalty $\\alpha \\rho(h)$: a regularization term counted per training example, where $\\rho(h,x)=1$ if the prediction path for $x$ touches any feature whose value is missing. It is carried through three instantiations. For decision trees, the split-selection criterion $C(\\ell,D;j,\\tau)$ is augmented by $\\alpha$ times the empirical fraction of samples in the leaf for which feature $j$ is missing, so splits are chosen that avoid frequently missing features while still reducing impurity; for Lasso, the coefficient penalty becomes $(\\lambda + \\alpha m_j)|\\theta_j|$, which drives frequently missing features out of the model through standard feature rescaling; and for boosted trees, per-sample weights $\\sigma_{i,j}$ are updated so that later trees prefer features already used by earlier trees, preventing independently fit trees from each re-introducing reliance. The theoretical companion is the class of ODDC rules, implications of the form 'whenever variables $T$ are observed and take values in $A$, variable $j$ is observed', which characterize when zero reliance and minimal risk can coexist.","core_discovery":"The paper's central claim is that missingness reliance can be moved from an architectural afterthought into the training objective. Reliance is defined as $\\rho(h) = \\mathbb{E}_p[\\max_j \\mathbb{1}[a_h(X,j)=1 \\wedge X_j = \\mathrm{na}]]$, the probability that evaluating $h$ on a random input needs at least one feature value that is absent, and the learning problem becomes minimizing $\\mathbb{E}_p[L(Y,h(X))] + \\alpha \\rho(h)$. The paper instantiates this objective for four model classes: trees gain a split-criterion term penalizing splits on features that are missing for samples in the node, Lasso gains a per-feature penalty $(\\lambda + \\alpha m_j)|\\theta_j|$ with $m_j$ the empirical missingness rate, random forests apply the tree penalty per estimator, and gradient boosting decays per-sample weights $\\sigma_{i,j}$ so later trees reuse features earlier trees already relied on. The theoretical core is the class of observed deterministic data collection rules, implications guaranteeing that a feature is observed whenever certain observed variables take certain values, under which a minimal-risk model with zero reliance provably exists (Corollary 1) and any rule-respecting tree has $\\rho(h)=0$ (Proposition 1). Empirically the trade-off is favorable: on NHANES, logistic regression and random forests rely on missing values for 100% of test predictions, while the MA versions rely on them for at most 0.4% at overlapping AUROC.","pith_inferences":["The same penalty could be carried into other model classes the paper leaves open, such as neural networks, generalized additive models, and rule lists, wherever 'requires the value of feature $j$' can be read off the architecture.","An alternative model-selection rule that caps the number of missing features any single prediction may rely on, rather than averaging reliance over the test set, would better match safety-critical per-patient guarantees.","If deployment itself decides what to measure next, the ODDC analysis suggests a natural loop: train the data-collection policy and the MA model jointly, since the model's zero-reliance structure is only as stable as the collection rules that guarantee observability.","A testable extension would quantify how quickly the benefit decays under shift: on a dataset with matched train/test collection rules but discontinued availability of one feature, measuring $\\rho$ and AUROC of a fixed MA model would give practitioners a degradation curve."],"forward_implications":["Practitioners can deploy MA-DT in settings like clinical risk scoring and still read off the decision path: the tree reaches patients with missing MRI results without ever branching on the scan.","In datasets whose collection process follows ODDC rules, the framework can find models that are simultaneously Bayes-optimal and never require a missing value, so zero-reliance predictions carry no accuracy cost.","The speed gap matters in practice: MA-LASSO trains in under a second, while the earlier rule-based baseline MINTY takes 18 to 292 seconds on the same tasks, making low-reliance prediction feasible at scale.","When features are missing completely at random and independently, Proposition 2's lower bound $\\rho(h) \\ge \\max_j \\mathbb{E}[a_h(X,j)]\\,p_j$ means no training trick can avoid reliance, so the benefit of MA learning is structural rather than universal.","Setting $\\alpha$ very high forces $\\rho \\approx 0$ but at a real accuracy cost, for instance MA-DT on LIFE falls from AUROC 0.90 to 0.67, so $\\alpha$ must be tuned as a trade-off parameter, which the paper's model-selection rule does automatically."],"supporting_citations":[{"why":"Introduced the missingness-reliance definition this paper formalizes, and supplied MINTY, the strongest low-reliance baseline and the speed comparison it must beat.","marker":"Stempfle & Johansson (2024)"},{"why":"XGBoost, the native-missingness tree baseline that MA-GBT is compared against and whose default-path handling the authors replace with explicit avoidance.","marker":"(Chen & Guestrin, 2016)"},{"why":"Shows when missingness indicators improve accuracy, framing the informative-missingness case in which the paper concedes MA learning may underperform.","marker":"(Van Ness et al., 2023)"},{"why":"M-GAM, the interpretable additive baseline with missingness indicators whose reliance levels the MA models improve on without adding mask features.","marker":"(McTavish et al., 2024)"},{"why":"Existence of large Rashomon sets is the theoretical motivation for why a missingness penalty can select among near-optimal models with minimal accuracy loss.","marker":"(Semenova et al., 2022)"},{"why":"Provides the L1-regularized Lasso that MA-LASSO extends with the feature-specific penalty $(\\lambda + \\alpha m_j)|\\theta_j|$.","marker":"(Tibshirani, 1996)"},{"why":"MIA, the native tree treatment of missingness as a separate category, which the MA trees' contextual avoidance is contrasted with.","marker":"(Twala et al., 2008)"}],"fun_headline_variants":["New penalty trains models to skip missing features","Bake missingness avoidance into your training objective","MA learning: models rarely need absent values at test","Cut missing-value reliance without hurting accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's practical benefit rests on the assumption that the missingness pattern at test time is stable and at least partly predictable from observed variables, so that a model trained to lean on features that tend to be present will still find those features present when deployed.","fun_headline_variants_meta":{"raw":{"variants":["New penalty trains models to skip missing features","Bake missingness avoidance into your training objective","MA learning: models rarely need absent values at test","Cut missing-value reliance without hurting accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000915,"raw_usage":{"total_tokens":3980,"prompt_tokens":1045,"completion_tokens":2935,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":2878}},"tokens_in":661,"tokens_out":2935,"duration_ms":19766,"temperature":1.0,"reasoning_tokens":2878,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:53:25.703752+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the NHANES benchmark again but alter the test-time missingness mask, for instance by dropping the survey-year feature so the deterministic collection rules no longer hold for a random subset of test rows, and measure $\\rho$ and AUROC of a fixed MA-DT. If reliance rises back toward the unregularized tree's level while AUROC falls with it, the claimed trade-off holds only under exactly matched missingness, contradicting the paper's implicit deployment claim.","supporting_citations":[{"cited_title":"and Johansson, F","cited_arxiv_id":null,"evidence_quote":"Introduced the missingness-reliance definition this paper formalizes, and supplied MINTY, the strongest low-reliance baseline and the speed comparison it must beat."},{"cited_title":"and Guestrin, C","cited_arxiv_id":null,"evidence_quote":"XGBoost, the native-missingness tree baseline that MA-GBT is compared against and whose default-path handling the authors replace with explicit avoidance."},{"cited_title":"M., Halpin-Gregorio, R., and Udell, M","cited_arxiv_id":null,"evidence_quote":"Shows when missingness indicators improve accuracy, framing the informative-missingness case in which the paper concedes MA learning may underperform."},{"cited_title":"Interpretable generalized additive models for datasets with missing values","cited_arxiv_id":null,"evidence_quote":"M-GAM, the interpretable additive baseline with missingness indicators whose reliance levels the MA models improve on without adding mask features."},{"cited_title":"On the existence of simpler machine learning models","cited_arxiv_id":null,"evidence_quote":"Existence of large Rashomon sets is the theoretical motivation for why a missingness penalty can select among near-optimal models with minimal accuracy loss."},{"cited_title":"Regression shrinkage and selection via the Lasso","cited_arxiv_id":null,"evidence_quote":"Provides the L1-regularized Lasso that MA-LASSO extends with the feature-specific penalty $(\\lambda + \\alpha m_j)|\\theta_j|$."},{"cited_title":"E., Jones, M., and Hand, D","cited_arxiv_id":null,"evidence_quote":"MIA, the native tree treatment of missingness as a separate category, which the MA trees' contextual avoidance is contrasted with."}],"review_version":1}