{"id":"d2be907e-cebb-4a08-ac24-f1a47e585229","arxiv_id":"2508.01957","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SEFA, a supervised latent-variable model with stochastic encoders and a gradient-based acquisition score, outperforms RL and mutual-information baselines on active feature acquisition benchmarks.","lead":"A new machine learning method, SEFA, chooses which missing data to collect next by sampling many possible 'latent' versions of the data and scoring which feature would most improve the prediction. In tests on 10 datasets including cancer classification, it beats existing active feature acquisition baselines, though the gains are sometimes small.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SEFA's acquisition score is evaluated under the feature's pre-acquisition imputed-latent distribution, not the post-measurement distribution; if this mismatch is uncalibrated, the claimed non-greedy mechanism is unsupported.","rationale":"The reader's weakest assumption identifies the acquisition heuristic as the soft spot; my concern is the same concern sharpened: the heuristic is not merely unproven but is computed under the wrong distribution. Eq. (2) scores feature i using the encoder's output for an imputed, masked feature, whereas the quantity of interest is the effect of observing the true feature value. The paper's own Proposition 4.2 uses p(x_U | x_O) over actual unobserved values, which is not what the stochastic encoder provides. This matters because the central contribution of the paper, and the stated reason SEFA is non-greedy, depends on the claim that stochastic latent sampling lets the method evaluate features across possible unobserved realizations. If that claim is false, SEFA is a heuristically motivated gradient-attribution policy; it may still work well empirically, but the mechanism distinguishing it from greedy CMI is unsupported. I do not reject the paper: the empirical evaluation is extensive, the code is released, and the ablations mostly support the design choices, even though some margins are small and a few ablations match or beat the full method. The concern affects the interpretation of the central mechanism rather than the raw results, so the appropriate outcome is to keep the reader's CONDITIONAL verdict and require either a formal argument or a targeted experiment connecting Eq. (2) to expected predictive gain. The proposed synthetic test with oracle value-of-information comparison would settle this directly and is feasible given the released code.","tokens_in":40656,"tokens_out":8358,"duration_ms":111208,"concrete_test":"Build a synthetic AFA task with known optimal order where the optimal next feature's masked/imputed latent representation has small or misleading gradient, while an irrelevant feature's masked representation has large gradient; for example, label Y = XOR(x1, x2) with a third irrelevant feature that is predictive of the imputation pattern but not of the label. Train SEFA and compare its first-acquisition ranking against the oracle value of information, computed from the true generative model as the expected reduction in log-loss from actually measuring each feature. Report the Spearman rank correlation between SEFA's scores and oracle gains across test instances and acquisition states. If the correlation is high and the optimal feature is selected, the distributional-mismatch concern is mitigated; if the correlation is low or the irrelevant feature is systematically preferred, Eq.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central conceptual claim is that SEFA is non-greedy because it uses stochastic encoders to reason across many possible unobserved realizations. But Eq. (2)-(3) do not implement this: for an unobserved candidate feature i, the score averages the normalized gradient norm over z ~ p_theta(z|x_O), where z_i is drawn from p_theta_i(z_i | imputed x_i, m_i=0). The distribution that matters after measuring i is p_theta_i(z_i | true x_i, m_i=1), which is not available at scoring time. Nothing in the training objective calibrates the masked/imputed distribution to the conditional distribution after observation; the information-bottleneck term compresses feature information and the predictive loss only trains the encoder to help predict y, not to model unobserved feature values. Therefore Eq. (2) is not an estimate of expected predictive gain from acquiring i. The stochasticity in the encoder is noise under the imputation model, not a conditional posterior over unobserved features. This distinguishes SEFA's objective from the corrected objective in Prop. 4.2, which explicitly integrates over p(x_U | x_O). If the gradient norm under the missing-imputation distribution is not a faithful proxy for the post-measurement effect, then SEFA's non-greedy advantage over CMI is not established; the empirical wins would rest on a heuristic whose mechanism is unexplained. This is more than a missing proof: the objective is evaluated under a distribution that is not the counterfactual one.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SEFA, a latent-variable method for active feature acquisition. The model trains per-feature stochastic encoders and a predictor with a negative log-likelihood loss plus an information-bottleneck regularization, then acquires features by scoring normalized latent-gradient norms of the class probability, averaged over latent samples and weighted by current class probabilities. The authors provide a theoretical critique of greedy conditional mutual information (CMI) maximization, including an indicator-feature example where CMI is provably suboptimal, and report extensive experiments on synthetic, tabular, image, and medical datasets claiming consistent improvements over RL-based and CMI-based baselines.","tokens_in":41012,"tokens_out":6047,"duration_ms":71893,"significance":"If the empirical claims hold, SEFA is a practically useful contribution: it avoids RL training instabilities and generative-model CMI estimation, and it ships with reproducible code, five-seed experiments, standard errors, and detailed ablations. The CMI critique in Section 4 is clear and the indicator example is instructive. However, the paper's central conceptual claim---that SEFA is non-greedy because it reasons over possible unobserved realizations---is not supported by the acquisition objective as written. The theoretical results concern a different, intractable objective, and the link between that theory and Eq. (2) is missing. The empirical wins are therefore best interpreted as evidence for a strong heuristic, not as validation of the stated mechanism.","major_comments":[{"comment":"The acquisition score averages r(c,z,i) over z ~ p_theta(z|x_O). For an unobserved feature i, z_i is drawn from the encoder conditioned on the imputed value and mask 0, not from a distribution over plausible true values of x_i and not from p(x_U|x_O). The training loss in Eq. (1) does not calibrate this imputation-conditional encoder to the post-measurement conditional distribution, so the claim that 'multiple possible latent realizations (including those associated with different unobserved feature values)' are considered is not implemented by the objective. The stochasticity is encoder noise under a fixed imputation, not a posterior over unobserved features. This undermines the 'non-greedy by design' claim and the Monte Carlo tree search analogy. The authors should either provide a formal derivation of what Eq. (2) optimizes or explicitly reposition SEFA as a heuristic whose non-greedy behavior is an empirical hypothesis rather than a proven property.","section":"Section 5.2, Eqs. (2)-(3)"},{"comment":"The paper motivates SEFA with the proof that greedy CMI fails on the indicator problem and that maximizing E_{p(x_U|x_O)} I(X_i; Y | x_O, x_U) is optimal. However, the text states that this objective is not used because it is intractable, and SEFA's Eq. (2) does not implement this or any other integral over p(x_U|x_O). The theoretical results therefore do not provide support for SEFA's acquisition mechanism; they only establish that some non-myopic objectives exist. The gap between the theoretical motivation and the actual acquisition objective should be stated explicitly, and the paper should not imply that Propositions 4.1-4.2 validate the heuristic in Eq. (2).","section":"Section 4, Propositions 4.1 and 4.2"},{"comment":"The claim that calculating the acquisition objective in latent space is superior because latent gradients are 'more meaningful and comparable' is not cleanly tested by the 'Feature Space Calculation' ablation. That ablation uses a separately trained VAE and MLP, which introduces a different imputation model and different predictor capacity, so the comparison conflates the representation space with the choice of generative model. An ablation that computes the same gradient-norm objective on features transformed to a comparable continuous scale (e.g., copula-transformed inputs) without a separate generative model would better isolate the effect of the latent space. Without this, the latent-space benefit remains plausible but not isolated.","section":"Section 5.3 and Table 3"}],"minor_comments":[{"comment":"The phrase 'reasoning about the features across many possible unobserved realizations in a stochastic latent space' overstates what Eq. (2) computes; consider rewording to describe sampling from the current encoder distribution.","section":"Abstract and Section 5.2"},{"comment":"On Cube the margin over DIME and Opportunistic RL is 0.003, and on MiniBooNE the margin over Fixed MLP is 0.003; the text should quantify these small margins rather than only emphasizing larger differences.","section":"Section 6.2, Table 2"},{"comment":"Hyperparameter configurations are selected by validation acquisition-curve area with three repeats, but main results use five seeds; please report whether the selected configurations were stable across the repeats.","section":"Appendix K"},{"comment":"The notation x_{S∩S'} is unclear because S is not defined as the full feature set; please clarify how the subsampling process interacts with the observed set during training.","section":"Eq. (1)"},{"comment":"The zoomed-in insets are too small to read; enlarge them or tabulate early-acquisition differences.","section":"Figure 3"},{"comment":"The gene-cancer association citations are post hoc and should be described as illustrative rather than as validation of acquisition optimality.","section":"Section 6.3"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is solid, but the manuscript oversells the non-greedy mechanism. The key fix is to reframe SEFA as a strong heuristic with clear empirical support, remove or qualify the MCTS/posterior language, and explicitly acknowledge that the theory in Section 4 does not apply to Eq. (2). If the authors make that reframing, a revised version could be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SEFA is a solid empirical contribution to active feature acquisition, with a clean new illustration of CMI's myopia and a reproducible implementation, but the paper overclaims the mechanism: the acquisition score is a gradient heuristic evaluated under the wrong distribution, so the non-greedy advantage is not actually established.\n\nWhat is new: the indicator example in Section 4 genuinely clarifies why greedy CMI fails and why considering unobserved feature values helps; Props 4.1 and 4.2 look correct. The SEFA combination—per-feature stochastic encoders, an information-bottleneck regularizer, and normalized latent gradient norms weighted by current class probabilities—is not in the prior work. The experiments are unusually thorough: 10 datasets, 5 seeds, standard errors, extensive ablations, and public code. The medical feature validation is a useful extra.\n\nThe soft spots are real. Most importantly, the stress-test concern lands. Equation (2) computes, for an unobserved feature i, gradient norms at z drawn from p_theta(z|x_O), where z_Gi comes from the encoder at the imputed value with m_i=0. After measuring i, the relevant distribution is p_theta_i(z_Gi | x_i_true, m_i=1). Nothing in the training loss calibrates the imputed latent distribution to the conditional posterior after observation. So the stochasticity is not actually reasoning over p(x_U | x_O) as Prop 4.2 requires. That is a gap between the stated mechanism and the objective, not just a missing proof. The paper does call the objective hand-crafted, which I credit, but then the abstract's 'non-greedy by design' is too strong.\n\nA second issue is the ablation overclaim. The paper says removing any novel component hurts, but their own Table 3 shows beta=0 and the copula removal sometimes match or beat SEFA, and differences like Cube 0.904 vs 0.901 are within noise. No significance tests are reported. Minor in the scheme of things, but worth fixing.\n\nWho this is for: people working on active feature acquisition. They will want to build on the empirical recipe and will get a clear baseline to beat. It deserves a serious referee, but the referee should push for a corrected description of the acquisition objective and for a direct test of the non-greedy mechanism, for example comparing against a deterministic encoder with the same number of samples or a variant that actually integrates over the post-measurement distribution.\n\nRecommendation: send to peer review, but expect a revision that narrows the claims.","headline":"Solid empirical AFA paper with a real gap between the claimed non-greedy mechanism and what the acquisition objective actually computes.","tokens_in":41509,"tokens_out":2905,"would_cite":true,"duration_ms":35097,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that latent-space gradient scores of a stochastically encoded predictor outperform RL and CMI-based active feature acquisition baselines on the evaluated tasks.","keywords":["active feature acquisition","dynamic feature selection","stochastic encoders","information bottleneck","latent space gradients","conditional mutual information","classification"],"falsifier":"Run SEFA on the paper's indicator problem with $d=4$ and record the first acquisition: if the indicator feature is not acquired first in a large fraction of instances, the stochastic expectation is not capturing long-term value as claimed. A complementary check is a synthetic task where a feature has a large latent gradient norm but no effect on prediction accuracy under any unobserved combination; SEFA should not systematically select it.","tokens_in":40482,"feed_emoji":"🎯","tokens_out":7635,"duration_ms":78439,"temperature":0.7,"pith_summary":"The paper claims that the standard recipes for active feature acquisition — reinforcement learning policies and greedy conditional-mutual-information maximization — can be replaced by a supervised-trained latent-variable model whose acquisition score reads off latent-space gradients of the predictor. The score, averaged over many stochastic encodings of the currently observed features, lets the model reason about unobserved feature values before committing to a measurement, so acquisitions are non-greedy by construction. The authors prove that any scoring function based on the marginal $p(x_i,y)$ fails on their indicator example, show that averaging conditional mutual information over unobserved features recovers the optimal first move, and then argue SEFA approximates that ideal in a tractable way. Across synthetic, tabular, image, and two cancer-classification benchmarks, SEFA attains the highest average acquisition-curve metric in the paper's comparisons, and ablations attribute the gain to the latent-space calculation, stochastic sampling, and class-probability weighting.","feed_headline":"Stochastic latent gradients beat myopic feature acquisition","feed_subtitle":"New SEFA method weighs each unobserved feature's effect across many possible worlds, outperforming RL and CMI baselines.","key_machinery":"The load-bearing object is the acquisition score of Eqs. (2)--(3): a class-probability-weighted expectation, over samples from the stochastic latent encoder, of the normalized squared Euclidean norm of the predictor's class-gradient in the latent block belonging to feature $i$. It is made tractable by factorizing the encoder into independent per-feature normal distributions (each feature gets $l$ latent components), by regularizing the latent space with a variational information-bottleneck KL term so gradients reflect label-relevant structure, and by taking many latent samples during scoring so unobserved feature values are implicitly considered.","core_discovery":"SEFA trains a per-feature stochastic encoder $p_\\theta(z|x_S)$ with a variational information-bottleneck term, then scores each candidate feature $i$ by $R(x_O,i)=\\sum_{c} p_{\\theta,\\phi}(Y=c|x_O)\\,\\mathbb{E}_{z\\sim p_\\theta(z|x_O)}\\, r(c,z,i)$, where $r(c,z,i)$ is the squared Euclidean norm of $\\nabla_z p_\\phi(Y=c|z)$ restricted to feature $i$'s latent block, normalized over all blocks. Because the encoders are stochastic, the expectation samples many possible realizations of unobserved features; because the latent blocks are per-feature, gradients can be attributed to individual features; because the sum is weighted by current class probabilities, acquisitions focus on disambiguating likely classes rather than merely lowering entropy. The paper's thesis is that this score is a faithful, tractable proxy for the long-term predictive value of measuring a feature, and that this is why SEFA outperforms both RL and CMI-based baselines on the evaluated tasks.","pith_inferences":["An untested consequence is that SEFA's score could be read as a gradient-based proxy for expected loss reduction; a direct comparison against one-step lookahead on expected 0-1 loss would clarify when the proxy over- or under-orders features.","Because the encoder factorizes per feature, correlations between features are only exploited through the predictor's gradients, so SEFA's margin over baselines on datasets with strongly correlated features is a natural stress test the paper does not run.","The paper's limitation that SEFA is classification-only suggests a discretized auxiliary classification head could extend the same acquisition score to regression; this is the authors' own suggested direction and can be tested without changing the scoring rule.","The gradient-norm heuristic may favour features with large but redundant latent gradients; measuring how often SEFA's first acquisition on the indicator task is the indicator itself would isolate the stochastic expectation's contribution from the heuristic's bias."],"forward_implications":["If the gradient-norm score is a faithful proxy, then one can build competitive AFA systems without RL training loops or generative CMI estimators, avoiding their known instabilities.","The paper's Proposition 4.2 implies that expectation over unobserved feature values — which SEFA approximates stochastically — is sufficient to make greedy acquisition optimal on the indicator problem, so the non-greedy behaviour is not incidental.","The ablation results imply each design choice (stochastic encoder, multiple latent samples, latent-space scoring, and probability weighting in multi-class settings) contributes to the measured gains, so simplified variants of SEFA should not match its full performance.","On multi-class tasks, probability weighting is the component that shifts acquisitions toward distinguishing likely classes; on binary tasks the paper proves the weighting cancels out, so any advantage there comes from the stochastic latent scoring itself."],"supporting_citations":[{"why":"Supplies the variational upper bound used to implement the information-bottleneck regularization of the latent space.","marker":"[Alemi et al., 2017]"},{"why":"Provides the information-bottleneck objective that motivates removing feature-level noise from the latent representations.","marker":"[Tishby et al., 1999]"},{"why":"Grounds the use of input gradients as local importance scores for classification decisions.","marker":"[Baehrens et al., 2010]"},{"why":"Establishes gradient-based saliency as a feature-attribution technique, which SEFA adapts to the latent space.","marker":"[Simonyan et al., 2014]"},{"why":"Supplies the synthetic classification tasks whose known optimal feature orderings are used to measure acquisition quality.","marker":"[Yoon et al., 2019]"},{"why":"Provides the Opportunistic RL baseline that SEFA must beat in the comparisons.","marker":"[Kachuee et al., 2019a]"},{"why":"Provides the GDFS CMI-maximization baseline that SEFA outperforms.","marker":"[Covert et al., 2023]"},{"why":"Provides the DIME CMI-estimation baseline used in the evaluation.","marker":"[Gadgil et al., 2024]"},{"why":"Provides the EDDI generative-CMI baseline used in the evaluation.","marker":"[Ma et al., 2019]"}],"fun_headline_variants":["SEFA: stochastic latent scoring beats myopic baseline","Stochastic encoders make myopic feature acquisition obsolete","SEFA: stochastic latent expectations pick better features","Active feature acquisition via stochastic latent scoring"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that the normalized Euclidean norm of the predictor's latent gradient for a feature, averaged over stochastic samples and weighted by current class probabilities, ranks features by how much measuring them will improve future predictions.","fun_headline_variants_meta":{"raw":{"variants":["SEFA: stochastic latent scoring beats myopic baseline","Stochastic encoders make myopic feature acquisition obsolete","SEFA: stochastic latent expectations pick better features","Active feature acquisition via stochastic latent scoring"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000707,"raw_usage":{"total_tokens":3143,"prompt_tokens":863,"completion_tokens":2280,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":2221}},"tokens_in":479,"tokens_out":2280,"duration_ms":19547,"temperature":1.0,"reasoning_tokens":2221,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:15:14.048081+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SEFA on the paper's indicator problem with $d=4$ and record the first acquisition: if the indicator feature is not acquired first in a large fraction of instances, the stochastic expectation is not capturing long-term value as claimed. A complementary check is a synthetic task where a feature has a large latent gradient norm but no effect on prediction accuracy under any unobserved combination; SEFA should not systematically select it.","supporting_citations":[],"review_version":1}