{"id":"c0245466-a291-496d-88a6-e5b7fb6c037b","arxiv_id":"2411.19923","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Under weak overlap, a latent confounder can be approximately identified from a single proxy or multiple unlabeled sources, and a reweighted mixture-of-experts model adapts to confounder shift.","lead":"This paper shows how to keep predictions accurate when an unmeasured background factor shifts between training and test data, using one extra observed clue or several unlabeled data sources. It matters because many real systems fail when hidden subpopulations become more or less common over time.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The identifiability guarantee collapses when Assumption 3 (weak overlap) is violated, yet the paper never measures η on any real dataset and the regularizer in Theorem 2 targets the extreme η=1 case.","rationale":"The reader identified Assumption 3 as the weakest assumption, and I agree: it is the condition that makes the factorization of P(S|X) approximately unique, and the paper provides no empirical verification of it. The theoretical argument inside Theorem 1 appears structurally coherent, but its conclusion is only as strong as η. Since Theorem 2 is proved only at η=1, the algorithm's regularizer is justified at the extreme point of the assumption, not in the regime where η is merely above 1/2. The real datasets are constructed so that Z is known, making a direct check of η feasible and decisive; without such a check, the claimed OOD robustness is a conditional result whose antecedent is unverified. The reader's conditional verdict already reflects the need to address this, so I do not move the verdict. I would not strengthen to reject because the theorem may still be true and the empirical results are suggestive; the missing verification is addressable.","tokens_in":23945,"tokens_out":26522,"duration_ms":244327,"concrete_test":"For each real dataset, use the known Z labels (available by construction: disability, age threshold, state, etc.) to compute holdout-calibrated estimates of η_i = max_{x in training} P(Z=i|X=x) for every latent class i, using a calibrated classifier with a validation split so that training overfitting does not inflate the maxima. If min_i η_i ≤ 1/2, Assumption 3 is violated and neither Theorem 1's identifiability bound nor Theorem 2's regularizer justification applies to that experiment; if all η_i are close to 1, the concern is mitigated and the conditional accept can stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that under Assumptions 1-3, the latent posterior P(Z|X) is approximately identifiable up to a permutation with error O((1-η)/(2η-1)), and that the proposed encoder-MoE-BBSE predictor can then adapt to P_te(Z) shift. Assumption 3 (Section 4.1) is load-bearing: it requires, for every latent class i, some x with P(Z=i|X=x) ≥ η > 1/2. If no such near-pure region exists, the factorization P(S|X)=Σ_z P(S|z)P(z|X) is not unique, and different valid factorizations can have very different posteriors; the Theorem 1 bound then degrades, becoming vacuous as η→1/2. Theorem 2, which justifies the Lvar regularizer, is proved only for η=1 exactly (the proof assumes ∃x* with φ*_i(x*)=1 for every i), so the method's selection rule rests on the extreme case. The paper never reports η for any real or synthetic dataset. In the real constructions, the assumption is not obviously satisfied: for ACS Employment, Z is disability status with P_tr(Z=1)=0.05 and the test flips to 0.95, so the existence of a training region where the minority class has posterior above 1/2 is asserted implicitly but never checked. The conclusion's limitation paragraph mentions only non-discrete confounders, not weak overlap. If Assumption 3 fails on these datasets, the theoretical guarantee does not apply to the reported experiments, and the empirical wins are left without the paper's stated justification.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies out-of-distribution generalization when an unobserved discrete confounder Z affects both X and Y and its marginal shifts between train and test. The main theoretical contribution is an approximate identifiability result (Theorem 1): under Markovian structure, full rank of P(S|Z), and a weak-overlap condition (Assumption 3), any two distributions matching the observed P(X,S) have latent posteriors P(Z|X) that agree up to permutation within O((1-η)/(2η-1)). The method then learns P(Z|X) with an encoder-decoder regularized by maximum row variance (Theorem 2), trains a mixture-of-experts predictor, and at test time reweights the gating using BBSE to estimate the shift in P(Z). Experiments on synthetic and ACS datasets show consistent gains over baselines.","tokens_in":24307,"tokens_out":14651,"duration_ms":111381,"significance":"If the theoretical results hold, the paper offers a valuable relaxation of prior proxy-based methods: only a single proxy or multiple unlabeled sources are needed, no test covariates during training, no concept variables, and no generative models. The approximate identifiability bound is a novel, clearly stated contribution, and the empirical gains are substantial (e.g., 0.709 vs 0.702 on ACS Employment, 0.883 vs 0.856 on ACS Income). However, the practical pipeline has gaps: the BBSE step is underspecified without Z labels, the weak-overlap assumption is unverified on real data, and the regularizer's justification covers only η=1. These gaps currently prevent the paper from fully delivering on its stated theoretical backing for the experiments.","major_comments":[{"comment":"The description of BBSE for estimating w = P_te(Z)/P_tr(Z) is incomplete. BBSE (Lipton et al., 2018) requires a classifier f(X) and a confusion matrix P(f(X)|Z) estimated from labeled examples. Since Z is unobserved, the paper never specifies how this confusion matrix is obtained; the only candidate is the inferred encoder posterior, whose calibration and consistency are not analyzed. Without this step, the test-time reweighting in Eq. (6) is not a well-defined algorithm. Please specify the estimator precisely (including the role of the encoder output and any consistency conditions) or state that the implementation uses an approximate confusion matrix and discuss the resulting bias.","section":"Section 5.2"},{"comment":"Assumption 3 (weak overlap) is load-bearing for Theorem 1, but the paper never verifies η on any dataset. For ACS Employment, P_tr(Z=1)=0.05; it is not evident that any region of the 54-dimensional feature space has P(Z=1|X=x) > 1/2, especially since the proxies (public insurance, independent living) are not part of X. The same concern applies to the other real datasets with extreme minority proportions (0.05). If Assumption 3 fails, the bound in Eq. (1) is inapplicable and the reported empirical advantage cannot be attributed to the paper's identifiability theory. Please report estimates of max_x P(Z=i|X=x) from the trained encoder on real and synthetic data, and discuss the implications for the theory.","section":"Section 4.1 and Section 6"},{"comment":"The proof claims '∃x* such that φ*_i(x*)=1 for all i', which is impossible for n_z>1 under a valid probability assignment. The argument only needs, for each i, some x_i with φ*_i(x_i)=1; with that correction the variance comparison goes through. More importantly, Theorem 2 is proved only for η=1, while the algorithm uses the L_var regularizer for general η>1/2. The paper should clarify whether the regularizer is a heuristic for η<1 or provide a justification (e.g., a continuity argument) for why the η=1 result supports the practical choice.","section":"Appendix A.4, proof of Theorem 2"},{"comment":"The proof asserts dim(span{P(S|X=x)}) = n_z, but this requires that the vectors P(Z|X=x) span R^{n_z} as x varies. Assumption 3 alone gives only coordinate-wise maxima; the proof should include an argument that these maxima imply the affine hull of {P(Z|X=x)} is the full simplex (e.g., by showing the points cannot lie in a proper affine subspace). This is likely true but is not established in the manuscript.","section":"Appendix A.1, proof of Theorem 1, Step 1"}],"minor_comments":[{"comment":"The notation in Eq. (4) uses M_{zs} while the proof writes Var(M_{z,:}); please harmonize the indexing.","section":"Appendix A.4"},{"comment":"Proposition 1 is proved only for binary Z; the extension to n_z > 2 should be stated or proven explicitly.","section":"Appendix A.3"},{"comment":"The left panel's y-axis is labeled 'Timing (seconds)' and the right panel's 'Memory Usage (MB)', but the two panels share a legend without a clear distinction of which curve corresponds to which metric; please make the axes self-contained.","section":"Figure 3"},{"comment":"The quotation 'all models are wrong, but some are useful' is attributed only implicitly through 'Inspired by Box's principle'; please add a citation to Box (1976) or rephrase to avoid an unattributed quote.","section":"Introduction"},{"comment":"The comparison table lists 'Existence of high-likelihood sample' as a difference from ProxyDA, but the practical implications of this assumption for the experiments are not discussed; a brief comment would help.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper is generally well-written and the theoretical idea is interesting, but the BBSE gap and the unverified weak-overlap assumption are significant. If the authors can specify the BBSE estimator and empirically assess Assumption 3, the contribution would be much stronger. The work fits the scope of AISTATS."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful paper with a strong empirical story, but the theory has real gaps—the test-time reweighting step is under-specified, and the regularizer's justification only covers the extreme η=1 case. I'd send it to serious peer review and ask for a revision, not a desk reject.\n\nWhat's actually new: the paper relaxes the assumptions needed for latent confounder shift by requiring only a single proxy (or multiple unlabeled sources), and it does not need the proxy or X^te at test time. Theorem 1's approximate identifiability bound O((1-η)/(2η-1)) is a solid contribution—it cleanly connects weak overlap to posterior identifiability. The max-variance regularizer is a simple, clever idea, and Theorem 2 gives it some support. Empirically, the method beats strong baselines on four of five ACS datasets, including the hard disability-status shift on Employment and the Travel Time shift; the scalability plot against ProxyDA makes its practical case.\n\nSoft spots, in order:\n\n1. The BBSE step is not derived. BBSE normally needs P(f(X)|Z), a confusion matrix, but Z is never observed. The paper never says how P(f(X)|Z) is estimated from the inferred posterior. This is load-bearing: without it, the test-time reweighting is hand-wavy. This should be fixable with an explicit algorithm.\n\n2. Theorem 2 proves the regularizer selects the true M only when η=1 exactly—the proof needs a point x* with φ*_i(x*)=1 for every class. Real datasets almost certainly have η<1. The paper never estimates η on any dataset, and on ACS Employment the minority class has P_tr=0.05, so a high-η region for that class is not obviously present. The theoretical justification for the regularizer is narrower than claimed.\n\n3. The paper never checks Assumption 3, and the bound is vacuous for η close to 0.5. Measuring η or reporting sensitivity to it would go a long way.\n\n4. Minor: the related-work section misses the anchor-point/separability line in topic models, which is the same idea as pure-point posterior recovery. Also one empirical win (ACS Mobility) is within noise. No code is released, but the method is simple enough to reimplement.\n\nOverall, the central claim—that a single proxy suffices for approximate identifiability under weak overlap—is plausible and worth taking seriously. The paper is clearly written and the experiments are informative. The gaps are addressable, not fatal. This is a good candidate for peer review, with a request for a careful derivation of the BBSE step and an empirical check of η.\n\nWho should read it: anyone working on OOD robustness for tabular or administrative data, and people studying latent shift with weak identifiability. I'd bring it to a reading group.","headline":"Useful method with strong experiments, but the BBSE test-time step is under-derived and the regularizer theory only covers η=1; send to review, ask for a revision.","tokens_in":24833,"tokens_out":4277,"would_cite":true,"duration_ms":37326,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single additional variable — a proxy or multiple sources — approximately identifies the unobserved confounder's posterior $P(Z|X)$ under three structural assumptions, and the recovery powers a scalable, OOD-robust mixture-of-experts…","keywords":["out-of-distribution generalization","unobserved confounder","latent shift","identifiability","proxy variable","mixture of experts","black box shift estimation","latent confounder shift"],"falsifier":"Build a synthetic proxy-task dataset whose true posterior is engineered to satisfy $\\max_x P(Z=i|X=x)=0.55$ for every class, and run the full encoder-decoder plus mixture-of-experts pipeline under the same style of train-to-test confounder shift used in Section 6; Theorem 1 then permits a constant-order identification error, so if the recovered posterior and test accuracy are essentially as strong as at $\\eta\\approx 1$, the theorem's bound is not the operative mechanism, while a collapse would indicate that the ACS results rely on near-pure inputs. A complementary observation is the trained encoder's per-class maximum posterior on the ACS training sets: a value at or below $1/2$ for any class would falsify Assumption 3 on real data.","tokens_in":23768,"feed_emoji":"🧩","tokens_out":13066,"duration_ms":98272,"temperature":0.7,"pith_summary":"The paper sets out to show that out-of-distribution robustness under a shifting unobserved confounder does not need the multiple proxies, concept variables, or test-domain access that prior methods require: one extra variable, either a discrete proxy of the confounder or a collection of training sources with different confounder mixes, is enough to identify the latent posterior $P(Z|X)$ up to a small error. The error is $O((1-\\eta)/(2\\eta-1))$, controlled by how close each latent class comes to having an input where its posterior reaches one, and the paper argues this error tends to zero for high-dimensional informative features. On this foundation the authors build a two-stage predictor: an encoder-decoder recovers $P^{\\mathrm{tr}}(Z|X)$, a mixture-of-experts classifier uses that posterior as its gating function, and at test time the gate is reweighted by a BBSE estimate of the confounder shift $P^{\\mathrm{te}}(Z)/P^{\\mathrm{tr}}(Z)$ computed from unlabeled test inputs alone. If these identifiability results hold, latent-shift OOD problems become solvable at roughly the computational cost of ordinary supervised learning, which is the concrete payoff the paper is after.","feed_headline":"One proxy suffices to adapt to hidden confounder shifts","feed_subtitle":"Proved identifiability plus a reweighted mixture-of-experts gate beats strong OOD baselines on ACS data.","key_machinery":"The load-bearing object is the factorization $P(S|X=x)=\\sum_z P(S|Z=z)\\,P(Z=z|X=x)$ that the Markov assumption $S \\perp X | Z$ makes valid: each observed proxy conditional is a convex combination of the rows of the proxy-confounder matrix $M=P(S|Z)$, weighted by the latent posterior. Because $M$ has full rank, the set of posteriors that could explain the observed data forms an $n_z$-dimensional subspace, so any two candidate posteriors are linked by a linear map $A$ with $P(Z|X=x)=A\\,Q(Z|X=x)$; weak overlap forces $A$ close to a permutation matrix, which is what turns matching observed marginals into matching posteriors. A second mechanism is the row-variance regularizer $L_{\\mathrm{var}}(M)=\\max_z \\frac{1}{n_s}\\sum_s (M_{zs}-1/n_s)^2$, which selects the true confounder-proxy matrix among all minimizers of the reconstruction loss, and a third is the BBSE estimator of the density ratio $w=P^{\\mathrm{te}}(Z)/P^{\\mathrm{tr}}(Z)$, computed from unlabeled test features, which reweights the mixture gate while the expert distributions stay fixed.","core_discovery":"Under Assumptions 1–3 (the proxy is conditionally independent of $X$ given $Z$; the proxy-confounder matrix $P(S|Z)$ is full rank; and each latent class has some input with posterior $P(Z=i|X=x) \\ge \\eta > 1/2$), Theorem 1 states that any two distributions matching the observed marginal $P(X,S)$ have latent posteriors that agree up to permutation, with $\\sup_x |P(Z=i|X=x) - Q(Z=\\pi(i)|X=x)| \\le O((1-\\eta)/(2\\eta-1))$. At $\\eta=1$ the bound vanishes and the posterior is fully identifiable (Corollary 1), and Proposition 1 argues that as the number of discriminative features grows the effective $\\eta$ approaches one, so high-dimensional data approaches full identifiability. Theorem 2 resolves the residual factorization ambiguity: among all matrices that minimize the reconstruction loss, the one belonging to a confounder satisfying weak overlap at $\\eta=1$ has the smallest maximum row variance, which is the justification for the regularizer used in training. The paper further claims that the resulting predictor, encoder-estimated gating plus class-specialized experts with BBSE gate reweighting, outperforms invariant, domain-adaptation, and proxy baselines on synthetic and ACS benchmark tasks and scales linearly in sample size.","pith_inferences":["A deployment-ready diagnostic follows from the paper's own logic: estimate $\\max_x P(Z=i|X=x)$ per class with the trained encoder; if any class never exceeds $1/2$, Assumption 3 is violated on that dataset and the theoretical support for the recovered gate lapses even if accuracy looks good.","The test-time reweighting inherits label-shift assumptions from BBSE: if the shift changes $P(X|Z)$ rather than only $P(Z)$, the reweighting formula is misspecified. A checkable consequence is that the encoder's posterior marginals on unlabeled test features should match the BBSE-estimated $P^{\\mathrm{te}}(Z)$; large discrepancies would indicate a shift outside the assumed invariance.","The recovery problem is structurally an anchor-based decomposition of the observed matrix $P(S|X)$, so decomposition algorithms with anchor guarantees could give recovery conditions at or below the $\\eta=1/2$ threshold and tighten the theorem's worst-case bound.","The paper selects $n_z$ by validation reconstruction loss; for a genuinely continuous confounder that is only binned, the chosen number of classes may overstate the true $n_z$ and the ordering information of the confounder is discarded, so extending the claims to continuous $Z$ would require a separate argument."],"forward_implications":["A single discrete proxy variable, or one labeled source plus unlabeled sources, is sufficient for OOD robustness under latent confounder shift; concept variables and multiple proxies are not needed.","The less the latent classes overlap, the tighter the recovered posterior: near-pure example inputs for every class make the approximation nearly exact, and high-dimensional discriminative features push toward full identifiability.","At test time the method needs only unlabeled inputs: BBSE estimates the shift ratio $P^{\\mathrm{te}}(Z)/P^{\\mathrm{tr}}(Z)$ from features of the test data, and adapting to a new confounder distribution reduces to reweighting the gating function.","The procedure scales linearly in training time and keeps memory constant as sample size grows, in contrast to kernel-based proxy methods with cubic time and growing memory.","On the reported synthetic and ACS benchmarks, the method achieves higher OOD accuracy than ERM, IRM, GroupDRO, VREx, DANN, DeepCORAL, and ProxyDA."],"supporting_citations":[{"why":"Establishes that a single proxy does not generally identify the latent confounder distribution, the gap this paper closes under its assumptions.","marker":"Pearl, 2010"},{"why":"Supplies the proximal-causal-inference framework from which the Markov and full-rank proxy matrix assumptions are drawn.","marker":"Miao et al., 2018"},{"why":"ProxyDA is the closest baseline and the work being relaxed, needing multiple sources plus a proxy and test-domain access.","marker":"Tsai et al., 2024"},{"why":"Provides Black Box Shift Estimation, the method-of-moments procedure used to estimate the confounder shift ratio from unlabeled test features.","marker":"Lipton et al., 2018"},{"why":"Its high-dimensional overlap result grounds Proposition 1, which argues the weak-overlap parameter eta tends to one as feature dimension grows.","marker":"D'Amour et al., 2021"},{"why":"Defines the latent-subgroup-shift task and the concept-and-proxy machinery this paper replaces with a single additional variable.","marker":"Alabdulmohsin et al., 2023"}],"fun_headline_variants":["One proxy suffices for OOD robustness","Single proxy achieves provable confounder shift adaptation","Provable OOD robustness with one proxy variable","Scalable OOD robustness via a single proxy","One proxy beats complex predictors on confounder shifts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is weak overlap (Assumption 3): for every latent class there must exist some training input whose posterior for that class is above one half, and the variance regularizer is justified at the extreme where some input reaches posterior exactly one; the paper never measures this quantity on the real datasets, and without it the factorization of the observed distribution into confounder posterior and proxy mechanism is not uniquely recoverable.","fun_headline_variants_meta":{"raw":{"variants":["One proxy suffices for OOD robustness","Single proxy achieves provable confounder shift adaptation","Provable OOD robustness with one proxy variable","Scalable OOD robustness via a single proxy","One proxy beats complex predictors on confounder shifts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000212,"raw_usage":{"total_tokens":1479,"prompt_tokens":1067,"completion_tokens":412,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":683,"completion_tokens_details":{"reasoning_tokens":342}},"tokens_in":683,"tokens_out":412,"duration_ms":4693,"temperature":1.0,"reasoning_tokens":342,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:42:20.636162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a synthetic proxy-task dataset whose true posterior is engineered to satisfy $\\max_x P(Z=i|X=x)=0.55$ for every class, and run the full encoder-decoder plus mixture-of-experts pipeline under the same style of train-to-test confounder shift used in Section 6; Theorem 1 then permits a constant-order identification error, so if the recovered posterior and test accuracy are essentially as strong as at $\\eta\\approx 1$, the theorem's bound is not the operative mechanism, while a collapse would indicate that the ACS results rely on near-pure inputs. A complementary observation is the trained encoder's per-class maximum posterior on the ACS training sets: a value at or below $1/2$ for any class would falsify Assumption 3 on real data.","supporting_citations":[{"cited_title":"On measurement bias in causal inference","cited_arxiv_id":null,"evidence_quote":"Establishes that a single proxy does not generally identify the latent confounder distribution, the gap this paper closes under its assumptions."},{"cited_title":"Tchetgen Tchetgen","cited_arxiv_id":null,"evidence_quote":"Supplies the proximal-causal-inference framework from which the Markov and full-rank proxy matrix assumptions are drawn."},{"cited_title":"Proxy methods for domain adaptation","cited_arxiv_id":null,"evidence_quote":"ProxyDA is the closest baseline and the work being relaxed, needing multiple sources plus a proxy and test-domain access."},{"cited_title":"Kusner, Stephen R","cited_arxiv_id":null,"evidence_quote":"Defines the latent-subgroup-shift task and the concept-and-proxy machinery this paper replaces with a single additional variable."}],"review_version":1}