Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift

Jiaye Teng; Shaoli Wang; Xin Liu; Zhen Zhang

REVIEW 2 major objections 4 minor 22 references

Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reweighted two-stage estimator attains the minimax rate for moment estimation under covariate shift.

desk verdict A sensible two-stage estimator attached to a minimax lower bound that isn't proved: the class is ill-posed without a norm bound, and the hard-instance constructions don't validly sit in the stated class. read the letter →

arxiv 2506.23453 v1 pith:UZ5UYZ67 submitted 2025-06-30 stat.ML cs.LG

Zhen Zhang , Xin Liu , Shaoli Wang , Jiaye Teng This is my paper · ORCID

classification stat.MLcs.LG MSC 62G0562G20

keywords covariateshiftmomentestimationminimaxlowerboundtwo-stageimportanceweightingcontrolvariatesdoublerobustnessSobolevspaces

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether an optimal function estimator trained under a source distribution can be calibrated to estimate the $q$-th moment under a different target distribution at the best possible statistical rate. The answer is yes: the paper proves a minimax lower bound of order $B\bar{b}\,n^{\max\{-q(s/d-1/p)-1,\,-1/2-s/d\}}$ for Sobolev functions, and constructs a two-stage estimator that matches it up to a logarithmic factor. The estimator first fits the function on half the sample, then uses likelihood-ratio reweighted residuals from the other half as a control-variate correction. A truncated plug-in version is shown to remain consistent even when the source and target densities are unknown, as long as either the function estimator or the likelihood-ratio estimator is consistent.

What carries the argument

The load-bearing device is the control-variate identity $E_{X\sim P^*}[f^q(X)] = E_{X\sim P^*}[\hat{f}^q(X)] + E_{X\sim P^\circ}[w(X)(f^q(X)-\hat{f}^q(X))]$, with $w=p^*/p^\circ$ the likelihood ratio. It converts the target moment into a term computable from the fitted function plus an importance-weighted residual that can be averaged over source samples. The oracle optimality of $\hat{f}$ under Assumption 1 controls the residual; the truncation map $\tau_T(\cdot)=\min\{\cdot,T\}$ is what removes the dependence on the worst-case shift constant $B$ and, when $w$ is plugged in, yields double robustness.

What would settle it

Read the Case I construction in the proof of Theorem 1: with $p^*$ uniform of density $B$ on a cube of volume $1/B$ and $g_1$ supported on a cube of volume $1/(200n)$, the requirement $w(x)\le B$ cannot hold on the bump for fixed $B$ as $n$ grows; verifying this incompatibility decides whether the claimed $B\bar{b}\,n^{-q(s/d-1/p)-1}$ lower bound is valid on the stated class.

Watch

Extended reading notes

Core claim

The paper's central claim is that, for $p\ge 2$, $q\le p\le 2q$, and functions in the Sobolev space $W^{s,p}(\Omega)$ with source density between $\underline{b}$ and $\bar{b}$ and likelihood ratio at most $B$, the minimax risk for estimating $I_f^q=E_{X\sim P^*}[f^q(X)]$ is $B\bar{b}\,n^{\max\{-q(s/d-1/p)-1,\,-1/2-s/d\}}$ up to logarithmic factors. The matching upper bound is attained by splitting the sample, computing an oracle-optimal nonparametric fit $\hat{f}_{S_1}$ on the first half, and forming $\hat{H}_B^q = E_{X\sim P^*}[\hat{f}_{S_1}^q(X)] + \frac{2}{n}\sum_{(x_i,y_i)\in S_2} w(x_i)(y_i^q-\hat{f}_{S_1}^q(x_i))$. The first summand estimates the leading moment of the fitted function; the weighted residual corrects the bias under the target measure. The lower bound is obtained from two fuzzy-hypotheses constructions, one based on a rare sharp bump amplified by covariate shift and one based on many small oscillations.

Load-bearing premise

The load-bearing premise is that the constructed target density, uniform with density $B$ on a subcube of volume $1/B$, can coexist with a function bump supported on a cube of volume $1/(200n)$ while satisfying $\sup w\le B$; for fixed $B$ and large $n$ this forces $B\ge 200n$ and pulls the instance out of the class.

Editorial extensions

If this is right

If the two theorems are right, practitioners can use any oracle-optimal function estimator from the source domain, add one calibration split, and obtain the best possible worst-case rate for target-domain moments.
The rate splits cleanly: $B$ is the unavoidable cost of covariate shift, $\bar{b}$ the cost of an irregular source density, and the exponent in $n$ comes from the interaction of smoothness, integrability, and moment order.
When densities are unknown, replacing $w$ by a truncated estimated ratio adds only the $m^{-1/2}$ Monte Carlo error from the unlabeled target sample and preserves consistency if either $\hat{f}$ or $\hat{w}$ is consistent.
Truncating at a threshold $T$ changes the rate to $\bar{b}\,r(n)^{\alpha/(\alpha+1)}$ under a tail condition $P(w>T)\le T^{-\alpha}$, which can beat the $B$-dependent bound when $B$ is large.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the two-stage form is taken as a template, the same control-variate correction should apply to other functionals of $f$, such as distributional or risk statistics, whenever the residual admits an importance-weight representation; the paper only proves the moment case.
The lower-bound geometry suggests that the worst-case cost of covariate shift is realized by rare, sharply localized features of $f$; a testable extension would be to check whether truncation or density-ratio smoothing gives better practical rates on distributions with unbounded ratios.
One could test the double-robustness claim directly: with a deliberately misspecified $\hat{f}$ that is still consistent and a badly initialized $\hat{w}$, the truncated estimator should remain consistent, while the untruncated version should fail.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague—

This paper has a good algorithmic idea wrapped in a minimax theorem that isn't proven. The two-stage estimator—fit f under the source distribution, then recalibrate with an importance-weighted control variate—is a natural extension of Blanchet et al., and the upper-bound analysis (Theorem 2) is probably sound: conditional on Assumption 1, the variance calculation goes through. The truncated variant for unknown likelihood ratios (Theorem 3) is also a sensible practical adaptation, and the simulations do show the qualitative effects of B, \bar b, and smoothness. I'd credit the authors for identifying a real gap in the literature and for a clean statement of what the rate ought to be.

The problems are in the lower bound and the problem class. Definition 1 puts f in W^{s,p}(Ω) without any bound on the Sobolev norm. Since W^{s,p} is a vector space, c f is in the class for any c>0, and the q-th moment scales by c^q. That makes the worst-case risk unbounded, so the "minimax lower bound" in Theorem 1 isn't a finite statement. The proof actually leans on this: Case II builds hard functions with amplitude (B\bar b)^{1/q}, which is a rescaling of a fixed bump family. If the intended class is a unit Sobolev ball, those functions leave the ball; if it isn't, the problem is ill-posed. Either way, the central claim fails.

The Case I construction in Appendix A.1 has a separate, more concrete problem. The target p* is uniform on a subcube of volume 1/B, but the proof computes the moment integral as if p* equals B on the whole cell Ω_i*. Only the subcube carries mass, so the integral is wrong unless the subcube is the whole cell, which requires B = m^d = 200n. For fixed B and growing n, the construction isn't in the class. (The stress-test note's inequality here is reversed—containment requires B ≥ m^d, not ≤—but the scaling objection above is the decisive flaw.) Assumption 1 has the same scale disease: an oracle bound independent of ||f|| can't hold for f and 2f.

What survives: the upper-bound calculations and the estimator itself. What doesn't survive: the claim of minimax optimality. The paper is honest about some limitations—the Appendix B.2 discussion of the truncated bound beating the lower bound is candid—but the mismatch is bigger than a log factor.

I'd send this to a serious referee, because the algorithmic idea is worth discussing and the authors may be able to repair the lower bound by defining the class properly (e.g., a unit ball) and redoing the hard instances. But as it stands, it should not be published as a claimed minimax result. If I were the editor, I'd invite a revision with the lower-bound proof as the central issue.

Referee Report

2 major / 4 minor

Summary. This paper studies minimax estimation of the q-th moment E_{P*}[f^q(X)] under covariate shift, where f lies in a Sobolev space, the source density p◦ is bounded above and below, and the likelihood ratio p*/p◦ is uniformly bounded by B. The paper claims a minimax lower bound of order B·\bar b·n^{max{-q(s/d-1/p)-1, -1/2-s/d}} (Theorem 1), proposes a two-stage control-variate estimator that attains this rate up to logarithmic factors (Theorem 2), and then introduces a truncated plug-in version for unknown densities with a double-robustness guarantee (Theorems 3-4 and Corollary 1). The central claim is the minimax optimality of the proposed two-stage procedure.

Significance. If the results were correct, they would give a sharp characterization of the cost of covariate shift for moment estimation and a practical estimator with attractive stability properties. The paper contains a detailed upper-bound proof, a concrete algorithm, and an explicit oracle assumption; the two-stage control-variate construction with truncation and double robustness is a sensible proposal. However, the main theorems are not well-posed as stated because the function class V is scale-invariant and unbounded, which makes the claimed minimax rate meaningless and the upper-bound oracle inconsistent. This is a load-bearing flaw that undermines the central optimality claim.

major comments (2)

[Section 3, Definition 1, and Theorem 1, Eq. (2)] The class V imposes no bound on the Sobolev norm of f. Since W^{s,p}(Ω) is a vector space, whenever f belongs to V, so does c f for every c>0. For any fixed target density p*, the q-th moment satisfies I_{cf}^q = c^q I_f^q, so the functional is unbounded on V. Moreover, using a bump supported on a cell of source probability at most m^{-d} as in Appendix A.1, with probability at least (1-m^{-d})^n the observed data under c f are exactly those under f=0; consequently any finite-valued estimator has worst-case error growing like c^q as c→∞. The true minimax risk over V is therefore infinite, and the finite lower bound in Eq. (2) is not the minimax rate. The lower-bound family in Appendix A.1 Case II, F = {(B\bar b)^{1/q}(M + Σ η_j f_j)}, places the factor (B\bar b)^{1/q} in the amplitude of the function, showing that the B coefficient in the claimed rate is an artifact of rescaling f rather than of covariate shift. This invalidates the central claim of minimax optimality.
[Assumption 1, Eq. (4), and Theorem 2, Eq. (5)] The oracle error bound in Eq. (4) is independent of the scale of f. The moving least squares construction described in Appendix B.3 is linear in the observed function values, so if an oracle satisfies Eq. (4) for f, the analogous construction for c f has error scaled by c. For c≠1, the same right-hand side cannot hold on the stated class V, which contains both f and c f. Thus Assumption 1 is not satisfiable on the class as defined, and the upper bound in Theorem 2, which relies on Assumption 1, is not established for the claimed problem. The claimed matching with Theorem 1 is therefore not meaningful.

minor comments (4)

[Appendix A.1, Case I] The reader's specific concern about the volume of the p* support appears reversed: the condition for the p* support cube to contain the cell Ω_{i*} is B ≤ m^d = 200n, which is satisfied for fixed B and n→∞. A separate issue is that the proof says 'let p◦(x) be any p.d.f.', but the uniform-B-bounded condition requires p◦ ≥ 1 on the support of p*; the construction should either choose p◦ accordingly (e.g., uniform density 1 on Ω) or justify that the constructed pair lies in V.
[Theorem 3, Eq. (7) and proof in Appendix A.3] In the statement of Theorem 3, the notation 'V ariance' should be 'Variance', and the proof's step 'Since \hat f_{S1} and f are in L^q(P*)' drops the dependence of the bias bound on the norms of f and \hat f_{S1}; on a normalized function class these norms would appear as constants, but on the stated unbounded class the bound is not justified.
[Section 6, numerical experiments] The experiments use a random forest as the first-stage learner, which does not satisfy Assumption 1; the simulations are therefore illustrative rather than a verification of the theoretical rates, and this should be stated explicitly.
[Appendix A.1, Case II, mixture formula] There is a typographical error in the expression for P1: one Dirac factor is written as δ_{M+f_j(x_i)}(y_i) instead of δ_{(B\bar b)^{1/q}(M+f_j(x_i))}(y_i); the surrounding text makes the intended expression clear.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: lower and upper bounds are derived by independent constructions, the oracle rate (Assumption 1) is sourced from external literature (Wendland; Blanchet et al.; Krieg & Sonnleitner; Reznikov & Saff), and the single self-citation (Teng et al. 2021) is descriptive and not load-bearing.

full rationale

The derivation chain is self-contained and not circular. The two-stage estimator (Eq. 3) follows from the exact identity in Eq. (6), E_{P*}[f^q(X)] = E_{P*}[f_hat^q(X)] + E_{P*}[f^q(X) - f_hat^q(X)], rewritten as E_{P*}[f_hat^q(X)] + E_{P deg}[w(X)(f^q(X) - f_hat^q(X))] via the change of measure; no target rate is injected into the estimator, and the proof of Theorem 2 first establishes unbiasedness and then bounds the variance. The upper bound rests on Assumption 1, whose rate (Eq. 4) is the standard nonparametric function-estimation rate attributed to external results (Wendland 2004; Blanchet et al. 2024; Krieg & Sonnleitner 2024; Reznikov & Saff 2016), with a construction in Appendix B.3; none of these are the present authors' works. The lower bound (Theorem 1) is built independently by the method of two fuzzy hypotheses (Tsybakov 2009): Case I uses a single Sobolev bump of amplitude b_bar^{1/q} m^{-s+d/p} with target density B on the bump cube, yielding B * b_bar * n^{-q(s/d - 1/p) - 1}; Case II uses an oscillation family (B * b_bar)^{1/q}(M + sum_j eta_j f_j) with kappa ~ n^{-1/2}, yielding B * b_bar * n^{-s/d - 1/2}. These exponents arise from standard bump/cube-volume and Rademacher-concentration arguments, not from feeding the claimed rate into the construction. The truncated bound (Theorem 3) is an explicit bias-variance balance, and the paper openly states (Remark after Theorem 3 and Appendix B.2, 'The estimator H^q_T in Theorem 3 may not be minimax optimal') that the resulting bound may beat the minimax lower bound when B is large, the opposite of engineering agreement with a target rate. The only self-citation is Teng et al. (2021) in the related-work list; it is descriptive and not load-bearing. Concerns about the well-posedness of V (no norm bound on f; the B factor entering Case II via function amplitude rather than covariate shift) are correctness/validity issues, not circularity, and do not affect this score.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The upper-bound analysis rests on Assumption 1, Sobolev embedding, and the tail condition in the truncated case. The lower-bound proof does not introduce new entities but implicitly assumes a target density that violates the fixed-B class, which is a logical gap rather than a free parameter.

free parameters (1)

truncation threshold T = T = O(r(n)^(-1/(α+1)))
T balances bias and variance in the truncated estimator; the paper suggests selecting it by grid search in practice, so it is a hand-chosen algorithmic parameter.

assumptions (4)

domain assumption Assumption 1: existence of an oracle function estimator satisfying the L^r convergence bound (4)
The upper bound in Theorem 2 relies on the existence of an estimator of f with the stated L^r error. The paper cites moving least squares as a construction, but the assumption itself is imported from prior work.
standard math Sobolev embedding theorem
Used throughout the proof of Theorem 2 to move between L^p norms and control moments of powers of f.
ad hoc to paper Tail condition g(T) ≤ T^{-α} on the likelihood ratio
Introduced in Theorem 3 to obtain the B-independent bound. The paper acknowledges this restricts the class V and may make the upper bound better than the minimax lower bound.
domain assumption Covering radius concentration for random points with bounded density (Lemma 4)
Used in Appendix B.3 to verify that moving least squares achieves the oracle rate under a bounded density, citing Reznikov and Saff.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift." pith.science (2026). https://pith.science/paper/UZ5UYZ67

@misc{pith2026250623453,
  author       = {Pith},
  title        = {Pith review of: Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UZ5UYZ67}},
  note         = {Machine review of arXiv:2506.23453}
}

read the original abstract

Covariate shift occurs when the distribution of input features differs between the training and testing phases. In covariate shift, estimating an unknown function's moment is a classical problem that remains under-explored, despite its common occurrence in real-world scenarios. In this paper, we investigate the minimax lower bound of the problem when the source and target distributions are known. To achieve the minimax optimal bound (up to a logarithmic factor), we propose a two-stage algorithm. Specifically, it first trains an optimal estimator for the function under the source distribution, and then uses a likelihood ratio reweighting procedure to calibrate the moment estimator. In practice, the source and target distributions are typically unknown, and estimating the likelihood ratio may be unstable. To solve this problem, we propose a truncated version of the estimator that ensures double robustness and provide the corresponding upper bound. Extensive numerical studies on synthetic examples confirm our theoretical findings and further illustrate the effectiveness of our proposed method.

Figures

Figures reproduced from arXiv: 2506.23453 by the authors.

**Figure 2.** Impact of covariate shift, sampling strategy, and function class on the error of the estimator, [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Comparison of the Monte Carlo estimator, one-stage estimator and two-stage estimator on synthetic data, with performance measured by the L1 norm between true and estimated values. Each subfigure presents boxplots of the errors for different methods across varying levels of covariate shift B. The Monte Carlo estimator performs better than the two-stage methods when there is no covariate shift (B = 1). However, as B i… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Effect of truncation. Each subfigure presents boxplots of errors for both untruncated and truncated estimators across different levels of covariate shift B. Truncation improves stability while preserving accuracy, with its impact becoming more pronounced as B increases…

**Figure 5.** Figure 5: Boxplots of errors for the Monte Carlo estimator, one-stage estimator, and two-stage [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 18 canonical work pages

[1]

In order to determine the separation between two priors µ0 and µ1, we need to derive the con- centration inequality of each prior first

= 1 − p. In order to determine the separation between two priors µ0 and µ1, we need to derive the con- centration inequality of each prior first. Define A := R Ωj B¯b · (M + fj(x))qdx and C :=R Ωj B¯b · (M − fj(x))qdx, we may derive the lower bound on the quantity ∆′ := A − C >0: ∆′ = Z Ωj B¯b · (M + fj(x))qdx − Z Ωj B¯b · (M − fj(x))qdx = B¯b · Z Ωj Z fj...

work page 2025
[2]

of the uniform distribution over [0, 1]d

(Case II) We proceed to construct the second lower bound instance: letp◦(x) and p∗(x) be the p.d.f. of the uniform distribution over [0, 1]d. For any 1 ≤ j ≤ md, consider fj defined as follows: fj(x) = m−sK(m(x − cj)) (x ∈ Ωj), 0 (otherwise), we further pick two constants α, Msatisfying α := ∥K∥L∞([− 1 2 , 1 2 ]d) and M = 3α. Now consider the following fi...

work page 2024
[3]

A.5 P ROOF OF COROLLARY 1 Corollary 1 (Double Robustness of Plug-in Truncated Estimator)

Thus, the convergence rate of ˜H q T is: E h ˜H q T (S ∪ S′) − I q f i ≲ E h ˆH q T (S) − I q f i + m− 1 2 . A.5 P ROOF OF COROLLARY 1 Corollary 1 (Double Robustness of Plug-in Truncated Estimator). If P(Ω− T ) → 0 as n → ∞, then the estimator ˜H q T exhibits double robustness. Specifically, this means that as long as either ˆw(x) or ˆfS1 (x) is consisten...

work page 2025
[7]

Instance weighting for neural machine translation domain adaptation

Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. Instance weighting for neural machine translation domain adaptation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1482–1488,

work page 2017
[10]

≥ 1 + ϵ 2 = 1 − β0, µ1(f ∈ W s,p(Ω) : I q f ≥ c + ∆) = µ1(I q f ≥ I q g1 ) ≥ 1 + ϵ 2 = 1 − β1. 14 Published as a conference paper at ICLR 2025 We proceed to compute the mixture distribution under µ0 and µ1: P0(⃗x, ⃗ y) = P(⃗x) · P0(⃗ y| ⃗x) = P(⃗x) · Z P(⃗ y| ⃗x, θ)µ0(dθ) = nY i=1 p◦(xi) ! ·   1 + ϵ 2 Y i:xi∈Ωi∗ δ0(yi) + 1 − ϵ 2 Y i:xi∈Ωi∗ δg1(xi)(yi) ...

work page 2025
[13]

Assume that p ≥ 2, q ≤ p ≤ 2q and s d > 1 p − 1 2q

A.2 P ROOF OF THEOREM 2 Theorem 2 (Upper Bound on Moment Estimation with Smoothness) . Assume that p ≥ 2, q ≤ p ≤ 2q and s d > 1 p − 1 2q . Let (f, p◦, p∗) ∈ V(b, ¯b, B, s, p) and we have sample S = {(xi, yi = f (xi))}n i=1. If ˆfS1 satisfies the Assumption 1, then the estimator ˆH q B constructed in equation 3 above satisfies ES h ˆH q B(S) − I q f i ≲ B...

work page 2024
[14]

This gives us the final upper bound under the assumption that s ∈ ( d(2q−p) p(2q−2) , d p ) as follows: ES1 Z Ω g2 S1 (x)f 2q−2(x) dx ≲ ¯b · n− 2s d

Thus, by applying Jensen’s inequality and choosing r = p in 4 of Assumption 1, we can upper bound the last term as follows: ES1 h ∥gS1 (x)∥2 Lp(Ω) i = ES1 ∥gS1 (x)∥L p 2 (Ω) 2 p ≤ES1 h ∥gS1 (x)∥p Lp(Ω) i 2 p = ES1 h ∥ ˆfS1 x − f (x)∥p Lp(Ω) i 2 p (Jensen’s inequality) ≲¯b · ( n 2 )− s d +( 1 p − 1 p )+ 2 ≲ ¯b · n− 2s d . This gives us the final upper boun...

work page 2025
[15]

(22) Combining the upper bounds derived in 19 20, 21 and 22 we may deduce that: ES1 ES2|S1 f q(X) − ˆf q S1 (X) 2 ≲ ¯b2 · h n2q( 1 p − s d )−1 + max{n− 2s d , n2q( 1 p − s d )−1} i

This gives us the final upper bound under the assumption that s ∈ ( d(2q−p) 2pq , d(2q−p) p(2q−2) ) as follows: ES1 Z Ω g2 S1 (x)f 2q−2(x) dx ≲ ¯b · n2q( 1 p − s d )−1. (22) Combining the upper bounds derived in 19 20, 21 and 22 we may deduce that: ES1 ES2|S1 f q(X) − ˆf q S1 (X) 2 ≲ ¯b2 · h n2q( 1 p − s d )−1 + max{n− 2s d , n2q( 1 p − s d )−1} i . (23) ...

work page 2025

Show all 22 references

[16]

Z Ω− T p∗(x) ˆf q S1 (x)dx + Z Ω− T T p◦(x) · (f q(x) − ˆf q S1 (x))dx − Z Ω− T p∗(x)f q(x)dx # =ES1

Proof. Decompose the mean square error ES ˆH q T (S) − I q f 2 into bias part ES [ ˆH q T (S)] − I q f 2 and variance part ES ˆH q T (S) − ES [ ˆH q T (S)] 2 . We first compute E h ˆH q T (S) | S1 i . Recall that Ω+ T := {x : w(x) ≤ T } and Ω− T = Ω/Ω+ T , then we may deduce t...

2025
[18]

To estimate the propensity score, we can apply a classification model that distinguishes between source and target samples

with m m+n and n m+n, respectively. To estimate the propensity score, we can apply a classification model that distinguishes between source and target samples. The propensity score estimator, denoted as ˆw(x), represents the estimated probability that a sample with feature x b...

2025
[19]

These potential outcomes are un- known functions of the covariate X

and control (T = 0). These potential outcomes are un- known functions of the covariate X. The difficulty is to estimate E [Y (0) | T = 1] which can be written as: E [Y (0) | T = 1] =EX|T =1 EY |X,T =1 [Y (0) | T = 1] =EX|T =1 [m0(X)] . But we only have sample n y(0) i , xi on0...

2025
[20]

(8) may outperform the minimax lower bound B · ¯b · r(n) when B is large

In fact, the upper bound ¯b · r(n) α 1+α in Eq. (8) may outperform the minimax lower bound B · ¯b · r(n) when B is large. • When the distributions are unknown, we do not have the minimax lower bound, and there- fore cannot determine whether the estimator is minimax-optimal. Ho...

2024
[21]

• ux(y) = 0 for any y ∈ Ω and x ∈ P with ∥x − y∥ ≥a2ρ(P, Ω)

• Pn i=1 |uxi (y)| ≤a1 for any y ∈ Ω. • ux(y) = 0 for any y ∈ Ω and x ∈ P with ∥x − y∥ ≥a2ρ(P, Ω). Based on the function uxi (1 ≤ i ≤ n) given in Lemma 3, we define a function estimator Kn = Kn({xi}n i=1, {f (xi)}n i=1) for f as Kn({xi}n i=1, {f (xi)}n i=1) := nX i=1 f (xi)uxi...

2001
[22]

However, as B increases, the two-stage estimator demonstrates greater accuracy and improved stability, highlighting its necessity under significant covariate shift

It indicates that the Monte Carlo estimator outperforms the two-stage methods in the absence of covariate shift (B = 1). However, as B increases, the two-stage estimator demonstrates greater accuracy and improved stability, highlighting its necessity under significant covariat...

2025
[2004]

Analysis of kernel mean matching under covariate shift

Yaoliang Yu and Csaba Szepesv´ari. Analysis of kernel mean matching under covariate shift. arXiv preprint arXiv:1206.4650,

arXiv
[2012]

Let Hf,q n denote the class of all the estimators using S = {(xi, yi = f (xi))}n i=1 to estimate the q-th moment of f under P∗

A A PPENDIX A.1 P ROOF OF THEOREM 1 Theorem 1 (Minimax Lower Bound on Estimating the Moment under Covariate Shift) . Let Hf,q n denote the class of all the estimators using S = {(xi, yi = f (xi))}n i=1 to estimate the q-th moment of f under P∗. For any given target p.d.f. p∗ ∈...

2019
[2017]

The covering radius of randomly distributed points on a manifold

Aleksandr Reznikov and Edward B Saff. The covering radius of randomly distributed points on a manifold. International Mathematics Research Notices, 2016(19):6065–6094,

2016
[2019]

Optimal transport for treatment effect estimation

12 Published as a conference paper at ICLR 2025 Hao Wang, Jiajun Fan, Zhichao Chen, Haoxuan Li, Weiming Liu, Tianqiao Liu, Quanyu Dai, Yichao Wang, Zhenhua Dong, and Ruiming Tang. Optimal transport for treatment effect estimation. Advances in Neural Information Processing Systems, 36,

2025
[2021]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805,

arXiv
[2022]

Convergence rates for non-log-concave sampling and log- partition estimation

David Holzm ¨uller and Francis Bach. Convergence rates for non-log-concave sampling and log- partition estimation. arXiv preprint arXiv:2303.03237,

arXiv
[2023]

Lipschitz density-ratios, structured data, and data-driven tuning

11 Published as a conference paper at ICLR 2025 Samory Kpotufe. Lipschitz density-ratios, structured data, and data-driven tuning. In Artificial Intelligence and Statistics, pp. 1320–1328. PMLR,

2025
[2024]

Few-shot learning via learning the representation, provably

Simon S Du, Wei Hu, Sham M Kakade, Jason D Lee, and Qi Lei. Few-shot learning via learning the representation, provably. arXiv preprint arXiv:2002.09434,

2002 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

In order to determine the separation between two priors µ0 and µ1, we need to derive the con- centration inequality of each prior first

= 1 − p. In order to determine the separation between two priors µ0 and µ1, we need to derive the con- centration inequality of each prior first. Define A := R Ωj B¯b · (M + fj(x))qdx and C :=R Ωj B¯b · (M − fj(x))qdx, we may derive the lower bound on the quantity ∆′ := A − C >0: ∆′ = Z Ωj B¯b · (M + fj(x))qdx − Z Ωj B¯b · (M − fj(x))qdx = B¯b · Z Ωj Z fj...

work page 2025

[2] [2]

of the uniform distribution over [0, 1]d

(Case II) We proceed to construct the second lower bound instance: letp◦(x) and p∗(x) be the p.d.f. of the uniform distribution over [0, 1]d. For any 1 ≤ j ≤ md, consider fj defined as follows: fj(x) = m−sK(m(x − cj)) (x ∈ Ωj), 0 (otherwise), we further pick two constants α, Msatisfying α := ∥K∥L∞([− 1 2 , 1 2 ]d) and M = 3α. Now consider the following fi...

work page 2024

[3] [3]

A.5 P ROOF OF COROLLARY 1 Corollary 1 (Double Robustness of Plug-in Truncated Estimator)

Thus, the convergence rate of ˜H q T is: E h ˜H q T (S ∪ S′) − I q f i ≲ E h ˆH q T (S) − I q f i + m− 1 2 . A.5 P ROOF OF COROLLARY 1 Corollary 1 (Double Robustness of Plug-in Truncated Estimator). If P(Ω− T ) → 0 as n → ∞, then the estimator ˜H q T exhibits double robustness. Specifically, this means that as long as either ˆw(x) or ˆfS1 (x) is consisten...

work page 2025

[4] [7]

Instance weighting for neural machine translation domain adaptation

Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. Instance weighting for neural machine translation domain adaptation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1482–1488,

work page 2017

[5] [10]

≥ 1 + ϵ 2 = 1 − β0, µ1(f ∈ W s,p(Ω) : I q f ≥ c + ∆) = µ1(I q f ≥ I q g1 ) ≥ 1 + ϵ 2 = 1 − β1. 14 Published as a conference paper at ICLR 2025 We proceed to compute the mixture distribution under µ0 and µ1: P0(⃗x, ⃗ y) = P(⃗x) · P0(⃗ y| ⃗x) = P(⃗x) · Z P(⃗ y| ⃗x, θ)µ0(dθ) = nY i=1 p◦(xi) ! ·   1 + ϵ 2 Y i:xi∈Ωi∗ δ0(yi) + 1 − ϵ 2 Y i:xi∈Ωi∗ δg1(xi)(yi) ...

work page 2025

[6] [13]

Assume that p ≥ 2, q ≤ p ≤ 2q and s d > 1 p − 1 2q

A.2 P ROOF OF THEOREM 2 Theorem 2 (Upper Bound on Moment Estimation with Smoothness) . Assume that p ≥ 2, q ≤ p ≤ 2q and s d > 1 p − 1 2q . Let (f, p◦, p∗) ∈ V(b, ¯b, B, s, p) and we have sample S = {(xi, yi = f (xi))}n i=1. If ˆfS1 satisfies the Assumption 1, then the estimator ˆH q B constructed in equation 3 above satisfies ES h ˆH q B(S) − I q f i ≲ B...

work page 2024

[7] [14]

This gives us the final upper bound under the assumption that s ∈ ( d(2q−p) p(2q−2) , d p ) as follows: ES1 Z Ω g2 S1 (x)f 2q−2(x) dx ≲ ¯b · n− 2s d

Thus, by applying Jensen’s inequality and choosing r = p in 4 of Assumption 1, we can upper bound the last term as follows: ES1 h ∥gS1 (x)∥2 Lp(Ω) i = ES1 ∥gS1 (x)∥L p 2 (Ω) 2 p ≤ES1 h ∥gS1 (x)∥p Lp(Ω) i 2 p = ES1 h ∥ ˆfS1 x − f (x)∥p Lp(Ω) i 2 p (Jensen’s inequality) ≲¯b · ( n 2 )− s d +( 1 p − 1 p )+ 2 ≲ ¯b · n− 2s d . This gives us the final upper boun...

work page 2025

[8] [15]

(22) Combining the upper bounds derived in 19 20, 21 and 22 we may deduce that: ES1 ES2|S1 f q(X) − ˆf q S1 (X) 2 ≲ ¯b2 · h n2q( 1 p − s d )−1 + max{n− 2s d , n2q( 1 p − s d )−1} i

This gives us the final upper bound under the assumption that s ∈ ( d(2q−p) 2pq , d(2q−p) p(2q−2) ) as follows: ES1 Z Ω g2 S1 (x)f 2q−2(x) dx ≲ ¯b · n2q( 1 p − s d )−1. (22) Combining the upper bounds derived in 19 20, 21 and 22 we may deduce that: ES1 ES2|S1 f q(X) − ˆf q S1 (X) 2 ≲ ¯b2 · h n2q( 1 p − s d )−1 + max{n− 2s d , n2q( 1 p − s d )−1} i . (23) ...

work page 2025