Pith. sign in

REVIEW 4 major objections 4 minor 30 references

Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A mixed-source/target SGD with an adaptively chosen sampling rate is shown to converge at the standard 1/√T rate to the better of the source-only and target-only risk levels.

desk verdict The adaptive sampling idea is genuinely new and the experiments are solid, but the convergence proof has a load-bearing gap that invalidates the claimed 1/√T guarantee. read the letter →

arxiv 2507.04194 v1 pith:NERECPK4 submitted 2025-07-06 stat.ML cs.AIcs.LG

Yuyang Deng , Samory Kpotufe This is my paper · ORCID
classification stat.MLcs.AIcs.LG MSC 68T0568Q3290C25
keywords supervisedtransferlearningmixed-sampleSGDadaptivesamplingconstrainedconvexprogramnegativelinearregressionconvergencerategeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the statistical guarantees of supervised transfer learning can be delivered by an actually efficient optimizer, without knowing in advance whether the source data helps or hurts the target. It proposes a mixed-sample SGD that at each step draws from source or target data with probability controlled by a bias parameter $\lambda_t$, updated online to track the usefulness of the source. The central result is an end-to-end bound: after $T$ iterations the returned model has target excess risk at most a constant times $\min\{\epsilon_Q,\ \lambda_{\max}(\Sigma_P^{-1}\Sigma_Q)\epsilon_P + E_Q(\theta_P^*)\}$, which is automatically the better of training on the target alone or transferring from the source. Convergence is at the standard $1/\sqrt{T}$ rate with total computation $O(dT)$ plus one final projection, and the same tracking argument is claimed to extend to general convex losses. The contribution, in short, is that adaptivity to unknown source quality can be built into ordinary SGD rather than purchased through cross-validation.

What carries the argument

The load-bearing object is the sequence of constrained convex programs (CPs): at each stage the procedure effectively solves $\min_\theta \hat{R}_P(\theta)$ subject to $\hat{R}_Q(\theta) \le \hat{R}_Q(\theta_{Q,t}) + 6\epsilon_Q$, whose Lagrangian saddle point $(\lambda^*, \tilde{\theta}_{PQ})$ is the target of the optimization. The mechanism that realizes it is the mixed-sample SGD update: a Bernoulli draw with source probability $1/(1+\lambda_t)$ decides which dataset feeds the stochastic gradient, while $\lambda_t$ is updated from stochastic constraint violations and simultaneously serves as the Lagrange multiplier and as the sampling bias; $\theta_{Q,t}$ is maintained by a separate parallel SGD, so the constraint set itself evolves. The analysis carries the argument by controlling how far the iterates drift from the constraint set defined by $\theta_{Q,t}$, using data-dependent quantities — the initial distance $\rho = \|\theta_0 - \tilde{\theta}_{PQ}\|$, Lipschitz constants $\hat{G}_{\theta}$ and $\hat{G}_{\lambda}$ on a neighborhood of radius $\sqrt{2}\rho$, the optimal multiplier $\lambda^*$, and a boundary-gradient lemma for the constraint set — which together replace the bounded-constraint-set assumption that earlier one-projection SGD analyses required.

What would settle it

On a synthetic linear-regression instance, compute the saddle point $(\lambda^*, \tilde{\theta}_{PQ})$ and engineer $P$ and $Q$ so that $\lambda^* > \rho = \|\theta_0 - \tilde{\theta}_{PQ}\|$ (for example, a very small target sample $n_Q$ with a source optimum far from the target optimum). Run Algorithm 1 and record $\max_t (\|\theta_t - \tilde{\theta}_{PQ}\|^2 + |\lambda_t - \lambda^*|^2)$: if it exceeds $2\rho^2$, the induction in Lemma 11 breaks and the stated convergence theorem does not cover the regime it claims to cover.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single stochastic-gradient procedure can track a sequence of constrained convex programs of the form $\min_\theta \hat{R}_P(\theta)$ subject to $\hat{R}_Q(\theta) \le \hat{R}_Q(\theta_{Q,t}) + 6\epsilon_Q$, where $\theta_{Q,t}$ is a parallel SGD estimate of the target ERM $\hat{\theta}_Q$, without paying for expensive projections onto the constraint sets at each iteration. The adaptive mixing weight $\lambda_t$ plays two roles at once: it is the running Lagrange multiplier of this program, updated as $\lambda_{t+1} = [(1-\gamma\eta)\lambda_t + \eta(\ell(\theta_t; x_t, y_t) - \ell(\theta_{Q,t}; x_t, y_t) - 6\epsilon_Q)]_+$, and it sets the sampling bias, with source data drawn with probability $1/(1+\lambda_t)$. The proven consequence is that the returned solution $\hat{\theta}_{PQ}$ satisfies $E_Q(\hat{\theta}_{PQ}) \lesssim \min\{\epsilon_Q,\ \lambda_{\max}(\Sigma_P^{-1}\Sigma_Q)\epsilon_P + E_Q(\theta_P^*)\}$ with probability $1-5\tau$, provided the number of iterations $T$ exceeds a sample-dependent threshold; $\epsilon_Q$ is the best rate achievable from target data alone and the second term is the best rate achievable by transferring from source data, so the bound interpolates between the two. The paper further claims the same tracking mechanism works for general convex losses, yielding rates expressed through Rademacher complexity and a weak-modulus link between source and target risks, and supports the theory with synthetic and real-data experiments.

Load-bearing premise

One premise carries the whole convergence argument — the induction in Lemma 11 that every iterate $(\theta_t, \lambda_t)$ stays within a radius-$\sqrt{2}\rho$ neighborhood of the saddle point $(\tilde{\theta}_{PQ}, \lambda^*)$ — and its base case silently requires the optimal bias $\lambda^*$ to be no larger than the initial distance $\rho$, a condition no theorem states and no assumption guarantees, so if $\lambda^*$ exceeds $\rho$ the convergence proof never gets started.

Editorial extensions

If this is right

  • The returned model's target excess risk is bounded by $\min\{\epsilon_Q,\ \lambda_{\max}(\Sigma_P^{-1}\Sigma_Q)\epsilon_P + E_Q(\theta_P^*)\}$, so the procedure automatically attains whichever is smaller: the rate of training on the target alone or the rate of transferring from the source.
  • The bias parameter is never chosen by cross-validation; it emerges from the online multiplier updates, so the per-iteration cost is a single stochastic gradient and total computation is $O(dT)$ plus one final projection.
  • The $1/\sqrt{T}$ optimization rate means the statistical transfer rate is reached as soon as $T$ is at least polynomial in the sample-dependent constants, making the guarantee truly end-to-end.
  • The same tracking argument is claimed to extend to general convex losses and to low-rank or over-parameterized regimes where the constraint sets are unbounded, which earlier projection-based SGD analyses could not handle.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The trajectory of $\lambda_t$ is itself an online estimate of the optimal bias $\lambda^*$, so the algorithm could double as a diagnostic: a persistently small $\lambda_t$ signals an uninformative source, while a rising $\lambda_t$ signals that source data is being exploited.
  • The unstated condition $\lambda^* \le \rho$ needed to start the induction suggests a warm-start phase that shrinks $\rho$ before the main loop would make the proof's regime cover more cases; the paper already warm-starts $\theta_{Q,t}$, and a warm-started $(\theta_0, \lambda_0)$ is the natural analogue.
  • The CP-tracking template could be carried to other transfer objectives — penalized formulations, domain-adversarial training — wherever the Lagrange multiplier of an inner program can be updated online and reused as a sampling rate.
  • One testable extension is whether the $\min\{\epsilon_Q, \text{ transfer term}\}$ bound is sharp: in a regime where the source and target are exactly equally informative, a tight analysis would predict excess risk of order $\epsilon_Q$, and the crossing point could be located empirically as $n_P$ grows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a mixed-sample SGD procedure for supervised transfer learning, in which each step samples from the source or target distribution with probability controlled by a running multiplier lambda_t, and lambda_t is updated to track a sequence of constrained convex programs of the form min_{theta} hat{R}_P(theta) subject to hat{R}_Q(theta) <= hat{R}_Q(theta_{Q,t}) + slack. The main linear-regression result, Theorem 1, claims that with high probability the returned solution achieves target excess risk bounded by a constant times min{ epsilon_Q, lambda_max(Sigma_P^{-1} Sigma_Q) epsilon_P + E_Q(theta^*_P) }, using only O(dT) computation plus one projection, with the number of iterations T polynomial in the problem parameters. The paper also gives a general convex-loss extension with Rademacher-complexity-based statistical guarantees and presents experiments on synthetic, School, Yearbook, CIFAR-10, and IoT data.

Significance. If the main theorems were correct, the paper would make a substantial contribution: it would provide an end-to-end optimization-plus-generalization analysis of transfer learning, avoid expensive cross-validation or repeated projections, and automatically interpolate between source-only and target-only rates without knowing the source quality in advance. The statistical transfer argument in Section 5.2 is largely independent of the optimization proof and appears coherent, and the appendix's extension to general strongly convex losses is a useful structural result. The experiments are also relevant and generally support the qualitative adaptivity claim. However, the central convergence proof contains load-bearing gaps: the induction invariant in Lemma 11 has an unstated base-case condition that fails exactly in the uninformative-source regime the paper emphasizes, and the displayed bound in Theorem 2 contains a T-independent term, so the claimed 1/sqrt(T) convergence is not established as stated.

major comments (4)
  1. [Section 12.1, Lemma 11] The induction invariant in Lemma 11 is ||theta_t - theta_tilde_PQ||^2 + |lambda_t - lambda^*|^2 <= 2 rho^2. At t = 0, with theta_0 = 0 and lambda_0 = 0, the base case requires rho^2 + (lambda^*)^2 <= 2 rho^2, i.e. lambda^* <= rho. No theorem or assumption in the paper states or implies this inequality; Lemma 4 only gives an upper bound on lambda^* involving lambda_max(Sigma_hat_P)/lambda_min^+(Sigma_hat_Q) plus a gradient term divided by sqrt(epsilon_Q), and it contains no comparison with rho. In a simple isotropic linear-regression instance with theta^*_P = 0, theta^*_Q = v, Sigma_P = Sigma_Q = I, and small epsilon_Q, one has lambda^* approximately ||v||/sqrt(6 epsilon_Q) - 1 while rho is of order ||v||, so lambda^* > rho whenever sqrt(6 epsilon_Q) < 1; this is precisely the regime where the source is uninformative and the adaptive guarantee is supposed to matter. Consequently the induction cannot start, and every later step that relies on the 2 rho^2 tube—Definition 3 of G_hat_theta and G_hat_lambda, the bound lambda_t <= lambda^* + sqrt(2) rho, the sub-Gaussian parameter in Lemma 10, and the telescoping argument—is unsupported. Theorem 2, and hence Theorem 1, do not apply as stated to the setting the paper claims to handle adaptively.
  2. [Theorem 2 (Section 5.1)] The final display of Theorem 2 has the form R_hat_P(theta_hat_PQ) - R_hat_P(theta_tilde_PQ) less-or-similar (G_hat_theta + G_hat_lambda sqrt(log 1/tau)) times [ G_hat_theta^2 / c_eta + terms that decay as 1/sqrt(T) ]. The first summand, G_hat_theta^2 / c_eta, is independent of T for fixed c_eta, so the right-hand side does not vanish as T grows. Since Theorem 1 chooses T so that R_hat_P(theta_hat_PQ) - R_hat_P(theta_tilde_PQ) <= epsilon_P, this nonvanishing constant means that for small epsilon_P the required T lower bound generally cannot be satisfied. The claim that the procedure converges at a 1/sqrt(T) rate is therefore not established by the displayed bound; the issue is not a typo in a single line, because the constant arises in the proof from the b/a terms in the quadratic inequality when bounding ||theta_bar_T - theta_hat_PQ||.
  3. [Theorem 1 and Section 4.1] The parameters of Algorithm 1 are declared to be chosen with c_eta 'sufficiently small as a function of (G_hat_theta, G_hat_lambda, lambda^*, rho)' and gamma >= G_hat_theta^2, but G_hat_theta, G_hat_lambda, lambda^*, and rho are defined in terms of the unknown saddle point (lambda^*, theta_tilde_PQ). Lemmas 1–4 provide sample-dependent upper bounds, yet no fully data-driven procedure is specified that would select c_eta, gamma, and T from the available samples before running the method. This is a gap in the algorithmic claim as stated, even setting aside the base-case problem in Lemma 11.
  4. [Lemma 9] Lemma 9 states a high-probability convergence bound for theta_{Q,t} for 'any t >= 0', but the proof invokes a result on noisy SGD whose constant depends on the sub-Gaussian parameter sigma_Q; the displayed definition of sigma_Q in the lemma differs from the definition used in the body and in Lemma 11 (one uses M_x M_y, the other M_x M_hat_y), and it is not clear which quantity bounds the gradient noise uniformly over all t. This is a secondary issue, but it should be clarified because Lemma 9 feeds directly into the constant C_PQ in Lemma 11.
minor comments (4)
  1. [Throughout] There are frequent typographical errors, including 'up tp' instead of 'up to' in the proof of Lemma 7 and inconsistent notation for M_y versus M_hat_y across Lemmas 3, 9, 11, and Theorem 2; these should be harmonized.
  2. [Section 1] The introduction states that the converged point inherits statistical guarantees 'at a typical rate of O(1/sqrt t)', but the actual optimization bound in Theorem 2 has a nonvanishing constant term; the abstract and introduction should be coordinated with the precise theorem statement.
  3. [Algorithm 1] In Algorithm 1, the text says 'Sample (x_t, y_t) uniformly from S_Q' for the lambda and theta_Q updates after having already sampled from S_Q for the theta update; the notation suggests a fresh independent draw, but this should be stated explicitly to avoid ambiguity.
  4. [Section 4.1, Lemma 1] Lemma 1 upper-bounds rho via a warm-up procedure, but the bound (5) uses quantities like nabla R_hat_P(theta_{Q,N}) that require the warm-up to have run for N steps; the dependence of the final T on N and on the accuracy of the warm-up is not tracked.

Circularity Check

0 steps flagged · score 2.0 of 10

No definitional or fitted-input circularity; the optimization and statistical claims do not reduce to their own inputs. Two serious proof gaps (Lemma 11 base case and a T-independent term in Theorem 2) are correctness risks, not circularity.

full rationale

I walked the derivation chain from Algorithm 1 through Lemma 11/Theorem 2 to Theorem 3/Theorem 1. The statistical transfer guarantee is conditional: if R̂_P(θ̂_PQ)−R̂_P(θ̃_PQ) ≤ ϵ_P, then E_Q(θ̂_PQ) ≤ 26 min{ϵ_Q, λ_max(Σ_P^{-1}Σ_Q)ϵ_P + E_Q(θ*_P)}; the min structure is obtained by case analysis on whether θ*_P lies in the constraint set, not by defining the target through the bound. The quantities ρ, λ*, Ĝ_θ, Ĝ_λ appear in the convergence rate and step-size conditions, but they are problem/sample parameters with data-dependent upper bounds (Lemmas 1–4), not fitted values renamed as predictions; the adaptivity claim is a theorem about unknown P,Q, not a tautology. The self-citations to [5] for the weak modulus and matrix concentration (Lemmas 5 and 6 use 'Lemma 3 of [5]') are real external support under the stated-assumption criterion, and the convergence analysis is new and not imported. Two correctness defects are worth flagging explicitly, though they are not circularity under the definitions used here. First, Lemma 11's induction invariant ∥θ_t−θ̃_PQ∥²+|λ_t−λ*|²≤2ρ² is started at t=0 with θ_0=0, λ_0=0, which algebraically requires (λ*)²+ρ²≤2ρ², i.e., λ*≤ρ; no assumption or lemma establishes this, and in an isotropic instance with uninformative source it can fail. Second, the displayed bound in Theorem 2 contains a T-independent first summand Ĝ_θ²/c_η, so the claimed 1/√T vanishing rate does not follow from the displayed inequality; this affects the 'provided T≳...' step of Theorem 1. These are unsupported proof steps, not reductions of the conclusion to the inputs, so the circularity score is low, reflecting only the minor self-citation flavor in the statistical lemmas.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The above ledger captures the main external inputs the central claim rests on: sub-Gaussian and boundedness assumptions, the full-rank source covariance, and crucially an unstated bound on the dual variable lambda* relative to the initial distance rho. The free parameters epsilon_Q and c_eta are not derived from data; they must be set by the user, so the procedure is not fully automatic.

free parameters (3)
  • epsilon_Q = c0 * sigma_y^2 * (d + log(1/tau)) / n_Q (input to Algorithm 1)
    Algorithm 1 requires epsilon_Q as input; it is defined through unknown constants c0 and sigma_y, so the user must supply an estimate of the target statistical rate.
  • c_eta = depends on G_hat_theta, G_hat_lambda, lambda*, rho, sigma_PQ (no explicit value)
    Stepsize constant in eta = c_eta / sqrt(T); the theorem requires it to be smaller than several quantities that depend on the unknown solution (lambda*, rho).
  • gamma = G_hat_theta^2 * eta
    Dual stepsize; set as G_hat_theta^2 eta where G_hat_theta depends on unknown rho.
assumptions (5)
  • domain assumption Assumption 1: Y - E[Y|X] is sigma_y-sub-Gaussian with zero mean, and X is bounded.
    Used to derive concentration and generalization bounds in Theorem 3.
  • domain assumption Assumption 2: Sigma_P is full rank.
    Needed to define lambda_max(Sigma_P^{-1} Sigma_Q) and to ensure the source is informative.
  • ad hoc to paper Unstated invariant condition: lambda* <= rho (or equivalently (lambda*)² + rho² <= 2rho²) to initialize the induction in Lemma 11.
    The proof of Lemma 11 assumes the iterates stay in a bounded set; the base case requires the initial dual distance to be small, which is not stated anywhere.
  • standard math Standard convex analysis (strong duality, KKT) for the Lagrangian in Eq (3).
    Used to define (lambda*, theta_tilde_PQ) and to characterize the projected solution.
  • domain assumption Assumption 3 (general losses): loss is m1-strongly convex and m2-smooth.
    Used for the general-loss results in Section 8; not needed for the linear regression main results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning." pith.science (2026). https://pith.science/paper/NERECPK4

@misc{pith2026250704194,
  author       = {Pith},
  title        = {Pith review of: Mixed-Sample SGD: an End-to-end Analysis of Supervised Transfer Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NERECPK4}},
  note         = {Machine review of arXiv:2507.04194}
}
abstract

Theoretical works on supervised transfer learning (STL) -- where the learner has access to labeled samples from both source and target distributions -- have for the most part focused on statistical aspects of the problem, while efficient optimization has received less attention. We consider the problem of designing an SGD procedure for STL that alternates sampling between source and target data, while maintaining statistical transfer guarantees without prior knowledge of the quality of the source data. A main algorithmic difficulty is in understanding how to design such an adaptive sub-sampling mechanism at each SGD step, to automatically gain from the source when it is informative, or bias towards the target and avoid negative transfer when the source is less informative. We show that, such a mixed-sample SGD procedure is feasible for general prediction tasks with convex losses, rooted in tracking an abstract sequence of constrained convex programs that serve to maintain the desired transfer guarantees. We instantiate these results in the concrete setting of linear regression with square loss, and show that the procedure converges, with $1/\sqrt{T}$ rate, to a solution whose statistical performance on the target is adaptive to the a priori unknown quality of the source. Experiments with synthetic and real datasets support the theory.

Figures

Figures reproduced from arXiv: 2507.04194 by the authors.

Figure 1
Figure 1. Simulation Results with Gaussian data, illustrating our guarantees that E( ˆθP Q) < min{E( ˆθP ), E( ˆθQ)}. ˆθHT L denotes the Hypothesis Transfer Learning (HTL). (Left) we fix P, Q while increasing nP , or (Middle) and (Right) we fix nP , and push P far from Q as controlled by λmax Σ −1 P ΣQ  and EQ(θ ∗ P ). The source is least informative, i.e. source ERM ˆθP is worse than target ERM ˆθQ, when either nP is too sm… view at source ↗
Figure 3
Figure 3. Linear regression results on the School dataset. (Left) Excess risk on Q. (Right) Runtime comparison. Mixed-Samples SGD ˆθP Q achieves Q-risk nearly the same of that of the ideal projection method ˆθPSGD, while achieving significantly faster runtime. 6 Experiments In this section, we present the experimental results of our algorithm and the baseline algorithms on both synthetic and real-world datasets. We implement … view at source ↗
Figure 2
Figure 2. Linear regression results on the synthetic dataset with low-rank ΣQ. The constraint set is then unbounded but our method still works well. We start with the results on the synthetic dataset. Throughout this subsection, we set the model dimen￾sion d to be 50. The distribution P and Q are set to be d-dimensional multivariate Gaussian with certain mean and covariance. The label is generated as y = θ ∗ µ ⊤ x + ε for µ ∈… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Binary classification results on the CIFAR Dog vs Cat dataset. We use linear classifier with logistic loss. Input features of dimension 512 are extracted by ResNet18. We fix nQ = 50 and increase nP . It verifies that our algorithm can work on the general loss. At last,…
Figure 5
Figure 5. Figure 5: Linear regression on the Yearbook dataset. Input features of dimension 512 are extracted by ResNet18. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Binary Classification on the CIFAR10 Dog vs Cat dataset. We fix [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Binary Classification on the Malware IoT dataset. We fix nQ = 100 (total samples from Q), nP,0 = 100 (normal samples from P) and increase nP,1 (abnormal samples) gradually. (Top-Left) Overall Target Error (Top-Right) Target Type-I Error (Bottom-Left) Target Type-II Err…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 23 canonical work pages

  1. [1]

    Stability and hypothesis transfer learning

    Ilja Kuzborskij and Francesco Orabona. Stability and hypothesis transfer learning. In International Conference on Machine Learning, pages 942–950. PMLR, 2013

    work page 2013

  2. [2]

    Fast rates by transferring from auxiliary hypotheses

    Ilja Kuzborskij and Francesco Orabona. Fast rates by transferring from auxiliary hypotheses. Machine Learning, 106:171–195, 2017

    work page 2017

  3. [3]

    On the hardness of domain adaptation and the utility of unlabeled target samples

    Shai Ben-David and Ruth Urner. On the hardness of domain adaptation and the utility of unlabeled target samples. In Algorithmic Learning Theory: 23rd International Conference, ALT 2012, Lyon, France, October 29-31, 2012. Proceedings 23, pages 139–153. Springer, 2012

    work page 2012

  4. [4]

    On the value of target data in transfer learning

    Steve Hanneke and Samory Kpotufe. On the value of target data in transfer learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  5. [5]

    A more unified theory of transfer learning

    Steve Hanneke and Samory Kpotufe. A more unified theory of transfer learning. arXiv preprint arXiv:2408.16189, 2024

    arXiv 2024

  6. [6]

    Stochastic gradient descent with only one projection

    Mehrdad Mahdavi, Tianbao Yang, Rong Jin, Shenghuo Zhu, and Jinfeng Yi. Stochastic gradient descent with only one projection. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012

    work page 2012

  7. [7]

    Learning bounds for domain adaptation

    John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman. Learning bounds for domain adaptation. Advances in neural information processing systems, 20, 2007

    2007

  8. [8]

    Domain adaptation: Learning bounds and algo- rithms

    Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algo- rithms. arXiv preprint arXiv:0902.3430, 2009

    arXiv 2009

Show all 30 references
  1. [9]

    A theory of learning from different domains

    Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79:151–175, 2010

    2010

  2. [10]

    New analysis and algorithm for learning with drifting distributions

    Mehryar Mohri and Andres Muñoz Medina. New analysis and algorithm for learning with drifting distributions. In Algorithmic Learning Theory: 23rd International Conference, ALT 2012, Lyon, France, October 29-31, 2012. Proceedings 23, pages 124–138. Springer, 2012

    2012

  3. [11]

    Density ratio estimation in machine learning

    Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cam- bridge University Press, 2012. 9 A PREPRINT - S EPTEMBER 22, 2025

    2012

  4. [12]

    Density-ratio matching under the bregman diver- gence: a unified framework of density-ratio estimation

    Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the bregman diver- gence: a unified framework of density-ratio estimation. Annals of the Institute of Statistical Mathematics , 64:1009–1044, 2012

    2012

  5. [13]

    Hypothesis transfer learning via transforma- tion functions

    Simon S Du, Jayanth Koushik, Aarti Singh, and Barnabás Póczos. Hypothesis transfer learning via transforma- tion functions. Advances in neural information processing systems, 30, 2017

    2017

  6. [14]

    Minimax lower bounds for transfer learning with linear and one-hidden layer neural networks

    Mohammadreza Mousavi Kalan, Zalan Fabian, Salman Avestimehr, and Mahdi Soltanolkotabi. Minimax lower bounds for transfer learning with linear and one-hidden layer neural networks. Advances in Neural Information Processing Systems, 33:1959–1969, 2020

    1959

  7. [15]

    A class of geometric structures in transfer learning: Minimax bounds and optimality

    Xuhui Zhang, Jose Blanchet, Soumyadip Ghosh, and Mark S Squillante. A class of geometric structures in transfer learning: Minimax bounds and optimality. In International Conference on Artificial Intelligence and Statistics, pages 3794–3820. PMLR, 2022

    2022

  8. [16]

    Maximum likelihood estimation is all you need for well-specified covariate shift

    Jiawei Ge, Shange Tang, Jianqing Fan, Cong Ma, and Chi Jin. Maximum likelihood estimation is all you need for well-specified covariate shift. arXiv preprint arXiv:2311.15961, 2023

    2023 arXiv

  9. [17]

    Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints

    Reese Pathak, Martin J Wainwright, and Lin Xiao. Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints. arXiv preprint arXiv:2303.12613, 2023

    2023 arXiv

  10. [18]

    High-dimensional kernel methods under covariate shift: Data-dependent implicit regularization

    Yihang Chen, Fanghui Liu, Taiji Suzuki, and V olkan Cevher. High-dimensional kernel methods under covariate shift: Data-dependent implicit regularization. arXiv preprint arXiv:2406.03171, 2024

    2024 arXiv

  11. [19]

    CVXPY: A Python-embedded modeling language for convex optimization

    Steven Diamond and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1–5, 2016

    2016

  12. [20]

    Uci machine learning repository, 2007

    Arthur Asuncion, David Newman, et al. Uci machine learning repository, 2007

    2007

  13. [21]

    A distribution-dependent analysis of meta learning

    Mikhail Konobeev, Ilja Kuzborskij, and Csaba Szepesvári. A distribution-dependent analysis of meta learning. In International Conference on Machine Learning, pages 5697–5706. PMLR, 2021

    2021

  14. [22]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009

    2009

  15. [23]

    Understanding machine learning: From theory to algorithms

    Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cam- bridge university press, 2014

    2014

  16. [24]

    Size-independent sample complexity of neural networks

    Noah Golowich, Alexander Rakhlin, and Ohad Shamir. Size-independent sample complexity of neural networks. In Conference On Learning Theory, pages 297–299. PMLR, 2018

    2018

  17. [25]

    Foundations of machine learning

    Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning

  18. [26]

    A century of portraits: A visual historical record of american high school yearbooks

    Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A Efros. A century of portraits: A visual historical record of american high school yearbooks. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1–7, 2015

    2015

  19. [27]

    Iot-23: A labeled dataset with malicious and benign iot network traffic

    Sebastian Garcia, Agustin Parmisano, and Maria Jose Erquiaga. Iot-23: A labeled dataset with malicious and benign iot network traffic. (No Title), 2020

    2020

  20. [28]

    Scikit-learn: Machine learning in python

    Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesna...

    2011

  21. [29]

    Revisiting the last-iterate convergence of stochastic gradient methods

    Zijian Liu and Zhengyuan Zhou. Revisiting the last-iterate convergence of stochastic gradient methods. In The Twelfth International Conference on Learning Representations

  22. [30]

    An optimal method for stochastic composite optimization

    Guanghui Lan. An optimal method for stochastic composite optimization. Mathematical Programming , 133(1):365–397, 2012. 10 A PREPRINT - S EPTEMBER 22, 2025 Algorithm 3: Mixed-Sample SGD (Restated for General Θ) Input: θ0 = θQ,0 = 0, λ0 = 0, stepsize {αt}T −1 t=0 and η, γ, ϵQ. ...

    2012

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.