Pith. sign in

REVIEW 4 major objections 5 minor 15 references

Multivariate and Online Transfer Learning with Uncertainty Quantification

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Joint transfer learning cuts prediction error and keeps 96% coverage

desk verdict A competent multivariate/online extension of RECaST with a real conditional-calibration soft spot; worth refereeing if the Appendix A mismatch and alpha interpretation are fixed. read the letter →

arxiv 2411.12555 v1 pith:UZJLHJWJ submitted 2024-11-19 stat.ME stat.ML

Jimmy Hickey , Jonathan P. Williams , Brian J. Reich , Emily C. Hector This is my paper · ORCID
classification stat.MEstat.ML MSC 62F1562H1262J07
keywords BayesiantransferlearningDentalrecordsInformativepriorOnlineRacialbiasMultivariateoutcomesUncertaintyquantificationNegative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper extends the RECaST Bayesian transfer-learning framework in two directions: it models several correlated outcomes at once, and it lets target datasets arrive one after another. The authors argue that modeling the outcomes jointly through a vector of source-to-target ratio random effects improves predictive accuracy relative to fitting each outcome separately, while the online extension borrows information from previous target datasets only when it helps. If the claims hold, medical and other settings with small, underrepresented target groups could get both sharper predictions and trustworthy credible intervals without ever sharing raw data.

What carries the argument

The central object is the source-to-target ratio random effect $\beta_{i,j}=g(\Theta_{T,j},x_i)/f(\Theta_{S,j},x_i)$, which replaces the unestimable target mapping with a scaled version of the fitted source model. Univariate RECaST models each ratio as Cauchy; this paper supplies two multivariate priors: a full multivariate Cauchy with location vector $\delta$ and scale $\Gamma$, and a Gaussian copula that keeps the univariate Cauchy marginals $\beta_j\sim\text{Cauchy}(\delta_j,\gamma_j)$ while coupling them through a correlation matrix $R$. The online extension stacks a learned posterior from a previous target into the prior for the next target as $\pi^\star(\Omega,\alpha)=\pi(\alpha)\{\alpha\,p(\Omega\mid T_1)+(1-\alpha)\pi(\Omega)\}$, with $\alpha\in[0,1]$ estimated from the new target's data so that uninformative transfer can be downweighted. MCMC sampling of the posterior and posterior predictive draws yields elliptical credible sets via Mahalanobis distances.

What would settle it

Simulate target data from a mechanism whose ratio to the source is far from Cauchy—for example, bounded, skewed outcomes such as the dental averages, or a nonlinear source-to-target map—and check whether the claimed 96–97% joint coverage falls well below the 95% nominal level. A specific test is to generate targets from a log-normal or beta regression and compute the empirical joint coverage of the multivariate Cauchy RECaST credible sets; if coverage drops below roughly 90%, the ratio-distribution assumption is doing the work.

Watch

Extended reading notes

Core claim

The paper's central claim is that transferring knowledge through the random ratio $\beta_{i,j} = g(\Theta_{T,j},x_i)/f(\Theta_{S,j},x_i)$—the factor by which the source model must be scaled to match the target—works for multivariate outcomes if $\boldsymbol\beta_i$ is given either a multivariate Cauchy distribution or a multivariate Gaussian copula with Cauchy marginals. In simulation, both multivariate RECaST variants beat univariate RECaST in every setting and beat a target-only ridge baseline whenever the target sample is small; joint coverage sits at 96–97% for a 95% nominal level, whereas univariate credible sets are often the whole outcome space. The online extension writes the prior for a new target as a convex combination $\alpha$ times the posterior from a previous target plus $(1-\alpha)$ times an uninformative prior, learning $\alpha$ from data; the paper finds no negative transfer from a distant previous target, with the online methods performing as well as or better than their offline counterparts. On dental outcomes (clinical attachment level and pocket depth), the multivariate and online methods improve Mahalanobis prediction error over ridge regression for the smallest underrepresented groups while maintaining slightly conservative joint coverage.

Load-bearing premise

The load-bearing premise is that the source-to-target ratio $\beta_i$ is adequately described by a multivariate Cauchy distribution or a Gaussian copula with Cauchy marginals; the paper itself notes this is only approximate even in the canonical linear setting, and for the real dental outcomes the approximation is untested.

Editorial extensions

If this is right

  • Practitioners can transfer a source model to a small target group with multiple correlated outcomes and get joint prediction regions that are near-nominally calibrated.
  • The online prior mixture means that when a new target dataset is very different from a previous one, the method can shrink back to the uninformative prior and avoid negative transfer.
  • Data privacy is preserved: only the source model and the learned posterior from prior targets are shared, never raw patient records.
  • Because the framework is agnostic to the source model, the same ratio-random-effect machinery could be applied atop neural-network or other flexible source fits.
  • The posterior mean of $\alpha$ gives a built-in, data-driven measure of how much information a previous target contributes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The learned weight $\alpha$ could be repurposed as a diagnostic for negative transfer: a posterior mean near the prior floor suggests the previous target is uninformative for the new one.
  • The copula variant under-covers slightly (93–94%) in the online setting at larger sample sizes; a natural next step is to test whether a heavier-tailed copula or a wider prior on $R$ restores nominal coverage.
  • The dental outcomes are bounded and skewed whole-mouth averages, so the Cauchy-ratio assumption is untested there; applying the method to synthetic data with bounded, skewed outcomes would show how much coverage degrades.
  • The 96–97% coverage in linear simulations suggests the credible sets are slightly conservative; calibrating them to exactly 95% could trade a small amount of over-coverage for tighter regions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper extends the RECaST Bayesian transfer learning framework in two directions: it replaces the univariate Cauchy random effect with a multivariate Cauchy or a Gaussian-copula-with-Cauchy-marginals distribution to model multiple outcomes jointly, and it introduces an online variant that combines the source-model prior with the posterior from previous target data sets through a learned convex weight alpha. The authors claim that the multivariate methods outperform univariate RECaST and target-only ridge regression in simulated and real periodontal data, that the online method mitigates negative transfer between sequentially arriving target sets, and that the posterior predictive credible sets achieve near-nominal coverage. The main methodological content is in Sections 4 and 5; simulation and data analyses are in Sections 6 and 7.

Significance. If the results hold, the paper offers a practically useful extension of a Bayesian transfer learning framework with attractive properties: privacy preservation (only models and posterior summaries are shared), scalability to small target samples, and built-in uncertainty quantification. The simulation study is reasonably extensive (100 replications, several covariance and similarity settings), and the dental data application addresses a socially relevant problem of demographic underrepresentation. The multivariate and online constructions are natural and the exposition is generally clear. However, the central uncertainty-quantification claim rests on an unexamined conditional approximation, and one appendix presents a sampler for a different model, so the current form needs substantive revision.

major comments (4)
  1. [Sections 3-4, Eqs. (2)-(3)] The model assumes the random effect beta_i is independent of the source prediction f_i = f(Theta_S, x_i), but in the motivating canonical linear case this is false. With (A_i, B_i) = (x_i^T Theta_S, x_i^T Theta_T) jointly Gaussian, the conditional law of beta_i = B_i / A_i given A_i = f_i is N(rho, sigma_W^2 / f_i^2), whose scale depends on the realized source prediction. Equations (2) and (3) instead assign every observation a common Cauchy scale gamma (or gamma_j), independent of f_i. Section 4.1 only acknowledges that the marginal multivariate law of beta_i is not generally multivariate Cauchy; it does not mention this conditional misspecification, which already occurs in the univariate setting and affects both the multivariate Cauchy and copula models. The paper reports only test-set-averaged coverage (e.g., 96-97% in Tables 1-4), so conditional miscalibration for observations with unusually small or large |f_i| is never probed. The authors should either demonstrate conditional calibration empirically (e.g., coverage stratified by |f_i|) or modify the prior to depend on f_i.
  2. [Appendix A and Section 4.2] The Gibbs sampler presented in Appendix A does not target the model in Equation (2). Its full conditional distributions are for the hierarchical model Y_i | mu, Sigma ~ N_m(mu, Sigma), mu | u, delta, Gamma ~ N_m(delta, u^{-1} Gamma), u ~ Gamma(nu/2, nu/2), i.e., a common multivariate t prior on a shared outcome mean mu. In contrast, the proposed RECaST model has observation-specific random effects via y_i = diag(beta_i) f(Theta_S, x_i) + U_i with beta_i ~ Cauchym. These are not equivalent unless f_i is constant and one interprets the t prior on a single common mean, which is not the setting of the paper. The sentence in Section 4.2 that a Gibbs sampler 'could also be used; the details ... are outlined in Appendix A' is therefore misleading. The appendix should be corrected to describe a sampler for Equation (2), or removed if it is not used in the numerical results.
  3. [Section 5, online alpha prior; Tables 11-13] The claim that alpha is 'learned' is overstated because the Uniform(0,1) prior restricts the posterior mean to the interval [1/3, 2/3]. The derivation in Section 5 shows E(alpha | data) = (2/3 k1 + 1/3 k2)/(k1 + k2), so alpha = 0.67 is the maximum possible value, not evidence that the posterior for T1 is 'more useful' than the uninformative prior. Tables 11-13 show posterior means concentrated at 0.67 (close setting) and around 0.34-0.47 (far setting), indicating that the procedure is often pushing against the prior boundary. The authors should present this as a deliberate bounded-influence design and discuss its implications: with a uniform prior the method cannot fully reject T1 information (lower bound 1/3) nor fully embrace it (upper bound 2/3). A sensitivity analysis with, e.g., Beta(1/a,1/a) priors would clarify how much the conclusions depend on this choice.
  4. [Section 6.1 and Tables 1-4] The comparison between multivariate and univariate RECaST is not fully specified. The paper computes a joint Mahalanobis distance using the test-set empirical covariance S, but for univariate RECaST only marginal posterior predictive distributions are provided (the paper reports marginal coverages of 100 for many settings, indicating that the univariate credible sets are uninformative). It is unclear how a joint mean vector and joint covariance matrix are formed from the univariate predictions to compute the reported Mahalanobis distances. Without this detail, the claim 'in all cases, the multivariate RECaST methods have better predictive performance than the univariate method' (Section 6.2) cannot be independently verified, and the comparison may be unfair. Please clarify the construction of the univariate joint predictive distribution or use a comparison metric that is well-defined for both methods.
minor comments (5)
  1. [Section 6.3] The phrase 'even when Θ_T2 if far from Θ_T1' appears several times; 'if' should be 'is'.
  2. [Table 4 header] The last column header reads 'MV Copulah' with a stray 'h'; this formatting error should be corrected.
  3. [Section 2] The sentence describing Maddox et al. (2021) ends with '...large covariance matrix in .' and is incomplete; please rephrase.
  4. [Section 4.1] The sentence 'considering The relationship between the outcomes...' has a capitalization/grammar error; 'The' should be 'the'.
  5. [General] The paper does not include a data or code availability statement. Given that the simulations are a key part of the claims, providing code or at least detailed random seeds would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the multivariate and online extensions are validated against external baselines and the cited Cauchy lemma is a derived, checkable result.

full rationale

The paper's core derivations are not circular. The multivariate random-effect models in Sections 4.2 and 4.3 are explicitly stated as modeling choices, and Section 4.1 concedes that beta_i is not multivariate Cauchy even in the canonical linear case, so the Cauchy/copula laws are not presented as forced consequences of the target outcomes. The univariate Cauchy result cited from Hickey et al. (2024) is restated in Section 3 as a ratio-of-correlated-normals identity with explicit location and scale parameters; it is a simple derived lemma rather than a uniqueness theorem or an assumed conclusion, and it does not forbid alternative random-effect distributions. The online extension in Section 5 estimates a weight alpha with disclosed prior bounds (E(alpha) in [1/3, 2/3] under the Uniform(0,1) prior), and the negative-transfer and performance claims are evaluated by out-of-sample Mahalanobis distances and coverage comparisons against ridge and univariate RECaST baselines, not read off from the fitted alpha. The reported 96-97% empirical coverages are out-of-sample evaluations rather than identities. The conditional beta_i|f_i scaling concern raised by the skeptic is a model-checking and calibration risk, not a circularity: the paper does not define its target conclusions into its modeling assumptions.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central method relies on the RECaST canonical lemma from prior work, on the approximation that the ratio vector follows the chosen heavy-tailed distribution, and on the convex prior construction for online learning. The only new free parameter is alpha, and its range is constrained by the prior. No new physical entities are introduced.

free parameters (2)
  • alpha (online weight) = 0.33-0.67 in simulations and dental analysis (Tables 11-13, 15-17)
    Learned weight in the convex prior combination; its posterior mean is constrained by the Uniform(0,1) prior to [1/3, 2/3], so values near 0.67 indicate maximal transfer under this prior.
  • delta, Gamma, Sigma (MV Cauchy) or delta_j, gamma_j, R, Sigma (copula) = Estimated from target data via MCMC; no point estimates reported
    These are the random effect and outcome covariance parameters of the Bayesian model, learned from target data. They are not physical constants but parameters of the statistical model.
assumptions (4)
  • standard math Lemma 1 of Hickey et al. (2024): in the canonical case with features iid N(0,I) and linear models, beta_i follows a univariate Cauchy distribution with parameters delta and gamma as in Eq (1).
    This is a cited theorem from the authors' prior work; used to motivate the Cauchy random effect family for the new multivariate and online extensions. It is a proven result, not an ad hoc assumption.
  • domain assumption The Cauchy random effect family remains practically adequate for source-target ratio distributions outside the canonical case (e.g., real features, nonlinear source models).
    Sections 4.2 and 4.3 state the models 'can be used regardless of the form of the source model' and rely on heavy tails to capture disparities, but no theorem guarantees this. The method's empirical performance depends on this assumption.
  • domain assumption Multivariate random effect beta_i can be modeled with either a multivariate Cauchy or a Gaussian copula with Cauchy marginals, and the outcome innovation is Gaussian: y_{T,i} = diag(beta_i) f(theta_S,x_i) + U_i, U_i ~ N(0,Sigma).
    Equation (2) and the posterior predictive algorithm. The true beta_i distribution is not MV Cauchy even canonically (Section 4.1), so this is a modeling choice, not a derived law.
  • domain assumption For online RECaST, the posterior p(Omega|T1) is a sufficient summary of T1 for constructing a prior for T2, and a convex combination of this posterior and the original prior with weight alpha is a valid prior; alpha has a Uniform(0,1) prior.
    Section 5. The weight prior bounds the posterior mean of alpha between 1/3 and 2/3, which shapes the reported alpha values. This is a modeling choice, not a theorem.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multivariate and Online Transfer Learning with Uncertainty Quantification." pith.science (2026). https://pith.science/paper/UZJLHJWJ

@misc{pith2026241112555,
  author       = {Pith},
  title        = {Pith review of: Multivariate and Online Transfer Learning with Uncertainty Quantification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UZJLHJWJ}},
  note         = {Machine review of arXiv:2411.12555}
}
read the original abstract

Untreated periodontitis causes inflammation within the supporting tissue of the teeth and can ultimately lead to tooth loss. Modeling periodontal outcomes is beneficial as they are difficult and time consuming to measure, but disparities in representation between demographic groups must be considered. There may not be enough participants to build group specific models and it can be ineffective, and even dangerous, to apply a model to participants in an underrepresented group if demographic differences were not considered during training. We propose an extension to RECaST Bayesian transfer learning framework. Our method jointly models multivariate outcomes, exhibiting significant improvement over the previous univariate RECaST method. Further, we introduce an online approach to model sequential data sets. Negative transfer is mitigated to ensure that the information shared from the other demographic groups does not negatively impact the modeling of the underrepresented participants. The Bayesian framework naturally provides uncertainty quantification on predictions. Especially important in medical applications, our method does not share data between domains. We demonstrate the effectiveness of our method in both predictive performance and uncertainty quantification on simulated data and on a database of dental records from the HealthPartners Institute.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 13 canonical work pages

  1. [1]

    Multivariate and Online Transfer Learning with Uncertainty Quantification

    Multivariate and Online Transfer Learning with Uncertainty Quantification Jimmy Hickey, Jonathan P. Williams, Brian J. Reich, Emily C. Hector Department of Statistics, North Carolina State University Abstract Untreated periodontitis causes inflammation within the supporting tissue of the teeth and can ultimately lead to tooth loss. Modeling periodontal ou...

    work page Pith review arXiv 2024

  2. [3]

    In Section 5, we develop an online extension to handle sequential target data sets

    We extend this framework in Section 4 to model multivari- ate outcomes, providing two natural modeling choices. In Section 5, we develop an online extension to handle sequential target data sets. We test these methods in Section 6 through 2 extensive simulation studies. Section 7 analyzes the periodontal disease outcomes. Section 8 concludes. Further math...

    work page 2010

  3. [4]

    , nT }, where YT,1,

    With this new β term, the target generating mechanism can be expressed as YT,i = h n g(θT , xT,i ), UT,i o = h n f (θS, xT,i ) f (θS, xT,i ) · g(θT , xT,i ), UT,i o = h n βi · f (θS, xT,i ), UT,i o for i ∈ {1, . . . , nT }, where YT,1, . . . , YT,n T is an independent sample of nT target outcomes with associated features xT,1, . . . ,xT,n T and βi = g(θT ...

    work page 2024

  4. [5]

    Unlike univariate RECaST, the multivariate random effect βi does not always follow a multivariate Cauchy distribution, even in the canonical case

    It is unclear what the expected behavior would be if a ratio of different outcomes were considered. Unlike univariate RECaST, the multivariate random effect βi does not always follow a multivariate Cauchy distribution, even in the canonical case. A vector of random variables follows a multivariate Cauchy distribution if and only if every linear combinatio...

    work page 2016

  5. [6]

    The Γ matrix captures the dependence between the elements of βi. For continuous responses y ∈ Rm×1, a natural choice for the h function is the Gaussian innovation function, yT,i = diag(βi)f ( bΘS, xT,i ) + U T,i , where U T,i ∼ Nm(0, Σ), Σ is an unknown m × m covariance matrix, and bΘS are parameter estimates from the source model. Denoting π(δ, Γ, Σ) as ...

    work page 2017

  6. [7]

    yTℓ,nTℓ , bΘS) ∝ π(α) n αℓ · π(Ω) + ℓ−1X k=1 αk · p(Ω | yTk,1,

    The online posterior distribution for Tℓ is given by p(Ω, α | yTℓ,1, . . .yTℓ,nTℓ , bΘS) ∝ π(α) n αℓ · π(Ω) + ℓ−1X k=1 αk · p(Ω | yTk,1, . . .yTk,nTk , bΘS) o · nTℓY i=1 Z Rm p(yTℓ,i | βi, bΘS) · p(βi | Ω) dβi, where π(α) is now a multivariate prior on α, for example a Dirichlet distribution. An analogous derivation for the posterior means of the elements...

    work page 2010

  7. [8]

    This will help leverage the most similar data to build reliable target models

    We perform three analyses corresponding to different target data sets to explore the relationship between racial groups. This will help leverage the most similar data to build reliable target models. 21 Table 14: HP Data feature and outcome summaries stratified by racial group. For age, CAL, and PD the 25 th, 50th, and 75 th percentiles are presented. Whi...

    work page

  8. [9]

    dα1 = c · n kℓ Z α1 · · · Z αℓ−1 π(α) dαℓ−1

    The normalizing constant is 1 = c · Z α1 · · · Z αℓ−1 π(α) · n 1 − ℓ−1X i=1 αi kℓ + ℓ−1X i=1 αiki o dαℓ−1 . . . dα1 = c · n kℓ Z α1 · · · Z αℓ−1 π(α) dαℓ−1 . . . dα1 + (ki − kℓ) Z α1 · · · Z αℓ−1 π(α) · αi dαℓ−1 . . . dα1 o = c · n kℓ + ℓ−1X i=1 (ki − kℓ)Eπ(α)(αi) o c = n kℓ + ℓ−1X i=1 (ki − kℓ)Eπ(α)(αi) o−1 . 26 The posterior mean of αj is E(αj | yTℓ,1, ...

    work page 2015

Show all 15 references
  1. [20]

    distance from the gingival margin to the base of the gingival sulcus or periodontal pocket

    In most cases, univariate RECaST provides marginal coverages of 100, indicating that the credible sets are not helpful at the 95% nominal level. For the online methods, Tables 11, 12, and 13 summarize the average estimatedα weights, stratified by Σ. For the multivariate Cauchy...

    2007

  2. [26]

    Tianxi Cai, M

    Curran Associates, Inc. Tianxi Cai, M. L. and Liu, M. (2024). Semi-supervised triply robust inductive transfer learning. Journal of the American Statistical Association . Weiss, K., Khoshgoftaar, T. M., and Wang, D. (2016). A survey of transfer learning. Journal of Big data , ...

    2024

  3. [413]

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. 27 Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzen...

    2009

  4. [1832]

    Kapoor, S., Karaletsos, T., and Bui, T. D. (2021). Variational auto-regressive Gaussian processes for continual learning. In International Conference on Machine Learning , pages 5290–5300. PMLR. Li, S., Cai, T., and Duan, R. (2023). Targeting underrepresented populations in pr...

    2021

  5. [2097]

    Y., and Koller, D

    Raina, R., Ng, A. Y., and Koller, D. (2006). Constructing informative priors using transfer learning. In Proceedings of the 23rd international conference on Machine learning , pages 713–720. Shwartz-Ziv, R., Goldblum, M., Souri, H., Kapoor, S., Zhu, C., LeCun, Y., and Wilson, ...

    2006 arXiv

  6. [2992]

    Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., and Zhang, G. (2015). Transfer learning using computational intelligence: A survey. Knowledge-Based Systems, 80:14–23. Maddox, W. J., Balandat, M., Wilson, A. G., and Bakshy, E. (2021). Bayesian optimization with high-dimensional...

    2015 arXiv

  7. [3256]

    A., Williams, J

    Abba, M. A., Williams, J. P., and Reich, B. J. (2024). A Bayesian shrinkage estimator for transfer learning. arXiv preprint arXiv:2403.17321 . Bakker, B. and Heskes, T. (2003). Task clustering and gating for bayesian multitask learning. Journal of Machine Learning Research , 4...

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.