REVIEW 4 major objections 5 minor 2 cited by
Deep Transfer $Q$-Learning for Offline Non-Stationary Reinforcement Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read When the reward gap between source and target tasks is smoother than the target's own Q-function, transferring source trajectories through a re-weighted targeting procedure provably improves offline Q-learning bounds.
desk verdict Useful extension of transferred Q-learning to neural networks and density ratio estimation, with coherent but conditional error bounds; the advertised gain hinges on an unverified smoothness gap and the paper needs cleanup before it can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing constructions are the re-weighted targeting (RWT) pseudo-response and the aggregated reward and Q-functions. RWT replaces the source Q-function in the pseudo-response with the target Q-function and multiplies the future term by the transition density ratio $\omega^{(k)}_t(s'|s,a) = p^{(0)}_t(s'|s,a)/p^{(k)}_t(s'|s,a)$, transporting source samples to the target distribution; the identity $\mathbb{E}^{(k)}[Y^{(rwt-k)}_{t,i}|s,a] = Q^{*(0)}_t(s,a) - \delta^{(k)}_t(s,a)$ turns transfer into a supervised regression problem with a known bias term. The debiasing step then estimates $\delta^{*\,agg}_t = r^{*\,agg}_t - r^{(0)}_t$ from target samples only. The rates are controlled by the hierarchical composition model's dimension-adjusted smoothness $\gamma^*(H) = \min_{(\beta,t)\in\mathcal P}\beta/t$, with $\gamma_1$ for Q-functions and $\gamma_2$ for reward gaps; Assumption 3 requires $\gamma_2>\gamma_1$, and the density-ratio estimators add terms involving $\gamma_3$ and $\gamma_4$.
What would settle it
Run the Section 5.2 two-stage experiment with a source task whose reward gap is deliberately chosen to be rough (so that $\gamma_2\le\gamma_1$) and compare transfer versus target-only regret; Theorem 8 predicts the transfer bound's leading term worsens relative to the target-only rate, so the regret advantage should shrink or reverse.
Extended reading notes
Core claim
The central claim is that transfer learning for offline non-stationary Q-learning succeeds through a two-step debiasing scheme. At each backward stage $t$, source pseudo-responses are built as $\widehat y^{(rwt-k)}_{t,i} = r^{(k)}_{t,i} + \gamma\,\widehat\omega^{(k)}_{t,i}\max_a \widehat Q^{(0)}_{t+1}$; because the source next-state distribution is re-weighted by the density ratio $\omega^{(k)}_t = p^{(0)}_t/p^{(k)}_t$, the conditional mean of these responses equals $Q^{*(0)}_t(s,a) - \delta^{(k)}_t(s,a)$, so the only remaining mismatch is the reward gap $\delta$. Pooling these responses gives a low-variance but biased pilot estimator of an aggregated Q-function; a second regression on target samples fits the reward-difference function $\delta^{*\,agg}_t$ and corrects the bias. The main theorem (Theorem 8) states that, with probability at least $1-7Te^{-u}$, the squared $\mathrm{L}^2$ error under the target distribution is bounded by $(T-t)\max\{\kappa,1\}^{T-t}$ times a sum of the nonparametric rate for estimating $\delta^{*\,agg}_t$ with $n_0$ samples, the rate for estimating the aggregated Q-function with $n_M$ samples, the transition-ratio estimation error, and a tail term. When the reward gap is smoother than the target Q-function ($\gamma_2>\gamma_1$) and $n_M\gg n_0$, this bound improves on target-only backward inductive Q-learning.
Load-bearing premise
The whole transfer advantage rests on Assumption 3: the source-target reward gap must be a smoother, lower-complexity function than the target Q-function itself; if $\gamma_2\le\gamma_1$, the leading error term from transfer can be slower than learning from target data alone.
Editorial extensions
If this is right
- Under total transition similarity ($\omega=1$), the transfer bound splits into a reward-difference rate in $n_0$ plus an aggregation rate in $n_M$; when $n_0\lesssim n_M$ and $\gamma_2>\gamma_1$, this improves on the target-only rate $(J\log n_0/n_0)^{2\gamma_1/(2\gamma_1+1)}$.
- When transition densities must be estimated, the transition-ratio error enters the Q-function bound; transfer still pays off with nontransferable transitions when $n_M\gg n_0$ and $\gamma_3\ge\gamma_1$, and with transferable transitions when $\gamma_4\ge\gamma_1$ as well.
- The analysis removes the function-class completeness assumption used in earlier deep Q-learning work and gives non-asymptotic bounds for deep-neural-network transition density and density-ratio estimators, so the tools apply to domain shift beyond value iteration.
- The same RWT sample construction is instantiated with kernel function classes and fits an online Explore-Then-Commit extension, giving a template for transfer RL beyond the offline batch setting.
Reading between the lines
- The paper leaves implicit that source tasks could be selected or weighted adaptively by estimating whether $\gamma_2>\gamma_1$ holds for each task pair; the theory predicts that sources with rough reward gaps should be downweighted or dropped.
- The debias-then-pool scheme is likely to carry over to policy evaluation ($Q^\pi$) and to supervised domain-shift problems, since the same density-ratio reweighting machinery appears there; a concrete follow-up would instantiate the same estimators and check whether the $n_0/n_M$ rate splitting survives.
- The horizon factor $(T-t)\max\{\kappa,1\}^{T-t}$ in the bound suggests the practical transfer advantage is largest for short-horizon decisions such as adaptive treatment regimes, although the paper does not make this comparative claim.
- Because the calibrated sepsis experiment uses sex as the source/target split, a natural robustness check is to repeat the evaluation on other subgroup splits where the reward-gap smoothness assumption is less likely to hold.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies transfer learning for offline, non-stationary finite-horizon MDPs with deep ReLU network function approximation. The authors propose a re-weighted targeting procedure that reweights source pseudo-responses by an estimated transition density ratio and then uses backward inductive Q-learning with a two-step estimator: a pooled pilot estimator of an aggregated Q* function and a target debiasing step that estimates the reward difference. They provide non-asymptotic L2 error bounds (Theorem 8, Corollaries 11–12) in terms of the target sample size n0, the total pooled sample size nM, function-class smoothness parameters γ1,γ2,γ3,γ4, and transition-ratio estimation error, and they evaluate the method on a synthetic two-stage MDP and a MIMIC-III sepsis management calibration. The main theoretical claim is that, when the aggregated reward-difference function is smoother than the target Q* function (γ2>γ1) and the source pool is large, the transfer estimator improves on target-only Q-learning.
Significance. If the stated assumptions hold, the paper offers a coherent framework and, to my knowledge, a novel non-asymptotic analysis of transfer deep Q-learning for non-stationary finite-horizon MDPs. The proof strategy—separating pilot aggregation error from debiasing error, handling temporal dependence without a completeness assumption, and bounding neural-network transition-density-ratio estimators—contains useful ideas that may be reusable. The empirical demonstration on a calibrated sepsis environment is a strength. However, the advertised transfer advantage is entirely driven by the unverified smoothness gap γ2>γ1 in Assumption 3, and the manuscript contains internal inconsistencies in sign conventions and in the definition of the pooled pilot estimator. The central derivation appears sound once the assumptions are taken at face value, but the paper is not acceptable in its current form.
major comments (4)
- [Section 3.1, Assumption 3; Section 4.1, Eq. (26)] The advertised advantage of transfer is entirely driven by the inequality γ2>γ1. Because f(γ)=2γ/(2γ+1) is increasing in γ, if γ2≤γ1 the debiasing term (J log n0/n0)^{2γ2/(2γ2+1)} in Eq. (26) is no smaller than the target-only rate (J log n0/n0)^{2γ1/(2γ1+1)} used in Remark 4, and the positive aggregation and transition-ratio terms make the bound strictly worse. The paper does not derive γ2>γ1 from primitive conditions: δ*_agg in Eq. (21) is Σ_k \barυ^(k) δ^(k) with mixing weights \barυ^(k)=υ_k p^(k)/P_agg, so even smooth reward differences can produce a rough aggregate when the transition densities are non-transferable. The experiments in Section 5 provide no check of γ2>γ1 or of the smoothness of δ*_agg. I am not treating Assumption 3 as circular; rather, it is a legitimate but unverified premise that carries the central claim. The manuscript should either derive γ2>γ1 from more basic conditions or provide an empirical diagnostic.
- [Section 2.5, Eqs. (19)-(21)] The sign conventions are internally inconsistent. With δ^(k)_t = r^(0)_t - r^(k)_t as in Eq. (8), Eqs. (14)-(15) give Σ_k \barυ^(k)_t E[Y^(rwt-k)_t | s,a] = Q*^(0)_t - Σ_k \barυ^(k)_t δ^(k)_t, whereas Eq. (19) writes this as Q*^(0)_t + Σ_k \barυ^(k)_t δ^(k)_t. Eq. (21) then defines δ*_agg = r*_agg - r^(0) and identifies it with Σ_k \barυ^(k) δ^(k), which has the opposite sign relative to (8). The ambiguity propagates to the debiasing step (24) and Lemma 16, where the estimator bδ_t is supposed to estimate Q*^(0)_t - Q*^agg_t. Please fix the sign convention and state explicitly whether δ*_agg in Assumption 3 is r*_agg - r^(0) or its negative, so that the approximation argument in Lemma 16 matches the estimand.
- [Algorithm 1, Eq. (22); Section 3, Eq. (24)] The pilot pooled estimator is defined over k∈[K] (source tasks only) in Algorithm 1, but over k∈{0}∪[K] in Eq. (24). The proof of Lemma 13 in Appendix B includes target samples, since it sets y^(rwt-0)=y^(0) and treats the task label as random over all tasks. The pilot rate (J log nM/nM)^{2γ1/(2γ1+1)} in Theorem 8 requires a pooled sample of size nM = n0 + Σ_k n_k. As written, Algorithm 1 computes a different object from the one analyzed in Theorem 8. Please align the algorithm with the analysis and with the definition of P^agg_t in Section 4, which includes k=0.
- [Section 4.3, Remark 8] The comparison rate for standard Q-learning is stated as (J log n0/n0)^{2γ3/(2γ3+1)}, but Remarks 4 and 9 and Corollary 12 use (J log n0/n0)^{2γ1/(2γ1+1)} for target-only backward inductive Q-learning. The inequality displayed in Remark 8 therefore does not establish the claimed advantage unless γ3≥γ1 is imposed. The text says this condition 'can be verified through equation (6)', but no proof or empirical verification is given. Please state the correct target-only rate and make the extra condition explicit.
minor comments (5)
- [Theorem 8] The theorem statement lists Assumptions 4(i), 5, 6, and 7, but the rate uses γ1 and γ2 from Assumption 3; please include Assumption 3 in the statement.
- [Section 2.6; supplementary index] The text mentions an RKHS/kernel instantiation in Appendix E, but the appended Appendix E is the MIMIC-III calibration and the RKHS material appears as 'Appendix ??'; the missing appendix should be supplied or the claim removed.
- [Section 5.2] There is an unresolved citation 'as introduced by ?' for the Deep & Cross Network blocks; please add the reference.
- [Assumption 4; Algorithm 4] Assumption 4(iii) is stated for 'one fixed source k', while Algorithm 4 and Corollary 12 use multiple sources; please clarify how the multi-source case is handled.
- [Section 2.5] The definition of \barυ^(k)_t(s,a) uses denominators P_t(s,a) that are not defined; please define them and reconcile this with the definition of P^agg_t in Section 4, which includes the target task.
Circularity Check
No load-bearing circularity: the transfer advantage is an explicitly stated smoothness assumption (γ2 > γ1), and the theorem rates are derived rather than assumed.
full rationale
The paper's central claim is conditional on Assumption 3, which asserts that the aggregated reward difference δ*_agg lies in a smoother hierarchical composition class than Q* (γ2 > γ1). This is a modeling assumption about the data-generating process, not a quantity that is defined in terms of the output being predicted. Theorem 8's bound is derived through least-squares oracle inequalities and neural-network approximation results; the three leading terms correspond to separate estimation targets (reward difference, reward aggregation, transition ratio), each analyzed with independent arguments. The transition-ratio estimators are studied separately in Theorems 9 and 10, with their own rates. The paper does not fit a parameter and then rename it as a prediction, nor does it invoke a self-cited uniqueness theorem to rule out alternatives. Self-citations such as Fan & Gu (2023) and Chen, Li & Jordan (2022) are used for approximation lemmas, relevant prior frameworks, or data-preprocessing details, and are not load-bearing for the new derivation. Whether γ2 > γ1 is plausible in practice, given that δ*_agg inherits transition-density weights through ¯υ^(k), is a correctness and assumption-verification concern, not a circularity. The derivation chain is therefore self-contained conditional on its stated assumptions, and no equation reduces to its own input by construction.
Assumptions & free parameters
free parameters (6)
- gamma1 =
not estimated, assumed
- gamma2 =
not estimated, assumed
- gamma3 =
not estimated, assumed
- gamma4 =
not estimated, assumed
- eta =
not estimated, assumed
- Upsilon1, Upsilon2 =
Upsilon1 assumed known or lower bounded
assumptions (7)
- domain assumption Q*_agg in H1 and delta*_agg in H2 with gamma1 < gamma2 (Assumption 3).
- domain assumption Transition densities are bounded below and above, smooth in H3, and their ratios are smooth in H4 with gamma3 <= gamma4 (Assumption 4).
- domain assumption The aggregated behavior policy has minimum action propensity c > 0 (Assumption 5).
- domain assumption The Radon-Nikodym derivative dP_agg/dP(0) is bounded between eta and 1/eta (Assumption 6).
- standard math Q*_t is bounded by 1 for all t, s, a (Assumption 7).
- ad hoc to paper The source sample sizes (n1,...,nK) follow a multinomial distribution with total nM (Section 2.5).
- standard math Neural network approximation rates and empirical process lemmas from Fan & Gu (2023), Wainwright (2019), Gyorfi et al. (2002).
Cite this review
Pith. "Pith review of Deep Transfer $Q$-Learning for Offline Non-Stationary Reinforcement Learning." pith.science (2026). https://pith.science/paper/WDEXDDR5
@misc{pith2026250104870,
author = {Pith},
title = {Pith review of: Deep Transfer $Q$-Learning for Offline Non-Stationary Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/WDEXDDR5}},
note = {Machine review of arXiv:2501.04870}
}
abstract
In dynamic decision-making scenarios across business and healthcare, leveraging sample trajectories from diverse populations can significantly enhance reinforcement learning (RL) performance for specific target populations, especially when sample sizes are limited. While existing transfer learning methods primarily focus on linear regression settings, they lack direct applicability to reinforcement learning algorithms. This paper pioneers the study of transfer learning for dynamic decision scenarios modeled by non-stationary finite-horizon Markov decision processes, utilizing neural networks as powerful function approximators and backward inductive learning. We demonstrate that naive sample pooling strategies, effective in regression settings, fail in Markov decision processes.To address this challenge, we introduce a novel ``re-weighted targeting procedure'' to construct ``transferable RL samples'' and propose ``transfer deep $Q^*$-learning'', enabling neural network approximation with theoretical guarantees. We assume that the reward functions are transferable and deal with both situations in which the transition densities are transferable or nontransferable. Our analytical techniques for transfer learning in neural network approximation and transition density transfers have broader implications, extending to supervised transfer learning with neural networks and domain shift scenarios. Empirical experiments on both synthetic and real datasets corroborate the advantages of our method, showcasing its potential for improving decision-making through strategically constructing transferable RL samples in non-stationary reinforcement learning contexts.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 2 Pith papers
-
Phase Transition in Nonparametric Minimax Rates for Covariate Shifts on Approximate Manifolds
Under covariate shift with target data near a smooth d-dimensional manifold in D dimensions, the minimax regression rate switches between a manifold-dominated and a noise-dominated regime at a threshold set by source ...
-
One-Step Bellman Alignment Enables Provably Efficient Transfer in Online RL
Re-weighting source Bellman targets by the transition density ratio and correcting the one-step reward difference makes source data reusable in online Q-learning, with regret controlled by the task shift.
Reference graph
Works this paper leans on
-
[1]
Agarwal, A., Song, Y., Sun, W., Wang, K., Wang, M. & Zhang, X. (2023), Provable ben- efits of representational transfer in reinforcement learning, in ‘The Thirty Sixth Annual Conference on Learning Theory’, PMLR, pp. 2114–2187. Bose, A., Du, S. S. & Fazel, M. (2024), ‘Offline multi-task transfer rl with representational penalization’, arXiv preprint arXiv...
arXiv 2023
-
[7]
From the optimality of our pooled estimator we have that nMX i=1 (by(rwt−ki) t,i − bQp t (s(ki) t,i , a(ki) t,i ))2 ≤ nMX i=1 (by(rwt−ki) t,i − gp t (s(ki) t,i , a(ki) t,i ))2. After some algebra we get that ∥ bQp t − Q∗ agg t ∥2 nM,bPagg t ≤ ∥gp t − Q∗ agg t ∥2 nM,bPagg t + 2 nM nMX i=1 (byrwt−ki t,i − Q∗ agg t,i ) · ( bQp t,i − gp t,i). By triangle ineq...
work page 2023
-
[10]
From Theorem 9, with probability at least 1 − e−u, E(k)(ρ(k) t − bρ(k) t )2 ≲ u nk + (log nk nk ) 2γ3 2γ3+1 . Therefore, from Theorem 10, E(k)(ω(k) t − bω(k) t )2 ≤ 1 η E0(ω(k) t − bω(k) t )2 ≲ 1 η E0|bρ(k) t − ρ(k) t |2 + 1 η ( log n0 n0 ) 2γ4 2γ4+1 + u n0η ≤ 1 η2 E(k)|bρ(k) t − ρ(k) t |2 + 1 η ( log n0 n0 ) 2γ4 2γ4+1 + u n0η ≤ 1 η2 ( log nk nk ) 2γ3 2γ3...
work page 2018
-
[13]
sub-Gaussian random variables with variance parameter σ
Let z1, · · ·, zn be fixed and ϵ1, · · ·, ϵn be i.i.d. sub-Gaussian random variables with variance parameter σ. Let eG be a subset of b-uniformly bounded functions and eg be a fixed function. Suppose for some quantity v, it holds that log(N∞(ϵ, G, zn 1 )) ≤ v log ebn ϵ . Then with probability at least 1 − c2 log(1/ϵ)e−t, we have for some constants c1, c2,...
work page 2023
-
[14]
Let z1, · · ·, zn ∈ Zbe i.i.d. copies of z, G be a b-uniformly-bounded function class satisfying log(N∞(ϵ, G, zn 1 )) ≤ v log ebn ϵ for some quantity v. Then there exists c1, c2, c3 such that as long as t ≥ c1 q v log n n , with probability at least 1 − c2e−c3nt2 , we have 3 ∥g∥2 n − ∥g∥2 2 ≤ 1 2 (∥g∥2 2 + t2), ∀g ∈ G. Proof. The proof consists of a stand...
work page 2019
-
[15]
Fan, J., Gu, Y. & Zhou, W.-X. (2024), ‘How do noise tails impact on deep relu networks?’, Annals of Statistics pp. 1845–1871. 38 Fan, J., Wang, Z., Xie, Y. & Yang, Z. (2020), A theoretical analysis of deep Q-learning, in ‘Learning for Dynamics and Control’, PMLR, pp. 486–489. Gu, T., Han, Y. & Duan, R. (2022), ‘Robust angle-based transfer learning in high...
arXiv 2024
-
[16]
+ u n0 + E p 0 (t). Proof. Note that although n0 is random, we can condition on fixed n0. The pipeline of this proof is similar to Lemma 13, with the bias now coming from pooling at the same time stage. Note that in this proof, all the neural network size is confined to be ( L2, N2, M2, B2) and sometimes we omit it. Again, using the approximation results ...
work page 2023
-
[17]
We now use the peeling argument to extend to uniform r
Therefore, by Talagrand’s concentration (Theorem 3.27 in Wainwright (2019)), we have for t ≥ c1 q v log n n , there exist some c1, c2, ec3 with probability at least 1 − c2e−ec3nt2 , sup g∈B(r,G) 1 n nX i=1 g(zi) − E[g(z1)] ≤ ζ 4 r2 + t2. We now use the peeling argument to extend to uniform r. That is, define Sm = {g ∈ G : 2mvn ≤ p E|g(z1)|2 ≤ 2m+1vn}. We ...
work page 2019
Show all 20 references
-
[18]
For g1, · · ·, gN being an ϵ-covering set of G, we claim that g2 1 − eg2, · · ·, g2 N − eg2 is an 2bϵ-covering set of ¯G
Define a new function class as ¯G = {g2 − eg2 : g ∈ G}. For g1, · · ·, gN being an ϵ-covering set of G, we claim that g2 1 − eg2, · · ·, g2 N − eg2 is an 2bϵ-covering set of ¯G. In fact, for any g ∈ G, there exists π(g) ∈ [N ] such that |g(zi) − gπ(g)(zi)| ≤ϵ. Therefore, |(g(z...
2023
-
[19]
In our dataset, the mortality rate is 24.21% for female and 22.71% for male
was issued if the patient died in hospital or within 90 days out of hospital. In our dataset, the mortality rate is 24.21% for female and 22.71% for male. For the intermediate rewards, we follow Prasad et al. (2017) and associates reward to the health measurement of a patient....
2017
-
[20]
Figure 5 in Chen, Li & Jordan (2022) presents mortality rates of different lengths
We adopt this method because the distribution of final status is similar across trajectories. Figure 5 in Chen, Li & Jordan (2022) presents mortality rates of different lengths. We see that while the numbers of trajectories differ a lot, the mortality rates do not vary much ac...
2022
-
[114]
B., Davidian, M
Zhang, Y., Laber, E. B., Davidian, M. & Tsiatis, A. A. (2018), ‘Interpretable dynamic treatment regimes’, Journal of the American Statistical Association 113(524), 1541–
2018
-
[343]
& Remlinger, C
Charpentier, A., Elie, R. & Remlinger, C. (2021), ‘Reinforcement learning in economics and finance’, Computational Economics pp. 1–38. Chen, E., Chen, X. & Jing, W. (2024), ‘Data-driven knowledge transfer in batch Q∗ learn- ing’, arXiv preprint arXiv:2404.15209 . Chen, E. Y., ...
2021
-
[640]
& Song, R
Shi, C., Zhang, S., Lu, W. & Song, R. (2022), ‘Statistical inference of the value function for reinforcement learning in infinite-horizon settings’, Journal of the Royal Statistical Society Series B: Statistical Methodology 84(3), 765–793. Shi, L., Li, G., Wei, Y., Chen, Y. & ...
2022
-
[651]
Cai, T. T. & Pu, H. (2022), ‘Transfer learning for nonparametric regression: Non- asymptotic minimax analysis and adaptive procedure’, arXiv preprint . Cai, T. T. & Wei, H. (2021), ‘Transfer learning for nonparametric classification: Minimax rate and adaptive classifier’, The ...
2022
-
[901]
Sutton, R. S. & Barto, A. G. (2018), Reinforcement Learning: An Introduction, MIT press. Tian, Y. & Feng, Y. (2022), ‘Transfer learning under high-dimensional generalized linear models’, Journal of the American Statistical Association pp. 1–14. Wainwright, M. J. (2019), High-d...
2018 arXiv
-
[1225]
(2012), Transfer in reinforcement learning: A framework and a survey, in ‘Reinforcement Learning’, Springer, pp
39 Lazaric, A. (2012), Transfer in reinforcement learning: A framework and a survey, in ‘Reinforcement Learning’, Springer, pp. 143–173. Li, G., Cai, C., Chen, Y., Wei, Y. & Chi, Y. (2024), ‘Is q-learning minimax optimal? a tight sample complexity analysis’, Operations Researc...
2012
-
[1549]
Deep Transfer Q-Learning for Offline Non-Stationary Reinforcement Learning
Zhu, Z., Lin, K., Jain, A. K. & Zhou, J. (2023), ‘Transfer learning in deep reinforcement learning: A survey’, IEEE Transactions on Pattern Analysis and Machine Intelligence . 42 SUPPLEMENTARY MATERIAL of “Deep Transfer Q-Learning for Offline Non-Stationary Reinforcement Learn...
2023
-
[2014]
Murphy, S. A. (2003), ‘Optimal dynamic treatment regimes’,Journal of the Royal Statistical Society: Series B (Statistical Methodology) 65(2), 331–355. 40 Murphy, S. A. (2005), ‘A generalization error for q-learning’, Journal of Machine Learning Research 6, 1073–1097. Nguyen, X...
2003
-
[3364]
Lu, R., Huang, G. & Du, S. S. (2021), ‘On the power of multitask representation learning in linear mdp’, arXiv preprint arXiv:2106.08053 . Ma, C., Pathak, R. & Wainwright, M. J. (2023), ‘Optimally tackling covariate shift in rkhs-based nonparametric regression’, The Annals of ...
2021 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.