Pith. sign in

REVIEW 4 major objections 8 minor 23 references

Modular Diffusion Policy Training: Decoupling and Recombining Guidance and Diffusion for Offline RL

T0 review · 4 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that in offline RL, the reward-guidance module and the diffusion policy can be trained separately and recombined freely, making diffusion policies modular.

desk verdict A genuinely novel training order with some real ablations, but the load-bearing modularity claim is supported more by assertion than evidence; worth reviewing, needs major revision. read the letter →

arxiv 2506.03154 v1 pith:HTPBKXPE submitted 2025-05-19 cs.LG

Zhaoyang Chen , Cody Fleming This is my paper · ORCID
classification cs.LG
keywords offlinereinforcementlearningdiffusionpoliciesclassifier-freeguidancemodulartrainingguidance-firstQ-learningcross-moduletransferabilityD4RLbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that in offline reinforcement learning, the reward-guidance module of a diffusion policy—the Q-function that steers sampling toward high-value actions—can be trained independently of the diffusion model, frozen, and then reused to guide a different diffusion model with no joint training. The motivation is that guidance depends only on the offline dataset (state, action, reward), not on the policy's own actions, so joint training is an unnecessary source of early-stage noise. The paper proposes Guidance-First Diffusion Training (GFDT): train the guidance module first, freeze it, and use classifier-free reward guidance during diffusion training. It reports across PyBullet D4RL tasks that this accelerates convergence, improves peak and area-under-curve performance (e.g., up to 11.9% peak and 25.2% AUC gains), cuts peak memory, and—by swapping in a differently seeded guidance module at inference—reduces reward variance (86% lower IQR). The payoff if true is a modular, reusable pipeline where value estimators and diffusion policies are independent building blocks.

What carries the argument

The central mechanism is Guidance-First Diffusion Training (GFDT), which decouples the Q-value guidance module from the diffusion policy. The guidance module is a Q-function Qϕ(s,a) trained on the offline dataset, frozen, and then injected into the denoising loop as a gradient perturbation: ϵθ(at,s,t) ← ϵθ(at,s,t) + α∇a_t Q(s,a_t). A second component, Double Guidance, uses a separately seeded copy of the same architecture at inference time to break the self-reinforcing bias loop between a policy and its own value estimates. The theoretical anchor is a positive-correlation argument: if ∇aQϕ is positively correlated with the ideal sampling direction, then small-step guided updates increase expected Q, and injected Gaussian noise averages to zero over many steps, so pretrained guidance can steer an untrained or differently trained diffusion model.

What would settle it

Measure the cosine similarity between ∇a Qϕ(s,a) and the true action-improvement direction on held-out states for a diffusion policy that was not used to train Qϕ; a value that is not reliably positive across the action distribution would refute the transferability claim.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is that a Q-value guidance module in diffusion-based offline RL is policy-independent: because both the guidance and the diffusion policy are learned from the same fixed dataset, the guidance signal can be computed from (s, a, r) data alone and does not need to be co-trained with the denoiser. The paper demonstrates this through GFDT, where the Q-network is trained first, frozen, and then used to perturb the denoising prediction as ϵθ(at,s,t) ← ϵθ(at,s,t) + α∇a_t Q(s,a_t). It further shows that the decoupling can be pushed to inference: plugging a guidance module trained for IDQL into a DQL diffusion model—two components that never trained together—yields performance at least matching the better baseline, while swapping in a separately seeded but architecturally identical guidance module at inference reduces variance (IQR down 86%).

Load-bearing premise

The load-bearing premise is that a Q-function trained on offline data gives gradients that remain accurate and positively correlated with reward improvement for whatever diffusion policy it is later attached to, even though it never saw that policy during training.

Editorial extensions

If this is right

  • Value estimators can be trained once on an offline dataset and reused across multiple diffusion policies, turning guidance into a shared, frozen asset.
  • Training memory drops because the guidance module is not updated during diffusion training, and convergence is faster because early training uses an already-accurate reward signal.
  • Diffusion policies and guidance modules become interoperable across algorithms: an IDQL-trained guidance can steer a DQL-trained diffuser, so new policy architectures can inherit existing value estimators.
  • Seed-to-seed stability improves by decoupling the guidance used in training from the guidance used at inference, suggesting that feedback coupling between value estimates and policy is a source of variance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If guidance is truly policy-independent, one could maintain a library of pretrained Q-modules per task and swap them freely as improved value estimators are developed; the paper gestures at this but does not build it.
  • The positive-correlation argument implies a measurable boundary condition: modular transfer should fail when the guided diffusion model samples actions where the Q-gradient is inaccurate, so testing on high-entropy or out-of-distribution policies would delimit the method's validity.
  • The two-seed variance reduction suggests a natural extension to ensembles: averaging gradients from several independently seeded guidance modules may reduce variance further than a single swap, a variant the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes "Guidance-First Diffusion Training" (GFDT), a modular training scheme for diffusion-based offline RL in which the reward-guidance (Q-function) module is trained first as a value estimator, frozen, and then used to guide a separately trained diffusion policy via classifier-free guidance. The authors claim three findings: guidance is not equally important at all training stages; freezing a pretrained guidance module accelerates training and improves final performance; and guidance modules can be transferred across different diffusion-policy algorithms (e.g., IDQL guidance with DQL diffusion) without retraining, reducing variance and sometimes improving early learning. A short theoretical argument in Section 3.2 is offered, and experiments are run on eight PyBullet D4RL tasks with three baseline algorithms (EDP, DQL, IDQL).

Significance. If the central modularity claim is correct, the paper would establish a practical and conceptually useful property: diffusion policies and their reward-guidance modules can be trained independently and recombined, which could reduce peak memory, enable reuse of value estimators, and improve sample efficiency and stability. Strengths of the manuscript include its explicit focus on an underexplored architectural design space, the use of multiple seeds and checkpoint-based evaluation, and the attempt to provide a theoretical rationale for decoupled training. However, the theoretical argument in Section 3.2 does not currently justify the central claim, and the empirical reporting has important gaps (no absolute normalized scores, post-hoc exclusion of Medium-Replay, contradictory statements about the plug-and-play hybrid). These issues are load-bearing because the paper's title and abstract promise a general principle of modular diffusion policy training.

major comments (4)
  1. [Section 3.2, Eqs. (10)-(12)] The theoretical justification for independent guidance rests entirely on the assumption, stated as an 'Observation', that the Q-gradient is positively correlated with the ideal sampling direction (cos(theta_t) > 0). Eq. (11) only shows that Q_phi increases along the guided path, not that the true expected return improves. Eq. (12) shows only that zero-mean noise accumulates to zero in expectation; with a fixed step size and persistent gradient noise, the standard conclusion is convergence to a noise ball, not to an optimum. The manuscript never verifies the cos(theta_t) > 0 condition for the trained Q-functions and target diffusion policies, and a Q-function accurate on the behavior distribution can be misleading off-distribution. This is a central load-bearing point: please either prove a sufficient condition from the offline-data support assumptions or measure the cosine similarity empirically over denoising trajectories.
  2. [Section 2.2, Eqs. (3)-(4), and Section 4.4] The abstract and Section 1 claim that guidance depends only on offline data and is independent of the policy module's behavior. This is not true for DQL: the Q-network is trained with temporal-difference targets that use actions sampled from the current diffusion policy, as shown in Eq. (3) and Eq. (4). The DQL Q-function therefore encodes information about the action distribution of the policy with which it was co-trained. The plug-and-play experiments in Figure 2b use IDQL guidance with DQL diffusion and vice versa, but they never test the case where a DQL-trained guidance module is transferred to a different diffusion policy, which is exactly the case where the independence premise fails. Please clarify which guidance modules satisfy the stated policy-independence property and provide direct evidence for that case.
  3. [Tables 1-2 and the Medium-Replay note] Absolute normalized scores are never reported; Tables 1 and 2 report only relative gains, and the relation between the two tables is unclear. Table 2 contains a row without a dagger despite the caption saying 'All differences marked with dagger are statistically significant', which is internally inconsistent. More importantly, the Medium-Replay environment is omitted 'due to significant performance degradation', and this exclusion appears to be made after observing the results. Medium-Replay is a standard D4RL data quality, so excluding it undermines the claim of consistent improvement. The note that the problem 'can be addressed by decreasing the ratio of loss_reward/loss_behavior_clone' also suggests task-specific tuning, which conflicts with the statement in Section 4.1 that no training-related components were modified. Please report full per-task absolute scores with standard deviations, clarify the table comparisons, and either include all tasks or justify the exclusion on a priori grounds.
  4. [Section 4.4 vs. Section 1] The paper gives contradictory descriptions of the plug-and-play hybrid: Section 1 says the forcibly combined IDQL-guidance/DQL-diffusion model 'outperformed both the standalone IDQL and DQL models', while Section 4.4 says it 'achieved final performance comparable to the DQL baseline'. Figure 2b is described without a per-task breakdown, confidence intervals, or seed counts. In addition, Section 4.4 claims that the composition 'requires only that the guidance delivers E[r(s,a)] with epsilon < 0.1 (per Theorem 1)', but Theorem 1 (from Fujimoto et al.) is about convergence of BCQL and contains no such epsilon condition. Please reconcile the contradictory statements, give the actual measured values, and either cite a theorem that supports the epsilon claim or remove it.
minor comments (8)
  1. [Section 4.1] The experiments are run on PyBullet D4RL environments, but the text says all models are trained using the 'D4RLMuJoCoTD Dataset'; please clarify the dataset/environment naming.
  2. [Section 4.2] The text says 'removing either guidance degrades performance' but earlier in the same section states that no-guidance diffusion improves faster in the early stage; please specify which stage and which algorithm each statement refers to.
  3. [Tables 1-2] The metric 'Early Gain' is not defined anywhere in the text; please define it and explain how it is computed.
  4. [Section 6] The sentence beginning 'Recent work by [Wang et al., 2024]...' is incomplete and the text ends mid-word with 'Because the lin'; please fix this truncated sentence.
  5. [Figure 2] The captions 'DDQL Double Guidance' and 'Plug-and-Play' do not describe axes, tasks, number of seeds, or aggregation; please provide full captions.
  6. [Table 3] The definitions of 'Same Model' and 'Different Guidance' should state whether the guidance is pretrained, frozen, and whether it is used during training or only inference; the row label 'policy configurations' also appears to refer to groups rather than policies.
  7. [Section 3.1] The text says Theorem 1 'motivates our design choice: pre-training a guidance policy on offline data', but BCQL is a batch-constrained Q-learning method, not a guidance policy; please rephrase to avoid conflating the two.
  8. [Throughout] The paper alternates between 'guidance module', 'classifier', and 'Q-network'; please use consistent terminology and clearly define each term at first use.

Circularity Check

1 steps flagged · score 6.0 of 10

The theoretical justification for modular guidance reduces to an assumption that the Q-gradient points toward better actions; empirical results remain external.

  1. self definitional [Section 3.2, Observation and Eq. (11)]
    "Observation: As long as this gradient is positively correlated with the ideal sampling direction (i.e., it roughly points toward higher-density or higher-reward regions), even a pretrained Q_phi can iteratively bias the sample path toward improved quality. ... E[Q_phi(x_{t+1}) - Q_phi(x_t)] ≈ η_t · cos θ_t · ||∇_x Q_phi|| + (noise)"

    The subsection's stated goal is to prove that an independently trained Q-network can guide a diffusion model. The 'Observation' assumes exactly the target property: the gradient of Q_phi points toward higher-reward regions. Equation (11) then derives an increase in Q_phi itself under that assumption, but it never shows that increasing Q_phi increases true expected return, nor that the diffusion policy's actions lie in the region where the offline-trained Q is accurate. Thus the conclusion that a pretrained Q-function can steer the diffusion process toward high-reward samples is the assumption restated, not an independent derivation. This is the load-bearing step for the paper's modularity and transferability claims.

full rationale

The main circular step is in Sec. 3.2: the theoretical justification for independent guidance assumes positive correlation between the Q-gradient and the ideal high-reward direction, then Eq. (11) only measures progress in Q_phi under that assumption. The convergence claim following Eq. (12) also overstates what averaging zero-mean Gaussian noise implies; it does not establish convergence to an optimal point. Separately, Sec. 4.4's requirement that 'the composition requires only that the guidance delivers E[r(s,a)] with ε<0.1 (per Theorem 1)' is not supported by the cited Fujimoto theorem, which contains no such epsilon condition; this is an unsupported citation rather than a circular reduction. The empirical benchmark comparisons on PyBullet D4RL (Tables 1-3, Fig. 2) are external benchmarks and do not reduce to fitted values, which is why the paper is only partially circular rather than wholly so. The internal contradiction between Sec. 1's claim that the hybrid 'outperformed both' models and Sec. 4.4's statement that it 'matched the performance baseline of DQL' weakens the empirical transfer claim but is not itself circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three types of unverified premises: the applicability of batch-constrained convergence assumptions to the benchmark tasks, the policy-invariance and positive-gradient-direction of the independently trained Q-function, and unspecified training hyperparameters (guidance coefficient, lambda, pretraining duration) that the results depend on. No new physical or architectural entities are introduced.

free parameters (3)
  • guidance coefficient alpha (Eq. 9) = not reported
    Scales the Q-gradient added during sampling and appears in the paper's central value-aware perturbation mechanism, but no value or schedule is provided.
  • guidance strength lambda (Eq. 8) = not reported
    Controls the balance between reward maximization and behavior cloning in the actor loss; the paper gives no value.
  • guidance pretraining duration = not reported
    The number of gradient steps used to train the guidance module before freezing is not specified, even though the paper says failing to lock the guidance erases the benefit.
assumptions (3)
  • domain assumption The offline environments satisfy the deterministic MDP and coherent batch conditions from Fujimoto et al. (2019).
    Invoked in Section 3.1 (Theorem 1) to motivate batch-constrained convergence, but the PyBullet D4RL datasets are not verified to meet these conditions.
  • ad hoc to paper The gradient of the independently trained Q-function remains positively correlated with the ideal sampling direction throughout denoising (cos(theta_t) > 0 for all t).
    Central to the proof sketch in Section 3.2, Eq. (11); the paper provides no evidence or conditions, and the actual update in Eq. (9) is not the process analyzed.
  • domain assumption Guidance depends solely on offline data and is independent of the diffusion policy's behavior.
    Stated in the abstract and Section 1; it is load-bearing for decoupling, but Q-values are policy-dependent and can be inaccurate on actions sampled by the diffusion model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Modular Diffusion Policy Training: Decoupling and Recombining Guidance and Diffusion for Offline RL." pith.science (2026). https://pith.science/paper/HTPBKXPE

@misc{pith2026250603154,
  author       = {Pith},
  title        = {Pith review of: Modular Diffusion Policy Training: Decoupling and Recombining Guidance and Diffusion for Offline RL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HTPBKXPE}},
  note         = {Machine review of arXiv:2506.03154}
}
read the original abstract

Classifier free guidance has shown strong potential in diffusion-based reinforcement learning. However, existing methods rely on joint training of the guidance module and the diffusion model, which can be suboptimal during the early stages when the guidance is inaccurate and provides noisy learning signals. In offline RL, guidance depends solely on offline data: observations, actions, and rewards, and is independent of the policy module's behavior, suggesting that joint training is not required. This paper proposes modular training methods that decouple the guidance module from the diffusion model, based on three key findings: Guidance Necessity: We explore how the effectiveness of guidance varies with the training stage and algorithm choice, uncovering the roles of guidance and diffusion. A lack of good guidance in the early stage presents an opportunity for optimization. Guidance-First Diffusion Training: We introduce a method where the guidance module is first trained independently as a value estimator, then frozen to guide the diffusion model using classifier-free reward guidance. This modularization reduces memory usage, improves computational efficiency, and enhances both sample efficiency and final performance. Cross-Module Transferability: Applying two independently trained guidance models, one during training and the other during inference, can significantly reduce normalized score variance (e.g., reducing IQR by 86%). We show that guidance modules trained with one algorithm (e.g., IDQL) can be directly reused with another (e.g., DQL), with no additional training required, demonstrating baseline-level performance as well as strong modularity and transferability. We provide theoretical justification and empirical validation on bullet D4RL benchmarks. Our findings suggest a new paradigm for offline RL: modular, reusable, and composable training pipelines.

Figures

Figures reproduced from arXiv: 2506.03154 by the authors.

Figure 1
Figure 1. Comparison between Half Cheetah Expert DQL and EDP with various guidance signals. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Comparison of DDQL Double Guidance, GFDT, and Baseline [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 9 canonical work pages

  1. [2]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2024. URL https://arxiv.org/abs/2303.04137

    arXiv 2024

  2. [3]

    Diffusion policies as an expressive policy class for offline reinforcement learning

    Zhendong Wang et al. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.06193, 2023

    arXiv 2023

  3. [5]

    Diffusion policies creating a trust region for offline reinforcement learning

    Tianyu Chen, Zhendong Wang, and Mingyuan Zhou. Diffusion policies creating a trust region for offline reinforcement learning. arXiv preprint arXiv:2405.19690, 2024. URL https://arxiv.org/abs/2405.19690

    arXiv 2024

  4. [6]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. URL https://arxiv.org/abs/2207.12598

    arXiv 2022

  5. [7]

    Refining generative process with discriminator guidance in score-based diffusion models

    Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang, and Il-Chul Moon. Refining generative process with discriminator guidance in score-based diffusion models. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 1--25. PMLR, 2023

    work page 2023

  6. [8]

    Towards controllable diffusion models via reward-guided exploration

    Hengtong Zhang and Tingyang Xu. Towards controllable diffusion models via reward-guided exploration. arXiv preprint arXiv:2304.07132, 2023. URL https://arxiv.org/abs/2304.07132

    arXiv 2023

  7. [9]

    Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review, 2025

    Masatoshi Uehara, Yulai Zhao, Chenyu Wang, Xiner Li, Aviv Regev, Sergey Levine, and Tommaso Biancalani. Inference-time alignment in diffusion models with reward-guided generation: Tutorial and review, 2025

    2025

  8. [10]

    Double q-learning

    Hado van Hasselt. Double q-learning. In Advances in Neural Information Processing Systems 23 (NeurIPS 2010), pages 2613--2621. Curran Associates, Inc., 2010

    work page 2010

Show all 23 references
  1. [11]

    Adding conditional control to diffusion models with reinforcement learning

    Yulai Zhao, Masatoshi Uehara, Gabriele Scalia, Sunyuan Kung, Tommaso Biancalani, Sergey Levine, and Ehsan Hajiramezanali. Adding conditional control to diffusion models with reinforcement learning. In Proceedings of the International Conference on Learning Representations (ICL...

    2025

  2. [12]

    Planning with diffusion for flexible behavior synthesis

    Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning, 2022 b . URL https://arxiv.org/abs/2205.09991

    2022 arXiv

  3. [13]

    Efficient diffusion policies for offline reinforcement learning

    Bingyi Kang, Xiao Ma, Chao Du, Tianyu Pang, and Shuicheng Yan. Efficient diffusion policies for offline reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2024. URL https://arxiv.org/abs/2310.03573

    2024 arXiv

  4. [14]

    Idql: Implicit q-learning as an actor-critic method with diffusion policies

    Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, and Sergey Levine. Idql: Implicit q-learning as an actor-critic method with diffusion policies. arXiv preprint arXiv:2304.10573, 2023 b . URL https://arxiv.org/abs/2304.10573

    2023 arXiv

  5. [15]

    Benchmarking batch deep reinforcement learning algorithms

    Scott Fujimoto, Edoardo Conti, Mohammad Ghavamzadeh, and Joelle Pineau. Benchmarking batch deep reinforcement learning algorithms. arXiv preprint arXiv:1910.01708, 2019

    1910 arXiv

  6. [16]

    D4rl: Datasets for deep data-driven reinforcement learning

    Justin Fu, Aviral Kumar, Ofir Nachum, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning. In Conference on Learning Representations (ICLR), 2020. URL https://arxiv.org/abs/2004.07219

    2020 arXiv

  7. [17]

    Cleandiffuser: Simple and clean diffusion-based offline rl

    Yifan Dong, Yining Xie, Yilun Chen, Zihan Liu, Mingyu Zhang, Yuhuang Wu, Lihong Zhao, Qi Wang, and Wen Sun. Cleandiffuser: Simple and clean diffusion-based offline rl. arXiv preprint arXiv:2401.13659, 2024. URL https://arxiv.org/abs/2401.13659

    2024 arXiv

  8. [18]

    Gymnasium: A standard api for reinforcement learning environments

    The Farama Foundation. Gymnasium: A standard api for reinforcement learning environments. https://github.com/Farama-Foundation/Gymnasium, 2024

    2024

  9. [19]

    D4rl: Datasets for deep data-driven reinforcement learning

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021. URL https://datasets-benchmarks-pro...

    2021

  10. [20]

    Diffusion-dice: In-sample diffusion guidance for offline reinforcement learning

    Liyuan Mao, Haoran Xu, Xianyuan Zhan, Weinan Zhang, and Amy Zhang. Diffusion-dice: In-sample diffusion guidance for offline reinforcement learning. In Advances in Neural Information Processing Systems, 2024. URL https://arxiv.org/abs/2407.20109

    2024 arXiv

  11. [21]

    Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning

    Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning. In International Conference on Machine Learning, 2023. URL https://arxiv.org/abs/2304.12824

    2023 arXiv

  12. [22]

    Modular multitask reinforcement learning with policy sketches

    Jacob Andreas, Dan Klein, and Sergey Levine. Modular multitask reinforcement learning with policy sketches. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017. URL https://arxiv.org/abs/1704.06643

    2017 arXiv

  13. [23]

    Composing complex skills by learning transition policies

    Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, and Sergey Levine. Composing complex skills by learning transition policies. In 2019 IEEE International Conference on Robotics and Automation (ICRA), 2019. URL https://arxiv.org/abs/1906.01068

    2019 arXiv

  14. [24]

    Analysis of classifier-free guidance weight schedulers

    Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. Transactions on Machine Learning Research, December 2024. URL https://arxiv.org/abs/2404.13040. R...

    2024 arXiv

  15. [25]

    Plug and play, model-based reinforcement learning, 2021

    Majid Abdolshah, Hung Le, Thommen George Karimpanal, Sunil Gupta, Santu Rana, and Svetha Venkatesh. Plug and play, model-based reinforcement learning, 2021

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.