Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

A teacher flow-matching model can supply both reward and behavioral regularization for online imitation learning, so a simple MLP policy learns stably and efficiently.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:36 UTC pith:NN7MTSKT

load-bearing objection FM-IRL is a competent, honest empirical IL paper — real but incremental delta over DRAIL/FPO — whose uncalibrated loss-difference reward and uneven baseline tuning make it a revise-and-resubmit rather than an accept. the 3 major comments →

arxiv 2510.09222 v3 pith:NN7MTSKT submitted 2025-10-10 cs.LG

FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learning

Zhenglin Wan , Jingxuan Wu , Xingrui Yu , Chubin Zhang , Mingcong Lei , Bo An , Ivor Tsang This is my paper
classification cs.LG
keywords flow matchinginverse reinforcement learningreward modelingpolicy regularizationimitation learningteacher-student architectureonline reinforcement learningsuboptimal demonstrations
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes treating a flow-matching generative model as a teacher that performs two jobs in online imitation learning: it converts distribution-level closeness to expert data into a scalar reward for a lightweight MLP student policy, and it generates expert-like behavior used to regularize the student. The student interacts with the environment and is updated with standard policy-gradient RL, avoiding the unstable and costly gradient computation that direct online tuning of flow-matching policies requires. The authors claim this is the first framework to use flow matching for both reward modeling and policy regularization in online RL. If true, imitation from demonstrations becomes more sample-efficient, generalizes to perturbed or unseen states, and remains robust when demonstrations are suboptimal; the deployed policy is also fast because only the MLP runs at inference time.

Core claim

The central claim is that a flow-matching model trained on the joint state-action distribution, conditioned on a binary label indicating expert or agent data, yields a per-sample flow-matching loss that measures distribution-level similarity. Taking a softmax over negative losses for the two labels produces a discriminator whose logit is used as a shaped reward; the same model, conditioned on the expert label, generates state-action pairs whose actions are matched to the student's actions by mean-squared error. This two-channel infusion lets the student policy benefit from the teacher's expressive distributional knowledge while keeping the online update in the space of simple MLP parameters.

What carries the argument

The key object is the per-sample conditional flow-matching distance Dist_theta(s,a|c): for a given state-action pair, the expected squared error between the learned vector field and the target velocity along a straight-line conditional path, conditioned on whether the pair is treated as expert or agent data. This distance plays two roles: through a softmax over negative distances it defines the reward model, and as an action generator for the expert condition it defines the behavioral regularizer. The same learned object therefore carries both the distribution-matching and the exploration-control signal.

Load-bearing premise

The reward and regularizer both depend on the assumption that the per-sample conditional flow-matching loss is a faithful, calibrated measure of how close a state-action pair is to the expert distribution, especially on states the agent visits that the expert never saw.

What would settle it

Construct an environment with two well-separated expert behavioral modes; compute Dist_theta for actions inside each mode, in the gap between modes, and outside the support. If the loss assigns a lower distance to interpolated or out-of-distribution actions than to genuine expert modes, the reward is not distribution-calibrated and the method's advantage should collapse; a direct way to check is to compare the learned reward to the true reward in an environment where the true reward is known.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A simple MLP policy updated with standard policy-gradient RL can surpass behavior-cloning policies on tasks where expert data does not cover all states, because the teacher's distribution-level reward gives guidance during active exploration.
  • The same flow-matching model serves as reward model and regularizer, so no separate discriminator or extra generative model is needed; regularization counteracts reward misestimation on out-of-distribution states.
  • The framework learns from scratch without pre-training or fine-tuning a flow policy, sidestepping backpropagation-through-time instability and reducing training and inference cost at deployment.
  • When expert demonstrations are suboptimal but above novice level, the framework can exceed expert performance, whereas cloning methods overfit and degrade.
  • The approach is most beneficial in settings with multi-modal expert behavior, since flow matching captures action multimodality and the student can imitate it through the reward and regularizer channels.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the teacher-student split is a general recipe; any generative model that exposes a per-sample training loss could replace the flow-matching teacher, so the same reward-plus-regularizer scheme may transfer to other density estimators.
  • Editorial inference: because the regularizer matches student actions to teacher-generated expert-like actions, it acts as an implicit conservatism penalty; adaptively tuning its strength per state or per training phase might further improve sample efficiency.
  • Editorial inference: the joint state-action modeling choice means the regularizer also constrains visited states, suggesting a testable extension where the FM teacher proposes states, not just actions, to guide exploration in sparse-reward environments.
  • Editorial inference: deployed inference cost scales with the MLP student alone, so the framework is a plausible candidate for real-time robotic control; a direct comparison of wall-clock deployment latency against flow-policy baselines would make this concrete.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes FM-IRL, a teacher-student imitation learning framework. A flow-matching (FM) model is trained as an AIL-style discriminator via a conditional FM loss (Eq. 9–10) and supplies the reward r(s,a)=log D - log(1-D) (Eq. 6). The same FM generator is then used to regularize an MLP student policy toward expert-like state-action pairs (Eq. 11). The method is evaluated on six MuJoCo/Gym environments against DP, FP, GAIL, VAIL, WAIL, AIRL, and DRAIL, with additional generalization, suboptimal-expert robustness, and wall-clock case studies. The paper claims this is the first framework to use flow matching for reward modeling and regularization in online RL, and that it yields faster, more stable, more generalizable learning with a low-cost MLP deployment policy.

Significance. If the empirical findings hold, FM-IRL is a practically valuable recipe: it decouples the expressive FM model from the policy, avoiding backpropagation through ODE solvers while retaining distribution-level expert knowledge in a cheap MLP policy. Strengths include an open-source code release, a broad baseline suite, explicit discussion of why FM policies are hard to update online (Appendix D), and an ablation of the regularization weight. The main technical uncertainty is the reward semantics: the paper does not establish that a per-sample conditional FM loss is a calibrated distributional distance, and the discriminator objective in Eq. 5 is non-standard. Because the reward and the regularizer share the same FM model, a miscalibrated Dist can propagate errors into both. Additional calibration analysis and a few reporting fixes are needed before the stability and generalization claims can be fully credited.

major comments (3)
  1. [§3.2, Eq. (5)] The discriminator objective is written as min L = E_E log(1-D) + E_π log D. This is not the standard AIL objective given in §2.2 (max E_E log D + E_π log(1-D)), nor is it the usual binary cross-entropy for an expert indicator. Minimizing the written loss pushes D→1 on expert data and D→0 on agent data, so it has the same separating direction as the standard objective, but the loss is unbounded below: log(1-D) and log D both go to -∞ at the separating extremes. If implemented literally, the reward logit in Eq. (6) will saturate and the reward scale becomes non-stationary. If the implementation uses a standard BCE with a different sign convention, then Eq. (5) does not describe the method. Since this objective defines the entire learning signal, please correct the equation and state precisely the loss actually minimized in the released code.
  2. [§3.3, Eqs. (8)–(10)] The reward is r = Dist_θ(s,a|c=0) - Dist_θ(s,a|c=1), which identifies an expected squared velocity residual along an OT path pinned to (s,a) with a distribution-level closeness measure. This is an uncalibrated heuristic borrowed from FPO; the paper supplies no proof, no calibration experiment, and no monotonicity check against true expert likelihood. If the loss is miscalibrated on out-of-distribution states, both the reward model and the regularizer built from the same G_θ(c=1) can reinforce each other's errors. I suggest adding a controlled-study calibration check (e.g., compare Dist_θ(c=1) against true density or against distance to the expert dataset) or an ablation replacing the FM-distance reward with an oracle reward to isolate the assumption.
  3. [§4.2, Figure 4; §4.4, Figure 5] The generalization study and the wall-clock/case study report single learning curves without error bars, confidence intervals, or seed-level variability, despite phrases such as 'consistently outperforms' and 'significantly more efficient.' The rest of the paper uses four seeds, so error bands should be straightforward to add. In addition, some baseline numbers in Table 3 (e.g., AIRL mean return -5.27, WAIL success 0.0000) are implausibly low and may indicate implementation issues; please verify these baselines or report their configuration and any failed runs.
minor comments (7)
  1. [§3.4, Eq. (11)] The regularization term writes a_π ∼ π_ϕ without specifying the conditioning state. It should be a_π ∼ π_ϕ(·|s_G), with s_G drawn from the FM generator's joint distribution; please clarify this notation.
  2. [Algorithm 1, line 7] The pseudo-code says 'Compute reward ∇_θ(s,a)'; this should be r_θ(s,a).
  3. [Table 2] Hyperparameters temperature and noise_scale are listed but their exact role in Eq. (30) is not documented; the regularization weight β is missing from Table 2. Please add the definitions and usage of temperature/noise_scale and include β.
  4. [§F.2] β=2 is selected from a Fetch-pick ablation and then reused across all six environments. Since the regularization term is a core component, a sensitivity study on at least one additional environment would strengthen the claim that the method is not overly sensitive to this choice.
  5. [§4.4] The case study compares FM-IRL (no true reward) with FPO/FM-PPO/FM-A2C (true reward available). This is a meaningful efficiency comparison, but the setting asymmetry should be stated explicitly in the main text, not only in the caption, to avoid over-interpretation.
  6. [§5] DiffAIL is discussed in related work but not included in the experimental comparisons. Either add it to the baseline suite or state why it is omitted (e.g., computational cost) in the experimental setup.
  7. [General] There are several typographical issues: §3.2 says 'scaler' instead of 'scalar'; Figure 1 contains '??,??' placeholders; and Eq. (5) uses L_FM for a loss with an unbounded minimum. A careful proofread is needed.

Circularity Check

0 steps flagged

No significant circularity: FM-distance reward is a fitted density proxy, and success is measured by external environment rewards; self-citations are motivational only.

full rationale

The central reward (Eq. 33) is r_theta(s,a) = Dist_theta(s,a|c=0) - Dist_theta(s,a|c=1) with D defined by the softmax in Eq. 10. This is a label-conditioned fitted error, not a parameter fitted to a subset and then renamed as a prediction of the same quantity. The teacher FM model is trained on expert and agent data by the FM regression (Eq. 7), and the resulting per-sample loss is used as a discrimination signal; this is the standard AIL/generative-model reward construction (also used by DRAIL and FPO), not a self-referential definition. The regularization term (Eq. 11) reuses the same G_theta(c=1) to produce target actions, but this is architectural reuse; it does not define the reward nor the evaluation metric. All headline results are measured by environment success rates and returns (Figures 3-4, Tables 1 and 3), which are external to the fitted FM losses. The only self-citations (Wan et al. 2024, 2025b; Yu et al. 2025) appear in Section 1 to motivate the claim that BC-based FM policies overfit to suboptimal data, and the paper additionally validates this in its own Section 4.3; they are not load-bearing for the derivation. The weakest point is the unverified FPO-inspired identification of per-sample FM loss with distribution-level closeness (Section 3.3), which is a calibration/robustness risk rather than a circularity: the paper does not reduce a predicted quantity to a fitted input. The paper's own admission that 'with highly deficient expert data, FM-IRL struggles to infer a meaningful reward model' (Section 4.3) is a stated limitation, not evidence of circular construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method introduces no new physical entities. It rests on the FPO heuristic that FM loss is a distribution-similarity measure, on the joint-modeling choice, on the MLP sufficiency assumption, and on several hand-set hyperparameters (beta, temperature, noise scale, MC samples).

free parameters (4)
  • beta (regularization weight) = 2
    Weight on the policy regularization term in Eq. 11. Selected via ablation on Fetch-pick (Appendix F.2) and reused for all main experiments.
  • temperature = 0.1
    Scale factor on the averaged distance in reward computation (Table 2); chosen by hand to stabilize reward magnitudes.
  • noise_scale = 0.5
    Standard deviation for x0's noise component when forming xt during Dist estimation (Table 2); chosen by hand.
  • num_samples (MC) = 100
    Monte Carlo samples S used to estimate Dist in Eq. 31 (Table 2); chosen as a speed-accuracy trade-off.
axioms (5)
  • standard math Conditional Flow Matching has the same gradients as Flow Matching, so optimizing the CFM objective trains the desired vector field.
    Invoked in Section 2.1 to justify the practical training objective (nabla L_FM = nabla L_CFM).
  • domain assumption The per-sample conditional FM loss Dist_theta(s,a|c) is a faithful measure of distribution-level similarity between (s,a) and the c-labeled data distribution.
    This is the central heuristic behind the FM-enhanced discriminator and reward (Section 3.3, Eq. 8-10), adopted from FPO without independent derivation or calibration analysis.
  • domain assumption Modeling the joint (s,a) distribution rather than a|s is appropriate and beneficial for reward modeling and regularization.
    Defended in Appendix B: the joint model enables state regularization by generating expert-like states, not just actions.
  • domain assumption A lightweight MLP student policy can capture expert behavior sufficiently well once guided by the FM teacher through reward and regularization.
    This is the design choice that makes online gradients stable and inference fast, but it assumes the distillation signal is rich enough; tested only empirically.
  • domain assumption The FM generator conditioned on c=1 produces state-action pairs representative of the expert distribution, so they can serve as regression targets in Eq. 11.
    Used in the policy regularization term; depends on the same distance-as-similarity assumption and on successful FM training.

pith-pipeline@v1.3.0-alltime-deepseek · 3711 in / 3842 out tokens · 519770 ms · 2026-08-04T10:36:38.810498+00:00 · methodology

0 comments
read the original abstract

Flow Matching (FM) has shown remarkable ability in modeling complex distributions and achieves strong performance in offline imitation learning for cloning expert behaviors. However, despite its behavioral cloning expressiveness, FM-based policies are inherently limited by their lack of environmental interaction and exploration. This leads to poor generalization in unseen scenarios beyond the expert demonstrations, underscoring the necessity of online interaction with environment. Unfortunately, optimizing FM policies via online interaction is challenging and inefficient due to instability in gradient computation and high inference costs. To address these issues, we propose to let a student policy with simple MLP structure explore the environment and be online updated via RL algorithm with a reward model. This reward model is associated with a teacher FM model, containing rich information of expert data distribution. Furthermore, the same teacher FM model is utilized to regularize the student policy's behavior to stabilize policy learning. Due to the student's simple architecture, we avoid the gradient instability of FM policies and enable efficient online exploration, while still leveraging the expressiveness of the teacher FM model. Extensive experiments show that our approach significantly enhances learning efficiency, generalization, and robustness, especially when learning from suboptimal expert data.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Training Diffusion Policies via Prior-Mapping Co-Evolution

    cs.LG 2025-12 conditional novelty 6.0

    GoRL outperforms Gaussian and generative RL baselines on continuous control by optimizing a Gaussian latent policy while a separately trained diffusion or flow decoder maps fixed noise to actions.

Reference graph

Works this paper leans on

26 extracted references · 24 linked inside Pith · cited by 1 Pith paper

  1. [2]

    Max Braun, Noémie Jaquier, Leonel Rozo, and Tamim Asfour

    URL https://arxiv.org/abs/2410.24164. Max Braun, Noémie Jaquier, Leonel Rozo, and Tamim Asfour. Riemannian flow matching policy for robot motion learning. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5144–5151,

    Pith/arXiv arXiv

  2. [7]

    Chun-Mao Lai, Hsiang-Chun Wang, Ping-Chun Hsieh, Frank Wang, Min-Hung Chen, and Shao-Hua Sun

    URLhttps://arxiv.org/abs/2412.03603. Chun-Mao Lai, Hsiang-Chun Wang, Ping-Chun Hsieh, Frank Wang, Min-Hung Chen, and Shao-Hua Sun. Diffusion- reward adversarial imitation learning.Advances in Neural Information Processing Systems, 37:95456–95487,

    Pith/arXiv arXiv

  3. [8]

    Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

    Pith/arXiv arXiv

  4. [9]

    Flow-GRPO: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470,

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-GRPO: Training flow matching models via online rl.arXiv preprint arXiv:2505.05470,

    Pith/arXiv arXiv

  5. [10]

    Flow matching policy gradients.arXiv preprint arXiv:2507.21053,

    David McAllister, Songwei Ge, Brent Yi, Chung Min Kim, Ethan Weber, Hongsuk Choi, Haiwen Feng, and Angjoo Kanazawa. Flow matching policy gradients.arXiv preprint arXiv:2507.21053,

    Pith/arXiv arXiv

  6. [12]

    Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, and Sergey Levine

    URLhttps://arxiv.org/abs/2301.10677. Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, and Sergey Levine. Variational discriminator bot- tleneck: Improving imitation learning, inverse RL, and gans by constraining information flow.arXiv preprint arXiv:1810.00821,

    Pith/arXiv arXiv

  7. [13]

    URLhttps://arxiv.org/abs/2312.11752. Allen Z. Ren, Justin Lidard, Lars L. Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization,

    Pith/arXiv arXiv

  8. [14]

    org/abs/2409.00588

    URL https://arxiv. org/abs/2409.00588. Moritz Reuss, Maximilian Li, Xiaogang Jia, and Rudolf Lioutikov. Goal-conditioned imitation learning using score- based diffusion policies,

    Pith/arXiv arXiv

  9. [15]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

    URLhttps://arxiv.org/abs/2304.02532. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms,

    Pith/arXiv arXiv

  10. [17]

    10 Andrew Szot

    URLhttps://arxiv.org/abs/2310.07896. 10 Andrew Szot. RL-toolkit.https://github.com/ASzot/rl-toolkit,

    Pith/arXiv arXiv

  11. [20]

    Wan: Open and advanced large-scale video generative models, 2025a

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, T...

    Pith/arXiv arXiv

  12. [21]

    Diversifying policy behaviors with extrinsic behavioral curiosity, 2025b

    Zhenglin Wan, Xingrui Yu, David Mark Bossens, Yueming Lyu, Qing Guo, Flint Xiaofeng Fan, Yew Soon Ong, and Ivor Tsang. Diversifying policy behaviors with extrinsic behavioral curiosity, 2025b. URL https://arxiv.org/ abs/2410.06151. Bingzheng Wang, Guoqiang Wu, Teng Pang, Yan Zhang, and Yilong Yin. Diffail: Diffusion adversarial imitation learning. InProce...

    Pith/arXiv arXiv

  13. [23]

    Huang Xiao, Michael Herman, Joerg Wagner, Sebastian Ziesche, Jalal Etesami, and Thai Hong Linh

    URLhttps://arxiv.org/abs/2208.06193. Huang Xiao, Michael Herman, Joerg Wagner, Sebastian Ziesche, Jalal Etesami, and Thai Hong Linh. Wasserstein adversarial imitation learning.arXiv preprint arXiv:1906.08113,

    Pith/arXiv arXiv 1906

  14. [24]

    Xingrui Yu, Zhenglin Wan, David Mark Bossens, Yueming Lyu, Qing Guo, and Ivor W

    URL https://arxiv.org/abs/2305.13122. Xingrui Yu, Zhenglin Wan, David Mark Bossens, Yueming Lyu, Qing Guo, and Ivor W. Tsang. Imitation from diverse behaviors: Wasserstein quality diversity imitation learning with single-step archive exploration,

    Pith/arXiv arXiv

  15. [25]

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu

    URL https://arxiv.org/abs/2411.06965. Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,

    Pith/arXiv arXiv

  16. [26]

    Fan Zhang and Michael Gienger

    URLhttps://arxiv.org/abs/2403.03954. Fan Zhang and Michael Gienger. Affordance-based robot manipulation with flow matching,

    Pith/arXiv arXiv

  17. [27]

    Qinglun Zhang, Zhen Liu, Haoqiang Fan, Guanghui Liu, Bing Zeng, and Shuaicheng Liu

    URL https: //arxiv.org/abs/2409.01083. Qinglun Zhang, Zhen Liu, Haoqiang Fan, Guanghui Liu, Bing Zeng, and Shuaicheng Liu. Flowpolicy: Enabling fast and robust 3d flow-based policy via consistency flow matching for robot manipulation. InProceedings of the AAAI Conference on Artificial Intelligence, pp. 14754–14762, 2025a. Tonghe Zhang, Chao Yu, Sichang Su...

    arXiv

  18. [2000]

    Flow q-learning.arXiv preprint arXiv:2502.02538,

    Seohong Park, Qiyang Li, and Sergey Levine. Flow q-learning.arXiv preprint arXiv:2502.02538,

    Pith/arXiv arXiv

  19. [2012]

    Behavioral cloning from observation.arXiv preprint arXiv:1805.01954,

    Faraz Torabi, Garrett Warnell, and Peter Stone. Behavioral cloning from observation.arXiv preprint arXiv:1805.01954,

    Pith/arXiv arXiv

  20. [2016]

    Vijay Konda and John Tsitsiklis

    URL https://arxiv.org/abs/ 1606.03476. Vijay Konda and John Tsitsiklis. Actor-critic algorithms.Advances in Neural Information Processing Systems, 12,

    Pith/arXiv arXiv

  21. [2017]

    Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine

    URLhttps://arxiv.org/abs/1707.06347. Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine. Nomad: Goal masked diffusion policies for navigation and exploration,

    Pith/arXiv arXiv

  22. [2018]

    Gymnasium: A standard interface for reinforcement learning environments.arXiv preprint arXiv:2407.17032,

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments.arXiv preprint arXiv:2407.17032,

    Pith/arXiv arXiv

  23. [2021]

    Scott Fujimoto, David Meger, and Doina Precup

    URLhttps://openreview.net/forum?id=px0-N3_KjA. Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International Conference on Machine Learning, pp. 2052–2062. PMLR,

    2052

  24. [2023]

    Jonathan Ho and Stefano Ermon

    URL https://arxiv.org/abs/2304.10573. Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning,

    Pith/arXiv arXiv

  25. [2024]

    URLhttps://arxiv.org/abs/2404.03729. Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohua...

    Pith/arXiv arXiv

  26. [2025]

    9 Justin Fu, Katie Luo, and Sergey Levine

    URL https://arxiv.org/abs/2502.06061. 9 Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adversarial inverse reinforcement learning. arXiv preprint arXiv:1710.11248,

    Pith/arXiv arXiv