Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

This paper shows that exact forward-backward representations cannot exist in continuous control with finite-dimensional representations, and that a simplified one-step variant converges to far smaller errors and improves zero-shot performan

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 00:09 UTC pith:UEFUG46N

load-bearing objection Useful paper for the zero-shot RL crowd: solid discrete analysis and a simpler method that works, but the headline impossibility claim for continuous domains is asserted rather than proved. the 3 major comments →

arxiv 2602.11399 v2 pith:UEFUG46N submitted 2026-02-11 cs.LG cs.AIcs.RO

Can We Really Learn One Representation to Optimize All Rewards?

Chongyi Zheng , Royina Karegoudra Jayanth , Benjamin Eysenbach This is my paper
classification cs.LG cs.AIcs.RO
keywords forward-backward representation learningsuccessor measurezero-shot reinforcement learningunsupervised pretrainingdensity ratio estimationcontraction mappinglow-rank approximationoffline RL
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Forward-backward (FB) representation learning promises to pre-train one representation that, for any reward, can produce the optimal policy. This paper argues that promise is broken: exact ground-truth FB representations require a representation dimension at least as large as the number of state-action pairs, so finite-dimensional representations cannot exist in continuous domains. The paper also shows why FB fails in practice: its objective is a temporal-difference density-ratio fit whose associated Bellman operator is not a contraction, and in simple testbeds it does not converge to its claimed fixed point. The authors then propose one-step FB, which fits the successor-measure ratio of a fixed behavioral policy and performs one step of policy improvement, and show it converges to much smaller errors and improves zero-shot performance by 24% on average across 10 control benchmarks. If the analysis holds, the practical goal of zero-shot RL shifts from 'optimal for any reward' to 'one improvement step from a behavioral prior.'

Core claim

The central claim is that forward-backward (FB) representation learning—which pre-trains F and B so that M^π/ρ = F(s,a,z)ᵀB(s',a') and then sets z = E[B r] to recover optimal policies—cannot in general achieve its promised fixed point. In any discrete CMP the successor measure matrix is full rank, so an exact factorization forces the representation dimension d to be at least the number of state-action pairs; taking the state-action space to be infinite, finite-dimensional ground-truth representations are impossible. The paper also rederives the FB objective as a temporal-difference variant of least-squares importance fitting and shows that the associated FB Bellman operator is not a γ-contra

What carries the argument

The load-bearing object is the successor measure ratio M^π(s', a' | s,a)/ρ(s',a'), the discounted occupancy of future state-action pairs under a policy, approximated as a bilinear factorization F(s,a,z)ᵀB(s',a'). In discrete CMPs the successor measure matrix is full rank, forcing any exact factorization to have dimension d ≥ |S × A| (Proposition 1). One-step FB makes the target static by fixing a behavioral policy π_β, so the factorization reduces to an SVD of a fixed matrix rather than a self-referential equation. The FB Bellman operator—the contraction-style map underlying the original algorithm—is shown not to be a γ-contraction, which is why the standard fixed-point proof fails.

Load-bearing premise

The impossibility result for continuous domains rests on carrying the finite-state rank bound d ≥ |S × A| over to infinite state-action spaces, and the didactic failure experiments substitute the Monte-Carlo loss for the actual temporal-difference objective.

What would settle it

Run the exact temporal-difference FB loss (Eq. 9), not the Monte-Carlo proxy, on the three-state and five-state didactic CMPs; if FB converges to a fixed point whose Q predictions obey the affine-equivariance property, the paper's central empirical failure claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Universal zero-shot optimality with finite representations is out of reach for large or continuous state-action spaces; exact FB needs one dimension per state-action pair.
  • The FB training objective is a temporal-difference version of least-squares importance fitting, so it is a fitted Q-evaluation for the successor-measure ratio rather than a standard value iteration; its Bellman operator lacks a contraction guarantee.
  • The one-step FB variant—fitting a fixed behavioral policy's successor measure and then taking one policy improvement step—converges to errors about 1e5 times smaller in didactic settings, and achieves +24% average zero-shot return over FB on 10 benchmark domains.
  • One-step FB's zero-shot policies serve as efficient initializations for further online fine-tuning, improving sample efficiency relative to learning from scratch.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: If exact universal representations are impossible, the field's objective for behavioral foundation models should be reframed from 'optimal for any reward' to 'one policy-improvement step from a behavioral prior', which is a strictly weaker and achievable target.
  • Inference: The non-contraction result likely generalizes to any representation-learning method whose policy and value target are coupled; such objectives should be checked for multiple fixed points (e.g., rotational symmetries of F and B) before assuming convergence.
  • Inference: The SVD view suggests one-step FB's quality is governed by the spectrum of the behavioral successor measure ratio, pointing to exploration strategies that shape that spectrum rather than try to cover all possible rewards.
  • Inference: A natural extension is to test one-step FB under limited coverage and stochastic or partially observed dynamics; the paper itself flags that performance may degrade when the behavioral policy covers the downstream task poorly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper analyzes forward-backward (FB) representation learning for zero-shot RL. It claims: (i) exact ground-truth FB representations require representation dimension at least |S×A| in finite CMPs, and are impossible in continuous CMPs with finite d (Proposition 1, Corollary 1); (ii) the FB objective can be derived as a temporal-difference least-squares importance fitting loss, and the associated FB Bellman operator is not a γ-contraction (Proposition 3); (iii) a simplified method, one-step FB, fits the successor measure ratio of a fixed behavioral policy and performs one step of policy improvement; didactic and continuous-control experiments show better convergence and zero-shot performance than FB. The paper includes proofs of the finite-state results, an implementation, and ablations.

Significance. If the finite-CMP results are accepted, the paper makes a useful conceptual contribution: it connects FB to density-ratio estimation and fitted Q-evaluation, shows a rank obstruction to exact FB representations, and identifies a circular-dependency issue in FB's Bellman operator. The proposed one-step FB method is simple, reproducible, and empirically competitive, with a clear interpretation as fitting a static successor-measure ratio. These positive aspects are partially offset by the fact that the headline negative answer to the title question rests on an unproven infinite-dimensional limit, and by a gap in the proof of Proposition 2. The empirical and algorithmic parts are stronger than the theoretical parts.

major comments (3)
  1. [§3.1, Corollary 1; Appendix C.1; §6 Limitations] Corollary 1 is the linchpin of the negative answer to the title question, but it is not established by Proposition 1. Proposition 1 proves d >= |S×A| for finite CMPs; Corollary 1 then lets |S×A| → ∞ and concludes that finite d is impossible for continuous CMPs. This limit is not a defined mathematical operation: no operator-theoretic definition of the successor-measure ratio on infinite-dimensional spaces is given, no rank theorem for infinite-dimensional kernels is proved, and rank is not continuous under such limits. Infinite-dimensional operators can have finite rank even when the index set is infinite, and the delta term in Eq. (1) may make the continuous object a distribution rather than a Hilbert–Schmidt kernel. The paper's own Limitations (Sec. 6) state that extending the analysis to continuous settings 'relies on approximation,' which contradicts the categorical wording of Coroll
  2. [§5.1, Eq. (7) vs Eq. (9); Appendix D.1] The didactic experiment that claims 'the FB algorithm fails to converge' optimizes the Monte-Carlo FB loss (Eq. 7), not the TD FB objective (Eq. 9) that the practical FB algorithm actually uses. The MC loss is a supervised regression to an analytically computed successor measure; the TD loss involves target networks and on-policy next-action sampling, and the two can have different fixed points and optimization dynamics. The paper calls Eq. (7) an 'analytical analogy,' but no formal relationship is given that would let the MC-loss behavior transfer to the TD objective. Since the practical convergence claim is central to the motivation for one-step FB, the experiment should be run with the TD objective (or with a precise equivalence theorem).
  3. [Appendix C.3, Proposition 2] The proof of Proposition 2 is incomplete. From the affine-equivariance of Q-values the authors derive, for every reward r, an equality F(s,a,z_{νr+ξ})ᵀ z_r = F(s,a,z_r)ᵀ z_r. This only shows equality of inner products with z_r. By Proposition 1, z_r = B(r ⊙ ρ) lies in the column space of B, which has dimension |S×A|; when d > |S×A|, these vectors do not span R^d, so the conclusion F(s,a,z_{νr+ξ}) = F(s,a,z_r) does not follow. The proposition as stated is therefore not proven unless d = |S×A| or an additional spanning assumption is imposed. This matters because Proposition 2 is the theoretical basis for the equivariance-error criterion used in the didactic experiments (Secs. 5.1 and 5.2).
minor comments (4)
  1. [Abstract] The phrase 'errors 105 smaller' should read '10^5 times smaller' (typesetting issue).
  2. [Table 1 / Table 2] The visual-domain rows in Tables 1 and 2 appear to contain fewer numeric entries than the six method columns in the header. This makes it hard to verify the claimed +24% average improvement over FB on image-based domains. Please reformat so each row has one entry per method.
  3. [§3.3, Definition 3] The FB Bellman operator TFB is defined on the inner product f(s,a,z)ᵀ b(sf,af) but its value depends on the policy induced by f, not only on the inner-product function. This should be stated explicitly to avoid ambiguity about what 'applying TFB to an inner product' means.
  4. [Appendix E.2] There are typos such as 'proprobability' and 'convergence transits'; please proofread the appendix.

Circularity Check

0 steps flagged

No significant circularity: central results derive from external rank/linear-algebra facts and held-out benchmarks; the only flagged issue is an underproved continuous limit, which is a rigor gap, not a circular reduction.

full rationale

The paper's central impossibility result (Proposition 1 and Corollary 1) is derived from the externally cited full-rank successor-measure lemma (Agarwal et al. 2019, Lemma 1.6/Corollary 1.5) plus standard linear algebra. The FB definitions are imported from Touati & Ollivier (2021), not from the authors' own prior work, and no load-bearing conclusion is justified by a self-citation. Proposition 3's non-contraction proof uses a rotation construction from Touati et al. 2022, which is external and does not assume the result. The didactic 'convergence' of one-step FB is transparently a supervised fit to a static target: Appendix D.2 states 'the target density ratio ... is fixed given πβ, the learning one-step FB reduces to solving a supervised learning problem,' and the reported errors are MSEs to that same target. This is a self-consistency check, not an external prediction, so it is not circular. The main benchmark claims are tested against held-out ExORL/OGBench tasks, so they do not reduce to the training objective. The one genuine weakness is Corollary 1: the finite matrix rank bound d >= |S x A| is extended to continuous CMPs by letting |S x A| -> infinity without defining the infinite-dimensional successor-measure operator or proving a rank theorem there. The paper's own Limitations (Sec. 6) concede 'extending it to continuous settings relies on approximation.' That is an unsupported extrapolation / correctness risk, not a circular definitional reduction; per the stated rules it does not raise the circularity score.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The central theoretical derivation has few free parameters; the fitted quantities are standard RL hyperparameters tuned per domain. No new physical or ontological entities are introduced: one-step FB is a new algorithm but not a new postulated thing. The main extra assumptions are the full-support rho condition and the unproven continuous extension of the rank argument.

free parameters (5)
  • lambda_BC (behavioral-cloning regularization coefficient) = 0 to 0.3 per domain (Table 4)
    Tuned per domain for one-step FB and baselines; zero-shot performance is sensitive to it (Fig. 10a).
  • lambda_ortho (orthonormalization coefficient) = 0 to 1 per domain (Table 4)
    Tuned per domain for one-step FB and baselines; ablations show strong sensitivity (Fig. 10b).
  • tau_reward (reward softmax temperature) = 3 to 300 per domain (Table 4)
    Tuned per domain; used in Eq. 34 to construct the reward-specific latent variable.
  • d (representation dimension) = 50 for ExORL; 128 or 512 for OGBench
    Chosen per benchmark; ablations show performance depends on d (Fig. 10d).
  • tau_policy (didactic softmax temperature) = 5e-3 during training, 1 during evaluation
    Fixed in didactic FB experiments; temperature ablations are reported in Appendix E.2.
axioms (4)
  • standard math The successor measure matrix M^pi is full rank with rank |S x A| (Lemma 1 of Agarwal et al. 2019).
    Used to set the rank of the aggregated successor measure and to derive all rank constraints in Proposition 1; proven for finite CMPs in Appendix B.1.
  • domain assumption The marginal measure rho has full support on S x A.
    Needed to define diag(rho)^{-1} and the ratio M^pi/rho; offline datasets are finite, so an empirical rho has zero support on most state-action pairs.
  • ad hoc to paper Finite-CMP rank results extend to continuous CMPs by letting |S x A| -> infinity.
    Corollary 1 depends on this limit; no measure-theoretic operator rank theorem is provided for the infinite-dimensional case.
  • ad hoc to paper In didactic experiments the Monte-Carlo FB loss (Eq. 7) is a faithful proxy for the TD FB objective (Eq. 9).
    The failure-mode evidence for FB is generated with the MC loss, described as an analytical analogy to the TD loss (Appendix D.1), but the TD loss is not run in the controlled setting.

pith-pipeline@v1.3.0-alltime-deepseek · 44904 in / 17595 out tokens · 188330 ms · 2026-08-03T00:09:30.799167+00:00 · methodology

0 comments
read the original abstract

As unsupervised pretraining becomes increasingly ubiquitous in reinforcement learning, a more thorough theoretical understanding of these methods becomes of equal importance to their empirical success. We focus on the setting of unsupervised learning via interaction, where the forward-backward (FB) representation learning serves as a prototypical and popular example. In this paper, we shed light on FB by formally contextualizing the method within a broader class of recent methods that use regression to obtain a low-rank approximation of a successor measure ratio. Our analysis clarifies when FB representations can exist and how the low-rank approximation converges in practice. Building upon the theory, we propose a variant of FB that is both more amenable to theoretical understanding and simpler to optimize in practice. Experiments in didactic settings, as well as in $10$ state-based and image-based continuous control domains, demonstrate that our method converges to desired representations with $10^5 \times$ smaller errors than FB, achieving $+24\%$ improved zero-shot performance on average. We also demonstrate that zero-shot policies inferred by our algorithm provide an efficient initialization if the user prefers further fine-tuning on downstream tasks. Our project website is available at https://chongyi-zheng.github.io/onestep-fb.

Figures

Figures reproduced from arXiv: 2602.11399 by Benjamin Eysenbach, Chongyi Zheng, Royina Karegoudra Jayanth.

Figure 1
Figure 1. Figure 1: How can we learn a library of policies to quickly maxi￾mize new rewards? (Left) Forward-backward representation learn￾ing (FB) (Touati & Ollivier, 2021) factorizes their successor mea￾sures into bilinear representations, and uses those representations to acquire new policies. (Right) Our theoretical analysis of this method reveals some optimization challenges, which are alleviated through a simplified meth… view at source ↗
Figure 2
Figure 2. Figure 2: The three-state CMP. Agents start from state s0 and take action ai (i = 0, 1, 2) to determinstically transit into state si. States s1 and s2 are both absorbing states. Sections 5.1 and 5.2 will use this simple MDP to study the convergence of the FB and the one-step FB algorithms. Does the practical FB algorithm converge to the fixed point characterized in Sec. 3.1? We test the convergence of the FB algorit… view at source ↗
Figure 3
Figure 3. Figure 3: Learning FB representations in the three-state CMP ( [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Domains for evaluation. (Top) ExORL domains (16 state-based tasks). (Bottom) OGBench domains (20 state-based tasks and 10 image-based tasks). • cheetah: run, run backward, walk, and walk backward. • quadruped: jump, run, stand, and walk. • jaco: reach bottom left, reach bottom right, reach top left, reach top right. For domains walker, cheetah, and quadruped, both the episode length and the maximum return … view at source ↗
Figure 5
Figure 5. Figure 5: Fine-tuning pre-trained agents on downstream tasks. After offline pre-training, we conduct online fine-tuning on various methods using the same off-the-shelf RL algorithm (TD3). One-step FB continues to provide higher sample efficiency (+40% on average) during fine-tuning, as compared with the original FB method. 0.0 0.2 0.4 0.6 0.8 1.0 gradient steps 1e5 10 4 10 3 10 2 10 1 10 0 10 1 error M / prediction … view at source ↗
Figure 6
Figure 6. Figure 6: Learning rate ablations for FB on the three-state CMP. We conduct ablations to study the effect of learning rate on FB. Using learning rates other than 10−4 results in higher equivariance error of Q-value prediction (ϵequiv > 10−4 ). Thus, the failure mode of FB is not explained by the choice of learning rate. FB against 3 baselines in our offline experiments: BYOL-γ, HILP, and FB, and also include the per… view at source ↗
Figure 7
Figure 7. Figure 7: Policy temperature ablations on the three-state CMP. We study the effect of policy temperature τpolicy on the convergence of FB: a decreasing τpolicy results in an increasing Q prediction equivariance error ϵequiv, suggeting that FB still fails to converge to the ground-truth fixed point. The policy temperature. Does the policy temperature τpolicy affect the convergence of FB? As mentioned in Appendix D.1,… view at source ↗
Figure 8
Figure 8. Figure 8: The five-state circular CMP. Agents start from state s0 and take action ai (i = 0, 1). At every state si, choosing a0 convergence transits to the next state s(i+1) mod 5, forming circular transitions. At every state si, choosing action a1 transits to state s(i−1) mod 5 with a proprobability 0.7 and stays in the same state with a probability of 0.3, forming the stochastic transitions. Appendix E.2 uses this… view at source ↗
Figure 9
Figure 9. Figure 9: Learning FB representations in the five-state circular CMP. (Left) After training for 105 gradient steps, FB fails to converge to a pair of ground-truth FB representations. (Right) Given a fixed policy, one-step FB exactly fits the ground-truth one-step FB representations within 4 × 104 gradient steps. These observations are consistent with our analysis on the three-state CMP (Sec. 5.1 and Sec. 5.2). withi… view at source ↗
Figure 10
Figure 10. Figure 10: Hyperparameter ablations. We conduct ablation experiments to study the effect of key components of one-step FB on walker, cheetah, antmaze large navigate, and scene play. 38 [PITH_FULL_IMAGE:figures/full_fig_p038_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Learning Object Manipulation from Scratch via Contrastive Interaction

    cs.RO 2026-06 unverdicted novelty 7.0

    IWR improves CRL sample efficiency and performance in interaction-rich manipulation by interaction-aware resampling that preserves mode boundaries, yielding 19.8% average gains and a real-world air-hockey agent.

  2. Switching Successor Measures for Hierarchical Zero-shot Reinforcement Learning

    cs.LG 2026-05 unverdicted novelty 7.0

    Switching successor measures extend classical successor measures to enable hierarchical zero-shot RL via the FB π-Switch algorithm that extracts subgoal-selection and control policies from forward-backward representations.

Reference graph

Works this paper leans on

16 extracted references · 5 linked inside Pith · cited by 2 Pith papers

  1. [1]

    The representation dimensiondis at least|S × A|, i.e.,d≥ |S × A|

  2. [2]

    The rank of the forward representation matrixF ⋆ Z is at least|S × A|and at mostd, i.e.,|S × A| ≤rank(F⋆ Z )≤d

  3. [3]

    Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., and Mordatch, I

    URL https://openreview.net/forum? id=H1lJJnR5Ym. Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., and Mordatch, I. Decision transformer: Reinforcement learning via se- quence modeling. In Beygelzimer, A., Dauphin, Y ., Liang, P., and Vaughan, J. W. (eds.),Advances in Neural Information Processing Systems, 2021. ...

    2021

  4. [4]

    cc/paper_files/paper/2019/file/ c8d3a760ebab631565f8509d84b3b3f1-Paper

    URL https://proceedings.neurips. cc/paper_files/paper/2019/file/ c8d3a760ebab631565f8509d84b3b3f1-Paper. pdf. Ernst, D., Geurts, P., and Wehenkel, L. Tree-based batch mode reinforcement learning.Journal of Machine Learn- ing Research, 6, 2005. Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V ., Ward, T., Doron, Y ., Firoiu, V ., Harley, T., Dunni...

    Pith/arXiv arXiv 2019

  5. [11]

    The rank of the backward representation matrixB ⋆ is equivalent to|S × A|, i.e., rank(B⋆) =|S × A|

  6. [12]

    18), and the successor measure matrix for each latentM π i (Eq

    For different latentszi(i= 1,· · ·,|Z|), the backward representation matrix B⋆, the forward representation matrix for each latentF ⋆ i (Eq. 18), and the successor measure matrix for each latentM π i (Eq. 16) must satisfy: B⋆ =F ⋆+ 1 M π 1 diag(ρ)−1 =F ⋆+ 2 M π 2 diag(ρ)−1 =· · ·=F⋆+ |Z| M π |Z|diag(ρ)−1, where X + denotes the pseudoinverse (Moore–Penrose ...

    1920

  7. [13]

    We define the successor measure ratio prediction error as the mean squared error (MSE) between the ratio predicted by the FB representations and the ground-truth ratio

    Successor measure ratio prediction error.This metric measures the fidelity of the learned FB representations in approximating the ground-truth successor measure ratio. We define the successor measure ratio prediction error as the mean squared error (MSE) between the ratio predicted by the FB representations and the ground-truth ratio. The ground-truth rat...

  8. [14]

    For each latent variable z with the corresponding reward vector r, the learned forward representation matrix FZ predicts the optimal Q-value as ˆQ⋆ r(z) =F zz∈R |S×A|

    Optimal Q-value prediction error.This metric measures the accuracy of the optimal Q-value predicted by the learned representation. For each latent variable z with the corresponding reward vector r, the learned forward representation matrix FZ predicts the optimal Q-value as ˆQ⋆ r(z) =F zz∈R |S×A|. On the other hand, we can compute the ground- truth optima...

  9. [15]

    Forward KL divergence (optimal policy).To evaluate the decision-making quality of the induced policy, we measure the forward KL divergence between the policy derived from ˆQr(z), ˆπ(a|s, z), and the optimal policy derived from Q⋆ r(z),π ⋆(a|s, z). We report the forward KL divergence averaged over all evaluation latents and all possible states: KLπ⋆ = 1 |S...

  10. [16]

    Q prediction equivariance error.This metric assesses whether the learned Q-values respect the affine equivariance property as discussed in Lemma 3 and Proposition 2. Specifically, given a latent variable z with the corresponding reward vector r, for a positive scalar, ν >0, and an offset, ξ∈R , the predicted Q-value should satisfy the equivariance ˆQr(zνr...

    2021

  11. [328]

    Russell, S., Norvig, P., and Intelligence, A

    Springer, 2005. Russell, S., Norvig, P., and Intelligence, A. A modern ap- proach.Artificial Intelligence. Prentice-Hall, Egnlewood Cliffs, 25(27):79–80, 1995. Savinov, N., Dosovitskiy, A., and Koltun, V . Semi- parametric topological memory for navigation.arXiv preprint arXiv:1803.00653, 2018. Schaul, T., Horgan, D., Gregor, K., and Silver, D. Universal ...

    Pith/arXiv arXiv 2005

  12. [2018]

    Version 0.3.13

    URL http://github.com/jax-ml/jax. Version 0.3.13. 9 Can We Really Learn One Representation to Optimize All Rewards? Brandfonbrener, D., Whitney, W., Ranganath, R., and Bruna, J. Offline rl without off-policy evaluation.Advances in neural information processing systems, 34:4933–4946, 2021. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwa...

    2021

  13. [2019]

    Bradbury, J., Frostig, R., Hawkins, P., Johnson, M

    URL https://openreview.net/forum? id=S1VWjiRcKX. Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S. et al. JAX: composable transfor- mations of Python+NumPy programs. GitHub repository,

  14. [2021]

    Zhang, J., Springenberg, J

    URL https://openreview.net/forum? id=-2FCwDKRREu. Zhang, J., Springenberg, J. T., Boedecker, J., and Bur- gard, W. Deep reinforcement learning with successor features for navigation across similar environments. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2371–2378. IEEE, 2017. Zheng, C., Eysenbach, B., Walke, H....

    Pith/arXiv arXiv 2017

  15. [2023]

    Mnih, V ., Kavukcuoglu, K., Silver, D., Rusu, A

    URL https://openreview.net/forum? id=oqOfLP6bJy. Mnih, V ., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fid- jeland, A. K., Ostrovski, G. et al. Human-level control through deep reinforcement learning.nature, 518(7540): 529–533, 2015. Mohamed, S. and Jimenez Rezende, D. Variational in- formation maxi...

    Pith/arXiv arXiv 2015

  16. [2025]

    Sikchi, H., Tirinzoni, A., Touati, A., Xu, Y ., Kanervisto, A., Niekum, S., Zhang, A., Lazaric, A., and Pirotta, M

    URL https://openreview.net/forum? id=xL7Bt4jS2U. Sikchi, H., Tirinzoni, A., Touati, A., Xu, Y ., Kanervisto, A., Niekum, S., Zhang, A., Lazaric, A., and Pirotta, M. Fast adaptation with behavioral foundation models. In Reinforcement Learning Conference, 2025. URLhttps: //openreview.net/forum?id=soeW8RGo1N. Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, ...

    Pith/arXiv arXiv 2025