Pith. sign in

REVIEW 5 major objections 5 minor 13 references

Fisher-Guided Selective Forgetting: Mitigating The Primacy Bias in Deep Reinforcement Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Periodically adding Fisher-information-scaled noise to network weights counteracts the primacy bias in deep reinforcement learning, improving final returns by up to 50% on complex control tasks.

desk verdict Interesting nuggets, but the paper's own Table 1 contradicts its headline claim; worth a serious review, but it needs major surgery on the empirical front. read the letter →

arxiv 2502.00802 v1 pith:BXIJM5NR submitted 2025-02-02 cs.LG cs.AI

Massimiliano Falzari , Matthia Sabatelli This is my paper · ORCID
classification cs.LGcs.AI
keywords primacybiasFisherinformationmatrixselectiveforgettingdeepreinforcementlearningplasticitymachineunlearningcontinuouscontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the primacy bias in deep reinforcement learning—the tendency of agents to overfit to early, suboptimal experiences—has a measurable geometric signature in the evolution of the trace of the Fisher Information Matrix (FIM): a sharp 'memorization' peak followed by a 'reorganization' decline. On that basis it proposes Fisher-Guided Selective Forgetting (FGSF), which periodically adds noise to network weights with magnitude scaled by $F^{-1/4}$, so that perturbations are small in parameter directions the current data marks as highly sensitive. Across DeepMind Control Suite tasks, FGSF is reported to beat both plain SAC and periodic network resets, with up to a 50% higher mean return in Humanoid and more stable learning in Quadruped. The paper also reports that scrubbing only the critic reproduces most of the benefit, that FGSF stays robust when replay ratios are raised, and that Fisher-scaled noise outperforms an isotropic Gaussian counterpart in stability.

What carries the argument

The central object is the empirical Fisher Information Matrix, $F = \frac{1}{N}\sum_i \nabla_\theta \log p(x_i|\theta)\nabla_\theta \log p(x_i|\theta)^T$, whose trace $\operatorname{Tr}(F)$ summarizes the network's local output sensitivity to parameter changes. The mechanism is the periodic scrubbing update $w \leftarrow w + (\lambda\sigma^2)^{1/4} F^{-1/4}\epsilon$, which injects noise preferentially into low-sensitivity directions (since $F^{-1/4}$ de-emphasizes directions the current batch marks sensitive), thereby imitating a selective-forgetting step from machine unlearning while skipping the gradient term. A Savitzky–Golay-filtered derivative of the trace identifies the transition between memorization and reorganization phases. The method is applied on top of any replay-based DRL algorithm, with SAC used throughout, and the FIM approximated by EKFAC for tractability.

What would settle it

Record the gradient of the loss on the earliest replay-buffer trajectories immediately after one FGSF scrubbing step, and compare it with the drop produced by an isotropic Gaussian perturbation of equal magnitude; if the drops are statistically indistinguishable, the Fisher structure plays no selective-forgetting role and FGSF reduces to noise injection.

Watch

Extended reading notes

Core claim

The central claim is that the primacy bias is not an inevitable form of overfitting but a phase in learning that can be identified and counteracted geometrically. Learning dynamics pass through a memorization phase, where the FIM trace explodes, and a reorganization phase, where it decays; FGSF intervenes with the update $S(w) = w + (\lambda\sigma^2)^{1/4} F^{-1/4}\epsilon$, taken from a selective-forgetting scrubbing rule with the gradient term removed. By applying this update every fixed number of steps to the actor and/or critic, the method keeps weight sensitivity low and prevents early experiences from locking in representations. The paper's evidence for effectiveness is the consistent performance advantage over baselines in complex environments, reduced FIM trace peaks, smoother update magnitudes, and maintained performance under higher replay ratios.

Load-bearing premise

The load-bearing premise is that perturbing weights with $F^{-1/4}$-scaled noise computed on the current replay batch actually removes the influence of early experiences, rather than merely adding unstructured regularization—something the paper does not directly measure.

Editorial extensions

If this is right

  • FIM trace evolution can be used as a diagnostic: an early sharp peak followed by decline flags primacy bias before final performance suffers.
  • Critic-only scrubbing gives most of the benefit of full scrubbing in complex tasks, suggesting cheaper mitigation for actor–critic methods.
  • FGSF retains performance at replay ratios of 2 and 4 where plain SAC degrades, opening the door to higher replay ratios for sample efficiency.
  • Because FGSF never resets parameters, it avoids the abrupt performance drops of periodic resetting while achieving similar or better final returns.
  • The FIM-scaled noise update is algorithm-agnostic and could be attached to any off-policy method, not just SAC.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the selective-forgetting interpretation is correct, FGSF should also reduce forward transfer interference in continual or transfer RL settings; this is a natural test the paper does not run.
  • The FIM trace's two-phase pattern suggests an adaptive schedule: apply scrubbing strongest during the memorization phase and weaken it as the trace decays, which could remove the $\lambda$ tuning burden.
  • The 15–20% computational overhead of EKFAC might be reduced by a diagonal FIM approximation; if performance holds, the method becomes nearly free to add to existing SAC pipelines.
  • The comparison to isotropic Gaussian noise suggests that part of the gain is pure regularization; isolating the $F^{-1/4}$ contribution would clarify whether the geometric structure matters beyond stability, which the paper's modest margin leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces Fisher-Guided Selective Forgetting (FGSF), a method that periodically adds FIM-scaled Gaussian noise to the weights of actor and critic networks during SAC training, adapted from the scrubbing rule of Golatkar et al. The authors propose a characterization of primacy bias through memorization and reorganization phases in the FIM trace, evaluate FGSF against baseline SAC and periodic network reset on ten DMC environments, and report ablations on scrubbed components, replay ratios, and Gaussian noise baselines.

Significance. If the headline claims were supported, FGSF would be a simple, algorithm-agnostic intervention for preserving plasticity in off-policy RL, with the FIM trace serving as a monitoring signal. The paper is also transparent about computational overhead and includes a limitations discussion. However, the evidence is not sufficient: the paper's own Table 1 contradicts the 'consistently outperforms' claim, the reported Humanoid numbers are internally inconsistent, the selective-forgetting mechanism is never directly measured, and the key hyperparameter appears to be tuned per environment. As it stands, the contribution reduces to an empirical study of a FIM-scaled noise regularizer whose advantage over simpler noise injection is not established.

major comments (5)
  1. [Abstract; §5.1; Table 1] The abstract and conclusion state that FGSF 'consistently outperforms baselines', but Table 1 reports that baseline SAC has higher final mean returns in five of ten environments: Acrobot (145.3 versus 6.5), Hopper (266.9 versus 148.0), Walker (758.4 versus 746.7), Finger (855.3 versus 824.2), and Pendulum (834.7 versus 770.5), with Swimmer essentially tied (324.7 versus 326.5). The empirical claim therefore is not supported by the paper's own data and must be revised to a per-environment claim or supported by a corrected protocol.
  2. [§5.1; Table 1] The Humanoid numbers are internally inconsistent. Section 5.1 reports FGSF mean return 150 ± 15 versus baseline SAC 95 ± 10 and calls this a 50% improvement, whereas Table 1 reports 136.6 ± 14.4 versus 68.5 ± 21.9, which is roughly a 99% improvement. The paper does not explain which numbers are the final results or how the 50% figure was derived, which undermines confidence in the reported performance gains.
  3. [§3.4; Algorithm 1] The selective-forgetting mechanism is not verified. The update S(w) = w + (λσ²)^{1/4} F^{-1/4} ε is a structured weight perturbation, and no experiment measures whether information from early trajectories is actually removed. The FIM in Algorithm 1 is computed on the current replay batch, which is not the set of early experiences to be forgotten and may itself contain early data. Because Section 5.3 shows that simple Gaussian noise injection achieves similar performance in several environments, the paper needs a direct test of forgetting (for example, measuring the influence of early transitions on outputs or gradients before and after scrubbing) to support the claim that FGSF performs selective forgetting rather than anisotropic regularization.
  4. [§3.3; §3.4] The derivation of the scrubbing update from Golatkar et al. is not justified. The removed term B^{-1}∇L_Dr(w) is not redundant with ordinary gradient descent: it is the term in the unlearning solution that preserves performance on the retained set, and ordinary gradient steps are not Hessian-scaled. Moreover, B is approximated by the empirical FIM of the current batch, whereas the unlearning framework requires the Hessian of the retained-data loss; no argument is given that these coincide. This leaves the method's theoretical grounding as an ad-hoc noise-scheduling rule rather than a selective-forgetting procedure.
  5. [§5.2; Table 3] The comparison is compromised by per-environment hyperparameter selection. Table 3 shows that the best λ varies across environments (Humanoid is best at 5e-7, Pendulum at 5e-8, Cheetah at 5e-8, Reacher at 5e-8, with Pendulum collapsing to 55.0 at 5e-6), and the text's claim that environment complexity has minimal influence on the optimal λ is difficult to reconcile with these large differences. The paper does not state whether the FGSF results in Table 1 use a single global λ or a per-environment selected value; if per-environment selection was used, the final-return comparison is partially fitted to the test environments.
minor comments (5)
  1. [Throughout] The affiliation is misspelled as 'Bernulli Institute' and should be 'Bernoulli Institute'; Section 2 also contains the typo 'in te form'.
  2. [§3.3] The phrases 'Achille et al..' and 'Golatkar et al..' appear with double periods and without proper inline citation pointers; please fix these to standard citations.
  3. [Algorithm 1] The symbol F is used both for the scrubbing frequency and for the Fisher Information Matrix, and line 8 writes the matrix name as 'F IM' with a space; rename the frequency variable (for example, f_scrub) to avoid confusion.
  4. [§5.3] The Gaussian noise baseline uses N(0, 0.001µ), but no attempt is made to match the perturbation magnitude to FGSF's update size, so the structured-versus-unstructured comparison is not controlled for noise scale.
  5. [§5.2] The phrase 'less sensible to the early, biased, experiences' should read 'less sensitive to the early, biased experiences'.

Circularity Check

3 steps flagged · score 6.0 of 10

PB mitigation is evidenced by the same FIM trace FGSF is built to reduce; lambda is tuned on the reported environments; and the 'selective forgetting' update reduces, by the paper's own equation, to current-batch FIM-scaled noise.

  1. self definitional [Section 3.2 (FIM-trace characterization), Section 5.1 (FIM-trace evidence), Algorithm 1]
    "'This pattern provides a metric to characterize and understand how early experiences disproportionately influence learning' and 'Based on our characterization, these FIM patterns provide evidence of FGSF's ability to mitigate the Primacy Bias.'"

    The paper operationalizes PB as the FIM-trace memorization/reorganization pattern, while FGSF is a procedure that periodically perturbs weights and is tuned by monitoring actor and critic FIM traces. The confirmation of PB mitigation is therefore the same quantity the method is designed to control. No independent measurement of what information was actually forgotten (e.g., influence of the early trajectories) is provided, so the FIM-trace evidence cannot confirm the forgetting mechanism; it only shows that the method changes its own target variable.

  2. fitted input called prediction [Section 5.2 (Hyperparameter Sensitivity), Table 3, Section 6 (Discussion)]
    "'For practical implementation, we recommend an initial λ value of 5 × 10−7, monitoring both actor and critic FIM traces, and adjusting λ based on observed learning stability.' See also 'we have investigated the impact of the hyperparameter λ and found optimal values around 5×10−7.'"

    lambda is selected per environment by inspecting the same learning curves and FIM traces used to report final performance. The headline 'up to a 50% increase in mean return' (Section 6, Table 1) is then a post-selection number on the environments used for tuning, not a prediction on held-out tasks. Table 3 confirms that lambda = 5e-7 is not universally best (e.g., Cheetah, Finger, Reacher, and Pendulum do better with other values), so the reported gains depend on per-environment tuning rather than a dataset-independent result.

1 more flagged steps
  1. renaming known result [Section 3.4 (FGSF update), Algorithm 1, Section 5.4 (Fisher vs Gaussian)]
    "'Note that, compared to the original formulation of Golatkar et al., we removed the term B−1∇LDr (w).' Algorithm 1 computes 'F IM = 1/N Σ ∇w log p(si|w)∇w log p(si|w)T' on the current batch; Section 5.4 states that 'even simple noise injection methods' achieve significant improvements."

    After dropping the gradient term from the scrubbing rule, S(w) = w + (lambda sigma^2)^(1/4) F^(-1/4) epsilon is a FIM-scaled random perturbation. The FIM is computed on the current replay batch, i.e., the data to be retained, not on the early trajectories that should be forgotten, so no early-experience information enters the update. The label 'selective forgetting' is thus a renaming of structured noise injection; the paper's own ablation showing that Gaussian noise nearly matches FGSF corroborates that perturbation, not selective erasure, carries the effect.

full rationale

The paper is not circular in the sense of deriving final performance numbers from a fitted target by closed-form construction: the SAC-FGSF results are genuine simulations and the FIM trace is measured rather than defined to equal the update. The circularity that is present is partial and located in the evaluation logic. First, PB is operationalized by the FIM-trace pattern, so the claim that FGSF 'mitigates' PB is confirmed by showing lower/reorganized FIM traces, the same variable the weight-perturbation step is designed to control; the forgetting mechanism itself is never tested against the early data. Second, the central quantitative claim is fitted: lambda is chosen by monitoring FIM traces and learning curves on each of the ten DMC environments, and the reported 'up to 50%' improvement uses those same environments. Third, the unlearning derivation has its selective content removed: the gradient term is dropped and the remaining update is pure FIM-scaled noise computed on the current batch, so the method is, by the paper's own equation, a regularizer/perturbation rather than a selective forgetting operator; the Gaussian-noise ablation corroborates this. There is no load-bearing self-citation chain (the cited FIM and unlearning works are external), so this is not a score-8 or 10 case. Independently of circularity, the abstract's 'consistently outperforms' is not supported by Table 1 (baseline SAC is better in Acrobot, Hopper, Walker, Finger, and Pendulum) and Section 5.1's Humanoid numbers disagree with Table 1, but those are correctness risks rather than circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central derivation is imported from Golatkar et al. and modified by dropping the gradient term; the main fitted quantity is lambda, chosen per environment. No new physical or conceptual entities are postulated.

free parameters (4)
  • forgetting coefficient lambda = 5e-7 recommended; per-environment optima range from 5e-8 to 5e-6 (Table 3)
    Tuned using final returns on the evaluation environments; the paper recommends 5e-7 as a default but reports different optimal values per task.
  • uncertainty scale sigma^2 = undefined
    Appears in the scrubbing update in Algorithm 1 but is never specified; in practice it is absorbed into the product (lambda sigma^2)^(1/4).
  • scrubbing frequency F = 10
    Set to 10 for the sake of simplicity in Section 3.4; not tuned.
  • Gaussian noise baseline scale = 0.001 times the mean of weights
    Heuristic choice for the ablation in Section 5.4; no justification is given for this specific scale.
assumptions (4)
  • domain assumption Golatkar et al.'s optimal scrubbing derivation is valid in the DRL setting (quadratic loss approximation, gradient flow, Gaussian noise bound).
    Section 3.3 adopts the forgetting Lagrangian and the resulting S(w) from Golatkar et al. without re-deriving or validating the approximations in the DRL setting.
  • domain assumption EKFAC gives an accurate enough approximation of the empirical FIM and its inverse fourth root.
    Section 4 states the empirical FIM is computed with NNGeometry's EKFAC; the accuracy of the approximation is not quantified.
  • ad hoc to paper The current batch's FIM can play the role of the retained-data Hessian B in the unlearning framework.
    Section 3.4 treats the current replay-buffer batch as Dr but never justifies why its FIM substitutes for the Hessian of the retained set in Golatkar's scrubbing rule.
  • ad hoc to paper Removing the B^(-1) grad L_Dr(w) term from the scrubbing rule is harmless for periodic in-training scrubbing.
    Section 3.4 argues the gradient step is redundant with SGD and a full gradient update is unrealistic; no experiment verifies that the omitted term is immaterial, and the resulting algorithm no longer targets a specific forget set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fisher-Guided Selective Forgetting: Mitigating The Primacy Bias in Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/BXIJM5NR

@misc{pith2026250200802,
  author       = {Pith},
  title        = {Pith review of: Fisher-Guided Selective Forgetting: Mitigating The Primacy Bias in Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BXIJM5NR}},
  note         = {Machine review of arXiv:2502.00802}
}
read the original abstract

Deep Reinforcement Learning (DRL) systems often tend to overfit to early experiences, a phenomenon known as the primacy bias (PB). This bias can severely hinder learning efficiency and final performance, particularly in complex environments. This paper presents a comprehensive investigation of PB through the lens of the Fisher Information Matrix (FIM). We develop a framework characterizing PB through distinct patterns in the FIM trace, identifying critical memorization and reorganization phases during learning. Building on this understanding, we propose Fisher-Guided Selective Forgetting (FGSF), a novel method that leverages the geometric structure of the parameter space to selectively modify network weights, preventing early experiences from dominating the learning process. Empirical results across DeepMind Control Suite (DMC) environments show that FGSF consistently outperforms baselines, particularly in complex tasks. We analyze the different impacts of PB on actor and critic networks, the role of replay ratios in exacerbating the effect, and the effectiveness of even simple noise injection methods. Our findings provide a deeper understanding of PB and practical mitigation strategies, offering a FIM-based geometric perspective for advancing DRL.

Figures

Figures reproduced from arXiv: 2502.00802 by the authors.

Figure 1
Figure 1. Example of the Primacy Bias characterization using the Tr(F) (blue) and ∆Tr(F) (black).The graphs represent the learn￾ing dynamics on the Quadruped and Swimmer environment respectively. From our characterization, the PB is present in the Quadruped while it is not present in the Swimmer. Best viewed in colors. These two phases indicate that initial experiences have a disproportionate impact on the model, while the fo… view at source ↗
Figure 5
Figure 5. Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 5.2
Figure 5.2. Evolution of FIM trace (Tr(F)) for actor networks across environ￾ments. Results compare baseline SAC (gold), reset method (teal), and FGSF (red). The plots demonstrate the differential impact of the PB mitigation techniques on actor network [PITH_FULL_IMAGE:figures/full_fig_p006_5_2.png] view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Performance comparison between FGSF and Gaussian noise injection across different environments. Shaded regions represent the minimum and maximum over 5 random seeds. Best viewed in colors. Despite these encouraging results, it is essential to acknowl￾edge the limitatio…
Figure 8
Figure 8. Figure 8: Evolution of FIM trace (Tr(F)) during training for critic networks across different environments. Results compare baseline SAC (gold), reset method (teal), and FGSF (red). Shaded regions represent the minimum and maximum over 5 random seeds. Best viewed in colors. 12 …
Figure 9
Figure 9. Figure 9: Local parameter update magnitudes measured by KL divergence across different environments. Lower values indicate a smaller parameter update. Spikes in the baseline (gold) and reset methods (teal) contrast with FGSF’s (red) more consistent update pattern. Best viewed in…
Figure 10
Figure 10. Figure 10: Fraction of dormant neurons during training across different environments. Plots compare baseline SAC (gold), reset method (teal), and FGSF (red). Best viewed in colors. In baseline SAC, critic networks exhibit a consistent increase in dormant neuron fraction, particu…
Figure 11
Figure 11. Figure 11 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Critic network FIM trace evolution under critic-only scrubbing (red) versus full network scrubbing (teal) for different environments. The traces show stronger regularization effects in critic-only scrubbing. Best viewed in colors. 17 [PITH_FULL_IMAGE:figures/full_fig…
Figure 13
Figure 13. Figure 13: FIM trace of the critic network under different scrubbing coefficients (λ ∈ [5 × 10−6 , 5 × 10−8 ]), illustrating the relationship between λ values and the FIM trace. The lighter the color the higher the coefficient. Best viewed in colors. 19 [PITH_FULL_IMAGE:figures…
Figure 14
Figure 14. Figure 14: Comparative analysis of cumulative training time across environments. The y-axis shows total computation time in seconds, demonstrating the computational overhead of different methods. Baseline SAC (gold), reset (teal) and FGSF (red). Best viewed in colors. FGSF shows…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 4 canonical work pages

  1. [6]

    Structural Neural Additive Models: Enhanced Interpretable Machine Learning

    Luber, M., Thielmann, A., and S¨afken, B. Structural neural additive models: Enhanced interpretable machine learn- ing. arXiv preprint arXiv:2302.09275,

    work page Pith review arXiv

  2. [7]

    Understanding and preventing capacity loss in reinforcement learning

    Lyle, C., Rowland, M., and Dabney, W. Understanding and preventing capacity loss in reinforcement learning. arXiv preprint arXiv:2204.09560, 2022a. Lyle, C., Rowland, M., Dabney, W., Kwiatkowska, M., and Gal, Y . Learning dynamics and generalization in deep reinforcement learning. In International Conference on Machine Learning, pp. 14560–14581. PMLR, 202...

    arXiv

  3. [8]

    Revisiting natural gradient for deep networks

    Pascanu, R. Revisiting natural gradient for deep networks. arXiv preprint arXiv:1301.3584,

    arXiv

  4. [10]

    Ramkumar, V . R. T., Zonooz, B., and Arani, E. The effec- tiveness of random forgetting for robust generalization. arXiv preprint arXiv:2402.11733,

    work page Pith review arXiv

  5. [11]

    On The Transferability of Deep-Q Networks

    Sabatelli, M. and Geurts, P. On the transferability of deep-q networks. arXiv preprint arXiv:2110.02639,

    work page Pith review arXiv

  6. [12]

    Tassa, Y ., Doron, Y ., Muldal, A., Erez, T., Li, Y ., Casas, D. d. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. Deepmind control suite. arXiv preprint arXiv:1801.00690,

    arXiv

  7. [13]

    Deep reinforcement learning and the deadly triad

    Van Hasselt, H., Doron, Y ., Strub, F., Hessel, M., Sonnerat, N., and Modayil, J. Deep reinforcement learning and the deadly triad. arXiv preprint arXiv:1812.02648,

    arXiv

  8. [2013]

    The primacy bias in model- based rl

    Qiao, Z., Lyu, J., and Li, X. The primacy bias in model- based rl. arXiv preprint arXiv:2310.15017,

    arXiv

Show all 13 references
  1. [2014]

    Hard tasks first: Multi-task reinforcement learning through task schedul- ing

    Cho, M., Park, J., Lee, S., and Sung, Y . Hard tasks first: Multi-task reinforcement learning through task schedul- ing. In Forty-first International Conference on Machine Learning. Dohare, S., Hernandez-Garcia, J. F., Rahman, P., Mahmood, A. R., and Sutton, R. S. Maintaining ...

    arXiv

  2. [2018]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    Golatkar, A., Achille, A., and Soatto, S. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9304–9312, 2020a. Golatkar, A., Achille, A., and Soatto, S. Forgetti...

    2020 arXiv

  3. [2022]

    C., and Bowling, M

    Farebrother, J., Machado, M. C., and Bowling, M. Gen- eralization and regularization in dqn. arXiv preprint arXiv:1810.00123,

    arXiv

  4. [2023]

    G., and Courville, A

    D’Oro, P., Schwarzer, M., Nikishin, E., Bacon, P.-L., Belle- mare, M. G., and Courville, A. Sample-efficient rein- forcement learning by breaking the replay ratio barrier. In Deep Reinforcement Learning Workshop NeurIPS 2022,

    2022

  5. [2024]

    Efficient deep reinforcement learning requires regulating overfit- ting

    Li, Q., Kumar, A., Kostrikov, I., and Levine, S. Efficient deep reinforcement learning requires regulating overfit- ting. arXiv preprint arXiv:2304.10466,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.