Pith. sign in

REVIEW 4 major objections 6 minor 37 references

Decorrelated Soft Actor-Critic for Efficient Deep Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that adding network-wide input decorrelation to the soft actor-critic algorithm speeds up deep reinforcement learning training in five of seven Atari games, improves final reward in two, and keeps performance level in the…

desk verdict A clean idea and honest reporting, but the speedup claim is confounded by batch size; the matched-batch-size control in Appendix G shows the effect mostly disappears. read the letter →

arxiv 2501.19133 v1 pith:R2U5G7VT submitted 2025-01-31 cs.LG cs.AI

Burcu Küçükoğlu , Sander Dalm , Marcel van Gerven This is my paper · ORCID
classification cs.LGcs.AI
keywords decorrelatedbackpropagationsoftactor-criticsampleefficiencyAtari100kcreditassignmentrepresentationlearningdeepreinforcement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sample efficiency is a major bottleneck in deep reinforcement learning. This paper proposes a simple modification to the soft actor-critic algorithm: before each neural network layer, multiply the layer input by a decorrelation matrix that is updated, in parallel with normal training, to suppress correlations between features. Applied to discrete SAC on seven hard Atari games, this decorrelated SAC (DSAC) trains faster in wall-clock time in five of the seven games, reduces training time by roughly half in the games where it also improves reward (Alien and Seaquest), and keeps performance level in the rest. The authors argue that decorrelation improves representation learning and credit assignment, which is why the agent learns more efficiently.

What carries the argument

The central object is the decorrelating matrix R inserted before each layer's forward pass, transforming the raw input z into x = Rz. R is updated by R ← R − η C R, where C is the empirical off-diagonal correlation of x with itself, a rule that minimizes the sum of squared off-diagonal correlation elements (the decorrelation loss) for that layer. The paper applies this to every layer of the policy network in a discrete-action version of SAC; the decorrelation update runs in parallel with the usual SAC loss updates, and in the main results only the policy network is decorrelated.

What would settle it

Run DSAC and SAC on the seven Atari games with identical batch sizes, identical learning rates, and identical wall-clock budgets, and apply a significance test to the difference in final reward and time-to-threshold; if no significant advantage remains when batch sizes are matched, the paper's attribution of the gains to decorrelation is not supported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that network-wide decorrelation of layer inputs, implemented with decorrelated backpropagation, can be integrated into discrete soft actor-critic as a separate parallel update, and that doing so reduces wall-clock training time in five of seven Atari 100k games (up to 76%), with statistically significant reward gains in two games (Alien +86%, Seaquest +6%) and no significant loss in the others. The paper also reports that the decorrelation loss of the policy network stays near zero throughout training for DSAC while rising beyond $10^{6}$ for the SAC baseline, and that the entropy-temperature loss converges faster, suggesting decorrelation supports the exploration objective. The authors attribute the gains to more effective credit assignment through better representation learning.

Load-bearing premise

The central claim assumes that the speedups and reward gains come from the decorrelation update rule itself, but in five of the seven games DSAC used a smaller batch size than the SAC baseline, so if batch-size differences rather than decorrelation drive the gains, the causal story collapses.

Editorial extensions

If this is right

  • If the central claim holds, deep RL agents on image-based tasks can be trained to the same reward in substantially less wall-clock time simply by adding a decorrelation step, without changing the RL update rules.
  • Because the decorrelation loss stays low during training and rises when the baseline converges, it can act as a signal of how novel the input is, which the paper suggests could guide exploration in other RL algorithms.
  • The method is applicable to both convolutional and fully-connected layers, so it can be dropped into other actor-critic or value-based RL algorithms beyond SAC.
  • The observation that decorrelation works best with smaller batch sizes suggests it may reduce the amount of data needed per gradient step, further improving sample efficiency.
  • Whitening (normalizing input variance as well as correlations) could extend DSAC's gains, since whitening has been shown to improve convergence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: The cleanest test of the paper's causal story is a matched-batch-size comparison with significance testing; the paper's own Appendix G only provides a qualitative, un-tested look at equal batch sizes, so the field would benefit from a pre-registered study that holds batch size and compute fixed and varies only the decorrelation update.
  • Editorial extension: If decorrelation is indeed what enables the smaller batch sizes, then the mechanism may be that decorrelated inputs allow the optimizer to take noisier gradient steps safely; this could be tested by measuring gradient variance with and without decorrelation.
  • Editorial extension: The decorrelation loss could be used as an intrinsic reward or exploration bonus in algorithms without SAC's built-in entropy term, which would be a different application than the paper's presentation.
  • Editorial extension: The paper's results are on Atari with discrete actions; continuous control with larger action spaces might see larger benefits because credit assignment is harder, a direction the paper mentions but does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper integrates decorrelated backpropagation (DBP) into discrete Soft Actor-Critic (SAC), adding per-layer decorrelation matrices that are updated with a separate learning rule alongside the usual RL losses. DSAC is evaluated against a BP-based discrete SAC baseline on seven Atari 100k games. The abstract claims faster training in five of seven games, improved rewards in two games, and a roughly 50% wall-clock reduction, and attributes these gains to network-wide decorrelation improving sample efficiency through more effective credit assignment.

Significance. If validated, the method would be a practically useful, architecture-agnostic add-on for off-policy deep RL, and the released code is a positive contribution. However, the current experimental design does not isolate the decorrelation mechanism from per-game hyperparameter differences, especially batch size. The paper is therefore significant only as a proposal with preliminary evidence; the central causal claim needs substantially stronger support before the results can be accepted.

major comments (4)
  1. [§3.1, Fig. 1, Table 2] The primary wall-clock speed-up claim is confounded with batch size. In all five games where DSAC is reported as faster, DSAC uses batch size 64 while SAC uses 256, and in ChopperCommand, the one game where DSAC uses batch size 256 against SAC's 64, DSAC is 288% slower. The Figure 1 caption itself states this pattern. Because wall-clock time per gradient step scales with batch size, these results do not establish that decorrelation, rather than the smaller batch size, causes faster training. The paper should report matched-batch-size wall-clock comparisons for the main results, together with per-step learning curves and statistical tests, and should not claim "five of seven games faster" based on the current comparisons.
  2. [Appendix G, Fig. 10] The matched-batch-size control is qualitative and undercuts the central claim. At identical batch sizes, DSAC loses its training-time advantage in Seaquest, Alien, PrivateEye, and BankHeist, retaining it only in BattleZone, and the reward advantage also disappears for BankHeist. The appendix does not report the number of seeds, error bars, or significance tests for these runs, and the configurations are selected as "the other batch size" rather than by a full factorial design. A quantitative controlled comparison with identical hyperparameters except for the presence of the decorrelation update is needed to support the causal attribution to decorrelation.
  3. [§3.4, Fig. 2; §2.1, Eq. (3)] The decorrelation-loss evidence is partly by construction. DSAC explicitly updates R using Eq. (3) to minimize the decorrelation loss in Eq. (5), while the SAC baseline has no such update, so the lower DSAC loss in Figure 2 is expected and does not by itself demonstrate more effective credit assignment or better sample efficiency. The paper needs a mechanistic quantity that is not directly optimized, such as gradient conditioning, effective rank of representations, or an ablation with a mismatched or random decorrelation update, to support the claim that decorrelation improves learning rather than merely that DSAC minimizes its own auxiliary objective.
  4. [§3.1, §2.5, Appendix A] The reward evidence for the headline claim consists of two nominally significant gains (p<0.05, N=10) out of seven games, with no correction for multiple comparisons, and each game uses a separately selected best hyperparameter configuration from the grid search. This selection procedure risks overfitting to final reward and makes the per-game comparisons non-independent. Report corrected p-values or false-discovery-rate control, include seed-level dispersion, and provide a sensitivity analysis with a common hyperparameter setting across games for both algorithms.
minor comments (6)
  1. [§2.5, Appendix A] The paper should clarify early that the main results decorrelate only the policy network, while the hyperparameter search decorrelated the policy and both Q-networks; the text currently oscillates between "all networks" and "policy network," which is confusing for reproducibility.
  2. [§2.2, Eq. (7)] The sample-size formula uses D_r and p without precise definitions; please define the dimensionality per patch and the number of patches before the equation so readers can interpret the downsampling rule.
  3. [Fig. 1 caption, §3.2] The batch-size confounding acknowledged in the Figure 1 caption and Section 3.2 is a central limitation and should be restated explicitly in the abstract and conclusions, where the causal phrasing currently overstates the evidence.
  4. [Table 2] Wall-clock times are reported as single values without confidence intervals or seed-level spread; given that the training-time claims are central, per-seed timing distributions should be reported.
  5. [§2.3.1] The statement that Leaky ReLU performs better than ReLU for the decorrelation procedure is anecdotal; either add supporting results or present it as a design choice without a performance claim.
  6. [References] The reference "Huangi et al. (2018)" appears to be a typo for "Huang et al. (2018)" (Decorrelated Batch Normalization); please correct it and unify the rendering of author names with diacritics throughout the bibliography.

Circularity Check

2 steps flagged · score 6.0 of 10

Decorrelation-loss evidence is by construction (Eq. 3 minimizes the plotted loss); reward gains are selected per-game via grid search on final reward; batch-size confounds block the causal speedup claim.

  1. self definitional [Section 3.4, Eq. (3), Algorithm 1, Figure 2 caption]
    "R ← R − ηCR ... We are thus aiming to minimize the off-diagonal elements of xx⊤ ... The most obvious and systematic difference emerges in the minimization of the decorrelation loss for the policy network, which is consistently kept at a minimum for DSAC throughout training ... These results show successful decorrelation in the DSAC policy networks."

    The quantity plotted in Figure 2 is the total decorrelation loss D defined in Eqs. (5)-(6), and Algorithm 1's update R ← R − ηCR (Eq. 3) is precisely a gradient step that minimizes D. DSAC is therefore guaranteed, by its own update rule, to keep this loss lower than a baseline that never updates R. Observing this difference confirms only that the optimizer optimizes its objective; it does not independently confirm that decorrelation causes the reported training speedups or reward gains.

  2. fitted input called prediction [Section 2.5, Appendix A, Section 3.1, Table 2]
    "For DSAC, additionally the learning rate for the decorrelation update of the policy network was treated as a hyperparameter ... The grid search results on final reward performance can be seen in Appendix A. ... It also significantly outperforms the SAC baseline ( p <0.05, N = 10) in 2 of the games tested"

    The per-game decorrelation learning rate (and batch size) for DSAC is selected by grid search on final reward in Appendix A, and the Section 3.1 claim of significant reward gains in two games is reported from those same selected configurations. The observation that decorrelation improves reward in those games is therefore a selection artifact: the configuration was chosen because it produced that reward, so the reported gain is not an independent prediction of the decorrelation rule. No held-out validation or correction for multiple comparisons is reported.

full rationale

The only strictly circular step is Section 3.4's use of decorrelation loss as evidence: Algorithm 1's update R ← R − ηCR is exactly the minimization of the loss plotted in Figure 2, so the gap between DSAC and SAC is entailed by the definitions. The reward claims are also partly fitted: per-game decorrelation learning rates and batch sizes were chosen on final reward in Appendix A, and the two significant wins are reported from those same selected runs. The batch-size confound acknowledged in Section 3.2 and the Figure 1 caption is a validity problem rather than circularity, but Appendix G's matched-batch-size results show most wall-clock advantages disappear, which further undermines the causal attribution. Citations to the authors' DBP work are legitimate external evidence (ImageNet benchmark) and are not invoked as a uniqueness theorem, so they are not counted as circular. Because one mechanistic prediction reduces by construction and the performance claims are partly selection-based, partial circularity is present (score 6), even though the core algorithmic contribution is not itself an identity with its input.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's central claim relies on per-game fitted hyperparameters (learning rate, batch size, decorrelation learning rate, scaling coefficient) and on the DBP update rule adopted from the authors' own prior work. No new speculative entities are introduced.

free parameters (5)
  • SAC learning rate = per game, 3e-5 to 3e-4
    grid searched per game and per algorithm (Section 2.5, Appendix D), so performance of both algorithms depends on per-game tuning.
  • Batch size = 64 or 256 per game
    selected per game per algorithm; DSAC uses 64 in most games where it is faster, and 256 in ChopperCommand where it is much slower (Table 2).
  • Decorrelation learning rate (policy) = 1e-4, 1e-3, 1e-2 or 0.01 per game
    grid searched per game (Appendix A, D), so the conclusion that decorrelation helps is conditioned on these fitted values.
  • Scaling coefficient b = 9
    chosen from preliminary analysis and fixed across experiments (Section 2.5).
  • Decorrelation learning rate for Q-networks = 1e-13 during search
    fixed ad hoc after preliminary search (Appendix A); Q-networks decorrelation omitted in main results.
assumptions (5)
  • domain assumption The update rule R ← R - η C R (Eq. 3) minimizes the decorrelation loss (Eq. 5) and thereby improves gradient descent in RL networks.
    Adopted from Dalm et al. (2024) without re-derivation; the causal benefit in the RL setting is the paper's hypothesis, not proven.
  • domain assumption Input decorrelation speeds up optimization and improves sample efficiency in deep RL.
    Motivating assumption (Section 1), supported only by prior work on supervised learning and by the paper's own confounded experiments.
  • ad hoc to paper Leaky ReLU performs better than ReLU for the decorrelation procedure in RL.
    Section 2.3.1 states this was changed 'due to performance improvement observed for the decorrelation procedure', no analysis provided.
  • domain assumption Downsampling with Eq. (7) and b=9 preserves enough correlational structure for decorrelation.
    Taken from DBP prior work and preliminary analysis (Section 2.2), with no sensitivity analysis.
  • ad hoc to paper The decorrelation loss value is a measure of how novel the visited states are.
    Discussion Section proposes this interpretation without empirical support.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decorrelated Soft Actor-Critic for Efficient Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/R2U5G7VT

@misc{pith2026250119133,
  author       = {Pith},
  title        = {Pith review of: Decorrelated Soft Actor-Critic for Efficient Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R2U5G7VT}},
  note         = {Machine review of arXiv:2501.19133}
}
read the original abstract

The effectiveness of credit assignment in reinforcement learning (RL) when dealing with high-dimensional data is influenced by the success of representation learning via deep neural networks, and has implications for the sample efficiency of deep RL algorithms. Input decorrelation has been previously introduced as a method to speed up optimization in neural networks, and has proven impactful in both efficient deep learning and as a method for effective representation learning for deep RL algorithms. We propose a novel approach to online decorrelation in deep RL based on the decorrelated backpropagation algorithm that seamlessly integrates the decorrelation process into the RL training pipeline. Decorrelation matrices are added to each layer, which are updated using a separate decorrelation learning rule that minimizes the total decorrelation loss across all layers, in parallel to minimizing the usual RL loss. We used our approach in combination with the soft actor-critic (SAC) method, which we refer to as decorrelated soft actor-critic (DSAC). Experiments on the Atari 100k benchmark with DSAC shows, compared to the regular SAC baseline, faster training in five out of the seven games tested and improved reward performance in two games with around 50% reduction in wall-clock time, while maintaining performance levels on the other games. These results demonstrate the positive impact of network-wide decorrelation in deep RL for speeding up its sample efficiency through more effective credit assignment.

Figures

Figures reproduced from arXiv: 2501.19133 by the authors.

Figure 1
Figure 1. DSAC speeds up training w.r.t. SAC in 5 of the 7 games tested for up to 76%, maintaining [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Decorrelation loss for the policy network (shown in log scale) constitutes the main difference [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Search for the best hyperparameter configuration for the BP-based discrete SAC baseline. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Search for the best hyperparameter configuration for DSAC. Values indicate the final [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Reward curves per game for top configurations of SAC baseline vs. DSAC with decorrela [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Decorrelation helps actor loss or policy network’s loss to stay around minimal values [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Decorrelation enables a faster convergence for the minimization of alpha loss that serves [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Average Q-loss from the two soft Q-networks. [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Time advantage of the DBP-based DSAC algorithm (with decorrelation in all trained [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Wall-clock time and reward performance of the highest ranking configurations in the [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 29 canonical work pages

  1. [1]

    Ahmad, N. (2024). Correlations are ruining your gradient descent. ArXiv preprint , ArXiv:2407.10780

    arXiv 2024

  2. [2]

    Ahmad, N., Schrader, E., and van Gerven, M. (2023). Constrained parameter inference as a principle for learning. Transactions on Machine Learning Research

    work page 2023

  3. [3]

    independent components

    Bell, A. J. and Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research , 37(23):3327--3338

    work page 1997

  4. [4]

    and Bergstra, J

    Bengio, Y. and Bergstra, J. (2009). Slow, decorrelated features for pretraining complex cell-like networks. In Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C., and Culotta, A., editors, Advances in Neural Information Processing Systems , volume 22. Curran Associates, Inc

    work page 2009

  5. [5]

    Christodoulou, P. (2019). Soft actor-critic for discrete action settings. ArXiv preprint , ArXiv:1910.07207

    arXiv 2019

  6. [6]

    Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science . Behavioral and Brain Sciences , 36(3):181--204

    work page 2013

  7. [7]

    Dalm, S., Offergeld, J., Ahmad, N., and van Gerven, M. (2024). Efficient deep learning with decorrelated backpropagation. ArXiv preprint , ArXiv:2405.02385

    work page arXiv 2024

  8. [8]

    de Ruyter van Steveninck, J., Nipshagen, M., van Gerven, M., Güçlü, U., Güçlütürk, Y., and van Wezel, R. (2024). Gaze-contingent processing improves mobility, scene recognition and visual search in simulated head-steered prosthetic vision. Journal of Neural Engineering , 21(2):026037

    work page 2024

Show all 37 references
  1. [9]

    Desjardins, G., Simonyan, K., Pascanu, R., and Kavukcuoglu, K. (2015). Natural neural networks. In Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 28. Curran Associates, Inc

    2015

  2. [10]

    Franke, K., Berens, P., Schubert, T., Bethge, M., Euler, T., and Baden, T. (2017). Inhibition decorrelates visual feature representations in the inner retina. Nature , 542

    2017

  3. [11]

    Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological sciences , 360(1456):815--836

    2005

  4. [12]

    Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018a). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Dy, J. G. and Krause, A., editors, ICML , volume 80 of Proceedings of Machine Learning Research , pages 1856--1865. PMLR

    2018

  5. [13]

    Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., and Levine, S. (2018b). Soft actor-critic algorithms and applications. ArXiv preprint , ArXiv:1812.05905

    2018 arXiv

  6. [14]

    Huang, S., Sun, Y., Hu, J., Guo, S., Chen, H., Chang, Y., Sun, L., and Yang, B. (2023). Learning generalizable agents via saliency-guided features decorrelation. In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  7. [15]

    and Rao, R

    Huang, Y. and Rao, R. P. N. (2011). Predictive coding. WIREs Cognitive Science , 2(5):580--593

    2011

  8. [16]

    Huangi, L., Yang, D., Lang, B., and Deng, J. (2018). Decorrelated batch normalization. In Proceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018 , Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recogniti...

    2018

  9. [17]

    H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., Mohiuddin, A., Sepassi, R., Tucker, G., and Michalewski, H

    Kaiser, L., Babaeizadeh, M., Milos, P., Osinski, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., Mohiuddin, A., Sepassi, R., Tucker, G., and Michalewski, H. (2020). Model based reinforcement learning for Atari . In 8th International Confer...

    2020

  10. [18]

    Küçükoğlu, B., Borkent, W., Rueckauer, B., Ahmad, N., Güçlü, U., and van Gerven, M. (2024). Efficient deep reinforcement learning with predictive processing proximal policy optimization. Neurons, Behavior, Data analysis, and Theory , pages 1--24

    2024

  11. [19]

    Küçükoğlu, B., Rueckauer, B., Ahmad, N., de Ruyter van Steveninck, J., Güçlü, U., and van Gerven, M. (2022). Optimization of neuroprosthetic vision via end-to-end deep reinforcement learning . International Journal of Neural Systems , 32(11):2250052

    2022

  12. [20]

    u ller, K.-R. (2012). Efficient backprop. In Montavon, G., Orr, G. B., and M \

    LeCun, Y. A., Bottou, L., Orr, G. B., and M \"u ller, K.-R. (2012). Efficient backprop. In Montavon, G., Orr, G. B., and M \"u ller, K.-R., editors, Neural Networks: Tricks of the Trade: Second Edition , pages 9--48, Berlin, Heidelberg. Springer Berlin Heidelberg

    2012

  13. [21]

    Lee, H., Lee, K., Hwang, D., Lee, H., Lee, B., and Choo, J. (2023). On the importance of feature decorrelation for unsupervised representation learning in reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning , ICML'23. JMLR.org

    2023

  14. [22]

    Linnainmaa, S. (1976). Taylor expansion of the accumulated rounding error. BIT , 16(2):146–160

    1976

  15. [23]

    Luo, P. (2017). Learning deep architectures via generalized whitened neural networks. In Precup, D. and Teh, Y. W., editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pages 2238--2246. PMLR

    2017

  16. [24]

    C., Bellemare, M

    Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., and Bowling, M. (2018). Revisiting the Arcade learning environment: Evaluation protocols and open problems for general agents . Journal of Artificial Intelligence Research , 61(1):523–562

    2018

  17. [25]

    Mavrin, B., Graves, D., and Chan, A. (2019a). Efficient decorrelation of features using Gramian in reinforcement learning . ArXiv preprint , ArXiv:1911.08610

    2019 arXiv

  18. [26]

    Mavrin, B., Yao, H., and Kong, L. (2019b). Deep reinforcement learning with decorrelation. ArXiv preprint , ArXiv:1903.07765

    2019 arXiv

  19. [27]

    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. A. (2013). Playing Atari with deep reinforcement learning . ArXiv preprint , ArXiv:1312.5602

    2013 arXiv

  20. [28]

    A., Veness, J., Bellemare, M

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015). Human-level control through deep reinforcement learning. Nature , 518(7540):529--533

    2015

  21. [29]

    and Meister, M

    Pitkow, X. and Meister, M. (2012). Decorrelation and efficient coding by retinal ganglion cells. Nature Neuroscience , 15:628--35

    2012

  22. [30]

    Ras, G., Xie, N., van Gerven, M., and Doran, D. (2022). Explainable deep learning: A field guide for the uninitiated. Journal of Artificial Intelligence Research , 73

    2022

  23. [31]

    Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence , 1:206--215

    2019

  24. [32]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. ArXiv preprint , ArXiv:1707.06347

    2017 arXiv

  25. [33]

    Y., Giladi, C., Gedalin, M., Rucci, M., Ben-Tov, M., Kushinsky, Y., Mokeichev, A., and Segev, R

    Segal, I. Y., Giladi, C., Gedalin, M., Rucci, M., Ben-Tov, M., Kushinsky, Y., Mokeichev, A., and Segev, R. (2015). Decorrelation of retinal response to natural scenes by fixational eye movements. Proceedings of the National Academy of Sciences , 112(10):3110--3115

    2015

  26. [34]

    V., Laughlin, S

    Srinivasan, M. V., Laughlin, S. B., and Dubs, A. (1982). Predictive coding: A fresh view of inhibition in the retina. Proceedings of the Royal Society of London. Series B. Biological Sciences , 216(1205):427--459

    1982

  27. [35]

    Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction . The MIT Press, second edition

    2018

  28. [36]

    Toromanoff, M., Wirbel, E., and Moutarde, F. (2019). Is deep reinforcement learning really superhuman on Atari? Leveling the playing field . ArXiv preprint , ArXiv:1908.04683

    2019 arXiv

  29. [37]

    Werbos, P. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences . PhD thesis, Harvard University, Cambridge, MA, USA

    1974

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.