Pith. sign in

REVIEW 3 major objections 4 minor 50 references

Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sparse-Reg claims that pruning 75–95% of network weights via a sensitivity score, refreshed early in training, prevents offline RL overfitting on 5k–100k transitions and improves BC, AWAC, TD3+BC, and IQL.

desk verdict The overfitting diagnosis is clean and the sparsity idea is plausible, but the headline gains rest on a sparsity schedule tuned on the same benchmark and on high-variance results without significance tests. read the letter →

arxiv 2506.17155 v2 pith:H3MR7ARL submitted 2025-06-20 cs.LG cs.AI

Samin Yeasar Arnob , Scott Fujimoto , Doina Precup This is my paper · ORCID
classification cs.LGcs.AI
keywords offlinereinforcementlearningsparsityregularizationsamplecomplexityoverfittingcontinuouscontrolSNIPD4RL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Offline reinforcement learning algorithms collapse when trained on small datasets, overfitting to a few thousand transitions and degrading after many gradient updates. This paper claims that a sparse regularization method, which masks off 75–95% of a network's weights using a gradient-sensitivity score and periodically refreshes the mask during the first 200k steps, dramatically improves episodic return across BC, AWAC, TD3+BC, and IQL on D4RL continuous-control tasks with 5k–100k transitions. The authors demonstrate the effect in learning curves, aggregated normalized scores, and a held-out validation-loss experiment, and they argue that sparsity is more reliable than dropout, weight decay, spectral normalization, and layer norm for limited-data offline RL. If correct, this makes offline RL practical for applications where collecting millions of transitions is prohibitive.

What carries the argument

The load-bearing object is the binary mask $m \in \{0,1\}^{|\theta|}$ built from the SNIP connection-sensitivity score $S(\theta_q) = |\theta_q \partial L/\partial \theta_q|$, evaluated on a minibatch and used to select the top-$k$ most loss-sensitive weights. The mask freezes all other weights at zero, and it is recomputed every 5k gradient steps during the first 200k steps, so the subnetwork can shift across layers as training proceeds. The same mask is applied to target networks whose weights are coupled to the online networks. This single-shot, gradient-based selection converts the dense architecture into a sparse subnetwork that retains representational capacity while drastically reducing the effective parameter count, which is the mechanism the paper argues prevents overfitting in small-data offline RL.

What would settle it

Run the sparsity-level sweep shown in Table 3 on a second environment, such as Walker2d-Expert, at 5k and 50k samples; if the best sparsity is not 95% or 75% respectively, or if the fixed ratios produce a loss relative to the dense baseline on any task, the central transferability assumption would be falsified.

Watch

Extended reading notes

Core claim

The central discovery is that reducing model complexity through unstructured sparsity acts as a targeted regularizer for offline RL in low-data regimes, and that this cannot be replaced by simply training a smaller dense network. Using the connection-sensitivity score $S(\theta_q) = |\theta_q \partial L/\partial \theta_q|$ from SNIP, Sparse-Reg computes a binary mask that keeps the top 5% of parameters (for 5k and 10k samples) or the top 25% (for 50k and 100k samples) in each actor, critic, and value network, and it recalculates the mask every 5k gradient steps for the first 200k steps. Across D4RL Expert and Medium datasets, the masked models consistently outperform dense baselines, with the largest gains at 5k and 10k samples. The paper attributes the improvement to reduced overfitting, supported by a validation-loss experiment in which dense BC training loss falls while validation loss rises, whereas the sparse model keeps validation loss low.

Load-bearing premise

The load-bearing premise is that the sparsity ratio chosen from the HalfCheetah-Expert ablation—95% for 5k/10k samples and 75% for 50k/100k samples—transfers to every other environment, dataset type, and sample size; if a different task requires a different ratio, the reported gains could reverse.

Editorial extensions

If this is right

  • At 5k–10k samples, keeping only 5% of parameters (95% sparsity) turns failed baselines into positive returns; for example, AWAC on HalfCheetah-Expert goes from −484 to 8735 at 10k samples.
  • The same recipe with 25% of parameters improves all four algorithms on both Expert and Medium D4RL subsets at 50k–100k samples.
  • Periodic mask updates outperform a fixed-at-initialization mask, and sparsity stabilizes after roughly 200k steps, making the refresh schedule an essential part of the method.
  • Sparse-Reg outperforms dropout, weight decay, spectral normalization, and layer norm in the comparisons run, suggesting it is a more reliable regularizer for limited-data offline RL.
  • Training a subnetwork of a large network gives more consistent gains than reducing hidden dimensions to 64 or 128, indicating the full architecture matters even when most weights are masked.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is that the optimal active-parameter fraction may scale monotonically with dataset size; a sweep across more sample sizes (e.g., 1k, 20k, 200k) could turn the two-point rule (5%/25%) into a general scaling law.
  • The periodic mask refresh behaves like a plasticity-restoring mechanism; it could be tested in online RL or non-stationary fine-tuning, where refreshing connections might prevent premature convergence even without data scarcity.
  • Because the sensitivity score is loss-based, the refresh interval and sparsity level may need to track the loss landscape's curvature; a testable extension is to vary the refresh interval (not just 5k during 200k) and see whether the best setting correlates with the rate of loss decrease.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Sparse-Reg, a regularization technique for offline reinforcement learning with small datasets. The method applies SNIP-style connection-sensitivity scoring to select a sparse subnetwork at initialization, and then periodically refreshes the masks during the first 200k gradient steps. The authors report that using 95% sparsity for 5k and 10k transitions and 75% sparsity for 50k and 100k transitions improves episodic return for BC, AWAC, TD3+BC, and IQL on D4RL Expert, Medium, and Replay datasets. They support the underlying overfitting diagnosis with a train/validation MSE experiment and compare sparsity against L1, dropout, weight decay, spectral normalization, and layer norm.

Significance. If the reported gains are reliable, the contribution is practically valuable: a simple, code-level regularization that improves offline RL in low-data continuous control, with released code and a clean validation-loss diagnostic. The paper's main mechanism, that limiting active parameters reduces overfitting on small datasets, is well motivated and supported by the controlled comparison in Figure 4. The method is also easy to integrate into existing actor-critic algorithms. However, the headline quantitative claims currently rest on a sparsity schedule selected on the same benchmark where performance is measured, with large standard deviations and no significance testing, so the empirical contribution needs additional validation before the claims can be accepted.

major comments (3)
  1. [Section 5.5 and Appendix Table 3] The sparsity schedule is selected after observing test returns on HalfCheetah-Expert with IQL, and Table 3 shows that the chosen operating point is a narrow peak. At 5k samples, moving from 95% sparsity to 75% changes IQL from 2454 to -322.96; at 10k, the change is from 8521.96 to 92.35. Since the same benchmark is then used for the headline comparisons in Table 1, the gains for other environments, dataset types, and algorithms may be inflated by selection on the test set. The periodic-update horizon (first 200k steps) is also justified heuristically from the same experiments. The authors should either fix the schedule on a validation split or leave-one-task-out selection, or report per-task selected sparsity along with a bias assessment.
  2. [Section 5.2, Table 1] The claim that Sparse-Reg "significantly improves performance across the entire range of training samples" is not supported by statistical evidence. Many comparisons show overlapping or very wide standard deviations; for example, TD3+BC on HalfCheetah at 5k goes from -260.51±136.23 to 494.55±796.65, and at 10k from 199.03±168.51 to 1038.07±1262.93. With only 5 seeds, the authors should report paired tests, bootstrap confidence intervals, or effect sizes for the Table 1 and Table 4 comparisons, and should avoid the word "significantly" without such tests.
  3. [Appendix, Table 5] The mixed-dataset results in Table 5 are not usable as printed: every row for HalfCheetah and Hopper is identical for both Medium-Replay and Expert-Replay at both 5k and 10k. This appears to be a data or formatting error, and it prevents assessment of the mixed-distribution claim. The table should be corrected or removed.
minor comments (4)
  1. [Section 5.5] The text says "In Figure 6, we conducted an ablation study where we vary the sparsity ratio," but Figure 6 as captioned compares BC-sparse and BC-baseline learning curves at 5k and 50k. The actual sparsity sweep is in Appendix Table 3, so the in-text reference and figure caption should be reconciled.
  2. [Abstract and Introduction] The abstract claims that the method outperforms "state-of-the-art baselines," but the experiments are compared only with standard offline RL algorithms (BC, AWAC, TD3+BC, IQL) and standard regularizers, not with recent small-data offline RL methods. The wording should be adjusted to avoid overclaiming.
  3. [Section 5.4] The validation-loss experiment uses a 200k held-out set for a policy trained on 50k samples, while the Appendix extends the curves to 5k and 10k training samples. The relationship between the held-out set and the training subsets should be stated more explicitly, including whether the same 200k validation tuples were used for every training size.
  4. [Tables 1 and 4] The tables contain formatting issues such as "AW AC" instead of "AWAC" and inconsistent decimal places. More importantly, the captions assert that performance losses are "insignificant" without reporting the test used; this claim should either be supported or removed.

Circularity Check

1 steps flagged · score 4.0 of 10

Sparsity schedule is tuned on test returns of the benchmark and then reused to report gains on that same benchmark, partially forcing the headline numbers.

  1. fitted input called prediction [Section 5.2 Table 1; Section 5.5 Table 3]
    "Consequently, for all experiments involving5k and 10k samples, we maintain a 95% sparsity, while for 50k and 100k samples, we employ a 75% sparsity. ... For dataset sizes of 50k and 100k, the best performance is achieved with 75% sparsity."

    The sparsity ratios used to generate the main Sparse-Reg results in Table 1 are chosen in Table 3 by maximizing test episodic return on HalfCheetah-Expert with IQL. The selected entries then reappear verbatim in Table 1: IQL/HalfCheetah at 5k is 2454.94 in both places, and at 10k it is 8521.96 in both places; the 50k/100k schedule (75%) is likewise the argmax over the Table 3 grid. Thus the headline improvement on the tuning benchmark is the maximum of the explored sparsity grid presented as the performance of a fixed method, rather than an out-of-sample prediction. The remaining rows and the validation-loss experiment provide independent support, so the circularity is partial rather than total.

full rationale

The paper is an empirical study with no closed-form derivation chain, so most definitional circularity patterns are absent. The one genuine self-referential step is the selection of the sparsity schedule: Section 5.5/Table 3 tunes sparsity ratios (95% for 5k/10k, 75% for 50k/100k) by maximizing test returns on HalfCheetah-Expert with IQL, and Section 5.2/Table 1 then reports Sparse-Reg gains using that schedule, with the IQL/HalfCheetah 5k and 10k values identical to the best grid entries. That makes the headline improvement on the tuning environment partly an artifact of selection on the test metric rather than an out-of-sample evaluation. The periodic-update design is justified both by a self-citation (Arnob et al., 2025) and by the paper's own SFI versus SPU ablation, so it is not load-bearing circularity. The validation-loss analysis in Section 5.4 and the other environments and algorithms provide independent evidence that sparsity can help, which prevents the entire central claim from reducing to the tuning step. Overall, the circularity is partial, arising from a fitted hyperparameter being reused as a prediction on the same benchmark.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central method imports SNIP pruning and a periodic update schedule from prior work; the only quantities tuned on the target benchmarks are the sparsity ratio and the update schedule. No new entities are introduced. The assumptions listed are the main premises the empirical claims rest on.

free parameters (2)
  • Sparsity ratio = 95% for 5k/10k samples; 75% for 50k/100k samples
    Selected via ablation on HalfCheetah-Expert (Table 3); the reported performance of Sparse-Reg depends directly on this choice.
  • Periodic mask update schedule = every 5k gradient steps for the first 200k steps
    Chosen heuristically because sparsity ratio stabilizes after 200k (Section 5.5); schedule affects final performance (Figure 5b).
assumptions (4)
  • domain assumption The gradient-based sensitivity score from SNIP (Lee et al., 2018) identifies parameters whose removal improves generalization, and pruning to 5% to 25% of parameters does not destroy the policy's representational capacity.
    This is the mechanism the entire method relies on; it is imported from supervised pruning literature and not re-derived for RL.
  • domain assumption The held-out validation loss, computed on transitions from the same behavior policy, is a faithful proxy for episodic return in the overfitting analysis.
    Section 5.4 uses validation MSE to conclude reduced overfitting; if validation distribution differs from deployment, the conclusion may not transfer.
  • domain assumption Subsampling D4RL datasets to 5k to 100k transitions preserves the same behavior policy and does not introduce distribution shift beyond sample size.
    The paper treats smaller sample sizes as the only change, without studying whether subsampled data is representative.
  • standard math The MDP framework and Bellman updates used by the baseline algorithms are correct.
    Background from Fujimoto et al. 2018 and Kostrikov et al. 2021; not in dispute.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity." pith.science (2026). https://pith.science/paper/H3MR7ARL

@misc{pith2026250617155,
  author       = {Pith},
  title        = {Pith review of: Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3MR7ARL}},
  note         = {Machine review of arXiv:2506.17155}
}
read the original abstract

In this paper, we investigate the use of small datasets in the context of offline reinforcement learning (RL). While many common offline RL benchmarks employ datasets with over a million data points, many offline RL applications rely on considerably smaller datasets. We show that offline RL algorithms can overfit on small datasets, resulting in poor performance. To address this challenge, we introduce "Sparse-Reg": a regularization technique based on sparsity to mitigate overfitting in offline reinforcement learning, enabling effective learning in limited data settings and outperforming state-of-the-art baselines in continuous control.

Figures

Figures reproduced from arXiv: 2506.17155 by the authors.

Figure 1
Figure 1. Learning curve of BC-baseline (blue) and BC-sparse (orange) for HalfCheetah-v2 with (a) 10k and (b) 100k training samples; Walker2d-v2 with (c) 10k and (d) 100k training samples over 1 million time steps. Performance mean and standard deviation of the episodic returns are estimated every 5k steps over 10 evaluations across 5 seeds. 5 Experiments In this section, we provide empirical evidence supporting the effective… view at source ↗
Figure 2
Figure 2. Comparison of normalized performance for various regularizers applied to IQL algorithm, trained with [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Aggregated normalized performance of IQL and IQL-Sparse-Reg across the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: To investigate overfitting, we compare the performance of BC-baseline ( [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: (a) Impact of applying network sparsity periodically [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Learning curve of BC-sparse and BC-baseline for [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Aggregated normalized performance of IQL and IQL-Sparse-Reg across [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Compare the MSE loss of behaviour-action prediction on the [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Compare the MSE loss of behaviour-action prediction on the [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Learning curve of BC-baseline (blue) and BC-sparse-reg (orange) for (a) HalfCheetah-v2, (b) Hopper, (c) Walker2d-v2 with varying training sample over 1 million time steps. Performance mean and standard deviation of the episodic returns are estimated every 5k steps ove…
Figure 11
Figure 11. Figure 11: Example code of applying sparsity. A.2 Libraries We run our algorithm in PyTorch-1.9.0 Paszke et al. (2019) and use following libraries: TD3-BC Fujimoto & Gu (2021), AWAC Tarasov et al. (2022), Implicit Q-learning (IQL) Thomas (2021) implementation. A.3 Alternate Regu…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 15 canonical work pages

  1. [1]

    Striving for simplicity in off-policy deep reinforcement learning

    Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. Striving for simplicity in off-policy deep reinforcement learning. CoRR, abs/1907.04543, 2019. URL http://arxiv.org/abs/1907.04543

    arXiv 1907

  2. [2]

    Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning

    Samin Yeasar Arnob, Riashat Islam, and Doina Precup. Importance of empirical sample complexity analysis for offline reinforcement learning. CoRR, abs/2112.15578, 2021 a . URL https://arxiv.org/abs/2112.15578

    work page Pith review arXiv 2021

  3. [3]

    Single-shot pruning for offline reinforcement learning

    Samin Yeasar Arnob, Riyasat Ohib, Sergey Plis, and Doina Precup. Single-shot pruning for offline reinforcement learning. arXiv preprint arXiv:2112.15579, 2021 b

    arXiv 2021

  4. [4]

    Efficient reinforcement learning by discovering neural pathways

    Samin Yeasar Arnob, Riyasat Ohib, Sergey Plis, Amy Zhang, Alessandro Sordoni, and Doina Precup. Efficient reinforcement learning by discovering neural pathways. Advances in Neural Information Processing Systems, 37: 0 18660--18694, 2025

    work page 2025

  5. [5]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016. URL https://arxiv.org/abs/1607.06450

    arXiv 2016

  6. [6]

    Pattern Recognition and Machine Learning

    Christopher M Bishop. Pattern Recognition and Machine Learning. Springer, 2006

    2006

  7. [7]

    What is the state of neural network pruning? Proceedings of machine learning and systems, 2: 0 129--146, 2020

    Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? Proceedings of machine learning and systems, 2: 0 129--146, 2020

    2020

  8. [8]

    Learning to prune deep neural networks via layer-wise optimal brain surgeon

    Xin Dong, Shangyu Chen, and Sinno Jialin Pan. Learning to prune deep neural networks via layer-wise optimal brain surgeon. arXiv preprint arXiv:1705.07565, 2017

    arXiv 2017

Show all 50 references
  1. [9]

    Schizophrenia: caused by a fault in programmed synaptic elimination during adolescence? Journal of psychiatric research, 17 0 (4): 0 319--334, 1982

    Irwin Feinberg. Schizophrenia: caused by a fault in programmed synaptic elimination during adolescence? Journal of psychiatric research, 17 0 (4): 0 319--334, 1982

    1982

  2. [10]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks, 2019

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks, 2019

    2019

  3. [11]

    D4RL: datasets for deep data-driven reinforcement learning

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL: datasets for deep data-driven reinforcement learning. CoRR, abs/2004.07219, 2020. URL https://arxiv.org/abs/2004.07219

    2004 arXiv

  4. [12]

    A minimalist approach to offline reinforcement learning

    Scott Fujimoto and Shixiang Shane Gu. A minimalist approach to offline reinforcement learning. CoRR, abs/2106.06860, 2021. URL https://arxiv.org/abs/2106.06860

    2021 arXiv

  5. [13]

    Off-policy deep reinforcement learning without exploration

    Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. CoRR, abs/1812.02900, 2018. URL http://arxiv.org/abs/1812.02900

    2018 arXiv

  6. [14]

    For sale: State-action representation learning for deep reinforcement learning

    Scott Fujimoto, Wei-Di Chang, Edward Smith, Shixiang Shane Gu, Doina Precup, and David Meger. For sale: State-action representation learning for deep reinforcement learning. Advances in Neural Information Processing Systems, 36, 2023

    2023

  7. [15]

    The state of sparsity in deep neural networks

    Trevor Gale, Erich Elsen, and Sara Hooker. The state of sparsity in deep neural networks. arXiv preprint arXiv:1902.09574, 2019

    1902 arXiv

  8. [16]

    Extreme q-learning: Maxent rl without entropy

    Divyansh Garg, Joey Hejna, Matthieu Geist, and Stefano Ermon. Extreme q-learning: Maxent rl without entropy. In The Eleventh International Conference on Learning Representations, 2022

    2022

  9. [17]

    The state of sparse training in deep reinforcement learning, 2022

    Laura Graesser, Utku Evci, Erich Elsen, and Pablo Samuel Castro. The state of sparse training in deep reinforcement learning, 2022

    2022

  10. [18]

    Rl unplugged: A suite of benchmarks for offline reinforcement learning

    Caglar Gulcehre, Ziyu Wang, Alexander Novikov, Thomas Paine, Sergio G \'o mez, Konrad Zolna, Rishabh Agarwal, Josh S Merel, Daniel J Mankowitz, Cosmin Paduraru, et al. Rl unplugged: A suite of benchmarks for offline reinforcement learning. Advances in Neural Information Proces...

    2020

  11. [19]

    Dynamic network surgery for efficient dnns

    Yiwen Guo, Anbang Yao, and Yurong Chen. Dynamic network surgery for efficient dnns. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL https://proceedings.neurips...

    2016

  12. [21]

    Learning both weights and connections for efficient neural networks

    Song Han, Jeff Pool, John Tran, and William J Dally. Learning both weights and connections for efficient neural networks. arXiv preprint arXiv:1506.02626, 2015 b

    2015 arXiv

  13. [22]

    Idql: Implicit q-learning as an actor-critic method with diffusion policies

    Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, and Sergey Levine. Idql: Implicit q-learning as an actor-critic method with diffusion policies. arXiv preprint arXiv:2304.10573, 2023

    2023 arXiv

  14. [23]

    Optimal brain surgeon and general network pruning

    Babak Hassibi, David G Stork, and Gregory J Wolff. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, pp.\ 293--299. IEEE, 1993

    1993

  15. [24]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. CoRR, abs/2110.06169, 2021 a . URL https://arxiv.org/abs/2110.06169

    2021 arXiv

  16. [25]

    Offline reinforcement learning with fisher divergence critic regularization

    Ilya Kostrikov, Jonathan Tompson, Rob Fergus, and Ofir Nachum. Offline reinforcement learning with fisher divergence critic regularization. CoRR, abs/2103.08050, 2021 b . URL https://arxiv.org/abs/2103.08050

    2021 arXiv

  17. [26]

    Conservative q-learning for offline reinforcement learning

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. CoRR, abs/2006.04779, 2020. URL https://arxiv.org/abs/2006.04779

    2006 arXiv

  18. [27]

    A workflow for offline model-free robotic reinforcement learning, 2021

    Aviral Kumar, Anikait Singh, Stephen Tian, Chelsea Finn, and Sergey Levine. A workflow for offline model-free robotic reinforcement learning, 2021

    2021

  19. [28]

    The challenges of exploration for offline reinforcement learning

    Nathan Lambert, Markus Wulfmeier, William Whitney, Arunkumar Byravan, Michael Bloesch, Vibhavari Dasagi, Tim Hertweck, and Martin Riedmiller. The challenges of exploration for offline reinforcement learning. arXiv preprint arXiv:2201.11861, 2022

    2022 arXiv

  20. [29]

    Optimal brain damage

    Yann LeCun, John S Denker, and Sara A Solla. Optimal brain damage. In Advances in neural information processing systems, pp.\ 598--605, 1990

    1990

  21. [30]

    Snip: Single-shot network pruning based on connection sensitivity

    Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340, 2018

    2018 arXiv

  22. [31]

    Sparse convolutional neural networks

    Baoyuan Liu, Min Wang, Hassan Foroosh, Marshall Tappen, and Marianna Pensky. Sparse convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 806--814, 2015

    2015

  23. [33]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https://arxiv.org/abs/1711.05101

    2019 arXiv

  24. [34]

    Spectral normalization for generative adversarial networks

    Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. CoRR, abs/1802.05957, 2018. URL http://arxiv.org/abs/1802.05957

    2018 arXiv

  25. [35]

    Pruning convolutional neural networks for resource efficient transfer learning

    Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient transfer learning. CoRR, abs/1611.06440, 2016. URL http://arxiv.org/abs/1611.06440

    2016 arXiv

  26. [36]

    Accelerating online reinforcement learning with offline datasets

    Ashvin Nair, Murtaza Dalal, Abhishek Gupta, and Sergey Levine. Accelerating online reinforcement learning with offline datasets. CoRR, abs/2006.09359, 2020. URL https://arxiv.org/abs/2006.09359

    2006 arXiv

  27. [37]

    Potluru, and Sergey Plis

    Riyasat Ohib, Nicolas Gillis, Niccolo Dalmasso, Sameena Shah, Vamsi K. Potluru, and Sergey Plis. Explicit group sparse projection with applications to deep learning and NMF . Transactions on Machine Learning Research, 2022. URL https://openreview.net/forum?id=jIrOeWjdpc

    2022

  28. [38]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  29. [39]

    Understanding the effects of dataset characteristics on offline reinforcement learning

    Kajetan Schweighofer, Markus Hofmarcher, Marius-Constantin Dinu, Philipp Renz, Angela Bitto-Nemling, and Sepp Hochreiter. Understanding the effects of dataset characteristics on offline reinforcement learning. arXiv preprint arXiv:2111.04714, 2021

    2021 arXiv

  30. [40]

    A dataset perspective on offline reinforcement learning

    Kajetan Schweighofer, Marius-constantin Dinu, Andreas Radler, Markus Hofmarcher, Vihang Prakash Patil, Angela Bitto-Nemling, Hamid Eghbal-zadeh, and Sepp Hochreiter. A dataset perspective on offline reinforcement learning. In Conference on Lifelong Learning Agents, pp.\ 470--5...

    2022

  31. [41]

    Dropout: A simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15 0 (56): 0 1929--1958, 2014. URL http://jmlr.org/papers/v15/srivastava14a.html

    1929

  32. [42]

    Rlx2: Training a sparse deep reinforcement learning model from scratch, 2023

    Yiqin Tan, Pihe Hu, Ling Pan, Jiatai Huang, and Longbo Huang. Rlx2: Training a sparse deep reinforcement learning model from scratch, 2023

    2023

  33. [43]

    Hidenori Tanaka, Daniel Kunin, Daniel L. K. Yamins, and Surya Ganguli. Pruning neural networks without any data by iteratively conserving synaptic flow. CoRR, abs/2006.05467, 2020. URL https://arxiv.org/abs/2006.05467

    2006 arXiv

  34. [44]

    CORL : Research-oriented deep offline reinforcement learning library

    Denis Tarasov, Alexander Nikulin, Dmitry Akimov, Vladislav Kurenkov, and Sergey Kolesnikov. CORL : Research-oriented deep offline reinforcement learning library. In 3rd Offline RL Workshop: Offline RL as a ''Launchpad'', 2022. URL https://openreview.net/forum?id=SyAS49bBcv

    2022

  35. [45]

    Implicit q-learning (iql) in pytorch

    Garrett Thomas. Implicit q-learning (iql) in pytorch. GitHub, 2021. URL https://github.com/gwthomas/IQL-PyTorch

    2021

  36. [46]

    Regression shrinkage and selection via the lasso

    Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58 0 (1): 0 267--288, 1996

    1996

  37. [47]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.\ 5026--5033. IEEE, 2012

    2012

  38. [48]

    Picking winning tickets before training by preserving gradient flow

    Chaoqi Wang, Guodong Zhang, and Roger Grosse. Picking winning tickets before training by preserving gradient flow. arXiv preprint arXiv:2002.07376, 2020

    2002 arXiv

  39. [49]

    Behavior regularized offline reinforcement learning

    Yifan Wu, George Tucker, and Ofir Nachum. Behavior regularized offline reinforcement learning. arXiv preprint arXiv:1911.11361, 2019

    1911 arXiv

  40. [50]

    Deephoyer: Learning sparser neural network with differentiable scale-invariant sparsity measures

    Huanrui Yang, Wei Wen, and Hai Li. Deephoyer: Learning sparser neural network with differentiable scale-invariant sparsity measures. arXiv preprint arXiv:1908.09979, 2019

    1908 arXiv

  41. [51]

    Don't change the algorithm, change the data: Exploratory data for offline reinforcement learning

    Denis Yarats, David Brandfonbrener, Hao Liu, Michael Laskin, Pieter Abbeel, Alessandro Lazaric, and Lerrel Pinto. Don't change the algorithm, change the data: Exploratory data for offline reinforcement learning. CoRR, abs/2201.13425, 2022. URL https://arxiv.org/abs/2201.13425

    2022 arXiv

  42. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.