Pith. sign in

REVIEW 4 major objections 5 minor 24 references

The paper argues that replacing MLP function approximators in reinforcement learning with SPAN, a low-rank B-spline network with a learnable preprocessing layer, improves sample efficiency by 30-50% and raises success rates by 1.3-9x under

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:08 UTC pith:WD36G5GQ

load-bearing objection Solid low-capacity RL function-approximator study, but the abstract's 30-50% claim and the HVAC/training-cost numbers do not survive contact with the paper's own tables. the 4 major comments →

arxiv 2601.23225 v2 pith:WD36G5GQ submitted 2026-01-30 cs.LG cs.AI

Agile Reinforcement Learning through Separable Neural Architecture and Applications

Rajib Mostakim , Reza T. Batley , Sourav Saha This is my paper
classification cs.LG cs.AI
keywords spline networkssample efficiencyreinforcement learningB-splineslow-rank tensor decompositionfunction approximationresource-constrained RLoffline RL
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that a spline-based function approximator with a low-rank tensor product structure learns control policies and value functions faster and more reliably than multilayer perceptrons when parameters and samples are scarce. If true, resource-constrained RL—robotics with limited trials, edge devices, production systems with uncertain training windows—would not have to wait for post-hoc compression. SPAN's central bet is that RL value functions are locally smooth, so a basis of local B-splines can concentrate capacity where it matters. The evidence includes faster progress to performance thresholds on PPO and SAC benchmarks, higher success rates at most thresholds, lower variance across seeds, and strong results on offline expert datasets. The paper frames SPAN as an intrinsically efficient alternative, not a compress-then-deploy fix.

Core claim

SPAN replaces MLP actor and critic networks with a learnable preprocessing layer followed by a low-rank separable tensor product of B-spline bases. Each input dimension is expanded into univariate B-splines, a small number of tensor-product modes combine those expansions, and a linear head produces policy or value outputs. The claim is that this local-support basis matches the local smoothness of control value functions, allowing expert-level performance with fewer environment interactions: 30-50% sample-efficiency gains, 1.3-9x higher success rates, lower variance, and, on offline expert demonstrations, an average 6.7x advantage over MLPs.

What carries the argument

The central object is the low-rank tensor product B-spline basis: per-dimension B-spline expansions are combined through tensor products into M separable modes, then linearly mapped to outputs. This keeps parameter count linear in input dimension instead of exponential, and the preprocessing sigmoid layer projects arbitrary observations into [0,1]^d, where the B-spline grid is defined. That combination is what lets a spline basis with local support—rather than a fully connected layer with globally distributed capacity—serve as the function approximator inside PPO, SAC, and IQL.

Load-bearing premise

The load-bearing premise is that RL value functions and policies in the tested environments are locally smooth—small changes in state produce small changes in predicted value or action; if a task has sharp discontinuities, the gains over MLPs could shrink or reverse, as the paper's own human-teleoperation results already hint.

What would settle it

Train SPAN and an equal-parameter MLP on a task with a deliberately nonsmooth optimal value function, such as a bang-bang controller with a sharp switching threshold, or on the Adroit Human datasets for Door and Hammer, where the paper already reports MLP scoring higher. If SPAN loses on such tasks while winning on smooth expert data, the smoothness mechanism is confirmed and its boundary is located.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • RL agents could learn usable policies with far fewer real-world trials in settings where each interaction costs time, energy, or hardware wear.
  • Strong control policies could be learned with only hundreds to low thousands of parameters, which matters for edge devices with strict memory budgets.
  • More consistent convergence across random seeds would reduce debugging and retraining; the paper estimates total training cost is 1.3-6.3x lower despite a slower per-step forward pass.
  • Offline and imitation learning from smooth expert demonstrations would benefit substantially, with the paper reporting large gains on several Adroit expert datasets.
  • Anytime performance means early training checkpoints are already usable, which is valuable when training may be interrupted or budgets are uncertain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A decisive next experiment the paper implicitly invites is to pit SPAN against an MLP on contact-rich or bang-bang tasks, where the optimal value function is not smooth; the paper's own smoothness explanation predicts SPAN's edge would shrink there.
  • Because SPAN's gains rely on concentrating capacity where value changes, a natural extension is to make the B-spline grid adaptive during training—adding knots where the learned function has high curvature—rather than fixing resolution in advance.
  • The paper's HVAC application suggests the mechanism generalizes to real engineering control loops; a practical extension is to test SPAN with noisy, partially observed sensor streams, where the preprocessing layer may need to be more expressive than a single dense sigmoid layer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SPAN, a tensor-product B-spline architecture with a learnable preprocessing layer, adapted from the authors' KHRONOS framework for use as a function approximator in RL. It is evaluated with PPO on classic control/Box2D tasks, with SAC on MuJoCo tasks, and with IQL on D4RL/Adroit offline benchmarks, using 20 seeds for online runs and matched parameter counts against MLP baselines. The paper claims a 30–50% improvement in sample efficiency and 1.3–9× higher success rates over MLPs, plus an HVAC application and substantial training-cost savings. The body supports a weaker version of the claim: SPAN often reaches performance thresholds faster and achieves higher success rates on many tasks, especially in low-capacity settings, but it does not support the specific numeric ranges stated in the abstract and conclusion, and the HVAC/total-training-cost results are absent from the main text.

Significance. If the central claims were supported, SPAN would be a valuable parameter-efficient, smooth function approximator for resource-constrained RL. The experimental design has genuine strengths: 20 seeds for online experiments, matched parameter counts, a sustained-solving sample-efficiency metric, broad coverage of discrete, continuous, and offline settings, and ablations over the main architectural hyperparameters. The paper also honestly acknowledges the smoothness limitation in Table 3's Human-dataset results. However, the significance is currently undermined by the mismatch between the headline quantitative claims and the reported tables, and by the unverifiable HVAC and training-cost statements. These issues are fixable but require substantial revision of the claims and either added experiments or explicit withdrawal of unsupported statements.

major comments (4)
  1. [Abstract; §1; §7; Tables 1–2] The headline claim of '30–50% improvement in sample efficiency and 1.3–9 times higher success rates' is contradicted by the paper's own data. In Table 1, CartPole at the 25% threshold: SPAN uses 36k steps vs MLP 20k steps, i.e., SPAN is 80% worse, not 30–50% better. At the 100% threshold, LunarLander shows 618k vs 810k (24% improvement) and HalfCheetah shows 698k vs 900k (22% improvement), both below the claimed 30% floor. Several success-rate entries are 100% for both methods (e.g., LunarLander 25%/50%, InvertedPendulum all thresholds), which falls outside the claimed 1.3–9× range. The conclusion and abstract should be rewritten to state the supported weaker claim—SPAN often reaches thresholds faster and achieves higher success on many low-capacity tasks—rather than a uniform numeric range.
  2. [Abstract; §5; §7] The abstract claims HVAC results—'SPAN reduces energy consumption in 9 of 12 months' and '1.1–3.4× reduction in thermal comfort violations'—and also claims 'expected total training cost 1.3–6.3× lower than MLP baselines.' No HVAC experiment, energy/comfort table, or total-training-cost analysis appears anywhere in the body or appendices. These are load-bearing quantitative claims in the abstract and must either be supported with full experimental detail or removed.
  3. [§5.2; Table 3; §7] The conclusion's claim that 'on offline expert datasets, SPAN outperforms the MLP baseline by an average factor of 6.7×' is not a reliable summary of Table 3. The 20× figure for Door is computed against a near-zero MLP baseline (0.4±0.0) with SPAN's own score 8.0±11.6; the Hammer ratio is 2.3× with huge standard deviations (44.1±38.3 vs 19.3±23.4); Pen shows a marginal difference (124.1 vs 119.6); only Relocate is comparatively clean. No significance tests or confidence intervals are reported. The average-of-ratios statistic is therefore misleading and should be replaced by a more careful, variance-aware characterization of the offline results.
  4. [§5.1; §5] The sample-efficiency metric depends on expert target scores, but for Hopper and HalfCheetah these are only described as 'scores indicative of a stable, high-velocity locomotion gait.' No numerical values are given for the 25%, 50%, etc., thresholds. Since the central '30–50% improvement' claim is computed against these thresholds, the missing targets make the results irreproducible and make it impossible for a reader to judge whether threshold choice drives the comparison. The concrete target values should be reported.
minor comments (5)
  1. [§4.2] The preprocessing layer adds d^2 + d parameters, so the total parameter count is O(M d N) + O(d^2), not strictly linear in dimension. This should be stated explicitly; it does not affect the empirical comparison but affects the complexity claim in §4.1.
  2. [Appendix B; Appendix C] The environment version is inconsistent: Table 6 and Figure 8 use 'LunarLander-v3,' while Appendix C text and Figure 8 caption refer to 'LunarLander-v2.' Please unify.
  3. [Figures 5 and 9; Appendix C] The text alternates between nmodes/nelems and krank/kelem for the same hyperparameters. Define the equivalence explicitly in the main text or captions to avoid confusion.
  4. [Appendix A; Table 4] The anytime-performance tables use checkpoints labeled '10%, 25%, ..., 100%' but the text also reports these as absolute step counts (100k, 250k, ...). Clarify the total budget for each environment so the percentages are interpretable.
  5. [Overall] No code, data, or reproducibility statement is provided. Given that the experiments are the main contribution, a code release or detailed configuration (seeds, environment wrappers, evaluation protocol) would substantially strengthen the paper.

Circularity Check

0 steps flagged

No circular derivation found; SPAN's central claims are empirical comparisons against external benchmarks.

full rationale

The paper's main derivation chain is architectural and empirical rather than inferential. Section 4 defines SPAN as a learnable preprocessing layer composed with the KHRONOS separable tensor-product B-spline architecture; Section 4.3 integrates it as a drop-in replacement for MLPs in SAC, PPO, and IQL. The claimed improvements in sample efficiency and success rates are then measured directly against Gymnasium/MuJoCo/D4RL baselines with matched parameter budgets (Tables 1-3 and Appendix B), not derived from any fitted parameter or prior result. No equation in the paper reduces to its own input: sample-efficiency thresholds are defined from environment-specific expert scores and measured, not predicted from the architecture's construction. The heavy self-citation to KHRONOS and related works by the same authors is background motivation and does not carry the central quantitative claim; the SPAN-versus-MLP results are tested in this paper against external environments. The smoothness assumption in Section 1 is an asserted inductive bias, not a circular step, and the paper itself acknowledges failure modes on human datasets with sharp transitions. Concerns such as the abstract's HVAC claims not appearing in the body, and some Table 1-2 cells (e.g., CartPole 25%: SPAN 36k vs MLP 20k) contradicting the stated 30-50% range, are correctness/completeness issues, not circularity. No self-citation is used to forbid alternatives or to substitute for the reported benchmark comparisons.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central results rest on hand-chosen architectural hyperparameters (rank, grid size, degree) and on the domain assumption that control value functions/policies are smooth. No new physical entities are introduced. The preprocessing layer is a learned component, not a free constant, but its sufficiency is assumed.

free parameters (4)
  • nmodes (tensor rank M) = 1-15 per environment
    Hand-selected per environment (e.g., CartPole M=1, Acrobot M=6, LunarLander M=11, InvertedPendulum M=2, Hopper/HalfCheetah M=10, Adroit M=15). The main results depend on these choices; ablation shows HalfCheetah improves with higher rank.
  • nelems (grid resolution N) = 2-8 per environment
    Hand-selected per environment (e.g., 2 for classic control, 8 for Hopper, 4 for HalfCheetah and Adroit). Ablation shows HalfCheetah and Hopper are sensitive to this parameter.
  • spline degree k = 1 or 2
    Degree 1 for classic control, 2 for MuJoCo and Adroit; chosen by hand. For HalfCheetah, degree fixed to 2 in 'optimized implementation'.
  • expert target scores for Hopper/HalfCheetah = unspecified 'stable, high-velocity locomotion gait' scores
    The expert thresholds for unbounded tasks are chosen by hand and are not referenced; changing them would change the reported sample-efficiency improvements.
axioms (6)
  • domain assumption RL value functions and policies in the tested environments are locally smooth; small state changes produce small value/action changes.
    Stated in Section 1 and 6 as 'the central insight'; if false, the B-spline inductive bias could be worse than MLP.
  • domain assumption A single linear layer followed by sigmoid maps the observation space into [0,1]^d without losing control-relevant information.
    Introduced in Section 4.2; the entire spline evaluation assumes inputs are in [0,1]^d.
  • domain assumption Low-rank tensor-product B-splines (KHRONOS) provide sufficient expressiveness for the tested value functions/policies at matched parameter counts.
    Assumed from prior work (Batley & Saha 2025); no expressiveness proof for RL functions is given.
  • domain assumption Standard RL algorithms (PPO, SAC, IQL) function correctly with the SPAN function approximator without algorithmic changes.
    The paper replaces MLP with SPAN inside standard algorithms; this assumes no interaction effects that break convergence.
  • domain assumption Gymnasium and D4RL benchmark scores are reliable and the used environments/datasets match the cited sources.
    The empirical claims rely on environment/dataset fidelity.
  • standard math B-spline basis functions have standard properties (local support, partition of unity, differentiability) as used.
    Standard spline theory; used in Section 4.1.

pith-pipeline@v1.3.0-alltime-deepseek · 23223 in / 18251 out tokens · 177010 ms · 2026-08-03T06:08:59.163650+00:00 · methodology

0 comments
read the original abstract

Deep reinforcement learning (RL) is increasingly deployed in resource-constrained environments, yet go-to function approximators - multilayer perceptrons (MLPs) - are often parameter-inefficient due to an imperfect inductive bias for the smooth structure of many value functions. This mismatch can also hinder sample efficiency and slow policy learning in this capacity-limited regime. Although model compression techniques exist, they operate post-hoc and do not improve learning efficiency. Spline-based architectures such as Kolmogorov-Arnold Networks (KANs) have been shown to offer parameter efficiency but are widely reported to exhibit significant computational overhead, especially at scale. In seeking to address these limitations, this work introduces SPAN (SPline-based Adaptive Networks) for RL. SPAN adapts the KHRONOS framework with a learnable preprocessing layer. SPAN is evaluated across discrete (PPO) and high-dimensional continuous (SAC) control tasks, offline settings (Minari/D4RL) and a real-world datacenter HVAC control application. SPAN achieves a 30-50% improvement in sample efficiency and 1.3-9 times higher success rates across benchmarks compared to MLP baselines. Despite incurring a per-step evaluation overhead of 1.2-1.8x, SPAN's superior convergence reliability yields an expected total training cost 1.3-6.3x lower than MLP baselines when accounting for convergence failures. In the HVAC application, SPAN reduces energy consumption in 9 of 12 months relative to MLP while simultaneously achieving a 1.1-3.4x reduction in thermal comfort violations across the evaluation year, demonstrating generalization to real-world engineering control. Furthermore, SPAN demonstrates superior anytime performance and robustness to hyperparameter variations, suggesting it as a viable, high-performance alternative for learning efficient policies in resource-limited settings.

Figures

Figures reproduced from arXiv: 2601.23225 by Rajib Mostakim, Reza T. Batley, Sourav Saha.

Figure 1
Figure 1. Figure 1: SPAN architecture for actor-critic reinforcement learn￾ing. Environment states, action or state-action pair pass through a preprocessing layer (Dense + Sigmoid) that normalizes inputs to [0, 1]d . Per-dimension B-spline basis expansions generate uni￾variate features, which are combined via tensor products (outer product) across dimensions to form M separable modes. A linear head produces task specific outp… view at source ↗
Figure 2
Figure 2. Figure 2: Reward curves for Classic Control and Box2D environments showing mean performance ± 1 standard deviation across 20 seeds [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Learning curves for MuJoCo continuous control environments showing mean episodic return ± one standard deviation across 20 random seeds. Hopper is a locomotion task in which a planar one legged robot must learn to hop forward efficiently while maintain￾ing balance. HalfCheetah models a planar bipedal agent with articulated joints that must learn to run forward at high speed [PITH_FULL_IMAGE:figures/full_f… view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study on Acrobot-v1 showing performance vs. nmode, nelem, and degree. 10 15 20 krank 4000 5000 6000 7000 Final Eval Return Effect of krank 2 4 6 8 kelem 3000 4000 5000 6000 7000 Final Eval Return Effect of kelem [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study on HalfCheetah-v5 showing performance vs. krank, kelem. 6. Discussion This work demonstrates that SPAN is particularly effective in reinforcement learning regimes where samples, parame￾ters, or training time are limited. Across both classic control, continuous control tasks, and high-dimensional adroit hand task SPAN consistently attains stronger policies with fewer environment interactions … view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study on CartPole-v1 showing performance vs. nmodes, nelems, and degree. D. Hardware Specification All experiments were conducted on a laptop computer with the following specifications: • Processor: Intel(R) Core(TM) i9-14900HX @ 2.20 GHz • RAM: 32.0 GB (31.7 GB usable) @ 5600 MT/s 14 [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation study on InvertedPendulum-v5 showing performance vs. nmodes, nelems, and degree. 5 7 9 11 13 nmodes 0 100 200 Final Eval Return Effect of nmodes 1 2 3 4 nelems 0 100 200 Final Eval Return Effect of nelems 1 2 3 degree 100 0 100 200 Final Eval Return Effect of degree [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation study on LunarLander-v3 showing performance vs. nmodes, nelems, and degree. 6 8 10 krank 500 1000 1500 2000 2500 3000 3500 Final Eval Return Effect of krank 4 6 8 kelem 500 1000 1500 2000 2500 3000 3500 Final Eval Return Effect of kelem [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Ablation study Hopper showing performance vs. krank and kelem • Graphics: NVIDIA GeForce RTX 5060 (8 GB VRAM) • Storage: 954 GB SSD • Operating System: Windows 11, 64-bit, x64-based processor 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 3 canonical work pages

  1. [7]

    Han, S., Mao, H., and Dally, W

    URL https://arxiv.org/abs/ 1812.05905. Han, S., Mao, H., and Dally, W. J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,

    Pith/arXiv arXiv

  2. [11]

    Hornik, K

    URL https: //arxiv.org/abs/1503.02531. Hornik, K. et al. Multilayer feedforward networks are uni- versal approximators.Neural Networks, 2(5):359–366,

    Pith/arXiv arXiv

  3. [13]

    Kostrikov, I., Nair, A., and Levine, S

    URL https: //arxiv.org/abs/2408.04841. Kostrikov, I., Nair, A., and Levine, S. Offline reinforcement learning with implicit q-learning,

    Pith/arXiv arXiv

  4. [14]

    Levine, S

    URL https: //arxiv.org/abs/2110.06169. Levine, S. et al. End-to-end training of deep visuomotor policies.Journal of Machine Learning Research, 17(39): 1–40,

    Pith/arXiv arXiv

  5. [15]

    Liu, Z., Wang, Y ., Vaidya, S., Ruehle, F., Halverson, J., Solja ˇci´c, M., Hou, T

    URL https://arxiv.org/abs/1509.02971. Liu, Z., Wang, Y ., Vaidya, S., Ruehle, F., Halverson, J., Solja ˇci´c, M., Hou, T. Y ., and Tegmark, M. Kan: Kolmogorov-arnold networks,

    Pith/arXiv arXiv

  6. [16]

    URL https:// arxiv.org/abs/2404.19756. Mao, H. et al. Resource management with deep reinforce- ment learning. InACM HotNets,

    Pith/arXiv arXiv

  7. [17]

    doi: https://doi.org/10.1016/j.ijfatigue.2025.109185

    ISSN 0142-1123. doi: https://doi.org/10.1016/j.ijfatigue.2025.109185. URL https://www.sciencedirect.com/ science/article/pii/S0142112325003822. Novikov, A., Podoprikhin, D., Osokin, A., and Vetrov, D. Tensorizing neural networks,

    arXiv 2025

  8. [18]

    Park, C., Saha, S., Guo, J., Zhang, H., Xie, X., Bessa, M

    URL https:// arxiv.org/abs/1509.06569. Park, C., Saha, S., Guo, J., Zhang, H., Xie, X., Bessa, M. A., Qian, D., Chen, W., Wanger, G. J., Cao, J., Hughes, T. J. R., and Liu, W. K. Unifying machine learning and interpolation theory via interpolating neural networks. Nature Communications, 16(1):8753,

    Pith/arXiv arXiv

  9. [20]

    org/abs/1710.05941

    URL https://arxiv. org/abs/1710.05941. Sainath, T. N., Kingsbury, B., Sindhwani, V ., Arisoy, E., and Ramabhadran, B. Low-rank matrix factorization for deep neural network training with high-dimensional output targets. In2013 IEEE international conference on acoustics, speech and signal processing, pp. 6655–6659,

    Pith/arXiv arXiv

  10. [21]

    URL https://arc.aiaa

    doi: 10.2514/6.2026-0043. URL https://arc.aiaa. org/doi/abs/10.2514/6.2026-0043. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algo- rithms,

    work page doi:10.2514/6.2026-0043 2026

  11. [22]

    Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P

    URL https://arxiv.org/abs/ 1707.06347. Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. High-dimensional continuous control using generalized advantage estimation,

    Pith/arXiv arXiv

  12. [23]

    org/abs/1506.02438

    URL https://arxiv. org/abs/1506.02438. Silver, D. et al. Mastering the game of go without human knowledge.Nature, 550:354–359,

    Pith/arXiv arXiv

  13. [24]

    U., De Cola, G., Deleu, T., Goul˜ao, M., Kallinteris, A., Krimmel, M., KG, A., et al

    Towers, M., Kwiatkowski, A., Terry, J., Balis, J. U., De Cola, G., Deleu, T., Goul˜ao, M., Kallinteris, A., Krimmel, M., KG, A., et al. Gymnasium: A standard interface for reinforcement learning environments.arXiv preprint arXiv:2407.17032,

    Pith/arXiv arXiv

  14. [1723]

    URL https: //doi.org/10.1038/s41467-025-63790-8

    doi: 10.1038/s41467-025-63790-8. URL https: //doi.org/10.1038/s41467-025-63790-8. Ramachandran, P., Zoph, B., and Le, Q. V . Searching for activation functions,

    work page doi:10.1038/s41467-025-63790-8

  15. [2015]

    Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D

    URL https:// arxiv.org/abs/1512.03385. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D. Deep reinforcement learning that mat- ters,

    Pith/arXiv arXiv

  16. [2016]

    He, K., Zhang, X., Ren, S., and Sun, J

    URL https: //arxiv.org/abs/1510.00149. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition,

    Pith/arXiv arXiv

  17. [2017]

    URL https: //arxiv.org/abs/1712.05877. Kahn, G. et al. Plato: Policy learning using adaptive trajec- tory optimization,

    Pith/arXiv arXiv

  18. [2018]

    Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V ., Zhu, H., Gupta, A., Abbeel, P., and Levine, S

    URL https: //arxiv.org/abs/1801.01290. Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V ., Zhu, H., Gupta, A., Abbeel, P., and Levine, S. Soft actor-critic algorithms and ap- plications,

    Pith/arXiv arXiv

  19. [2019]

    Fu, J., Kumar, A., Nachum, O., Tucker, G., and Levine, S

    URL https://arxiv.org/abs/1904.12901. Fu, J., Kumar, A., Nachum, O., Tucker, G., and Levine, S. D4rl: Datasets for deep data-driven reinforcement learning,

    Pith/arXiv arXiv 1904

  20. [2021]

    Guo, H., Li, F., Li, J., and Liu, H

    URL https://arxiv.org/abs/ 2004.07219. Guo, H., Li, F., Li, J., and Liu, H. Kan v.s. mlp for offline reinforcement learning,

    Pith/arXiv arXiv 2004

  21. [2023]

    Hinton, G., Vinyals, O., and Dean, J

    URL https://arxiv.org/abs/ 1606.08415. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network,

    Pith/arXiv arXiv

  22. [2024]

    org/abs/2409.09653

    URL https://arxiv. org/abs/2409.09653. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    Pith/arXiv arXiv

  23. [2025]

    Batley, R

    URL https://arxiv.org/abs/ 2505.13315. Batley, R. T. and Saha, S.A Unified Generative-Predictive Framework for Deterministic Inverse Design

    Pith/arXiv arXiv

  24. [2026]

    URL https://arc.aiaa

    doi: 10.2514/6.2026-0365. URL https://arc.aiaa. org/doi/abs/10.2514/6.2026-0365. Dulac-Arnold, G., Mankowitz, D., and Hester, T. Chal- lenges of real-world reinforcement learning,

    work page doi:10.2514/6.2026-0365 2026