Pith. sign in

REVIEW 3 major objections 5 minor 52 references

SMOSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A top-1 sparse mixture of shallow linear experts can outperform recent interpretable controllers on continuous control benchmarks and approach the performance of neural Soft Actor-Critic policies.

desk verdict SMOSE is a clean, plausible interpretable-RL architecture, but its headline outperformance claim rests on visually transcribed baselines and a suspicious duplicated table entry. read the letter →

arxiv 2412.13053 v1 pith:6WHRBU4Z submitted 2024-12-17 cs.LG cs.AI

Mátyás Vincze , Laura Ferrarotti , Leonardo Lucio Custode , Bruno Lepri , Giovanni Iacca This is my paper · ORCID
classification cs.LGcs.AI
keywords interpretablereinforcementlearningmixtureofexpertscontinuouscontrolSoftActor-CriticdecisiontreesMuJoCosparseactivationlinearpolicies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SMOSE is a policy architecture for continuous control that decomposes behavior into a small set of linear sub-policies, or experts, and a linear router that selects exactly one expert per state. The paper argues that this sparse, fully linear mixture can be trained with Soft Actor-Critic and load-balancing penalties, and that the resulting controller is both high-performing and directly interpretable. On six MuJoCo benchmarks, SMOSE outperforms the interpretable baselines CGP, LGP, and Metric-40 in five of six environments, and its mean returns land close to neural SAC policies of comparable active parameter count. The authors also distill decision trees from the router weights to make the task-allocation logic readable without sacrificing the policy's inherent transparency.

What carries the argument

The central object is the top-1 sparse mixture $\pi(s)=\sum_{m=1}^M [\mathrm{TOP1}(\mathrm{softmax}(\Theta s))]_m \, (\theta_m s)$, where each expert is a linear map from state to action and the router is also linear, so every decision is a single linear control law selected by a linear preference score. Training uses Soft Actor-Critic for exploration and adds the importance loss and load-balancing loss from sparse MoE literature, with Gaussian noise on the router logits, to prevent expert collapse and keep the experts evenly used. After training, the router is distilled into depth-limited decision trees by labeling replay-buffer states, producing a readable approximation of when each expert takes control.

What would settle it

Re-run CGP, LGP, and Metric-40 under the same MuJoCo v4 environments, ten seeds, one million training timesteps, and 100 evaluation episodes, then compare mean episodic rewards; if the transcribed baseline values are materially higher or lower than the original runs, the claimed five-of-six dominance over interpretable methods is not established.

Watch

Extended reading notes

Core claim

The central claim is that a sparse Mixture-of-Experts policy with a top-1 hard gate, linear experts $\pi_m(s)=\theta_m s$, and a linear router $\hat{g}(s|\Theta)=\Theta s$ can be trained end-to-end with SAC to reach competitive performance on continuous control tasks while remaining interpretable. The router uses $g(s|\Theta)=\mathrm{TOP1}(\mathrm{softmax}(\Theta s))$, so in any state only a single linear controller is active. This design outperforms recent interpretable baselines on Walker2d-v4, Hopper-v4, Ant-v4, HalfCheetah-v4, and Reacher-v4, and it narrows the gap to non-interpretable SAC networks of comparable size, sometimes matching or exceeding them. The paper further shows that the learned router weights can be read as scoring functions that assign each expert to a recognizable regime, and that decision trees distilled from those router scores provide an additional compact explanation layer.

Load-bearing premise

The headline comparison against interpretable baselines rests on performance numbers transcribed by eye from plots in earlier papers, which may have been produced under different environment versions, episode counts, or seeds than SMOSE's runs.

Editorial extensions

If this is right

  • SMOSE shows that a sparse combination of linear policies, with only one expert active per state, can outperform the selected interpretable baselines on five of six MuJoCo tasks.
  • With far fewer active parameters than the neural SAC baselines, SMOSE achieves returns close to SAC-M and SAC-S, narrowing the gap between interpretable and closed-box controllers.
  • The linear router weights give a state-space decomposition into identifiable skill regimes, which suggests a practical route for auditing or debugging a learned controller at a per-expert level.
  • The router distillation into decision trees provides a compact, human-readable map of when each expert is invoked, supporting verification and human oversight without needing to inspect the full weight matrices.
  • Because the architecture allows TOP k with $k>1$, the same training scheme can trade interpretability for expressiveness while keeping the policy linear and inspectable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same router-plus-linear-experts decomposition may transfer to higher-dimensional control if the router remains sparse and the experts are kept shallow, since the interpretability burden stays localized in the router's decision boundaries rather than in a monolithic network.
  • Editorial inference: the failure on Swimmer-v4, where all SAC-based policies are weak, hints that the top-1 gating structure struggles when the optimal behavior requires smooth blending of several velocity-dependent strategies; a testable extension would be to compare TOP1 against TOP2 or TOP3 on velocity-dominated environments.
  • Editorial inference: distilling decision trees from the router rather than from the whole policy creates a path toward formal verification: a bounded-depth tree over linear thresholds could be model-checked or converted to rules for certification, which the paper only gestures at by citing imitation-learning extensions as future work.
  • Editorial inference: one could replace the linear experts with other interpretable classes, such as single-step lookahead or proportional-derivative controllers, and keep the same training pipeline, which would test whether linearity or modularity is the main driver of the observed performance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SMOSE, an interpretable reinforcement learning policy architecture in which a sparse top-1 mixture of linear experts is selected by a linear router, trained end-to-end with SAC and load-balancing losses, and followed by distillation of decision trees from the router. The method is evaluated on six MuJoCo continuous control tasks. The authors report that SMOSE outperforms the interpretable baselines CGP, LGP, and Metric-40 in five of six environments, and that its performance is close to neural SAC policies of comparable size. The interpretability of the learned policies is demonstrated through per-expert weight visualizations and decision trees for Reacher-v4, with additional interpretations for the other environments in the appendix.

Significance. If the reported results are correct, SMOSE would be a valuable contribution to interpretable RL: a simple, trainable top-1 mixture-of-linear-experts policy that narrows the gap to neural policies while remaining directly interpretable through its linear router weights. The paper's strengths include a 10-seed evaluation protocol, a dedicated ablation on the number of experts, parameter-count-aware comparisons with SAC variants, and unusually detailed qualitative interpretations for all six environments. However, the central outperformance claim is contingent on baseline numbers that were visually transcribed from plots in other papers, and no code or data is provided for independent verification. The absence of a fidelity measure for the distilled decision trees also leaves part of the interpretability contribution unsupported.

major comments (3)
  1. [Table 1, Policy evaluation] The paper's headline claim that SMOSE 'consistently outperforms its competitors' among interpretable baselines rests on CGP and LGP values that the authors state were 'visually derived from the plots reported in the original papers.' Plot transcription is imprecise, and the original papers may have used different environment versions (e.g., v2/v3 vs. v4), different episode horizons, or different evaluation protocols, which would make the absolute rewards non-comparable even with perfect transcription. Moreover, Table 1 contains a clear internal anomaly: the Metric-40 entry is identical (2210.50 ± 175.50) for both Ant-v4 and HalfCheetah-v4, which is either a copy-paste error or an extremely unlikely coincidence. Since the comparison to CGP/LGP/Metric-40 is the empirical basis for the main contribution, the authors should re-run these baselines under their own protocol (or otherwise obtain trustworthy numbers with confirmed environment versions) and correct Table 1. Without this, the primary outperformance claim is not established.
  2. [Method, Eqs. (3)-(4)] The load-balancing losses in Eqs. (3) and (4) use π_m(s|θ_m, σ_m) as if it were a scalar router logit: Eq. (3) applies softmax to it, and Eq. (4) compares it to τ(s_k)=max_m π_m(...). However, π_m is defined just above as a Gaussian distribution N(θ_m·s, σ_m^2). As written, softmax of a Gaussian distribution and the threshold comparison in the load term are not well-defined, so the training objective cannot be reproduced from the text. The authors should clarify whether ĝ(s|Θ), the inner router output, or the expert mean θ_m·s is intended, and rewrite the equations accordingly.
  3. [Decision Trees as a support for interpretation] The paper distills binary decision trees from the learned router and presents these trees as a tool that 'significantly improves the ease of interpretation.' However, no fidelity metric is reported: the authors do not measure how often the depth-limited DTs agree with the continuous router, nor the performance impact of replacing the router with the DTs. Since the trees are post-hoc surrogates, the reader cannot tell whether the depicted splits faithfully represent the router's actual decisions. Please report at least one agreement/fidelity metric (e.g., percentage of states where the DT selects the same expert as the router, or the average return when the DT router is deployed), or moderate the claim accordingly.
minor comments (5)
  1. [Abstract / Results] The abstract says SMOSE 'outperforms recent interpretable baselines' without the caveat that appears in the Results section, where Table 1 shows SMOSE is far worse than CGP/LGP on Swimmer-v4 (45.40 vs. 280.00/278.50). Please either add the caveat in the abstract or weaken the wording.
  2. [Figure 2 caption] The caption reads 'SMoSE' but the method is consistently spelled 'SMOSE' elsewhere; please unify the spelling.
  3. [Figures 7-14] The decision-tree figures for Reacher-v4 are extremely low-resolution and essentially unreadable in the manuscript; they also appear to contain 'Powered by TCPDF' artifacts. Please re-render these figures at higher quality.
  4. [Appendix, Decision Trees] The notation in the distillation description is inconsistent: the text says 'DT_m(Θ_m·s)=1' but the discussion of inputs and labels could be clarified, particularly whether the trees are trained on raw states s or on the projected router scores Θ_m·s. Please state this explicitly.
  5. [Table 1 footnote] The footnote says 'magenta = best score per environment,' but the table appears monochrome in the submitted PDF; please use boldface or another visible marker.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SMOSE's performance claims rest on environment reward evaluation of a learned policy, not on fitted quantities reinserted into a derivation.

full rationale

The paper's derivation chain is: define a sparse top-1 MoE policy with linear experts and a linear router (Eqs. 1-2), train it with SAC plus load-balancing losses (Eqs. 3-4), and evaluate the resulting deterministic policy on MuJoCo environments. None of these stages equates a prediction to an input by construction. The router and expert weights are optimized against environment rewards; the reported episodic rewards are measured outcomes, not re-fitted parameters. The decision trees distilled from the router are explicitly post-hoc interpretation aids: they are trained on router-labeled replay-buffer states and are not used to produce actions or to compute any performance number, so they cannot feed back into the results. The load-balancing losses are adopted from external prior work (Riquelme et al. 2021; Shazeer et al. 2017) as standard regularization, not as a source of the claimed predictive content. The comparisons with CGP/LGP use numbers that are 'visually derived from the plots reported in the original papers' (Table 1 footnote), and Table 2 contains a suspicious duplicated Metric-40 entry; these are evidence-quality and reproducibility concerns, not circularity, because the baseline values are external inputs rather than outputs of this paper's own fitted model. Self-citations (Custode and Iacca 2021, 2023, 2024) appear only as related-work context and do not carry any load-bearing assumption in the method or evaluation. No step reduces to a self-citation chain, and no 'prediction' is a renamed fit. The central outperform-claim could fail on baseline-transcription or protocol-comparison grounds, but that is a correctness risk, not a circularity risk.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on training hyperparameters (M, lambda, DT depth) chosen by the authors, on the MuJoCo simulator as ground truth, on SAC as the learning algorithm, and on an unverified assumption that the distilled depth-3 decision trees represent the router well. No new physical entities are postulated.

free parameters (3)
  • M (number of experts) = 8
    Chosen empirically from ablation on the same six MuJoCo environments to balance performance and interpretability (Table 5); larger M improves performance in most environments.
  • lambda (load-balancing loss weight) = 0.1
    Tuned on a three-dimensional grid between 0.01 and 1.0 to ensure fair expert usage and reduce expert collapse.
  • DT maximum depth = 3
    Set for the distilled decision trees (Figures 7-14); no fidelity metric is reported to justify this depth.
assumptions (5)
  • domain assumption MuJoCo benchmarks are valid performance proxies for continuous control.
    The paper evaluates on six MuJoCo environments as representative continuous control tasks, which is standard practice in RL.
  • domain assumption Soft Actor-Critic with the listed hyperparameters produces well-performing policies for comparison and for SMOSE.
    SAC is used as the base RL algorithm and its hyperparameters are taken from CleanRL without tuning, assuming they are appropriate for these tasks.
  • ad hoc to paper Top-1 hard routing is sufficient to represent good control policies on these tasks.
    The method intentionally sets k=1 to maximize interpretability, and the paper does not compare against top-k with k>1 in the main results.
  • ad hoc to paper CART decision trees of depth 3 faithfully approximate the learned linear router for interpretation purposes.
    The paper distills depth-3 decision trees but reports no accuracy or fidelity measure of these trees against the router they approximate.
  • domain assumption Load-balancing losses from Riquelme et al. 2021 transfer to the linear-expert setting and do not hurt policy quality.
    The importance and load-balancing losses are adopted from sparse MoE literature without an analysis of their effect in the shallow linear setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SMOSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks." pith.science (2026). https://pith.science/paper/6WHRBU4Z

@misc{pith2026241213053,
  author       = {Pith},
  title        = {Pith review of: SMOSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6WHRBU4Z}},
  note         = {Machine review of arXiv:2412.13053}
}
read the original abstract

Continuous control tasks often involve high-dimensional, dynamic, and non-linear environments. State-of-the-art performance in these tasks is achieved through complex closed-box policies that are effective, but suffer from an inherent opacity. Interpretable policies, while generally underperforming compared to their closed-box counterparts, advantageously facilitate transparent decision-making within automated systems. Hence, their usage is often essential for diagnosing and mitigating errors, supporting ethical and legal accountability, and fostering trust among stakeholders. In this paper, we propose SMOSE, a novel method to train sparsely activated interpretable controllers, based on a top-1 Mixture-of-Experts architecture. SMOSE combines a set of interpretable decisionmakers, trained to be experts in different basic skills, and an interpretable router that assigns tasks among the experts. The training is carried out via state-of-the-art Reinforcement Learning algorithms, exploiting load-balancing techniques to ensure fair expert usage. We then distill decision trees from the weights of the router, significantly improving the ease of interpretation. We evaluate SMOSE on six benchmark environments from MuJoCo: our method outperforms recent interpretable baselines and narrows the gap with noninterpretable state-of-the-art algorithms

Figures

Figures reproduced from arXiv: 2412.13053 by the authors.

Figure 1
Figure 1. SMOSE. Schematic summary of the proposed architecture. ples (S, A,P, R, γ, S0), where S is the set of the states in the problem, A ∈ R na is the set of (continuous) ac￾tions, P(s, a, s′ ) : S × A × S → [ 0, 1 ] associates a probability to each transition from (s, a) to each state s ′ ; R(s, a, s′ ) : S × A × S → R + assigns a reward to each triplet (s, a, s′ ); γ is a discount factor, used for denoting the importanc… view at source ↗
Figure 2
Figure 2. Performance in training. SMOSE compares to non-interpretable models of the same size, considering the number of overall (SAC-M) and active (SAC-S) parameters. Swimmer-v4 (described in Appendix), which are commonly used as benchmarks in the RL field. The following sub￾sections detail the experimental setup, performance eval￾uation, and comparative analysis with interpretable and non-interpretable baselines, highlight… view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (11 more)
Figure 5
Figure 5. Figure 5: Visual representation of the Hopper-v4 and [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 4
Figure 4. Figure 4: Visual representation of the Reacher-v4 and [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 6
Figure 6. Figure 6: Visual representation of the HalfCheetah-v4 and [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 11
Figure 11. Figure 11: Reacher-v4. Decision tree for Expert 5. F T NO YES NO YES Expert 6 NO NO YES Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 9
Figure 9. Figure 9: Reacher-v4. Decision tree for Expert 3. F T YES NO YES Expert 4 NO NO YES Powered by TCPDF (www.tcpdf.org) Powered by TCPDF (www.tcpdf.org) [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Reacher-v4. Decision tree for Expert 4. Additional policy interpretations In this section, we will interpret the best solutions ob￾tained for the Walker2d-v4, Hopper-v4, Swimmer-v4, HalfCheetah-v4, and Ant-v4 environments. Walker2d-v4 In this environment, the policy m…
Figure 15
Figure 15. Figure 15: Walker2d-v4. Visual representation of the learned weights for each expert and of the corresponding column of the router’s weight matrix. • right thigh’s torque: τrt • right leg’s torque: τrl • right foot’s torque: τrf • left thigh’s torque: τlt • left leg’s torque: τl…
Figure 16
Figure 16. Figure 16: Hopper-v4. Visual representation of the learned weights for each expert and of the corresponding column of the router’s weight matrix. The thigh rotor (τth) aims to reduce the torso’s angle by moving the thigh rotor (which is connected to the torso) ex￾ploiting a larg…
Figure 17
Figure 17. Figure 17: Swimmer-v4. Visual representation of the learned weights for each expert and of the corresponding column of the router’s weight matrix. the first joint itself. The torque τ1 is positively influenced by the angle at that joint. This suggests that when θ1 increases, τ1 …
Figure 18
Figure 18. Figure 18: HalfCheetah-v4. Visual representation of the learned weights for each expert and of the corresponding column of the router’s weight matrix. • angular velocity of the angle between back right links: ωbr2 and has to control the following variables: • Torque applied on t…
Figure 19
Figure 19. Figure 19: Ant-v4. Visual representation of the learned weights for each expert and of the corresponding column of the router’s weight matrix [PITH_FULL_IMAGE:figures/full_fig_p025_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 29 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Akrour, R.; Tateo, D.; and Peters, J. 2021. Continuous action reinforcement learning from a mixture of interpretable experts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10): 6795--6806

    work page 2021

  4. [4]

    B.; D \' az-Rodr \' guez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garc \' a, S.; Gil-L \'o pez, S.; Molina, D.; Benjamins, R.; et al

    Arrieta, A. B.; D \' az-Rodr \' guez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garc \' a, S.; Gil-L \'o pez, S.; Molina, D.; Benjamins, R.; et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion, 58: 82--115

    2020

  5. [5]

    Bastani, O.; Pu, Y.; and Solar-Lezama, A. 2019. Verifiable Reinforcement Learning via Policy Extraction. arXiv:1805.08328

    arXiv 2019

  6. [6]

    A Comparative Study of Faithfulness Metrics for Model Interpretability Methods

    Chan, C. S.; Kong, H.; and Liang, G. 2022. A Comparative Study of Faithfulness Metrics for Model Interpretability Methods. arXiv:2204.05514

    work page Pith review arXiv 2022

  7. [7]

    Cheng, G.; Dong, L.; Cai, W.; and Sun, C. 2023. Multi-task reinforcement learning with attention-based mixture of experts. IEEE Robotics and Automation Letters, 8(6): 3812--3819

    work page 2023

  8. [8]

    L.; and Iacca, G

    Custode, L. L.; and Iacca, G. 2021. A co-evolutionary approach to interpretable reinforcement learning in environments with continuous action spaces. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), 1--8. IEEE

    work page 2021

Show all 52 references
  1. [9]

    L.; and Iacca, G

    Custode, L. L.; and Iacca, G. 2023. Evolutionary Learning of Interpretable Decision Trees. IEEE Access, 11: 6169--6184

    2023

  2. [10]

    L.; and Iacca, G

    Custode, L. L.; and Iacca, G. 2024. Social Interpretable Reinforcement Learning. arXiv preprint arXiv: 2401.15480

    2024 arXiv

  3. [11]

    Delfosse, Q.; Shindo, H.; Dhami, D.; and Kersting, K. 2023. Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction. arXiv:2306.01439

    2023 arXiv

  4. [12]

    Delfosse, Q.; Sztwiertnia, S.; Rothermel, M.; Stammer, W.; and Kersting, K. 2024. Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents. arXiv:2401.05821

    2024 arXiv

  5. [13]

    Dhebar, Y.; Deb, K.; Nageshrao, S.; Zhu, L.; and Filev, D. 2020. Interpretable- AI Policies using Evolutionary Nonlinear Decision Trees for Discrete Action Systems . arXiv preprint arXiv:2009.09521

    2020 arXiv

  6. [14]

    Dwivedi, R.; Dave, D.; Naik, H.; Singhal, S.; Omer, R.; Patel, P.; Qian, B.; Wen, Z.; Shah, T.; Morgan, G.; et al. 2023. Explainable AI (XAI): Core ideas, techniques, and solutions. ACM Computing Surveys, 55(9): 1--33

    2023

  7. [15]

    Glanois, C.; Weng, P.; Zimmer, M.; Li, D.; Yang, T.; Hao, J.; and Liu, W. 2024. A survey on interpretable reinforcement learning. Machine Learning, 1--44

    2024

  8. [16]

    H.; Kovach, T.; Miller, K.; and Dubrawski, A

    Good, J. H.; Kovach, T.; Miller, K.; and Dubrawski, A. 2023. Feature Learning for Interpretable, Performant Decision Trees. In Advances in Neural Information Processing Systems

    2023

  9. [17]

    Haarnoja, T.; Zhou, A.; Abbeel, P.; and Levine, S. 2018. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. CoRR, abs/1801.01290

    2018 arXiv

  10. [18]

    He, S. 2021. Who is liable for the UBER self-driving crash? Analysis of the liability allocation and the regulatory model for autonomous vehicles. Autonomous Vehicles: Business, Technology and Law, 93--111

    2021

  11. [19]

    Huang, S.; Dossa, R. F. J.; Ye, C.; Braga, J.; Chakraborty, D.; Mehta, K.; and Ara \ A s jo, J. G. 2022. Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research, 23(274): 1--18

    2022

  12. [20]

    Huang, X.; Kroening, D.; Ruan, W.; Sharp, J.; Sun, Y.; Thamo, E.; Wu, M.; and Yi, X. 2020. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability. Computer Science Review, 37: 100270

    2020

  13. [21]

    T.; and Alpayd n, E

    Irsoy, O.; Y ld z, O. T.; and Alpayd n, E. 2012. Soft decision trees. In International Conference on Pattern Recognition, 1819--1822. IEEE

    2012

  14. [22]

    Jiang, Z.; and Luo, S. 2019. Neural Logic Reinforcement Learning. arXiv:1904.10729

    2019 arXiv

  15. [23]

    J.; Tatsubori, M.; Munawar, A.; and Gray, A

    Kimura, D.; Ono, M.; Chaudhury, S.; Kohita, R.; Wachi, A.; Agravante, D. J.; Tatsubori, M.; Munawar, A.; and Gray, A. 2021. Neuro-Symbolic Reinforcement Learning with First-Order Logic. arXiv:2110.10963

    2021 arXiv

  16. [24]

    Kohler, H.; Delfosse, Q.; Akrour, R.; Kersting, K.; and Preux, P. 2024. Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning. arXiv:2405.14956

    2024 arXiv

  17. [25]

    Liu, G.-T.; Hu, E.-P.; Cheng, P.-J.; yi Lee, H.; and Sun, S.-H. 2023. Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs. arXiv:2301.12950

    2023 arXiv

  18. [26]

    McCallum, A. K. 1996. Reinforcement learning with selective perception and hidden state. University of Rochester

    1996

  19. [27]

    McGough, M. 2018. How bad is Sacramento’s air, exactly? Google results appear at odds with reality, some say. Sacramento Bee, 7

    2018

  20. [28]

    Nadizar, G.; Medvet, E.; and Wilson, D. G. 2024. Naturally Interpretable Control Policies via Graph-Based Genetic Programming. In European Conference on Genetic Programming (Part of EvoStar), 73--89. Springer

    2024

  21. [29]

    K.; Precup, D.; and Castro, P

    Obando-Ceron, J.; Sokar, G.; Willi, T.; Lyle, C.; Farebrother, J.; Foerster, J.; Dziugaite, G. K.; Precup, D.; and Castro, P. S. 2024. Mixtures of experts unlock parameter scaling for deep rl. arXiv preprint arXiv:2402.08609

    2024 arXiv

  22. [30]

    C.; Tseng, H

    Paleja, R.; Chen, L.; Niu, Y.; Silva, A.; Li, Z.; Zhang, S.; Ritchie, C.; Choi, S.; Chang, K. C.; Tseng, H. E.; et al. 2023. Interpretable Reinforcement Learning for Robotics and Continuous Control. arXiv preprint arXiv:2311.10041

    2023 arXiv

  23. [31]

    D.; Howe, A

    Pyeatt, L. D.; Howe, A. E.; et al. 2001. Decision tree function approximation in reinforcement learning. In International Symposium on Adaptive Systems: Evolutionary Computation and Probabilistic Graphical Models, volume 2, 70--77

    2001

  24. [32]

    Ren, J.; Li, Y.; Ding, Z.; Pan, W.; and Dong, H. 2021. Probabilistic mixture-of-experts for efficient deep reinforcement learning. arXiv preprint arXiv:2104.09122

    2021 arXiv

  25. [33]

    S.; Keysers, D.; and Houlsby, N

    Riquelme, C.; Puigcerver, J.; Mustafa, B.; Neumann, M.; Jenatton, R.; Pinto, A. S.; Keysers, D.; and Houlsby, N. 2021. Scaling Vision with Sparse Mixture of Experts. CoRR, abs/2106.05974

    2021 arXiv

  26. [34]

    J.; and Bagnell, J

    Ross, S.; Gordon, G. J.; and Bagnell, J. A. 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. arXiv:1011.0686

    2011 arXiv

  27. [35]

    M.; Topin, N.; Jamshidi, P.; and Veloso, M

    Roth, A. M.; Topin, N.; Jamshidi, P.; and Veloso, M. 2019. Conservative q-improvement: Reinforcement learning for an interpretable decision-tree policy. arXiv preprint arXiv:1907.01180

    2019 arXiv

  28. [36]

    Rudin, C. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5): 206--215

    2019

  29. [37]

    Rudin, C.; Chen, C.; Chen, Z.; Huang, H.; Semenova, L.; and Zhong, C. 2021. Interpretable Machine Learning : Fundamental Principles and 10 Grand Challenges . arXiv preprint arXiv:2103.11251

    2021 arXiv

  30. [38]

    Rudin, C.; Wang, C.; and Coker, B. 2019. The age of secrecy and unfairness in recidivism prediction. arXiv preprint arXiv:1811.00731

    2019 arXiv

  31. [39]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  32. [40]

    Sha, J.; Shindo, H.; Delfosse, Q.; Kersting, K.; and Dhami, D. S. 2024. EXPIL: Explanatory Predicate Invention for Learning in Games. arXiv:2406.06107

    2024 arXiv

  33. [41]

    Shazeer, N.; Mirhoseini, A.; Maziarz, K.; Davis, A.; Le, Q.; Hinton, G.; and Dean, J. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. arXiv preprint arXiv:1701.06538

    2017 arXiv

  34. [42]

    Silva, A.; Killian, T.; Rodriguez, I. D. J.; Son, S.-H.; and Gombolay, M. 2020. Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learning . In International Conference on Artificial Intelligence and Statistics , 1855--1865. PMLR

    2020

  35. [43]

    Smyth, J.; Ulahannan, A.; Florek, F.; Shaw, E.; and Mansfield, N. 2021. Understanding misuse of partially automated vehicles--A discussion of NTSB’s findings of the 2018 mountain view Tesla crash . Technical report, Chartered Institute of Ergonomics and Human Factors (CIEHF)

    2021

  36. [44]

    M.; Kosut, R

    Tao, K. M.; Kosut, R. L.; and Aral, G. 1994. Learning feedforward control. In American Control Conference, volume 3, 2575--2579. IEEE

    1994

  37. [45]

    Timofeev, R. 2004. Classification and regression trees (CART) theory and applications. Humboldt University, Berlin, 54: 48

    2004

  38. [46]

    Todorov, E.; Erez, T.; and Tassa, Y. 2012. Mujoco: A physics engine for model-based control. In International Conference on Intelligent Robots and Systems, 5026--5033. IEEE

    2012

  39. [47]

    R.; and Alemzadeh, H

    Varshney, K. R.; and Alemzadeh, H. 2017. On the safety of machine learning: Cyber-physical systems, decision sciences, and data products. Big Data, 5(3): 246--255

    2017

  40. [48]

    M.; Yue, Y.; and Chaudhuri, S

    Verma, A.; Le, H. M.; Yue, Y.; and Chaudhuri, S. 2021. Imitation-Projected Programmatic Reinforcement Learning. arXiv:1907.05431

    2021 arXiv

  41. [49]

    Verma, A.; Murali, V.; Singh, R.; Kohli, P.; and Chaudhuri, S. 2019. Programmatically Interpretable Reinforcement Learning. arXiv:1804.02477

    2019 arXiv

  42. [50]

    Videau, M.; Leite, A.; Teytaud, O.; and Schoenauer, M. 2022. Multi-objective genetic programming for explainable reinforcement learning. In European Conference on Genetic Programming (Part of EvoStar), 278--293. Springer

    2022

  43. [51]

    Wexler, R. 2017. When a computer program keeps you in jail. The New York Times, 13: 1

    2017

  44. [52]

    Willi, T.; Obando-Ceron, J.; Foerster, J.; Dziugaite, K.; and Castro, P. S. 2024. Mixture of Experts in a Mixture of RL settings. arXiv:2406.18420

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.