Pith. sign in

REVIEW 3 major objections 4 minor 87 references

When to Plan: Learning to Select Between Reactive Control and Deliberative Planning

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reinforcement-learned meta-policy can decide when to plan by watching a cheap uncertainty signal, and it beats fixed strategies that always react or always plan.

desk verdict Competent empirical study of RL-based arbitration between reactive control and planning; the mechanism claim is plausible but the uncertainty signal needs calibration before the joint-training result can be taken at face value. read the letter →

arxiv 2607.16421 v1 pith:DTZBFXYX submitted 2026-07-17 cs.AI

Adam Labiosa , Josiah P. Hanna This is my paper
classification cs.AI
keywords meta-reasoningadaptivecomputationreinforcementlearninguncertaintyestimationensembledisagreementplanningreactivecontrolcomputeallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks when an agent should act instantly from a learned reactive policy rather than spend time planning, and claims an RL-trained meta-policy can answer that question by watching a cheap uncertainty signal. The meta-policy conditions on an ensemble-disagreement score from the reactive policy, the distance to the goal, and a short history, then chooses between one reactive step and plans of different horizons. Across five motion-planning and navigation domains, the learned controller achieves higher average return than fixed baselines that always react or always plan, measured by time to reach the goal. Ablations show that the uncertainty score and the observation history are the load-bearing inputs. The paper also demonstrates that when the reactive policy improves during joint training, the meta-policy tracks that improvement and shifts toward fully reactive control.

What carries the argument

The carrying mechanism is the meta-MDP with options: a meta-policy selects among a one-step reactive option and three planning options of fixed horizon, each converted to a common time-based reward (a cost of -1 per action, plus -αk for a plan of length k). The meta-observation is built from the reactive policy's ensemble-disagreement score u(s)—variance across an ensemble of M behavior-cloned networks for continuous actions, or entropy of the averaged categorical distribution for discrete actions—plus distance to goal, previous meta-action, and a history of recent observations. This uncertainty score is what lets the meta-policy estimate reactive competence without search and without the ra

What would settle it

Compute, on held-out tasks in any of the five environments, the rank correlation between the ensemble-disagreement score and the suboptimality gap V*(s) - V^{πr}(s). If the correlation is weak or negative, the mechanism the paper attributes to uncertainty is not operating. Alternatively, replacing u(s) with a random scalar in the meta-observation and showing performance barely changes would falsify the claim that the uncertainty signal carries the adaptive behavior.

Watch

Extended reading notes

Core claim

The central claim is that a model-free RL meta-policy can learn adaptive computation allocation—deciding at each moment whether to emit a fast reactive action or invoke a slower planner—using only a task-agnostic uncertainty signal from the reactive policy together with distance-to-goal, previous action, and a short history. The uncertainty signal is the disagreement of an ensemble of behavior-cloned networks: variance of predicted action means for continuous control, entropy of the averaged action distribution for discrete control. On all five environments the learned meta-policy achieves higher average return than always-reactive, always-short-plan, always-medium-plan, and always-long-plan

Load-bearing premise

The load-bearing premise is that the ensemble-disagreement score u(s) is a valid, task-agnostic proxy for when the reactive policy is incompetent; the paper does not calibrate it against actual reactive value loss, so if high disagreement does not align with out-of-distribution states where planning helps, the meta-policy's advantage should not transfer.

Editorial extensions

If this is right

  • If the central claim holds, autonomous systems can deploy a fast learned policy most of the time and reserve expensive planning for the states where it matters, reducing average time-to-goal without sacrificing reliability.
  • Because the meta-observation is computed without search, the method imposes only a small overhead on the reactive baseline.
  • The joint-training result implies that planning can be used as a data generator to improve the reactive policy, and the meta-policy will automatically reallocate computation as competence grows.
  • The same framing applies wherever a cheap policy and an expensive deliberative routine coexist, including language-model reasoning and cloud-robot control, since the tradeoff is time versus quality.
  • The ablations imply that a history of uncertainty and progress is needed, not just instantaneous values, so meta-decisions are made from short trends rather than a single snapshot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the method assumes the planner is reliable everywhere; with a learned world model, the planner's own uncertainty would need to enter the meta-observation, and the same ensemble trick could be applied to the world model.
  • Editorial inference: because the uncertainty score is only a proxy, tasks with high aleatoric noise but good reactive actions might waste compute planning; adding a calibration step against actual reactive returns would test whether the score can be trusted beyond these five domains.
  • Editorial inference: the joint-training loop suggests a self-improvement curriculum—planning in out-of-distribution states generates the very data that makes planning unnecessary—which could be studied as a general mechanism for reducing deliberation over an agent's lifetime.
  • Editorial inference: the paper leaves open whether the meta-policy can track a worsening reactive policy; capacity limits or catastrophic forgetting are plausible failure modes that would require the meta-policy to relearn when to plan.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies meta-reasoning in goal-reaching tasks: a PPO-trained meta-policy chooses at each meta-step between a fast reactive policy and several planning options of different horizons, with planning incurring a time cost. The meta-observation is the ensemble-disagreement uncertainty of the reactive policy, distance to goal, previous meta-action, and a short history of these features. Experiments on five motion-planning/navigation domains compare against fixed-compute baselines (always react, always short/medium/long plan), ablate observation components, vary planning cost, in-distribution proportion, and stochasticity, and finally study joint training in which the reactive policy is trained online from planner rollouts. The paper reports that the meta-policy outperforms all fixed baselines in all five domains, that uncertainty and history are the most critical observation components, and that an uncertainty-conditioned meta-policy shifts toward fully reactive control as the reactive policy improves.

Significance. If the results hold, this is a useful empirical contribution to adaptive computation allocation. The options-MDP formulation with an explicit time-based planning cost is clean, and the experimental protocol is solid: 30 seeds, 300 evaluation episodes per seed, bootstrap confidence intervals, ablations, and sensitivity sweep. The central novelty—an RL meta-policy conditioned on an ensemble-disagreement uncertainty score—is timely and relevant to the broader compute-allocation literature. The principal limitations are the unvalidated link between ensemble disagreement and reactive-policy incompetence, and the use of a perfect world model; both are acknowledged or implicit in the text but need to be addressed before the mechanism claims can be accepted at face value.

major comments (3)
  1. [Section 5, Eqs. for u(s); Section 6.6] The load-bearing premise is that the ensemble-disagreement score u(s) is a valid, task-agnostic proxy for reactive-policy incompetence. The paper asserts that 'high uncertainty corresponds to out-of-distribution inputs where planning may be beneficial,' but it never calibrates u(s) against the actual advantage of planning, e.g., V*(s)-V^{pi_r}(s) or the expected return difference between reactive and planning options. This is especially important because the central claim attributes the meta-policy's success to conditioning on this signal. Please add a calibration analysis (correlation or ranking of u(s) against measured planning advantage on held-out ID/OOD states) and, ideally, an oracle-ablation where the meta-policy receives the true competence signal, to establish that the uncertainty score is adding the intended information rather than a generic novelty signal.
  2. [Section 6.4, Table 3] The summary 'the two most important components are history and uncertainty scores' is not supported by the reported numbers. In Doorkey, removing distance causes the largest degradation (-34.3 vs -25.1), while removing history has almost no effect (-26.1). In Box Push, removing distance improves return (-20.9 vs -24.3), contradicting the idea that distance is a helpful component. In Maze, removing uncertainty degrades from -19.3 to -25.0, which is substantial, yet the text says there is no substantial drop-off without uncertainty in the 'other three environments.' Please re-analyze Table 3 with per-environment pairwise comparisons and correct the text; the current selective summary misrepresents which observation components matter and in which environments.
  3. [Section 6.6, Figure 4] The joint-training experiment is confounded with the training procedure. The reactive policy is trained only on state-action pairs collected from planning options, so ensemble disagreement is mechanically reduced as those specific states are fit, regardless of whether the reactive policy has actually become competent in a broader sense. The paper concludes that the uncertainty signal 'reflects the strength of the reactive policy,' but the experiment as reported cannot separate genuine competence improvement from distribution fitting. Please add a control condition—for example, training the same reactive policy on an equal number of ID-only or random states—and report the reactive policy's actual success/value on held-out ID and OOD states. This would clarify whether the meta-policy is tracking true competence or merely a decreasing ensemble-variance artifact.
minor comments (4)
  1. [Section 6.1/Table 4] The text says planning horizons are 15%, 30%, and 50% of the task horizon, but the table values correspond to approximately 16-17%, 32-33%, and 50% for most environments. Please say 'approximately' or reconcile the fractions.
  2. [Table 1 caption] The caption states that bold entries overlap in CI with the best performer, but no boldface is visible in the provided text. Please render the bold in the final version.
  3. [Section 6.4] The phrase 'model-free uncertainty' is confusing; the uncertainty score comes from an ensemble, not from model-free RL. Consider renaming to 'reactive-policy ensemble uncertainty'.
  4. [Section 6.5] In the stochasticity experiment, the text says the ratio of plan lengths remains roughly constant as alpha increases, but the accompanying figure shows some shifts; a quantitative statement or per-environment figure would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: results are empirical comparisons against external fixed-compute baselines; the uncertainty proxy is an assumption, not a circular reduction.

full rationale

The paper's central claims are established by direct RL training and evaluation against fixed-compute baselines (Table 1), not by fitting parameters to the target and then 'predicting' it. The uncertainty score u(s) is an input feature computed from an ensemble of reactive networks; the load-bearing premise that high ensemble disagreement indicates states where planning helps is an explicit design assumption (Section 5), supported by ablations (Table 3), not a circular reduction. In the joint-training experiment (Section 6.6), the reactive policy is trained on planner rollouts, so a drop in ensemble disagreement is expected; however, the meta-policy's shift toward reactive control is a learned behavior under the RL reward, not an identity forced by the definition of u(s). The only self-citation (Hanna & Corrado 2025) appears in Related Work and is not load-bearing. Limitations such as the perfect world model assumption are external-validity concerns, not circularity. No equation-level reduction between input and output was found.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-picked cost and horizon parameters, a perfect world model, an engineered in-distribution/out-of-distribution competence gap, and an uncalibrated ensemble-disagreement uncertainty signal. No new physical or conceptual entities are introduced beyond standard constructs (meta-policy, options, uncertainty score). The reader must grant these assumptions for the results to transfer beyond the five simulated environments.

free parameters (4)
  • Plan cost scaling alpha = 0.5 (default), swept over {0.1, 0.5, 1, 5}
    Chosen by hand in Section 6.2 to make planning neither trivially cheap nor prohibitively expensive. The learned reactive/planning balance is directly controlled by this parameter.
  • Planning horizons as fractions of task horizon = 15% / 30% / 50% of task horizon (e.g., 5/10/15 for Box Push)
    Hand-chosen in Section 6.2 to span low, medium, and high compute. The authors explicitly leave sensitivity to these values for future work, yet the meta-policy behavior depends on them.
  • Ensemble size M = 4
    All experiments use M=4 reactive networks; the uncertainty score is computed from this ensemble and is a core input to the meta-policy.
  • Meta-observation history length n = 4
    Set in Section 6.2. The ablation shows history is one of the most critical components, so the choice of n affects the central results.
assumptions (4)
  • domain assumption A perfect, accurate world model is available for planning.
    Section 6.2: 'Planning is performed using a given accurate world model which isolates the compute allocation problem from the model learning problem.' Acknowledged in Section 7 as unrealistic in most real-world settings.
  • domain assumption The reactive policy is near-optimal on in-distribution states and poor on out-of-distribution states.
    This competence gap is engineered by behavior cloning from planner trajectories on S_ID (Section 4). The entire value of planning in the setup depends on this gap.
  • domain assumption Ensemble disagreement (variance or entropy) is a valid proxy for reactive-policy incompetence.
    Section 5 defines u(s) from deep-ensemble disagreement and asserts that high uncertainty corresponds to out-of-distribution inputs where planning may be beneficial. The paper does not calibrate u(s) against reactive value loss.
  • domain assumption A* with a fixed search-depth limit returns a near-optimal action sequence for the chosen horizon.
    Section 6.2 uses depth-limited A* as the planner. The meta-policy's planning options are only as good as these approximations, and no error analysis of the depth-limited plans is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When to Plan: Learning to Select Between Reactive Control and Deliberative Planning." pith.science (2026). https://pith.science/paper/DTZBFXYX

@misc{pith2026260716421,
  author       = {Pith},
  title        = {Pith review of: When to Plan: Learning to Select Between Reactive Control and Deliberative Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DTZBFXYX}},
  note         = {Machine review of arXiv:2607.16421}
}
read the original abstract

It has long been recognized that humans have the ability to switch between fast, reactive decision-making and slower, deliberative planning. In this paper, we study the question of how to learn this ability, known as meta-reasoning, in artificial agents. We model reactive decision-making as a policy that directly maps state observations to actions. Such policies can be trained with reinforcement learning (RL) or imitation learning, but may generalize poorly outside of their training distribution. Alternatively, model-based decision-time planning is more likely to produce good actions across a broader set of states but requires additional computation time, which delays acting. In this work, we introduce an RL method for training a meta-reasoning policy that allocates computation by conditioning on a reactive-policy uncertainty score. This score enables it to predict when the reactive policy is likely to perform poorly and when planning is needed. We conduct an empirical study on motion planning and navigation environments, showing that this design enables the meta-reasoning policy to learn when the reactive policy provides a good-enough action versus when decision-time planning is needed. Additionally, we show that our design enables the meta-agent to shift toward fully reactive control as the reactive policy improves.

Figures

Figures reproduced from arXiv: 2607.16421 by the authors.

Figure 1
Figure 1. Example images of our environment suite. Environments vary in state space type (discrete [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The meta reasoning agent. The reactive policy (ensemble of neural networks), outputs an [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average meta-action taken after training across all environments for different environment [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Average return, reactive action percent and reactive actor uncertainty value as the meta [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Reactive policy in-distribution percentage experiment. Individual environment results when varying the percent of in-distribution tasks for the reactive policy. 20 [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Plan cost experiment. Individual environment results when varying the planning cost α. 21 [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Action stochasticity experiments. Individual environment results when varying the en￾vironment stochasticity. 22 [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 18 canonical work pages

  1. [1]

    Philosophical Transactions of the Royal Society B: Biological Sciences , author =

    Cortical mechanisms of action selection: the affordance competition hypothesis , volume =. Philosophical Transactions of the Royal Society B: Biological Sciences , author =. 2007 , pages =. doi:10.1098/rstb.2007.2054 , abstract =

    arXiv 2007

  2. [2]

    Selecting

    Hay, Nicholas and Russell, Stuart and Tolpin, David and Shimony, Solomon Eyal , month = aug, year =. Selecting. doi:10.48550/arXiv.1408.2048 , abstract =

    arXiv doi:10.48550/arxiv.1408.2048

  3. [3]

    Simple and

    Lakshminarayanan, Balaji and Pritzel, Alexander and Blundell, Charles , month = nov, year =. Simple and. doi:10.48550/arXiv.1612.01474 , abstract =

    arXiv doi:10.48550/arxiv.1612.01474

  4. [4]

    Thinking Fast and Slow in AI

    Booch, Grady and Fabiano, Francesco and Horesh, Lior and Kate, Kiran and Lenchner, Jon and Linck, Nick and Loreggia, Andrea and Murugesan, Keerthiram and Mattei, Nicholas and Rossi, Francesca and Srivastava, Biplav , month = dec, year =. Thinking. doi:10.48550/arXiv.2010.06002 , abstract =

    work page Pith review arXiv doi:10.48550/arxiv.2010.06002 2010

  5. [5]

    Li, Zhong-Zhi and Zhang, Duzhen and Zhang, Ming-Liang and Zhang, Jiaxin and Liu, Zengyan and Yao, Yuxuan and Xu, Haotian and Zheng, Junhao and Wang, Pei-Jie and Chen, Xiuyi and Zhang, Yingying and Yin, Fei and Dong, Jiahua and Li, Zhiwei and Bi, Bao-Long and Mei, Ling-Rui and Fang, Junfeng and Liang, Xiao and Guo, Zhijiang and Song, Le and Liu, Cheng-Lin ...

    arXiv doi:10.48550/arxiv.2502.17419

  6. [6]

    , month = feb, year =

    Lee, Sang Wan and Shimojo, Shinsuke and O’Doherty, John P. , month = feb, year =. Neural. Neuron , publisher =. doi:10.1016/j.neuron.2013.11.028 , language =

    doi:10.1016/j.neuron.2013.11.028 2013

  7. [7]

    2011 , publisher=

    Thinking, fast and slow , author=. 2011 , publisher=

    2011

  8. [8]

    arXiv preprint arXiv:2302.04840 , year=

    What are the mechanisms underlying metacognitive learning? , author=. arXiv preprint arXiv:2302.04840 , year=

    arXiv

Show all 87 references
  1. [9]

    Rodrigo de Lazcano and Kallinteris Andreas and Jun Jet Tai and Seungjae Ryan Lee and Jordan Terry , title =

  2. [10]

    CoRR , volume =

    Maxime Chevalier-Boisvert and Bolun Dai and Mark Towers and Rodrigo de Lazcano and Lucas Willems and Salem Lahlou and Suman Pal and Pablo Samuel Castro and Jordan Terry , title =. CoRR , volume =

  3. [11]

    Journal of Machine Learning Research , year =

    Antonin Raffin and Ashley Hill and Adam Gleave and Anssi Kanervisto and Maximilian Ernestus and Noah Dormann , title =. Journal of Machine Learning Research , year =

  4. [12]

    The quarterly journal of economics , pages=

    A behavioral model of rational choice , author=. The quarterly journal of economics , pages=. 1955 , publisher=

    1955

  5. [13]

    British Journal of Psychology , volume=

    Heuristic and analytic processes in reasoning , author=. British Journal of Psychology , volume=. 1984 , publisher=

    1984

  6. [14]

    Learning

    Paglieri, Davide and Cupiał, Bartłomiej and Cook, Jonathan and Piterbarg, Ulyana and Tuyls, Jens and Grefenstette, Edward and Foerster, Jakob Nicolaus and Parker-Holder, Jack and Rocktäschel, Tim , month = sep, year =. Learning. doi:10.48550/arXiv.2509.03581 , abstract =

    doi

  7. [15]

    Current Opinion in Behavioral Sciences , author =

    The structure of reinforcement-learning mechanisms in the human brain , volume =. Current Opinion in Behavioral Sciences , author =. 2015 , pages =. doi:10.1016/j.cobeha.2014.10.004 , abstract =

    2015 doi

  8. [16]

    Deployable

    Wang, Dong and Beltrame, Giovanni , month = apr, year =. Deployable. doi:10.48550/arXiv.2401.09286 , abstract =

    doi arXiv

  9. [17]

    and Corrado, Nicholas E

    Hanna, Josiah P. and Corrado, Nicholas E. , month = oct, year =. When. doi:10.48550/arXiv.2506.17124 , abstract =

    doi

  10. [18]

    , month = jul, year =

    Sheikhnezhad Fard, Farzaneh and Trappenberg, Thomas P. , month = jul, year =. A. Frontiers in Neurorobotics , publisher =. doi:10.3389/fnbot.2019.00052 , abstract =

    2019

  11. [19]

    Psychological Science , author =

    The. Psychological Science , author =. 2013 , pages =. doi:10.1177/0956797612463080 , abstract =

    2013 doi

  12. [20]

    and Trappenberg, Thomas P

    Fard, Farzaneh S. and Trappenberg, Thomas P. , month = jul, year =. Mixing. 2018. doi:10.1109/IJCNN.2018.8489122 , abstract =

    2018

  13. [21]

    , month = jun, year =

    Lugosch, Loren and Nowrouzezahrai, Derek and Meyer, Brett H. , month = jun, year =. Surprisal-. doi:10.48550/arXiv.2006.01659 , abstract =

    2006 doi arXiv

  14. [22]

    Journal of Cognitive Neuroscience , author =

    Planning. Journal of Cognitive Neuroscience , author =. 2018 , keywords =. doi:10.1162/jocn_a_01263 , abstract =

    2018 doi

  15. [23]

    Liao, Baohao and Xu, Yuhui and Dong, Hanze and Li, Junnan and Monz, Christof and Savarese, Silvio and Sahoo, Doyen and Xiong, Caiming , month = jun, year =. Reward-. doi:10.48550/arXiv.2501.19324 , abstract =

    doi arXiv

  16. [24]

    Thinkless:

    Fang, Gongfan and Ma, Xinyin and Wang, Xinchao , month = jun, year =. Thinkless:. doi:10.48550/arXiv.2505.13379 , abstract =

    doi arXiv

  17. [25]

    and Kadous, M

    Ong, Isaac and Almahairi, Amjad and Wu, Vincent and Chiang, Wei-Lin and Wu, Tianhao and Gonzalez, Joseph E. and Kadous, M. Waleed and Stoica, Ion , month = feb, year =. doi:10.48550/arXiv.2406.18665 , abstract =

    doi arXiv

  18. [26]

    and Johansson, Karl H

    Yoo, Jaehyun and Molin, Adam and Jafarian, Matin and Esen, Hasan and Dimarogonas, Dimos V. and Johansson, Karl H. , month = dec, year =. Event-triggered model predictive control with machine learning for compensation of model uncertainties , url =. 2017. doi:10.1109/CDC.2017.8...

    2017

  19. [27]

    Ocean Engineering , author =

    Predictive compensator based event-triggered model predictive control with nonlinear disturbance observer for unmanned surface vehicle under cyber-attacks , volume =. Ocean Engineering , author =. 2022 , keywords =. doi:10.1016/j.oceaneng.2022.111868 , abstract =

    2022

  20. [28]

    IEEE Transactions on Cybernetics , author =

    Learning-. IEEE Transactions on Cybernetics , author =. 2025 , keywords =. doi:10.1109/TCYB.2025.3536606 , abstract =

    2025

  21. [29]

    Automatica , author =

    Event-based model predictive control for nonlinear systems with dynamic disturbance , volume =. Automatica , author =. 2022 , keywords =. doi:10.1016/j.automatica.2022.110533 , abstract =

    2022

  22. [30]

    IEEE/ASME Transactions on Mechatronics , author =

    Tracking of. IEEE/ASME Transactions on Mechatronics , author =. 2020 , keywords =. doi:10.1109/TMECH.2019.2962099 , abstract =

    2020

  23. [31]

    Reinforcement

    Chen, Jun and Meng, Xiangyu and Li, Zhaojian , month = jun, year =. Reinforcement. 2022. doi:10.23919/ACC53348.2022.9867347 , abstract =

    2022

  24. [32]

    IEEE Transactions on Intelligent Vehicles , author =

    Event-. IEEE Transactions on Intelligent Vehicles , author =. 2024 , keywords =. doi:10.1109/TIV.2023.3329785 , abstract =

    2024

  25. [33]

    and Griffiths, Thomas L

    Callaway, Frederick and Gul, Sayan and Krueger, Paul M. and Griffiths, Thomas L. and Lieder, Falk , month = aug, year =. Learning to select computations , url =. doi:10.48550/arXiv.1711.06892 , abstract =

    doi arXiv

  26. [34]

    and Xia, Jerry and Chung, Stephen and Greenwald, Amy , month = apr, year =

    Wang, Kevin A. and Xia, Jerry and Chung, Stephen and Greenwald, Amy , month = apr, year =. Dynamic

  27. [35]

    Manvi, Rohin and Hong, Joey and Seyde, Tim and Labonne, Maxime and Lechner, Mathias and Levine, Sergey , month = dec, year =. Zero-. doi:10.48550/arXiv.2512.01457 , abstract =

    doi

  28. [36]

    PLoS computational biology , author =

    Speed/accuracy trade-off between the habitual and the goal-directed processes , volume =. PLoS computational biology , author =. 2011 , keywords =. doi:10.1371/journal.pcbi.1002055 , abstract =

    2011 doi

  29. [37]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Stop!. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2024 , keywords =. doi:10.1609/aaai.v38i18.29983 , abstract =

    2024 doi

  30. [38]

    The European Journal of Neuroscience , author =

    Adaptive. The European Journal of Neuroscience , author =. 2025 , pages =. doi:10.1111/ejn.70024 , abstract =

    2025 doi

  31. [39]

    , month = aug, year =

    Ackerman, Rakefet and Thompson, Valerie A. , month = aug, year =. Meta-. Trends in Cognitive Sciences , publisher =. doi:10.1016/j.tics.2017.05.004 , language =

    2017 doi

  32. [40]

    , month = may, year =

    Gläscher, Jan and Daw, Nathaniel and Dayan, Peter and O'Doherty, John P. , month = may, year =. States versus. Neuron , publisher =. doi:10.1016/j.neuron.2010.04.016 , language =

    2010 doi

  33. [41]

    Inference

    Wu, Yangzhen and Sun, Zhiqing and Li, Shanda and Welleck, Sean and Yang, Yiming , month = mar, year =. Inference. doi:10.48550/arXiv.2408.00724 , abstract =

    doi arXiv

  34. [42]

    Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , month = aug, year =. Scaling. doi:10.48550/arXiv.2408.03314 , abstract =

    doi arXiv

  35. [43]

    Heuristic and analytic processes in reasoning* -

  36. [44]

    and Chess, Benjamin and Child, Rewon and Gray, Scott and Radford, Alec and Wu, Jeffrey and Amodei, Dario , month = jan, year =

    Kaplan, Jared and McCandlish, Sam and Henighan, Tom and Brown, Tom B. and Chess, Benjamin and Child, Rewon and Gray, Scott and Radford, Alec and Wu, Jeffrey and Amodei, Dario , month = jan, year =. Scaling. doi:10.48550/arXiv.2001.08361 , abstract =

    2001 doi arXiv

  37. [45]

    OpenAI and Jaech, Aaron and Kalai, Adam and Lerer, Adam and Richardson, Adam and El-Kishky, Ahmed and Low, Aiden and Helyar, Alec and Madry, Aleksander and Beutel, Alex and Carney, Alex and Iftimie, Alex and Karpenko, Alex and Passos, Alex Tachard and Neitz, Alexander and Prok...

    doi arXiv

  38. [46]

    2025 , note =

    Nature , author =. 2025 , note =. doi:10.1038/s41586-025-09422-z , abstract =

    2025 doi

  39. [47]

    Sensors (Basel, Switzerland) , author =

    A. Sensors (Basel, Switzerland) , author =. 2024 , pages =. doi:10.3390/s24154808 , abstract =

    2024 doi

  40. [48]

    Baniodeh, Mustafa and Goel, Kratarth and Ettinger, Scott and Fuertes, Carlos and Seff, Ari and Shen, Tim and Gulino, Cole and Yang, Chenjie and Jerfel, Ghassen and Choe, Dokook and Wang, Rui and Charrow, Benjamin and Kallem, Vinutha and Casas, Sergio and Al-Rfou, Rami and Sapp...

    doi arXiv

  41. [49]

    doi:10.48550/arXiv.1812.03079 , abstract =

    Bansal, Mayank and Krizhevsky, Alex and Ogale, Abhijit , month = dec, year =. doi:10.48550/arXiv.1812.03079 , abstract =

    doi arXiv

  42. [50]

    and Tirumala, Dhruva and Wulfmeier, Markus and Humplik, Jan and Tunyasuvunakool, Saran and Siegel, Noah Y

    Haarnoja, Tuomas and Moran, Ben and Lever, Guy and Huang, Sandy H. and Tirumala, Dhruva and Wulfmeier, Markus and Humplik, Jan and Tunyasuvunakool, Saran and Siegel, Noah Y. and Hafner, Roland and Bloesch, Michael and Hartikainen, Kristian and Byravan, Arunkumar and Hasencleve...

  43. [51]

    Transportation Research Part F: Traffic Psychology and Behaviour , author =

    Driving on familiar roads:. Transportation Research Part F: Traffic Psychology and Behaviour , author =. 2013 , keywords =. doi:10.1016/j.trf.2013.03.008 , abstract =

    2013 doi

  44. [52]

    Proximal

    Schulman, John and Wolski, Filip and Dhariwal, Prafulla and Radford, Alec and Klimov, Oleg , month = aug, year =. Proximal. doi:10.48550/arXiv.1707.06347 , abstract =

    doi arXiv

  45. [53]

    Botea, Adi and Müller, Martin and Schaeffer, Jonathan , editor =. Using. Computers and. 2003 , pages =. doi:10.1007/978-3-540-40031-8_24 , abstract =

    2003 doi

  46. [54]

    Nicolò De and Sumers, Theodore R

    Sabbata, C. Nicolò De and Sumers, Theodore R. and AlKhamissi, Badr and Bosselut, Antoine and Griffiths, Thomas L. , month = jun, year =. Rational. doi:10.48550/arXiv.2410.05563 , abstract =

    doi arXiv

  47. [55]

    Biological Cybernetics , author =

    Path planning versus cue responding: a bio-inspired model of switching between navigation strategies , volume =. Biological Cybernetics , author =. 2010 , keywords =. doi:10.1007/s00422-010-0400-z , abstract =

    2010 doi

  48. [56]

    Hafez, Muhammad Burhan and Weber, Cornelius and Kerzel, Matthias and Wermter, Stefan , month = jul, year =. Curious. 2019. doi:10.1109/IJCNN.2019.8852254 , abstract =

    2019

  49. [57]

    Kästner, Linh and Cox, Johannes and Buiyan, Teham and Lambrecht, Jens , month = sep, year =. All-in-. doi:10.48550/arXiv.2109.11636 , abstract =

    doi arXiv

  50. [60]

    Qu, Xiaoye and Li, Yafu and Su, Zhaochen and Sun, Weigao and Yan, Jianhao and Liu, Dongrui and Cui, Ganqu and Liu, Daizong and Liang, Shuxian and He, Junxian and Li, Peng and Wei, Wei and Shao, Jing and Lu, Chaochao and Zhang, Yue and Hua, Xian-Sheng and Zhou, Bowen and Cheng,...

    doi

  51. [61]

    Cognition , author =

    The best of both worlds:. Cognition , author =. 2022 , keywords =. doi:10.1016/j.cognition.2022.105118 , abstract =

    2022

  52. [62]

    Current Opinion in Behavioral Sciences , author =

    Doing more with less: meta-reasoning and meta-learning in humans and machines , volume =. Current Opinion in Behavioral Sciences , author =. 2019 , pages =. doi:10.1016/j.cobeha.2019.01.005 , language =

    2019 doi

  53. [63]

    , year =

    Lieder, Falk and Griffiths, Thomas L. , year =. Strategy selection as rational metareasoning , volume =. Psychological Review , publisher =. doi:10.1037/rev0000075 , abstract =

    doi

  54. [64]

    and Markman, Arthur B

    Gershman, Samuel J. and Markman, Arthur B. and Otto, A. Ross , year =. Retrospective revaluation in sequential decision making:. Journal of Experimental Psychology: General , publisher =. doi:10.1037/a0030844 , abstract =

    doi

  55. [65]

    Artificial Intelligence , author =

    Between. Artificial Intelligence , author =. 1999 , keywords =. doi:10.1016/S0004-3702(99)00052-1 , abstract =

    1999 doi

  56. [66]

    Javed, Khurram and Sutton, Richard S , year=. The

  57. [67]

    Young, Kenny and Ramesh, Aditya and Kirsch, Louis and Schmidhuber, Jürgen , month = jul, year =. The. doi:10.48550/arXiv.2211.02222 , abstract =

    doi arXiv

  58. [68]

    Artificial Intelligence Review , author =

    Monte. Artificial Intelligence Review , author =. 2023 , keywords =. doi:10.1007/s10462-022-10228-y , abstract =

    2023 doi

  59. [69]

    Computation and action under bounded resources / , abstract =

    Horvitz, Eric , year =. Computation and action under bounded resources / , abstract =

  60. [70]

    Artificial Intelligence , author =

    Principles of metareasoning , volume =. Artificial Intelligence , author =. 1991 , pages =. doi:10.1016/0004-3702(91)90015-C , abstract =

    1991 doi

  61. [71]

    He, Ruiqi and Jain, Yash Raj and Lieder, Falk , month = jan, year =. Have. doi:10.48550/arXiv.2201.00764 , abstract =

    doi arXiv

  62. [72]

    Measuring and modelling how people learn how to plan and how people adapt their planning strategies to the structure of the environment , abstract =

  63. [73]

    Rupam and Jagersand, Martin and Tosatto, Samuele , month = oct, year =

    Karimi, Amirmohammad and Jin, Jun and Luo, Jun and Mahmood, A. Rupam and Jagersand, Martin and Tosatto, Samuele , month = oct, year =. Dynamic. 2023. doi:10.1109/IROS55552.2023.10342408 , abstract =

    2023

  64. [74]

    Honda, Kohei and Yonetani, Ryo and Nishimura, Mai and Kozuno, Tadashi , month = may, year =. When to. 2024. doi:10.1109/ICRA57147.2024.10611474 , abstract =

    2024

  65. [75]

    Sharma, Vishnu Dutt and Lee, Jeongran and Andrews, Matthew and Hadžic, Ilija , year =. Hybrid

  66. [76]

    Proceedings of the Annual Meeting of the Cognitive Science Society , author =

    Enhancing metacognitive reinforcement learningusing reward structures and feedback , volume =. Proceedings of the Annual Meeting of the Cognitive Science Society , author =

  67. [77]

    Rational metareasoning and the plasticity of cognitive control , doi =

    Lieder, Falk and Shenhav, Amitai and Musslick, Sebastian and Griffiths, Thomas , month = feb, year =. Rational metareasoning and the plasticity of cognitive control , doi =

  68. [78]

    How do people learn how to plan? , url =

    Jain, Yash Raj and Gupta, Sanit and Rakesh, Vasundhara and Dayan, Peter and Callaway, Frederick and Lieder, Falk , year =. How do people learn how to plan? , url =. 2019. doi:10.32470/CCN.2019.1313-0 , abstract =

    2019

  69. [79]

    Thinker:

    Chung, Stephen and Du, Wenyu and Fu, Jie , month = may, year =. Thinker:. doi:10.48550/arXiv.2505.21097 , abstract =

    doi

  70. [80]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Dynamic. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2017 , keywords =. doi:10.1609/aaai.v31i1.10918 , abstract =

    2017 doi

  71. [81]

    Metelli, Alberto Maria and Mazzolini, Flavio and Bisi, Lorenzo and Sabbioni, Luca and Restelli, Marcello , month = nov, year =. Control. Proceedings of the 37th

  72. [82]

    Orenstein, Adrian and Chen, Jessica and Santos, Gwyneth Anne Delos and Sapara, Bayley and Bowling, Michael , month = oct, year =. Toward. doi:10.48550/arXiv.2510.22833 , abstract =

    doi

  73. [83]

    Learning to

    Sharma, Sahil and Srinivas, Aravind and Ravindran, Balaraman , month = sep, year =. Learning to. doi:10.48550/arXiv.1702.06054 , abstract =

    doi arXiv

  74. [84]

    2025 , note =

    Gemini. 2025 , note =. doi:10.48550/arXiv.2503.20020 , abstract =

    2025 doi arXiv

  75. [85]

    Artificial intelligence , volume=

    Real-time heuristic search , author=. Artificial intelligence , volume=. 1990 , publisher=

    1990

  76. [86]

    Advances in neural information processing systems , volume=

    Thinking fast and slow with deep learning and tree search , author=. Advances in neural information processing systems , volume=

  77. [87]

    arXiv.org , author =

    Metacontrol for. arXiv.org , author =

  78. [88]

    and Mirza, Mehdi and Xiao, Da and Courville, Aaron and Bengio, Yoshua , month = mar, year =

    Goodfellow, Ian J. and Mirza, Mehdi and Xiao, Da and Courville, Aaron and Bengio, Yoshua , month = mar, year =. An. doi:10.48550/arXiv.1312.6211 , abstract =

    doi arXiv

  79. [89]

    and Koop, Anna and Silver, David , month = jun, year =

    Sutton, Richard S. and Koop, Anna and Silver, David , month = jun, year =. On the role of tracking in stationary environments , isbn =. Proceedings of the 24th international conference on. doi:10.1145/1273496.1273606 , abstract =

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.