Pith. sign in

REVIEW 4 major objections 5 minor 64 references

Free-Gate: Planning, Control And Policy Composition via Free Energy Gating

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Free-Gate provably computes the optimal composition of control primitives via free energy gating, with a per-step convex problem even in nonlinear stochastic environments.

desk verdict Free-Gate gives a clean per-step convex formulation for composing control primitives, but the central optimality proof quietly assumes state-dependent weights that Problem 1 never states. read the letter →

arxiv 2412.06636 v3 pith:B72Q6HGP submitted 2024-12-09 math.OC cs.ROcs.SYeess.SY

Francesca Rossi , Émiland Garrabé , Giovanni Russo This is my paper · ORCID
classification math.OCcs.ROcs.SYeess.SY MSC 90C2593E20
keywords FreeEnergyMinimizationPolicyCompositionAutonomousSystemsDecisionMakingOptimalControlconvexoptimizationvariationalrobotnavigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Free-Gate is a computational rule for combining a library of simple control policies—'primitives'—into a single policy that can perform a task. The paper's central claim is that the optimal weighting of these primitives is the solution to a finite-horizon optimal control problem whose cost is variational free energy, and that this problem can be solved exactly. It proves that the per-time-step weight optimization is convex even when the environment is nonlinear and stochastic and the task cost is nonconvex, so the optimal composition can be computed by a backward recursion. The practical payoff is that a robot with only basic motor primitives (move left, right, up, down) can reach a goal and avoid obstacles, which no primitive could do individually.

What carries the argument

The central machinery is the chain rule for the KL divergence, which splits the free energy functional of Problem 1 into a sum of per-step terms and enables the backward dynamic-programming recursion of Algorithm 1. At each step the decision variable is the weight vector on the probability simplex, and the per-step objective is the variational free energy of the joint state–action distribution, with the optimal cost-to-go folded into the expected loss. Convexity is proven by computing the Hessian of this objective: it is an outer product of the vector of primitives scaled by a nonnegative fraction, hence positive semidefinite, and it is positive definite when the primitives are linearly independent.

What would settle it

Run Algorithm 1 on a small discrete MDP with nonconvex costs, then brute-force Problem 1 by exhaustive search over a fine discretization of the weight simplex; any mismatch between the algorithm's output and the brute-force optimum beyond numerical tolerance would refute the claimed optimality.

Watch

Extended reading notes

Core claim

The paper establishes that the problem of optimally composing control primitives via free energy gating—minimizing $D_{\mathrm{KL}}(p_{0:N}\|q_{0:N})$ plus expected state and action costs, subject to the policy being a convex combination of primitives with weights on the probability simplex—can be solved exactly by Algorithm 1. The algorithm runs a backward recursion in which each step solves a convex optimization over the current weight vector, with the optimal value of the next step's problem embedded as a cost-to-go. Property 1 proves that this recursion returns the global optimum of the finite-horizon problem, and Property 2 proves that each per-step problem is convex in the weights, strictly convex when the primitives are linearly independent. From strict convexity and an interior optimum, the paper derives a transcendence result: the composed policy strictly outperforms every individual primitive, so tasks that are infeasible with the primitives alone become feasible.

Load-bearing premise

The optimality guarantee holds only for the exact transition density and the fixed generative model supplied to the algorithm; if the real environment differs from that model in any way, the composed policy is optimal only for the model, and the paper provides no adaptation or robustness bound for model error.

Editorial extensions

If this is right

  • The optimal composition of primitives can be computed exactly by solving a sequence of per-time convex problems, so standard convex solvers can be used online.
  • Free-Gate gives a normative justification for gating in control: the weights are the solution of a well-defined optimization, not an arbitrary softmax heuristic or a learned policy.
  • Because the per-step cost includes the optimal cost-to-go, the composition is plan-aware: even a myopic per-step solver incorporates future consequences.
  • The transcendence property guarantees that a combination of simple primitives strictly outperforms each primitive alone, so individually infeasible tasks become feasible through composition.
  • The formulation holds for nonlinear, non-stationary, stochastic environments and nonconvex costs, without stationarity or discounting assumptions, so it applies broadly in planning and control.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The experiments use a heuristic one-step lookahead for the cost-to-go, not the exact backward recursion, so the demonstrated performance is for the approximate version; whether the full recursion improves it in practice is not tested in the paper.
  • The full-support assumption (every primitive is stochastic everywhere) is stronger than typical deterministic controllers; relaxing it, perhaps by Gaussian smoothing, would make the method applicable to deterministic primitives while preserving the convexity proof.
  • Since the optimality statement is model-relative, Free-Gate would benefit from a learning or adaptation layer for the transition density and generative model; the paper does not analyze robustness to model error.
  • The free-energy gating rule could be transplanted into deep mixture-of-experts architectures as a principled alternative to learned gating; whether it scales and yields better expert selection is an open question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Free-Gate, a mechanism for composing control primitives by minimizing a variational free-energy functional. The composition problem is cast as a finite-horizon optimal control problem (Problem 1) whose decision variables are the primitive weights, and Algorithm 1 solves it by backward recursion. The paper proves a dynamic-programming-style optimality property (Property 1) and a per-step convexity property (Property 2), then validates the approach on a Robotarium navigation task with four directional primitives that individually cannot reach the goal. The appendix contains self-contained proofs based on the KL chain rule and an explicit Hessian computation.

Significance. If the formalization is corrected, Free-Gate offers a principled normative account of policy composition: the per-step weight optimization is convex even for nonconvex costs and nonlinear, stochastic, non-stationary dynamics, which is a useful step beyond stationary mixture-of-experts and ad hoc soft-max gating. The paper supplies self-contained proofs, a documented code repository, and both simulation and hardware demonstrations. However, the central optimality theorem currently has a formal gap concerning state dependence of the weights, the global convexity claim in the abstract is stronger than what is proved, and the experiments use a heuristic cost-to-go rather than the exact recursion in Algorithm 1. These issues are fixable within the scope of the manuscript, but they need to be addressed before the claims can be accepted as stated.

major comments (4)
  1. [§3.1 (Problem 1) and Appendix A] The decision variables in Problem 1 are defined as a sequence of vectors {w_k}_{1:N} with no dependence on the state, yet the proof of Property 1 minimizes the final-stage objective pointwise in x_{N-1}, writing the optimal cost as E_{p(x_{N-1})}[l*_N(X_{N-1})] with l*_N obtained from a pointwise minimization. This step is valid only if w_N is allowed to be a function of x_{N-1}. If w_N is instead a single vector common to all states, then min_{w_N} E[...] is not equal to E[min_{w_N} ...] in general, so Algorithm 1 solves a different problem from the one stated. The running text and Fig. 2(d) indicate that state-feedback weights are intended; Problem 1 and Algorithm 1 should be rewritten so that w_k = w_k(x_{k-1}) with an explicit measurability condition, and Property 1 should be stated as a dynamic programming result for feedback policies.
  2. [Abstract, §3.2 (Property 2)] The abstract claims that the finite-horizon composition problem 'remains convex,' but the proof establishes only that, for each fixed x_{k-1}, the per-time problem (2) is convex in w_k. If Problem 1 is interpreted with state-feedback weights, the global problem is an infinite-dimensional optimization and no convexity argument is given for it; if it is interpreted with a single vector per time step, the pointwise Hessian computation does not establish convexity of the expectation over states. The convexity claim should be restricted to the per-step, per-state optimization problem (2), and the abstract and contributions should be reworded accordingly.
  3. [§4, Validation paragraph] The experiments replace the exact cost-to-go l*_{k+1} computed by Algorithm 1 with a heuristic approximation, described as 'the cost obtained by applying twice the same input.' Therefore the in-silico and hardware demonstrations do not validate the optimality guarantee of Algorithm 1; they validate a variant with an approximate planning value. The paper should either implement the full backward recursion on the discretized state space or explicitly state that the experiments test an approximate version and discuss the implications for the optimality claims.
  4. [§3.2, Remark 3] The 'transcendence' claim in Remark 3 does not follow from strict convexity. Strict convexity gives J_k(w) < Σ_i w_i J_k(e_i), but a weighted average of primitive costs can be larger than the minimum primitive cost. For example, with two primitives satisfying J_k(e_1)=10 and J_k(e_2)=0 and w=(0.5,0.5), strict convexity only implies J_k(w)<5, not J_k(w)<0. The remark should be corrected or removed unless an additional condition, such as equal primitive costs, is imposed.
minor comments (5)
  1. [Appendix A] The proof header reads 'Proof of Proposition 1,' but the statement being proved is Property 1 in Section 3.2; please correct the cross-reference.
  2. [Algorithm 1] Algorithm 1 writes l*_{N+1}(X_N) ← 0 and later uses l*_{k+1}(X_k), but it does not specify how these value functions are represented or computed over a continuous state space. The discretization used in the experiments should be described either as part of the algorithm or as a numerical approximation.
  3. [§4, implementation details] The text says 'CVX [7] is used to solve the optimization problem,' but reference [7] is the CVXPY paper; please align the name with the cited software.
  4. [Fig. 2(d)] The weight trajectories in Fig. 2(d) are not clearly labeled; adding a legend or axis labels identifying each primitive would help the reader connect the plot to the four cardinal-direction primitives.
  5. [Assumption 3 and §4] Assumption 3 requires each primitive to have full support over U, while the primitives are introduced as Gaussian densities on R^2. After restricting or discretizing to the 7×7 action grid this is likely satisfied, but the paper should state explicitly how the Gaussian support is reconciled with the compact action space.

Circularity Check

0 steps flagged · score 0.0 of 10

No meaningful circularity: the optimality and convexity proofs are self-contained and do not reduce to their inputs.

full rationale

Free-Gate's central claims are derived in the paper itself rather than assumed. Property 1's optimality proof uses the KL chain rule and a backward recursion: each per-step problem (2) is minimized over wk, and the cost-to-go l*_{k+1} is constructed as the optimal value of the subsequent step; this is a standard dynamic-programming argument, not a circular one. Property 2's convexity proof explicitly computes the Hessian and shows it is a positive semi-definite outer-product matrix, with strict convexity following from linear independence of the primitives; again, the conclusion is proved, not assumed. No fitted parameter is relabeled as a prediction: the experimental costs, dynamics, noise covariances, and primitives are chosen inputs, and the reported trajectories are consequences of the algorithm, not calibration targets. Citations to the authors' prior work (e.g., [13], [14]) are motivational or attributive; the KL-chain-rule derivation is self-contained and standard. One formal concern is that the proof of Property 1 in Appendix A appears to treat wk as state-dependent when swapping the outer expectation with the pointwise minimization, whereas Problem 1 writes {wk} as a fixed sequence; this is a mathematical-rigor gap in the optimality theorem, but it is not a circularity, because the target result is not inserted as an assumption and the claimed optimality does not reduce by construction to an earlier unverified claim. Overall, the derivation chain is self-contained and no circular step is present.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central mathematical claim is parameter-free: the only quantities chosen by hand appear in the experimental instantiation. The proofs rely on the Markov assumption plus Assumptions 1 to 3. No new physical entities are introduced.

free parameters (4)
  • cost coefficients in navigation experiment = 150 for obstacle penalties, 30 for boundary penalties
    Hand-chosen weights for the state cost c_x in Section 4; not fitted to data but tuned by the authors and not varied in sensitivity analysis.
  • generative model covariances = Sigma_tilde_x = 0.002 I2, Sigma_tilde_u = 0.005 I2
    Hand-selected covariances defining q in the experiments; the theory allows arbitrary q, but these values affect the demonstrated behavior.
  • primitive covariance and primitive mean controllers = Sigma_u = 0.005 I2, means from proportional controllers
    User-specified primitives, four cardinal-direction Gaussian policies; the composition depends on this primitive set, and no criterion for a good primitive set is given, which the paper lists as future work.
  • heuristic cost-to-go horizon = two time steps
    Experiments approximate the value function by applying the same input twice instead of running the full backward recursion of Algorithm 1; this changes the objective actually optimized on the robot.
assumptions (6)
  • domain assumption Environment is Markovian and described by conditional densities p_x(x_k|x_{k-1},u_k), possibly nonlinear, stochastic, and non-stationary.
    Used throughout Section 3 to define the closed-loop distribution p0:N and to justify the backward recursion.
  • domain assumption Primitives are bounded densities (Assumption 1).
    Used to ensure integrals in the convexity proof are well defined, as noted in Remark 1.
  • domain assumption Feasible weight set W is non-empty and the optimal free energy is finite (Assumption 2).
    Standard in entropic optimal transport; ensures Problem 1 has a finite minimizer and the recursion is well posed.
  • domain assumption Each primitive has full support over the action space, S(pi) = U (Assumption 3).
    Used to guarantee the gradient of the cost is well defined and for strict convexity; excludes deterministic or compact-support primitives.
  • standard math KL chain rule and convexity of KL divergence in the first argument, plus standard calculus, are used in the proofs.
    Appendix A derives the dynamic programming decomposition and the convexity of the per-step problem from these standard facts.
  • domain assumption The generative model q0:N factorizes over time.
    Defined in Section 3.1; without this factorization, the per-step KL decomposition and Algorithm 1 do not follow.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Free-Gate: Planning, Control And Policy Composition via Free Energy Gating." pith.science (2026). https://pith.science/paper/B72Q6HGP

@misc{pith2026241206636,
  author       = {Pith},
  title        = {Pith review of: Free-Gate: Planning, Control And Policy Composition via Free Energy Gating},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B72Q6HGP}},
  note         = {Machine review of arXiv:2412.06636}
}
read the original abstract

We consider the problem of optimally composing a set of primitives to tackle planning and control tasks. To address this problem, we introduce a free energy computational model for planning and control via policy composition: Free-Gate. Within Free-Gate, control primitives are combined via a gating mechanism that minimizes variational free energy. This composition problem is formulated as a finite-horizon optimal control problem, which we prove remains convex even when the cost is not convex in states/actions and the environment is nonlinear, stochastic and non-stationary. We develop an algorithm that computes the optimal primitives composition and demonstrate its effectiveness via in-silico and hardware experiments on an application involving robot navigation in an environment with obstacles. The experiments highlight that Free-Gate enables the robot to navigate to the destination despite only having available simple motor primitives that, individually, could not fulfill the task.

Figures

Figures reproduced from arXiv: 2412.06636 by the authors.

Figure 1
Figure 1. In Free-Gate control primitives are dynamically combined by a free energy gating mech￾anism. This mechanism, given the state, action/state costs and the generative model, computes the optimal weights for the primitives composition by minimizing variational free energy. Con￾sequently, the Free-Gate possibly non-stationary policy, p ⋆ u,k (uk|xk−1), is a linear combination of these primitives with the optimal weights … view at source ↗
Figure 2
Figure 2. (a) trajectories when the robot is controlled by the 4 primitives described in Section 4; (b) heat map of the cost used in the experiments; (c) robot trajectories when controlled by Free￾Gate; (d) optimal primitives’ weights corresponding to the experiment in blue in panel (c). Free￾Gate allows the robot to idle at the goal position despite lacking a stopping primitive. matching the work area, and uk ∈ U := [−0.2, 0… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 59 canonical work pages

  1. [1]

    Advances in neural information processing systems 6 (1993)

    Cacciatore, T., Nowlan, S.: Mixtures of controllers for jump linear and non-linear plants. Advances in neural information processing systems 6 (1993)

    work page 1993

  2. [2]

    IEEE Transactions on Knowledge and Data Engineering pp

    Cai, W., Jiang, J., Wang, F., Tang, J., Kim, S., Huang, J.: A survey on mixture of experts in large language models. IEEE Transactions on Knowledge and Data Engineering pp. 1–20 (2025)

    work page 2025

  3. [3]

    SIAM Journal on Mathematical Analysis 49(2), 1385–1418 (2017)

    Carlier, G., Duval, V ., Peyré, G., Schmitzer, B.: Convergence of Entropic Schemes for Optimal Transport and Gradient Flows. SIAM Journal on Mathematical Analysis 49(2), 1385–1418 (2017)

    work page 2017

  4. [4]

    Neural Computation 36(6), 1163–1197 (2024)

    Centorrino, V ., Gokhale, A., Davydov, A., Russo, G., Bullo, F.: Positive Competitive Net- works for Sparse Reconstruction. Neural Computation 36(6), 1163–1197 (2024)

    work page 2024

  5. [5]

    Cambridge University Press (2006)

    Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press (2006)

    work page 2006

  6. [6]

    Wiley-Interscience, USA (2006)

    Cover, T.M., Thomas, J.A.: Elements of Information Theory (Wiley Series in Telecommuni- cations and Signal Processing). Wiley-Interscience, USA (2006)

    work page 2006

  7. [7]

    Journal of Machine Learning Research 17(83), 1–5 (2016)

    Diamond, S., Boyd, S.: CVXPY: A Python-embedded modeling language for convex opti- mization. Journal of Machine Learning Research 17(83), 1–5 (2016)

    work page 2016

  8. [8]

    Neural Computation 14(6), 1347–1369 (2002)

    Doya, K., Samejima, K., Katagiri, K.i., Kawato, M.: Multiple Model-Based Reinforcement Learning. Neural Computation 14(6), 1347–1369 (2002)

    work page 2002

Show all 64 references
  1. [9]

    Friston, K.: The free-energy principle: a rough guide to the brain? Trends in Cognitive Sci- ences 13(7), 293–301 (2009)

    2009

  2. [10]

    Friston, K.: The free-energy principle: a unified brain theory? Nature reviews neuroscience 11(2), 127–138 (2010)

    2010

  3. [11]

    Physics Reports 1024, 1–29 (2023)

    Friston, K., Da Costa, L., Sajid, N., Heins, C., Ueltzhöffer, K., Pavliotis, G.A., Parr, T.: The free energy principle made simpler but not too simple. Physics Reports 1024, 1–29 (2023)

    2023

  4. [12]

    Neural computation 29(1), 1–49 (2017)

    Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P., Pezzulo, G.: Active inference: a process theory. Neural computation 29(1), 1–49 (2017)

    2017

  5. [13]

    Automatica 137, 110121 (2022)

    Gagliardi, D., Russo, G.: On a Probabilistic Approach to Synthesize Control Policies from Example Datasets. Automatica 137, 110121 (2022)

    2022

  6. [14]

    Automatica173, 112015 (2025)

    Garrabe, E., Jesawada, H., Vecchio, C.D., Russo, G.: On convex data-driven inverse optimal control for nonlinear, non-stationary and stochastic systems. Automatica173, 112015 (2025)

    2025

  7. [15]

    IEEE Control Systems Letters 7, 2557–2562 (2023)

    Garrabé, É., Lamberti, M., Russo, G.: Optimal decision-making for autonomous agents via data composition. IEEE Control Systems Letters 7, 2557–2562 (2023)

    2023

  8. [16]

    Annual Reviews in Control 54, 81–102 (2022)

    Garrabé, E., Russo, G.: Probabilistic design of optimal sequential decision-making algo- rithms in learning and control. Annual Reviews in Control 54, 81–102 (2022)

    2022

  9. [17]

    Neural networks 6(4), 485–497 (1993)

    Gomi, H., Kawato, M.: Recognition of manipulated objects by motor learning with modular architecture networks. Neural networks 6(4), 485–497 (1993)

    1993

  10. [18]

    In: International conference on machine learning

    Haarnoja, T., Tang, H., Abbeel, P., Levine, S.: Reinforcement learning with deep energy- based policies. In: International conference on machine learning. pp. 1352–1361. PMLR (2017)

    2017

  11. [19]

    In: 2023 IEEE International Conference on Robotics and Automation (ICRA)

    Hansel, K., Urain, J., Peters, J., Chalvatzaki, G.: Hierarchical policy blending as inference for reactive robot control. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 10181–10188. IEEE (2023)

    2023

  12. [20]

    Neural computation 13(10), 2201–2220 (2001)

    Haruno, M., Wolpert, D.M., Kawato, M.: Mosaic model for sensorimotor learning and con- trol. Neural computation 13(10), 2201–2220 (2001)

    2001

  13. [21]

    Basic Books (2021)

    Hawkins, J.: A thousand brains: A new theory of intelligence. Basic Books (2021)

    2021

  14. [22]

    Proceedings of the National Academy of Sciences 121(17) (2024) 14 F

    Heins, C., Millidge, B., Da Costa, L., Mann, R.P., Friston, K.J., Couzin, I.D.: Collective behavior from surprise minimization. Proceedings of the National Academy of Sciences 121(17) (2024) 14 F. Rossi et al

    2024

  15. [23]

    Neural computation 14(8), 1771–1800 (2002)

    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural computation 14(8), 1771–1800 (2002)

    2002

  16. [24]

    Machine Learning and Knowledge Extraction 4(1), 172–221 (2022)

    Hutsebaut-Buysse, M., Mets, K., Latré, S.: Hierarchical reinforcement learning: A survey and open research challenges. Machine Learning and Knowledge Extraction 4(1), 172–221 (2022)

    2022

  17. [25]

    Neural computation 3(1), 79–87 (1991)

    Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural computation 3(1), 79–87 (1991)

    1991

  18. [26]

    In: Proceedings of the 37th International Conference on Neural Informa- tion Processing Systems

    Jaiswal, P., Honnappa, H., Rao, V .A.: On the statistical consistency of risk-sensitive bayesian decision-making. In: Proceedings of the 37th International Conference on Neural Informa- tion Processing Systems. NIPS ’23, Curran Associates Inc., Red Hook, NY , USA (2024)

    2024

  19. [27]

    Machine learning 37, 183–233 (1999)

    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational meth- ods for graphical models. Machine learning 37, 183–233 (1999)

    1999

  20. [28]

    IEEE Signal Processing Magazine 38(2), 120–125 (2021)

    Jose, S.T., Simeone, O.: Free Energy Minimization: A Unified Framework for Modeling, Inference, Learning, and Optimization [Lecture Notes]. IEEE Signal Processing Magazine 38(2), 120–125 (2021)

    2021

  21. [29]

    IEEE Control Systems Letters (2023)

    Kouw, W.M.: Information-seeking polynomial NARX model-predictive control through ex- pected free energy minimization. IEEE Control Systems Letters (2023)

    2023

  22. [30]

    Kouw, W.M.: Planning to Avoid Ambiguous States Through Gaussian Approximations to Non-linear Sensors in Active Inference Agents, p. 195–208. Springer Nature Switzerland (2024)

    2024

  23. [31]

    Annals of Mathematical Statistics 22, 79–87 (1951)

    Kullback, S., Leibler, R.: On information and sufficiency. Annals of Mathematical Statistics 22, 79–87 (1951)

    1951

  24. [32]

    Behavioral and brain sciences 40, e253 (2017)

    Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behavioral and brain sciences 40, e253 (2017)

    2017

  25. [33]

    Frontiers in Neurorobotics 16, 840658 (2022)

    Van de Maele, T., Verbelen, T., Çatal, O., Dhoedt, B.: Embodied object representation learn- ing and recognition. Frontiers in Neurorobotics 16, 840658 (2022)

    2022

  26. [34]

    Artificial Intelli- gence Review 42, 275–293 (2014)

    Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artificial Intelli- gence Review 42, 275–293 (2014)

    2014

  27. [35]

    In: Ranzato, M., Beygelzimer, A., Dauphin, Y ., Liang, P., Vaughan, J.W

    Mazzaglia, P., Verbelen, T., Dhoedt, B.: Contrastive Active Inference. In: Ranzato, M., Beygelzimer, A., Dauphin, Y ., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Infor- mation Processing Systems. vol. 34, pp. 13870–13882. Curran Associates, Inc. (2021)

    2021

  28. [36]

    In: International workshop on active inference

    Millidge, B., Tschantz, A., Seth, A.K., Buckley, C.L.: On the relationship between active inference and control as inference. In: International workshop on active inference. pp. 3–11. Springer (2020)

    2020

  29. [37]

    Mountcastle, V .B.: An organizing principle for cerebral function: the unit module and the distributed system (1978), https://api.semanticscholar.org/CorpusID: 59731439

    1978

  30. [38]

    MIT Press (2023),http: //probml.github.io/book2

    Murphy, K.P.: Probabilistic Machine Learning: Advanced Topics. MIT Press (2023),http: //probml.github.io/book2

    2023

  31. [39]

    Lecture notes (2020)

    Nutz, M.: Introduction to Entropic Optimal Transport. Lecture notes (2020)

    2020

  32. [40]

    Biological Cybernetics 113(5–6), 495–513 (2019)

    Parr, T., Friston, K.J.: Generalised free energy and active inference. Biological Cybernetics 113(5–6), 495–513 (2019)

    2019

  33. [41]

    The MIT Press (2022)

    Parr, T., Pezzulo, G., Friston, K.J.: Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. The MIT Press (2022)

    2022

  34. [42]

    ACM Computing Surveys (CSUR)54(5), 1–35 (2021)

    Pateria, S., Subagdja, B., Tan, A.h., Quek, C.: Hierarchical reinforcement learning: A com- prehensive survey. ACM Computing Surveys (CSUR)54(5), 1–35 (2021)

    2021

  35. [43]

    Entropy 26(6), 484 (2024)

    Paul, A., Isomura, T., Razi, A.: On predictive planning and counterfactual learning in active inference. Entropy 26(6), 484 (2024)

    2024

  36. [44]

    Expert Systems with Applications 253, 124315 (2024) Planning, Control And Policy Composition via Free Energy Gating 15

    Paul, A., Sajid, N., Da Costa, L., Razi, A.: On efficient computation in active inference. Expert Systems with Applications 253, 124315 (2024) Planning, Control And Policy Composition via Free Energy Gating 15

    2024

  37. [45]

    Progress in neurobiology 134, 17–35 (2015)

    Pezzulo, G., Rigoli, F., Friston, K.: Active inference, homeostatic regulation and adaptive behavioural control. Progress in neurobiology 134, 17–35 (2015)

    2015

  38. [46]

    Trends in cognitive sciences 22(4), 294–306 (2018)

    Pezzulo, G., Rigoli, F., Friston, K.J.: Hierarchical active inference: a theory of motivated control. Trends in cognitive sciences 22(4), 294–306 (2018)

    2018

  39. [47]

    Science Robotics 8(78), eadg6014 (2023)

    Prescott, T.J., Wilson, S.P.: Understanding brain functional architecture through robotics. Science Robotics 8(78), eadg6014 (2023)

    2023

  40. [48]

    org/abs/2405.15164

    Russin, J., McGrath, S.W., Williams, D.J., Elber-Dorozko, L.: From Frege to chatGPT: Com- positionality in language, cognition, and deep neural networks (2024), https://arxiv. org/abs/2405.15164

    2024 arXiv

  41. [49]

    Electronics and Communications in Japan (Part III: Funda- mental Electronic Science) 89(9), 54–69 (2006)

    Samejima, K., Katagiri, K., Doya, K., Kawato, M.: Multiple model-based reinforcement learning for nonlinear control. Electronics and Communications in Japan (Part III: Funda- mental Electronic Science) 89(9), 54–69 (2006)

    2006

  42. [50]

    Neural Computation 26(12), 2669–2691 (2014)

    Sanger, T.D.: Risk-Aware Control. Neural Computation 26(12), 2669–2691 (2014)

    2014

  43. [51]

    Shafiei, A., Jesawada, H., Friston, K., Russo, G.: Robust decision-making via free energy minimization (2025), https://arxiv.org/abs/2503.13223

    2025

  44. [52]

    Foundations and Trends® in Signal Processing 12(3–4), 200–431 (2018)

    Simeone, O.: A Brief Introduction to Machine Learning for Engineers. Foundations and Trends® in Signal Processing 12(3–4), 200–431 (2018)

    2018

  45. [53]

    Sutton, R.S., Barto, A.G., et al.: Reinforcement learning: An introduction, vol. 1. MIT press Cambridge (1998)

    1998

  46. [54]

    Neural Networks 12(7-8), 1131–1141 (1999)

    Tani, J., Nolfi, S.: Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems. Neural Networks 12(7-8), 1131–1141 (1999)

    1999

  47. [55]

    In: International conference on machine learning

    Vezhnevets, A.S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., Kavukcuoglu, K.: Feudal networks for hierarchical reinforcement learning. In: International conference on machine learning. pp. 3540–3549. PMLR (2017)

    2017

  48. [56]

    Science Robotics10(98) (2025)

    Vijayaraghavan, P., Queißer, J.F., Flores, S.V ., Tani, J.: Development of compositionality through interactive learning of language and action of robots. Science Robotics10(98) (2025)

    2025

  49. [57]

    Journal of the Royal Society Interface 3(9), 471–482 (2006)

    Vincent, J.F., Bogatyreva, O.A., Bogatyrev, N.R., Bowyer, A., Pahl, A.K.: Biomimetics: its practice and theory. Journal of the Royal Society Interface 3(9), 471–482 (2006)

    2006

  50. [58]

    arXiv preprint arXiv:2406.18420 (2024)

    Willi, T., Obando-Ceron, J., Foerster, J., Dziugaite, K., Castro, P.S.: Mixture of Experts in a Mixture of RL settings. arXiv preprint arXiv:2406.18420 (2024)

    2024 arXiv

  51. [59]

    IEEE Control Systems Magazine 40(1), 26–44 (2020)

    Wilson, S., Glotfelter, P., Wang, L., Mayya, S., Notomista, G., Mote, M., Egerstedt, M.: The Robotarium: Globally Impactful Opportunities, Challenges, and Lessons Learned in Remote- Access, Distributed Control of Multirobot Systems. IEEE Control Systems Magazine 40(1), 26–44 (2020)

    2020

  52. [60]

    Neural networks 11(7-8), 1317–1329 (1998)

    Wolpert, D.M., Kawato, M.: Multiple paired forward and inverse models for motor control. Neural networks 11(7-8), 1317–1329 (1998)

    1998

  53. [61]

    In: 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN)

    Yoo, J., Kim, H.J., Johansson, K.H.: Mapless indoor localization by trajectory learning from a crowd. In: 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN). pp. 1–7. IEEE (2016)

    2016

  54. [62]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Yu, X., Liu, T., Gong, M., Batmanghelich, K., Tao, D.: An efficient and provable approach for mixture proportion estimation using linear independence assumption. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4480–4489 (2018)

    2018

  55. [63]

    In: Glober- son, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C

    Zhang, E., Zhu, V ., Saphra, N., Kleiman, A., Edelman, B.L., Tambe, M., Kakade, S., Malach, E.: Transcendence: Generative models can outperform the experts that train them. In: Glober- son, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Ad- va...

    2024

  56. [64]

    In: Proceedings of the 23rd National Conference on Artificial Intelligence - V olume 3

    Ziebart, B.D., Maas, A., Bagnell, J.A., Dey, A.K.: Maximum Entropy Inverse Reinforce- ment Learning. In: Proceedings of the 23rd National Conference on Artificial Intelligence - V olume 3. p. 1433–1438. AAAI’08, AAAI Press (2008)

    2008

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.