Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

LLM-Guided Probabilistic Program Induction for POMDP Model Estimation

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a POMDP's transition, observation, reward, and initial-state functions can be learned as short probabilistic programs proposed and repaired by an LLM, and that the resulting model plans better than tabular POMDP…

desk verdict A genuinely useful method for learning POMDP models from few demos, but the coverage metric obscures how much the learned model deserves the credit. read the letter →

arxiv 2505.02216 v2 pith:FE4VYZDZ submitted 2025-05-04 cs.AI

Aidan Curtis , Hao Tang , Thiago Veloso , Kevin Ellis , Joshua Tenenbaum , Tomás Lozano-Pérez , Leslie Pack Kaelbling This is my paper · ORCID
classification cs.AI
keywords POMDPmodellearningprobabilisticprograminductionLLM-guidedsynthesiscoverage-basedselectionpartiallyobservableplanningmobilerobotsearchMiniGridsample-efficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a Partially Observable Markov Decision Process (POMDP) model can be learned from just ten demonstrations when an LLM is used as a prior: the LLM proposes short probabilistic programs for the transition, observation, reward, and initial-state components, the programs are scored against observed data, and failures are fed back to the LLM for repair. A sympathetic reader would care because POMDPs are powerful but famously hard to specify by hand, so a data-driven way to write interpretable models could make partial observability practical in robotics and other sequential decision problems. The paper tests this on classical POMDP benchmarks, five simulated MiniGrid environments, and two real mobile-robot search tasks, reporting that the resulting models match or outperform tabular model learning, behavior cloning, and direct LLM action selection in every simulated domain, and outperform them on the real robot. If the claim is right, the bottleneck in applying POMDPs shifts from hand-engineering world models to writing a small code-level API plus collecting a handful of trajectories.

What carries the argument

The central object is the short probabilistic program written in Pyro: executable Python code that samples stochastic structure, one program per POMDP component. The mechanism that carries the argument is the LLM proposal-and-repair loop paired with coverage-based selection. Coverage is the fraction of empirical transitions that receive nonzero probability under the candidate model, and it gives the LLM a concrete, interpretable training signal: a repair prompt lists conditions and outcomes that the current program cannot produce, alongside samples the program does produce. Thompson sampling over Beta distributions on each candidate's coverage decides which program to refine next, and the highest-coverage program is eventually returned. Because the program uses symbolic ranges and conditions rather than enumerating states, it can generalize to initial states and situations that never appeared in the demonstration data, which is precisely where tabular models and behavior cloning fail.

What would settle it

A decisive check is to re-run the Unlock and Lava domains while selecting candidate models by their planned expected return instead of by coverage; if a model with low initial-state test coverage still drives successful search, coverage is not the binding constraint and the reported gains should be attributed to the planner and online repair, whereas if planning collapses, the induction step is doing the work attributed to it.

Watch

Extended reading notes

Core claim

The central discovery is that the components of a POMDP—the initial-state distribution, transition function, observation function, and reward function—can be treated as low-complexity probabilistic programs, and that an LLM-guided proposal-and-repair loop can find those programs from data. The method, POMDP Coder, gives the LLM a code template, a state/action/observation API, and a small set of demonstration transitions; the LLM writes an initial program for each component, and the program is kept only if it gives nonzero probability to the observed data under a coverage metric. Candidate programs form a tree, Thompson sampling decides which candidate to refine next, and the LLM repairs a program using the specific empirical outcomes it failed to cover. The final programs are coupled to a belief-space planner with particle-filter belief updates. Across Tiger, RockSample, five MiniGrid tasks, and two real Boston Dynamics Spot search scenarios, POMDP Coder matched or outperformed all baselines in the simulated domains and produced higher expected discounted reward and more successful episodes on the robot, with the learned initial-state distribution providing more efficient exploration than a uniform one.

Load-bearing premise

The load-bearing premise is that the coverage metric—the fraction of recorded transitions the model can produce with nonzero probability—reliably tracks how well the model supports good planning; the paper itself concedes that coverage can reward overly broad distributions and reports initial-state test coverage as low as 0.24 in the Unlock domain.

Editorial extensions

If this is right

  • Ten demonstrations can be enough to obtain a usable POMDP model in domains with structured state spaces, because probabilistic programs generalize far beyond the enumerated training examples.
  • Learning a model rather than a policy separates the task of understanding the world from the task of acting in it, so the same learned model can be replanned with any POMDP solver and does not inherit the quality limitations of the demonstrations.
  • The online repair loop lets the agent extend and correct its model from its own experience, fixing gaps that offline-only learning misses, such as the consequence of stepping into lava or the effect of using a key.
  • Because the learned model is code, it is inspectable and debuggable in a way that learned tables or neural policies are not, which matters when the model must be validated for safe robot deployment.
  • Direct LLM action selection is not enough in partially observable settings: it can loop or ignore obstacle constraints, whereas the LLM only needs to write local generative code within POMDP Coder, leaving global reasoning to the planner.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the coverage metric actively biases the search toward broader distributions, so a promising cheap extension is to add a model-complexity or distribution-width penalty to the selection objective and test whether planning quality improves on the Lava, Rooms, and Unlock domains where initial-state coverage was lowest.
  • The success on the real robot suggests a testable generalization: if the learned model is replaced by an equally good hand-written one, planning performance should be nearly identical; comparing those two cases would isolate how much of the reported gain comes from the induction procedure itself versus from the belief-space planner and online repair.
  • A further consequence not explored in the paper is that the same LLM-guided induction could learn belief updates or abstraction layers, not just the four POMDP components, since the code-API interface seems to be the main bottleneck; richer scene graphs or learned predicates would likely extend the approach to more complex manipulation tasks.
  • The post-hoc full-observability assumption means the method is most directly applicable where episode states can be recovered after the fact, such as object-search tasks; extending the repair loop to actively ask for missing state information during execution is a natural next step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces POMDP Coder, a method that learns low-complexity POMDP models from data by using an LLM to propose and iteratively repair short probabilistic programs for the initial-state, transition, observation, and reward components. Candidate programs are selected according to a coverage metric that measures the fraction of empirical transitions with nonzero probability under the model. The learned model is then used by a determinized belief-space planner with particle-filter belief updates. Experiments are reported on two classical POMDPs (Tiger, RockSample), five modified MiniGrid domains, and two real Spot robot search tasks, comparing against an oracle model, direct LLM planning, behavior cloning, tabular model learning, a random baseline, and offline/online ablations. The central claim is that LLM-guided construction of a low-complexity POMDP model can be more effective than tabular POMDP learning, behavior cloning, or direct LLM planning.

Significance. If the attribution holds, this is a valuable step toward data-efficient, interpretable POMDP model learning, combining the LLM's prior knowledge with an empirical coverage check to produce executable models. The paper's strengths include the use of multiple seeds and error bars in simulation, ablations that separate offline and online learning, real-robot evaluation, and a promised public code release. The interpretable probabilistic-program representation is a genuine advantage over tabular approaches. However, the current evidence does not fully establish that the learned model, rather than the planner or the online repair loop, is what drives the reported gains; the coverage-based selection metric is weak and its reported test values are not independent generalization estimates. The significance is therefore contingent on additional analysis demonstrating sensitivity of planning performance to learned-model quality.

major comments (4)
  1. [§4.1 and Appendix B, Algorithm 2] The 'test' coverage reported in Table 3 is not an independent generalization estimate. Algorithm 2 evaluates coverage on both Dtrain and Dtest (Lines 5 and 11) and returns the model maximizing coverage over this combined evaluation (Line 15). Thus the 'Test' column in Table 3 measures the selection objective itself, not holdout generalization, and the paper's claim of avoiding overfitting through a train/test split is not supported by the reported numbers. I recommend using a three-way split (train/validation/test), selecting on train and validation, and reporting coverage only on the truly held-out test set, or otherwise clarifying that the reported test coverage is part of the selection criterion.
  2. [§7 and Table 3] The paper's central attribution is undermined by the decoupling between coverage and task reward. Section 7 concedes that the coverage metric can reward overly broad distributions, and Table 3 shows Initial Test coverage of only 0.24 for Unlock, meaning the learned initial-state program assigns zero probability to roughly three-quarters of test initial states; yet Figure 3 reports strong task performance in Unlock. This indicates that planning success is not sensitive to the accuracy of the learned model in the way the central claim requires, and the gains could be carried by the determinized belief-space planner (Appendix C) or by the online repair loop of Algorithm 1. To support the claim that LLM-guided construction of the low-complexity POMDP model is what makes the method effective, the paper should provide evidence that task reward is sensitive to learned-model quality, for example by substituting oracle components for learned ones one at a time, or by showing that across candidate models, higher coverage is associated with higher planning reward.
  3. [§5.4 and Table 1] The real-robot results are based on only ten runs per condition with high variance and no significance tests. For example, in Large-Tables, the Direct LLM baseline has 2 successes out of 3 runs, and the standard deviations (e.g., 0.42 vs. 0.08 for Ours) are large relative to the means. The claim that POMDP Coder achieves 'more efficient and accurate exploration' is not statistically supported. I request confidence intervals or a significance test (e.g., a permutation or bootstrap test), or additional runs, and a discussion of the low effective sample size for baselines with fewer than ten runs.
  4. [Appendix D] The planning hyperparameters (λ, α, H, and others) are tuned to work best with the oracle ground-truth models. Because the same planner is used with the learned and tabular models, this tuning may advantage methods whose learned models resemble the oracle; for instance, the information-gain weight λ was set to 0.1 for all domains without checking whether this choice favors certain model classes. A sensitivity analysis over λ and α, or at least an explicit argument that the comparison is fair across baselines, is needed to rule out planner-tuning bias as an explanation of the results.
minor comments (5)
  1. [Appendix B, Algorithm 2] The text in Appendix B refers to 'Line 26' when returning the candidate with the highest coverage, but the pseudocode shown has only 15 lines; this cross-reference should be corrected.
  2. [Table 3] The Initial Test coverage of 0.24 for Unlock is a striking number that deserves explicit discussion in the main text, not only in a table, since it bears directly on the paper's attribution claim.
  3. [Appendix E.4] The direct LLM baseline prompt contains typos ('aciton', 'Explan') that should be fixed for reproducibility since the prompt is part of the baseline definition.
  4. [References] References [38] and [43] are the same paper (Smith and Simmons, 'Heuristic Search Value Iteration for POMDPs') and should be merged or renumbered.
  5. [Figure 3] Normalization of discounted rewards by the oracle's performance should be described more precisely, including how domains are handled if the oracle's expected reward is near zero.

Circularity Check

1 steps flagged · score 3.0 of 10

Central reward comparison is independent, but Table 3's 'test coverage' is the model-selection objective itself, not a held-out prediction.

  1. fitted input called prediction [Section 4.1 / Appendix B, Algorithm 2 lines 5, 11, 15; Table 3]
    "During evaluation, we estimate the model's coverage on a combination of the training and testing sets, evaluating models based on their ability to generalize beyond the training examples (Line 5). ... coverage,Derrors← Eval(θprev,Dtest,Dtrain) ... coverage′,D′errors← Eval(θnew,Dtest,Dtrain) ... return argmaxM(coverage)[0]"

    Algorithm 2 evaluates candidate programs on Dtest at lines 5 and 11, and line 15 returns the program with the highest coverage computed over that same Dtest. The 'Test' coverage reported in Table 3 is therefore the value of the selection objective, not an independent generalization estimate. Presenting it as 'testing coverage' is a fitted quantity renamed as a predictive check. This does not force the main reward-based comparison, but it makes Table 3 non-evidential as a measure of model fidelity.

full rationale

POMDP Coder's central claim—that LLM-guided construction of low-complexity POMDP models can outperform tabular learning, behavior cloning, and direct LLM planning—is evaluated by expected discounted reward in Figures 3 and Table 1. That comparison is an empirical outcome and is not derived algebraically from the coverage objective; no equation in the paper reduces task reward to coverage. The one concrete circularity is secondary: Algorithm 2 selects the returned program by maximizing coverage on Dtest as well as Dtrain, so the 'Test' coverage numbers in Table 3 are the optimization target itself rather than a held-out generalization measure. Section 7 further concedes that the coverage metric can reward overly broad distributions, and the Unlock row (Initial Test coverage 0.24) shows coverage can be low while task performance is good, so coverage should not be read as evidence of planning accuracy. The reliance on prior work [37] for LLM repair and Thompson sampling is a self-citation, but it is not load-bearing in the circular sense: [37] is an independent prior algorithm with its own empirical validation, and this paper extends it to POMDPs rather than deriving the method's validity from it. Overall, the central derivation is self-contained against the empirical baseline comparisons, and the circularity is confined to a reported prediction-like statistic.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method contributes a search procedure, not a derivation. Its support rests on the empirical validity of the LLM prior, the coverage objective, and the post-hoc observability assumption. The learned programs are not fitted parameters in the classical sense, but the planner hyperparameters are hand-chosen per domain.

free parameters (4)
  • lambda (information-gain weight) = 0.1 (all domains)
    Planner hyperparameter tuned to work best for the oracle models; balances reward and information gain in belief-space search (Appendix C, Table D).
  • alpha (entropy coefficient) = 0.0 (Classical/MiniGrid), 1.0 (Spot)
    Planner hyperparameter tuned per domain; weights belief entropy in the search cost (Appendix C, Table D).
  • C (Thompson smoothing constant) = 25
    Smoothing constant in the Beta distribution update during LLM repair; selected based on prior work [37] (Appendix B, Table D).
  • M (max refinements) = 25
    Iteration budget for the LLM repair loop; chosen under computational constraints (Appendix B, Table D).
assumptions (5)
  • domain assumption Post-hoc full observability: after each episode the agent receives the intermediate states.
    Section 4 states this assumption; without it the transition and observation programs cannot be trained on state-labeled demonstrations in real domains.
  • domain assumption The LLM can propose and repair probabilistic programs that are close enough to the true model within the provided code API.
    The learning loop depends on the LLM's prior to generate plausible code; no formal guarantee is given, and the paper notes that the LLM sometimes proposes overly broad distributions (Section 7).
  • ad hoc to paper Coverage, defined as the proportion of data with nonzero probability under the model, is a sufficient objective for model selection.
    The paper adopts coverage as the objective (Section 4.1) and concedes it can reward overly broad distributions, which can lead to less efficient behavior (Section 7).
  • domain assumption The state space is finite and discrete, and the POMDP components can be expressed as short probabilistic programs.
    The method targets a subclass of POMDPs; Section 7 notes the restriction to discrete state/action spaces.
  • domain assumption Particle filtering provides adequate belief updates for the chosen number of particles.
    The method uses particle filtering as the belief-updating mechanism across all domains (Section 3.1); Section 7 notes it does not scale to large state spaces.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-Guided Probabilistic Program Induction for POMDP Model Estimation." pith.science (2026). https://pith.science/paper/FE4VYZDZ

@misc{pith2026250502216,
  author       = {Pith},
  title        = {Pith review of: LLM-Guided Probabilistic Program Induction for POMDP Model Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FE4VYZDZ}},
  note         = {Machine review of arXiv:2505.02216}
}
read the original abstract

Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty. While there are many approaches to approximately solving POMDPs, we aim to address the problem of learning such models. In particular, we are interested in a subclass of POMDPs wherein the components of the model, including the observation function, reward function, transition function, and initial state distribution function, can be modeled as low-complexity probabilistic graphical models in the form of a short probabilistic program. Our strategy to learn these programs uses an LLM as a prior, generating candidate probabilistic programs that are then tested against the empirical distribution and adjusted through feedback. We experiment on a number of classical toy POMDP problems, simulated MiniGrid domains, and two real mobile-base robotics search domains involving partial observability. Our results show that using an LLM to guide in the construction of a low-complexity POMDP model can be more effective than tabular POMDP learning, behavior cloning, or direct LLM planning.

Figures

Figures reproduced from arXiv: 2505.02216 by the authors.

Figure 1
Figure 1. An architecture diagram for our POMDP coder method [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. A visualization of the final belief state for each of the MiniGrid tasks. The green square is [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Experimental results for the MiniGrid and Classical POMDP domains. We show the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The two real-world experimental setups wherein a robot is searching for an apple in a [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisions

    cs.AI 2025-11 conditional novelty 6.0 of 10

    MAPs is a new amusement-park simulator benchmark on which frontier LLM agents score 7–15% of human performance, exposing persistent gaps in long-horizon planning, active learning, spatial reasoning, and handling stoch...

Reference graph

Works this paper leans on

43 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [37]

    H. Tang, K. Hu, J. P. Zhou, S. Zhong, W.-L. Zheng, X. Si, and K. Ellis. Code re- pair with llms gives an exploration-exploitation tradeoff. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neu- ral Information Processing Systems, volume 37, pages 117954–117996. Curran Associates, Inc., 2024. URL ht...

    work page 2024

  2. [1]

    W. Song, G. Xiong, and H. Chen. Intention-aware autonomous driving decision-making in an uncontrolled intersection. Mathematical Problems in Engineering, 2016:1–15, 01 2016. doi:10.1155/2016/1025349

    work page doi:10.1155/2016/1025349 2016

  3. [2]

    Ghorbel, J

    M. Ghorbel, J. Pineau, R. Gourdeau, S. Javdani, and S. Srinivasa. A decision-theoretic approach for the collaborative control of a smart wheelchair. International Journal of Social Robotics, 10, 01 2018. doi:10.1007/s12369-017-0434-7

    work page doi:10.1007/s12369-017-0434-7 2018

  4. [3]

    Pajarinen and V

    J. Pajarinen and V . Kyrki. Robotic manipulation of multiple objects as a pomdp. Artificial Intel- ligence, 247:213–228, 2017. ISSN 0004-3702. doi:https://doi.org/10.1016/j.artint.2015.04.001. URL https://www.sciencedirect.com/science/article/pii/S0004370215000570. Special Issue on AI and Robotics

    doi:10.1016/j.artint.2015.04.001 2017

  5. [4]

    Zheng, A

    K. Zheng, A. Paul, and S. Tellex. A system for generalized 3d multi-object search. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1638–1644, 2023. doi:10.1109/ICRA48891.2023.10161387

    arXiv 2023

  6. [5]

    T. M. Moerland, J. Broekens, and C. M. Jonker. Model-based reinforcement learning: A survey. CoRR, abs/2006.16712, 2020. URL https://arxiv.org/abs/2006.16712

    arXiv 2006

  7. [6]

    Learning nonsingular phylogenies and hidden Markov models

    E. Mossel and S. Roch. Learning nonsingular phylogenies and hidden markov models. CoRR, abs/cs/0502076, 2005. URL http://arxiv.org/abs/cs/0502076

    work page Pith review arXiv 2005

  8. [7]

    S. Ross, B. Chaib-draa, and J. Pineau. Bayes-adaptive pomdps. In J. Platt, D. Koller, Y . Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007. URL https://proceedings.neurips.cc/paper_files/paper/ 2007/file/3b3dbaf68507998acd6a5a5254ab2d76-Paper.pdf

    work page 2007

Show all 43 references
  1. [8]

    Boots, S

    B. Boots, S. M. Siddiqi, and G. J. Gordon. Closing the learning-planning loop with predictive state representations, 2009. URL https://arxiv.org/abs/0912.2385

    2009 arXiv

  2. [9]

    C. Jin, S. M. Kakade, A. Krishnamurthy, and Q. Liu. Sample-efficient reinforcement learning of undercomplete pomdps. CoRR, abs/2006.12484, 2020. URL https://arxiv.org/abs/ 2006.12484

    2006 arXiv

  3. [10]

    Makino and J

    T. Makino and J. Takeuchi. Apprenticeship learning for model parameters of partially observable environments, 2012. URL https://arxiv.org/abs/1206.6484

    2012 arXiv

  4. [11]

    Thomson, F

    B. Thomson, F. Jurˇc´ıˇcek, M. Gaˇsi´c, S. Keizer, F. Mairesse, K. Yu, and S. Young. Parameter learning for pomdp spoken dialogue models. In 2010 IEEE Spoken Language Technology Workshop, pages 271–276, 2010. doi:10.1109/SLT.2010.5700863

    2010

  5. [12]

    Bingham, J

    E. Bingham, J. P. Chen, M. Jankowiak, F. Obermeyer, N. Pradhan, T. Karaletsos, R. Singh, P. A. Szerlip, P. Horsfall, and N. D. Goodman. Pyro: Deep universal probabilistic programming. CoRR, abs/1810.09538, 2018. URL http://arxiv.org/abs/1810.09538

    2018 arXiv

  6. [13]

    M. F. Cusumano-Towner, F. A. Saad, A. K. Lew, and V . K. Mansinghka. Gen: a general- purpose probabilistic programming system with programmable inference. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, page 221–2...

    2019

  7. [14]

    N. D. Goodman, V . Mansinghka, D. M. Roy, K. A. Bonawitz, and J. B. Tenenbaum. Church: a language for generative models. CoRR, abs/1206.3255, 2012. URL http://arxiv.org/abs/ 1206.3255. 10

    2012 arXiv

  8. [15]

    Ellis, C

    K. Ellis, C. Wong, M. I. Nye, M. Sabl ´e-Meyer, L. Cary, L. Morales, L. B. Hewitt, A. Solar- Lezama, and J. B. Tenenbaum. Dreamcoder: Growing generalizable, interpretable knowledge with wake-sleep bayesian program learning. CoRR, abs/2006.08381, 2020. URL https: //arxiv.org/ab...

    2006 arXiv

  9. [16]

    B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332–1338, 2015. doi:10.1126/science. aab3050. URL https://doi.org/10.1126/science.aab3050

    2015 doi

  10. [17]

    M. Y . Li, E. B. Fox, and N. D. Goodman. Automated statistical model discovery with language models, 2024. URL https://arxiv.org/abs/2402.17879

    2024 arXiv

  11. [18]

    L. Wong, G. Grand, A. K. Lew, N. D. Goodman, V . K. Mansinghka, J. Andreas, and J. B. Tenenbaum. From word models to world models: Translating from natural language to the probabilistic language of thought, 2023. URL https://arxiv.org/abs/2306.12672

    2023 arXiv

  12. [19]

    Grand, V

    G. Grand, V . Pepe, J. Andreas, and J. B. Tenenbaum. Loose lips sink ships: Asking questions in battleship with language-informed program sampling, 2024. URL https://arxiv.org/abs/ 2402.19471

    2024 arXiv

  13. [20]

    Liang, N

    Y . Liang, N. Kumar, H. Tang, A. Weller, J. B. Tenenbaum, T. Silver, J. F. Henriques, and K. Ellis. Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning, 2025. URL https://arxiv.org/abs/2410.23156

    2025 arXiv

  14. [21]

    H. Tang, D. Key, and K. Ellis. Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment, 2024. URL https://arxiv.org/abs/ 2402.12275

    2024 arXiv

  15. [22]

    Curtis, N

    A. Curtis, N. Kumar, J. Cao, T. Lozano-P´erez, and L. P. Kaelbling. Trust the proc3s: Solving long-horizon robotics problems with llms and constraint satisfaction, 2024. URL https: //arxiv.org/abs/2406.05572

    2024 arXiv

  16. [23]

    Y . Hao, Y . Chen, Y . Zhang, and C. Fan. Large language models can plan your travels rigorously with formal verification tools. In arxiv preprint, 2024. URL https://arxiv.org/abs/2404. 11891

    2024

  17. [24]

    X. Ye, Q. Chen, I. Dillig, and G. Durrett. Satlm: Satisfiability-aided language models using declarative prompting. Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https://arxiv.org/pdf/2305.09656

    2024 arXiv

  18. [25]

    Thrun, W

    S. Thrun, W. Burgard, and D. Fox. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, 2005. ISBN 0262201623

    2005

  19. [26]

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1):99–134, 1998. ISSN 0004-3702. doi: https://doi.org/10.1016/S0004-3702(98)00023-X. URL https://www.sciencedirect.com/ science/art...

    1998 doi

  20. [27]

    Mundhenk, J

    M. Mundhenk, J. Goldsmith, and E. Allender. The complexity of policy evaluation for finite- horizon partially-observable markov decision processes. In I. Pr´ıvara and P. Ruˇziˇcka, editors, Mathematical Foundations of Computer Science 1997, pages 129–138, Berlin, Heidelberg, 1...

    1997

  21. [28]

    M. Littman. The witness algorithm: Solving partially observable markov decision processes. 02 1995

    1995

  22. [29]

    A. R. Cassandra, M. L. Littman, and N. L. Zhang. Incremental pruning: A simple, fast, exact method for partially observable markov decision processes. CoRR, abs/1302.1525, 2013. URL http://arxiv.org/abs/1302.1525. 11

    2013 arXiv

  23. [30]

    Silver and J

    D. Silver and J. Veness. Monte-Carlo planning in large POMDPs. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010

    2010

  24. [31]

    Curtis, L

    A. Curtis, L. Kaelbling, and S. Jain. Task-directed exploration in continuous pomdps for robotic manipulation of articulated objects, 2022. URL https://arxiv.org/abs/2212.04554

    2022 arXiv

  25. [32]

    Sunberg and M

    Z. Sunberg and M. J. Kochenderfer. POMCPOW: an online algorithm for pomdps with continuous state, action, and observation spaces. CoRR, abs/1709.06196, 2017. URL http: //arxiv.org/abs/1709.06196

    2017 arXiv

  26. [33]

    S. W. Yoon, A. Fern, and R. Givan. Ff-replan: A baseline for probabilistic planning. In International Conference on Automated Planning and Scheduling, 2007. URL https://api. semanticscholar.org/CorpusID:15013602

    2007

  27. [34]

    Kaelbling and T

    L. Kaelbling and T. Lozano-Perez. Integrated task and motion planning in belief space. The International Journal of Robotics Research , 32:1194–1227, 08 2013. doi:10.1177/ 0278364913484072

    2013

  28. [35]

    Curtis, G

    A. Curtis, G. Matheos, N. Gothoskar, V . Mansinghka, J. Tenenbaum, T. Lozano-P´erez, and L. P. Kaelbling. Partially observable task and motion planning with uncertainty and risk awareness,

  29. [36]

    Pinto, M

    L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel. Asymmetric actor critic for image-based robot learning. CoRR, abs/1710.06542, 2017. URL http://arxiv.org/ abs/1710.06542

    2017 arXiv

  30. [39]

    Chevalier-Boisvert, B

    M. Chevalier-Boisvert, B. Dai, M. Towers, R. de Lazcano, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. CoRR, abs/2306.13831, 2023

    2023 arXiv

  31. [40]

    Polycam – capture and share 3d models, 2025

    Polycam. Polycam – capture and share 3d models, 2025. URLhttps://poly.cam/. Accessed: 2025-04-30

    2025

  32. [41]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  33. [42]

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024

    2024

  34. [46]

    "" Input: empty_state (MiniGridState): An empty state with only the walls filled into the grid,→ Returns: state (MiniGridState): the initial state of the environment

    URL http://arxiv.org/abs/1207.4166. 12 A Code Release The full implementation, including all code, trained models, and experiment configurations, will be released publicly upon publication to ensure reproducibility and facilitate future research. B Model Learning The learning ...

    arXiv

  35. [2024]

    URL https://arxiv.org/abs/2403.10454

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.