Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Maps

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A model-based agent can navigate a never-before-seen maze layout zero-shot by reading an abstract 2-D top-down map, without exploring or retraining.

desk verdict A solid empirical methods paper whose central zero-shot result holds up, but the unstated map-generation process undercuts the stronger 'novel layout' generalization claim. read the letter →

arxiv 2412.12024 v1 pith:27FP4AMJ submitted 2024-12-16 cs.LG cs.AIcs.RO

Linfeng Zhao , Lawson L.S. Wong This is my paper · ORCID
classification cs.LGcs.AIcs.RO
keywords zero-shotnavigationmodel-basedreinforcementlearninghypernetworkMuZerotop-downmapcompositionalgeneralizationmazehindsightexperiencereplay
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MMN is a model-based navigation agent that treats a 2-D top-down map as an image and uses it, along with start and goal markers, to generate the weights of its learned transition model. The claim is that after training on only 20 maze layouts, the agent can plan and execute near-optimal routes through unseen layouts on its first visit, with no exploration, no explicit localization, and no hand-coded path planning. This matters because it turns map-based navigation into a compositional generalization problem: local wall and corridor patterns learned in training maps recombine to describe novel layouts. In the experiments the map-conditioned model-based agent outperforms a map-conditioned model-free baseline, especially on long routes, and degrades gracefully when the map or the agent's reported location is noisy.

What carries the argument

The load-bearing object is the task-conditioned hypermodel: a meta-network $h_\psi$ maps a task context $c$ (downsampled global occupancy map, cropped local occupancy map, and one-hot start/goal grids) to the parameter vector $\phi$ of a small transition network $f_\phi(s,a)=(s',r)$. Because $\phi$ is generated per map-goal pair rather than selected from a finite set, the transition model can recombine local wall/gap patterns seen in training maps into a dynamics model for an unseen layout. The second component is MuZero-style MCTS over this hypermodel, with a task-conditioned value/policy network $g_\theta(s,c)$; training uses an $n$-step hindsight goal-relabeling scheme and an auxiliary transition-prediction loss that maximizes mutual information between the context and predicted trajectories. The use of the agent's own action space inside the latent model is what lets planned trajectories transfer to the environment without explicit localization.

What would settle it

Generate a separate set of evaluation mazes using a different procedural rule that creates local motifs absent from the 20 training maps, such as diagonal wall segments, isolated pillars, or staggered T-junctions, then run MMN zero-shot on them; the central claim is falsified if success rate and SPL drop to roughly the random baseline on these maps while remaining high on maps that merely recombine training patches.

Watch

Extended reading notes

Core claim

On a 3-D maze simulator with continuous pose and velocity states, the paper trains a MuZero-style planner whose transition network weights are produced by a hypermodel $h_\psi(c) = \phi$ from the abstract map and goal context $c = (m,g)$. Planning runs Monte Carlo tree search in a learned latent state space that keeps the agent's original action space, so the resulting action sequences can be executed directly without solving the 2-D-to-3-D correspondence problem. At zero-shot evaluation on 20 unseen $13\times 13$ maps, MMN beats the map-conditioned Ape-X HER DQN baseline MAH, especially for start-goal distances beyond the training range; with a landmark oracle providing subgoals, MMN reaches 16 of 20 global goals, including all 9 that MAH reaches. The paper also reports that MMN remains effective when the abstract map is flipped with noise, the map-to-environment scaling is perturbed, or the reported position is corrupted, whereas a deterministic planner with perfect information fails under the same localization noise.

Load-bearing premise

The method assumes that every never-before-seen maze is built from local wall and corridor patterns that already appeared in the 20 training maps, so that a model trained on those patches can predict how the new maze behaves.

Editorial extensions

If this is right

  • On 20 unseen $13\times 13$ maps, MMN achieves higher zero-shot success than the model-free MAH baseline across start-goal distances, with the gap widening at longer distances even though training only used distances of 1 to 5.
  • With landmark-generated subgoals, MMN reaches 16 of 20 global goals and achieves SPL around 0.6, covering all 9 successes of MAH; MAH reaches 9 of 20 with SPL around 0.28.
  • When trained on larger maps ($15\times 15$ up to $21\times 21$), MMN shows the same pattern: local training transfers to unseen layouts but distant goals remain harder.
  • Under map flips, perspective/scaling perturbations, and localization noise up to 50 units (half a cell width), MMN's SPL degrades gradually, while a hand-crafted deterministic planner with perfect information gets stuck when localization is noisy.
  • Both map-conditioned agents outperform a single-task DQN and random actions, indicating that the map and goal context, not the base algorithm alone, carry the zero-shot transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The same weight-prediction mechanism should transfer to other multi-task model-based settings where a family of MDPs is indexed by a compact context, such as manipulation with different object layouts or navigation from raw pixels, a direction the paper flags as future work.
  • Inference: The compositional premise suggests a testable scaling law: as the vocabulary of local wall/corridor patches in training maps grows, zero-shot success on recombined layouts should improve, and deliberately constructed maps with out-of-distribution local motifs should define the method's failure boundary.
  • Inference: Because MMN tolerates half-cell localization noise without training on noise, the learned planner appears to rely on coarse map structure rather than precise metric alignment; adding pose noise during training would likely extend this tolerance further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies zero-shot navigation in novel maze layouts in DeepMind Lab. The agent is given an abstract 2-D occupancy map plus start/goal indicators and must reach the goal in an unseen layout without exploration. The proposed method, MMN, extends MuZero with a hypermodel h_psi that maps the task context (map and goal grids) to the weights of a latent transition model, and planning is done by MCTS in that learned model. A model-free baseline MAH (map-conditioned Ape-X DQN with HER) and a single-task DQN are compared. On 20 held-out 13x13 maps, MMN outperforms MAH especially for long distances; with a landmark oracle for hierarchical navigation, MMN reaches 16/20 global goals versus 9/20 for MAH. Appendix B studies robustness to map flips, action-mapping perturbations, and localization noise, comparing MMN to a deterministic planner. The paper claims MMN generalizes to novel layouts via compositional generalization over local map patches and is more robust to noise.

Significance. If the claims hold, the contribution is a meaningful step: it demonstrates that a model-based planner conditioned on an abstract map can transfer to unseen layouts in a continuous 3-D environment without exploration, with a large empirical gap over a map-conditioned model-free baseline. The hierarchical result (16/20 vs 9/20, including all MAH successes) is a strong and clean comparison. The auxiliary-loss derivation in Appendix A.2 is a standard variational lower bound for mutual information and is not circular; the headline result is measured on unseen maps, so it is not a test-set-fitting artifact. The main limitations are that the map-generation process is unspecified, so the novelty of the held-out layouts is not established, and the robustness comparison in the abstract is supported only against a hard-coded planner, not against the learned baseline.

major comments (3)
  1. [§4, §5.1] The central generalization claim rests on challenge (ii) in §4: the method must achieve compositional generalization from existing map patches to novel combinations of patches. However, the paper never specifies the map-generation algorithm or a characterization of the local patch distribution; §3 and §5.1 only state that maps are 'generated' and made fully connected. If the 20 evaluation maps are fresh draws from the same generator as the 20 training maps, their local n×n wall/gap patterns may substantially overlap with training patches, and the 16/20 success rate in §5.3 would demonstrate within-distribution generalization rather than recombination to genuinely novel local structures. Please provide the generator (or code) and an analysis of train/eval patch coverage, e.g., the fraction of local n×n patches in the evaluation maps that also appear in training maps for the patch sizes used by the hypermodel.
  2. [Abstract, Appendix B] The abstract claims that MMN is 'more robust to noise', but Appendix B compares MMN only with a hard-coded deterministic planner that assumes perfect localization and landmarks; there is no comparison with MAH under the same map-flip, action-mapping, or location-noise perturbations. Since the robustness claim is one of the two headline contributions, it should be evaluated against the same learned baseline, or the claim should be restricted to robustness relative to the deterministic planner. In addition, Table 2 and Figure 6 report no confidence intervals or significance tests, so the qualitative 'graceful degradation' statement is not quantified.
  3. [§5.1, Appendix A] The experimental section omits most implementation details needed to assess or reproduce the method: the hypermodel and encoder architectures, the dimensionality of the latent space and of the predicted weight vector, the number of MCTS simulations, the HER horizon n, the training hyperparameters, and the perturbation magnitudes (flip ratios, noise levels) are not given. Without these, the reader cannot judge whether the reported gap between MMN and MAH could be sensitive to, for example, the MCTS budget or the HER horizon; at minimum, an ablation over the MCTS budget and n is needed to support the claim that the improvement comes from model-based planning rather than from more favorable tuning of the model-based agent.
minor comments (5)
  1. [§3] The text says 'nagivable spaces'; this should be 'navigable spaces'.
  2. [Figure 1 caption] The caption contains a duplicated phrase: 'The map provides a rough solution solution'; one 'solution' should be removed.
  3. [References] The reference list contains several typos: 'Schrittweiser' in Beattie et al., 'Davod Silver' in Wayne et al., and 'W ong' in the author affiliation appears to have an inserted space.
  4. [§4.1] The transition function is first described as predicting state and reward, but then defined as f_phi: S×A→S only; please clarify whether the reward is part of the hypermodel output or computed separately in the MCTS node.
  5. [Table 1] The evaluation-metrics paragraph in §5.1 mentions 95% confidence intervals, but Table 1 does not report intervals for the success rate; please add the intervals or state the per-cell sample size.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central zero-shot claim is evaluated on unseen maps, and the auxiliary losses are standard variational/transition-prediction objectives rather than inputs renamed as outputs.

full rationale

I walked the paper's claimed derivation chain: a hypermodel h_psi maps an abstract map and goal context to weights of a latent transition model; MCTS plans with that learned model; training uses n-step hindsight relabeling plus an auxiliary model loss. The auxiliary model loss is derived as a variational lower bound on mutual information I(c_T; tau_T) and reduces to minimizing the prediction error between predicted and true next states; this is a standard bounding argument, not a result that is equivalent to its input by construction. Zero-shot evaluation is conducted on 20 unseen maps with no further training, so the headline result (16/20 hierarchical goals) is not a fitted value on the evaluation set. The hierarchical experiment chooses landmark distance 5 because the agent was trained on start-goal distances <= 5; this does place subgoal navigation within the training-distance distribution, but completing a chain of subgoals on novel layouts is still an independent behavioral claim and is not forced by construction. The numerous self-citations to Zhao et al. appear in related-work and background passages and are not load-bearing for the method's central premise; no uniqueness theorem or prior-work mandate is invoked to forbid alternatives. The unspecified map-generation process is a legitimate generalization-support concern, but under the stated rules it is a missing-support issue rather than circularity. No self-definitional step, fitted-input-called-prediction step, load-bearing self-citation, ansatz-smuggled-via-citation step, or renaming of a known result was identified.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the environment being deterministic, the abstract map encoding the task dynamics, compositional generalization of local map patches, and the capacities of the hypermodel and learned encoder. None of these are derived; they are domain assumptions or design postulates. The quantitative inputs are experimental hyperparameters, many of which are unreported.

free parameters (8)
  • MCTS simulation budget
    Number of MCTS simulations per action is not reported; the quality of zero-shot planning depends directly on this budget.
  • n-step HER horizon n
    The n-step relabeling window is not given; it controls reward density and learning stability.
  • Goal area size = 100x100 world units per abstract cell
    Section 5.1 defines the goal region as a 100x100 block; success and reward depend on this size.
  • Action repeat = 10
    Section 5.1 sets action repeat to 10, which determines the effective step duration of the agent.
  • Number of training maps = 20
    All agents train on 20 maps per size; the hypermodel's compositional generalization is only tested under this small training set.
  • Landmark distance for hierarchical navigation = 5
    Section 5.3 sets subgoal spacing to 5 cells, which matches the training distance range; hierarchical results depend on this choice.
  • Network architectures
    Encoder, hypermodel, transition, policy, and value network sizes and activations are unspecified.
  • Training hyperparameters
    Learning rates, batch sizes, optimizer, and convergence criteria are not reported.
assumptions (6)
  • domain assumption Transitions are deterministic in the learning setting.
    Section 3 states 'In the learning, we assume transitions are deterministic.' MuZero-style planning and the auxiliary transition loss rely on this.
  • domain assumption The abstract 2-D occupancy map and one-hot start/goal grids contain sufficient information to specify the task-specific transition and reward structure.
    Section 3 defines task context c=(m,g) and treats it as fully specifying dynamics and reward. If this correspondence is not recoverable, zero-shot navigation is impossible.
  • ad hoc to paper Unseen maps can be handled by compositional generalization of local map patches seen in 20 training maps.
    Section 4 states that maps often exhibit common local patterns and require the ability to generalize to recombination of known patterns, which is the key learning premise behind the hypermodel.
  • ad hoc to paper A neural hypermodel h_psi can map task context to transition-network weights such that the resulting transition model predicts the next latent state accurately.
    Section 4.1 postulates the soft weight-sharing manifold: parameter vectors live in a low-dimensional manifold. This is an architectural assumption, not proven.
  • ad hoc to paper The learned encoder maps observed 12-dimensional joint states into a latent space where the transition model is accurate and MCTS planning transfers to the real environment.
    Section 4.2 describes encoding observations into latent states; the paper assumes this latent space supports reliable value/policy prediction and plan execution.
  • domain assumption A landmark oracle is available and provides correct subgoals for hierarchical navigation when used.
    Section 5.3 requires an additional landmark oracle to generate sequences of subgoals; zero-shot success in that setting depends on oracle quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Maps." pith.science (2026). https://pith.science/paper/27FP4AMJ

@misc{pith2026241212024,
  author       = {Pith},
  title        = {Pith review of: Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Maps},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/27FP4AMJ}},
  note         = {Machine review of arXiv:2412.12024}
}
abstract

Learning navigation capabilities in different environments has long been one of the major challenges in decision-making. In this work, we focus on zero-shot navigation ability using given abstract $2$-D top-down maps. Like human navigation by reading a paper map, the agent reads the map as an image when navigating in a novel layout, after learning to navigate on a set of training maps. We propose a model-based reinforcement learning approach for this multi-task learning problem, where it jointly learns a hypermodel that takes top-down maps as input and predicts the weights of the transition network. We use the DeepMind Lab environment and customize layouts using generated maps. Our method can adapt better to novel environments in zero-shot and is more robust to noise.

Figures

Figures reproduced from arXiv: 2412.12024 by the authors.

Figure 1
Figure 1. We develop an agent that can perform zero-shot navigation on unseen maps T (in DeepMind Lab, blue box), without needing to first explore the new 3-D environment. Instead, the agent is given the top-down view as additional guidance: an abstract 2-D occupancy map, and a goal and start position (bottom-left black dot and top-right gray dot). The map provides a rough solution solution, the path cannot be directly follow… view at source ↗
Figure 2
Figure 2. Applying the hypermodel hψ on map m1 and m2 outputs two sets of transition network weights ϕ1 = hψ(m1, g1) and ϕ2 = hψ(m2, g2). Each transition network uses their weight ϕi to predict the next state f(s, a; ϕi) = s ′ , illustrated at the bottom. Since the maps may share local patterns at some scales (illustrated by the cropped 3 × 3 patches in light blue), they can be captured by the hypermodel hψ. 4 Learning to nav… view at source ↗
Figure 3
Figure 3. The planning/learning process. Yellow boxes indicate predictions; grey boxes come from actual interactions. (Left) Inference: search with learned model. Applying MCTS with hypermodel to search for policy and value, and act with a sampled action. (Right) Training: building learning targets. Computing targets and backpropagating from loss. The dark blue line indicates n-step relabelling. We only illustrate backpropaga… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: (Left) Zero-shot evaluation performance on 13 × 13 maps. Local navigation with different distances between start and goal, from 1 to 15. (Right) Performance of our method on larger maps. generate a set of landmarks with a fixed distance of 5 between them and provide th…
Figure 5
Figure 5. Figure 5: Trajectories from hierarchical navigation in zero-shot on 13 × 13 maps. The top row is for MMN and bottom row is for MAH. Since there is a fixed scaling factor from maps to environments, we can compute the corresponding location on the abstract map and visualize trajec…
Figure 6
Figure 6. Figure 6: Violin plots show the SPL of MMN with different map flip ratio [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: (left pair) MMN visualized with perturbed locations; even though the provided state is noisy, MMN successfully reaches the goal. (right pair) Deterministic planner is unable to reach the goal when the provided state is noisy. (We only show the unperturbed locations in …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 37 canonical work pages

  1. [1]

    Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, and Amir R. Zamir. On evaluation of embodied navigation agents. arXiv:1807.06757, 2018

    arXiv 2018

  2. [2]

    Hindsight experience replay

    Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. In Neural Information Processing Systems, 2017

    work page 2017

  3. [3]

    Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J. Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik, Stephen Gaffney, Helen King, Koray Kavukcuoglu...

    work page 2018

  4. [4]

    Deep M ind lab

    Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich K \"u ttler, Andrew Lefrancq, Simon Green, V \' ctor Vald \'e s, Amir Sadik, Julian Schrittweiser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg, and Stig Petersen. Deep M ind lab. arXiv:1612.03801, 2016

    arXiv 2016

  5. [5]

    Teaching a machine to read maps with deep reinforcement learning

    Gino Brunner, Oliver Richter, Yuyi Wang, and Roger Wattenhofer. Teaching a machine to read maps with deep reinforcement learning. In AAAI Conference on Artificial Intelligence , 2018

    work page 2018

  6. [6]

    Learning to explore using active neural SLAM

    Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning to explore using active neural SLAM . In International Conference on Learning Representations, 2020

    work page 2020

  7. [7]

    Learning exploration policies for navigation

    Tao Chen, Saurabh Gupta, and Abhinav Gupta. Learning exploration policies for navigation. In International Conference on Learning Representations, 2019

    work page 2019

  8. [8]

    Deep reinforcement learning in a handful of trials using probabilistic dynamics models

    Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. In Neural Information Processing Systems, 2018

    work page 2018

Show all 45 references
  1. [9]

    Learning to act by predicting the future

    Alexey Dosovitskiy and Vladlen Koltun. Learning to act by predicting the future. In International Conference on Learning Representations, 2017

    2017

  2. [10]

    Leibo, and Charles Blundell

    Meire Fortunato, Melissa Tan, Ryan Faulkner, Steven Hansen, Adri\` a Puigdom\` e nech Badia, Gavin Buttimore, Charlie Deck, Joel Z. Leibo, and Charles Blundell. Generalization of reinforcement learners with working and episodic memory. In Neural Information Processing Systems, 2019

    2019

  3. [11]

    Cognitive mapping and planning for visual navigation

    Saurabh Gupta, Varun Tolani, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. Cognitive mapping and planning for visual navigation. International Journal on Computer Vision, 128: 0 1311--1330, 2020

    2020

  4. [12]

    Hyper N etworks

    David Ha, Andrew Dai, and Quoc V Le. Hyper N etworks. In International Conference on Learning Representations, 2017

    2017

  5. [13]

    Learning latent dynamics for planning from pixels

    Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning, 2019

    2019

  6. [14]

    Distributed prioritized experience replay

    Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado van Hasselt, and David Silver. Distributed prioritized experience replay. In International Conference on Learning Representations, 2018

    2018

  7. [15]

    Equivariant single view pose prediction via induced and restriction representations

    Owen Howell, David Klee, Ondrej Biza, Linfeng Zhao, and Robin Walters. Equivariant single view pose prediction via induced and restriction representations. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing ...

    2023

  8. [16]

    Continual model-based reinforcement learning with hypernetworks

    Yizhou Huang, Kevin Xie, Homanga Bharadhwaj, and Florian Shkurti. Continual model-based reinforcement learning with hypernetworks. In IEEE International Conference on Robotics and Automation , 2021

    2021

  9. [17]

    Reinforcement learning with unsupervised auxiliary tasks

    Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, and Koray Kavukcuoglu. Reinforcement learning with unsupervised auxiliary tasks. In International Conference on Learning Representations, 2017

    2017

  10. [18]

    Open-vocabulary pick and place via patch-level semantic maps

    Mingxi Jia, Haojie Huang, Zhewen Zhang, Chenghao Wang, Linfeng Zhao, Dian Wang, Jason Xinyu Liu, Robin Walters, Robert Platt, and Stefanie Tellex. Open-vocabulary pick and place via patch-level semantic maps. 2024. URL https://openreview.net/forum?id=cY3jXubzpR&referrer=

    2024

  11. [19]

    Differentiable algorithm networks for composable robot learning

    Peter Karkus, Xiao Ma, David Hsu, Leslie Pack Kaelbling, Wee Sun Lee, and Tom\' a s Lozano-P\' e rez. Differentiable algorithm networks for composable robot learning. In Robotics: Science and Systems, 2019

    2019

  12. [20]

    Practice makes perfect: Planning to learn skill parameter policies

    Nishanth Kumar, Tom Silver, Willie McClinton, Linfeng Zhao, Stephen Proulx, Tomás Lozano-Pérez, Leslie Pack Kaelbling, and Jennifer Barry. Practice makes perfect: Planning to learn skill parameter policies. arXiv preprint arXiv: 2402.15025, 2024

    2024 arXiv

  13. [21]

    Playing FPS games with deep reinforcement learning

    Guillaume Lample and Devendra Singh Chaplot. Playing FPS games with deep reinforcement learning. In AAAI Conference on Artificial Intelligence , 2017

    2017

  14. [22]

    Gated path planning networks

    Lisa Lee, Emilio Parisotto, Devendra Singh Chaplot, Eric Xing, and Ruslan Salakhutdinov. Gated path planning networks. In International Conference on Machine Learning, 2018

    2018

  15. [23]

    Discriminative particle filter reinforcement learning for complex partial observations

    Xiao Ma, Peter Karkus, David Hsu, Wee Sun Lee, and Nan Ye. Discriminative particle filter reinforcement learning for complex partial observations. In International Conference on Learning Representations, 2020

    2020

  16. [24]

    Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, Dharshan Kumaran, and Raia Hadsell

    Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J. Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, Dharshan Kumaran, and Raia Hadsell. Learning to navigate in complex environments. In International Conference on Learning ...

    2017

  17. [25]

    Learning to navigate in cities without a map

    Piotr Mirowski, Matthew Koichi Grimes, Mateusz Malinowski, Karl Moritz Hermann, Keith Anderson, Denis Teplyashin, Karen Simonyan, Koray Kavukcuoglu, Andrew Zisserman, and Raia Hadsell. Learning to navigate in cities without a map. In Neural Information Processing Systems, 2018

    2018

  18. [26]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, ...

    2015

  19. [27]

    Goal-directed planning via hindsight experience replay

    Lorenzo Moro, Amarildo Likmeta, Enrico Prati, and Marcello Restelli. Goal-directed planning via hindsight experience replay. In International Conference on Learning Representations, 2022

    2022

  20. [28]

    Value prediction network

    Junhyuk Oh, Satinder Singh, and Honglak Lee. Value prediction network. In Neural Information Processing Systems, 2017

    2017

  21. [29]

    Neural map: Structured memory for deep reinforcement learning

    Emilio Parisotto and Ruslan Salakhutdinov. Neural map: Structured memory for deep reinforcement learning. In International Conference on Learning Representations, 2018

    2018

  22. [30]

    Learning symmetric embeddings for equivariant world models

    Jung Yeon Park, Ondrej Biza, Linfeng Zhao, Jan Willem van de Meent, and Robin Walters. Learning symmetric embeddings for equivariant world models. In ICML, 2022

    2022

  23. [31]

    Temporal difference models: Model-free deep RL for model-based control

    Vitchyr Pong, Shixiang Gu, Murtaza Dalal, and Sergey Levine. Temporal difference models: Model-free deep RL for model-based control. In International Conference on Learning Representations, 2018

    2018

  24. [32]

    Mastering A tari, G o, chess and shogi by planning with a learned model

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, and David Silver. Mastering A tari, G o, chess and shogi by planning with a learned model. Nat...

    2020

  25. [33]

    Value iteration networks

    Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. In Neural Information Processing Systems, 2016

    2016

  26. [34]

    Probabilistic Robotics

    Sebastian Thrun, Wolfram Burgard, and Dieter Fox. Probabilistic Robotics. The MIT Press, Cambridge, MA, 2005

    2005

  27. [35]

    Grewe, and Jo \ a o Sacramento

    Johannes von Oswald, Christian Henning, Benjamin F. Grewe, and Jo \ a o Sacramento. Continual learning with hypernetworks. In International Conference on Learning Representations, 2020

    2020

  28. [36]

    Greg Wayne, Chia-Chun Hung, David Amos, Mehdi Mirza, Arun Ahuja, Agnieszka Grabska-Barwinska, Jack Rae, Piotr Mirowski, Joel Z. Leibo, Adam Santoro, Mevlana Gemici, Malcolm Reynolds, Tim Harley, Josh Abramson, Shakir Mohamed, Danilo Rezende, David Saxton, Adam Cain, Chloe Hill...

    2018 arXiv

  29. [37]

    Clara De Paolis Kaluza, Linfeng Zhao, Lawson Wong, and Rose Yu

    Fan Xie, Alexander Chowdhury, M. Clara De Paolis Kaluza, Linfeng Zhao, Lawson Wong, and Rose Yu. Deep imitation learning for bimanual robotic manipulation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Syste...

    2020

  30. [38]

    Toward compositional generalization in object-oriented world modeling

    Linfeng Zhao, Lingzhi Kong, Robin Walters, and Lawson LS Wong. Toward compositional generalization in object-oriented world modeling. In ICML, 2022

    2022

  31. [39]

    Linfeng Zhao, Owen Howell, Jung Yeon Park, Xupeng Zhu, Robin Walters, and Lawson L. S. Wong. Can euclidean symmetry be leveraged in reinforcement learning and planning? arXiv preprint arXiv: 2307.08226, 2023 a

    2023 arXiv

  32. [40]

    Linfeng Zhao, Huazhe Xu, and Lawson L.S. Wong. Scaling up and stabilizing differentiable planning with implicit differentiation. In International Conference on Learning Representations, 2023 b

    2023

  33. [41]

    Linfeng Zhao, Xupeng Zhu, Lingzhi Kong, Robin Walters, and Lawson L.S. Wong. Integrating symmetry into differentiable planning with steerable convolutions. In International Conference on Learning Representations, 2023 c

    2023

  34. [42]

    Linfeng Zhao, Owen Lewis Howell, Xupeng Zhu, Jung Yeon Park, Zhewen Zhang, Robin Walters, and Lawson L.S. Wong. Equivariant action sampling for reinforcement learning and planning. In The 16th International Workshop on the Algorithmic Foundations of Robotics, 2024 a . URL http...

    2024

  35. [43]

    Linfeng Zhao, Hongyu Li, Taşkın Padır, Huaizu Jiang, and Lawson L.S. Wong. E(2) -equivariant graph planning for navigation. IEEE Robotics and Automation Letters, 9 0 (4): 0 3371--3378, 2024 b . doi:10.1109/LRA.2024.3360011

    2024

  36. [44]

    RTFM : Generalising to new environment dynamics via reading

    Victor Zhong, Tim Rockt\"aschel, and Edward Grefenstette. RTFM : Generalising to new environment dynamics via reading. In International Conference on Learning Representations, 2020

    2020

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.