Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Encoding physics priors as symbolic DSL programs and using them to guide PPO yields consistently shorter paths on unseen indoor maps while cutting training GPU hours by 26–83%.

desk verdict PiPRL is a solid engineering contribution whose empirical gains look real, but the paper's importance-sampling correction is biased as written and the sample-efficiency comparison is muddied by an early-stopping protocol. read the letter →

arxiv 2506.22365 v1 pith:DAEGA3QJ submitted 2025-06-27 cs.LG cs.RO

Tao Li , Haozhe Lei , Mingsheng Yin , Yaqi Hu This is my paper · ORCID
classification cs.LGcs.RO
keywords reinforcementlearningsymbolicprogrampriordomain-specificlanguagewirelessindoornavigationmmWavesignalpropagationzero-shotgeneralizationphysics-informedRLimportancesampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that physics priors for wireless indoor navigation can be written as human-readable symbolic programs in a domain-specific language and then used both as an executable high-level policy and as a guide for reinforcement learning. The authors claim that this physics-informed program-guided RL framework, PiPRL, learns with fewer episodes and GPU hours than a plain neural RL baseline, and that the learned policy generalizes to indoor maps never seen in training. Across six unseen indoor maps, PiPRL reports normalized path lengths near 1.01 in line-of-sight conditions, 1.13–1.73 in first-order non-line-of-sight conditions, and 2.13–2.65 in higher-order non-line-of-sight conditions, consistently better than the three baselines. If the claim holds, it would mean that a general user can inject domain knowledge into RL agents simply by writing readable programs, without handcrafting rewards, architectures, or features.

What carries the argument

The load-bearing object is the physics-informed symbolic program written in RLang, a domain-specific language for describing partial world knowledge to RL agents. The program has three active parts: a symbolic reverse-angle-of-arrival policy that follows the strongest mmWave path to a waypoint; an ActionRestriction element that forbids neural-policy actions deviating more than 10 degrees from the signal-to-noise ascent direction; and an Effect element that terminates an episode when the estimated link state worsens, enforcing monotonic decrease. Together they form a meta-program that dispatches the symbolic policy for link states up to 2 and the program-guided PPO policy for worse link states, with a cost-correction Effect that reweights rewards by the ratio of the neural policy's probability to the uniform resampling probability.

What would settle it

Instrument the training loop to log, for each step, whether the neural action was kept or resampled from the compliant set, then compare the empirical distribution of executed actions with the assumed uniform resampling; if the corrected-cost estimator's expectation under the actual execution distribution differs from the ordinary cost, the unbiasedness claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that physics knowledge too weak to prescribe a complete policy can still be encoded in a DSL program and used to constrain and guide RL, yielding zero-shot generalization in wireless indoor navigation. The meta-program switches between a symbolic reverse-angle-of-arrival policy for line-of-sight and low-order NLOS conditions and a PPO neural policy for higher-order NLOS, where the program restricts the neural policy's actions to those aligned with the signal-to-noise ascent direction and terminates episodes when the estimated link state increases. The reward for a replaced action is corrected by an importance-sampling weight, which the paper claims makes the guided cost unbiased. In the reported results, PiPRL achieves normalized path lengths close to the theoretical minimum in line-of-sight conditions and substantially lower than all baselines in 2+-NLOS, while reducing training GPU hours by 26–83% compared with the non-physics neural baseline.

Load-bearing premise

The method's reward-correction formula assumes that whenever a non-compliant action is replaced, the replacement is drawn uniformly from the compliant set, and that the formula's importance weight accounts for all ways an action can be executed; the paper's own description also keeps the original action when it is already compliant, which the formula omits.

Editorial extensions

If this is right

  • If PiPRL is right, zero-shot deployment of wireless indoor navigation is feasible: agents trained on a set of indoor maps navigate structurally different unseen maps without fine-tuning.
  • Physical priors that only state constraints or desiderata, such as signal-strength ascent and link-state monotonicity, can be injected into RL as symbolic programs, reducing the need for reward shaping or handcrafted features.
  • The modular design means the same DSL-prior mechanism could be combined with any policy-based or value-based RL algorithm, not just PPO.
  • Training-time savings grow over the course of training, from roughly 26% GPU-hour reduction on early maps to more than 80% on later maps, because the neural policy increasingly inherits prior-compliant behavior.
  • The symbolic reverse-angle-of-arrival policy alone handles line-of-sight and first-order NLOS near-optimally, so the neural component is only needed where the prior is insufficient, which is how the paper explains its sample-efficiency gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader could extend the recipe to other physical control domains where a symbolic high-level policy is available for part of the state space and a learned policy is needed for the rest; the DSL mechanism is not tied to wireless signals.
  • If the cost-correction estimator is biased as the paper's own description suggests, some of the reported gains may come from an implicit reward distortion rather than from the physics priors themselves; re-running with a provably unbiased correction would separate the two effects.
  • A natural testable extension is to degrade the DSL priors deliberately, for example by using an incorrect signal-strength gradient, to measure how much of the benefit comes from the prior itself versus from the program-guided exploration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes PiPRL, a neuro-symbolic framework for wireless indoor navigation that encodes physics priors in the RLang domain-specific language. A meta-program selects a symbolic reverse-AoA policy when the estimated link state is favorable, and otherwise lets a PPO policy propose waypoints subject to an SNR-based action restriction. The paper's second contribution is a claimed unbiased importance-sampling correction that substitutes physics-compliant actions and reweights costs, plus a link-state monotonicity termination condition. Experiments on Gibson maps report consistently lower normalized path length than NPRL, WAN, and V-SLAM on six unseen maps, and reduced GPU hours relative to NPRL. The central theoretical claim, however, rests on an importance-sampling formula that is not unbiased under the behavior policy described in the text.

Significance. If the empirical results hold, the framework is a useful demonstration that hand-coded physics priors expressed in a DSL can improve sample efficiency and zero-shot generalization in a realistic navigation task. The paper's strengths include the use of externally stated physics principles rather than fitted constants, a fairly extensive evaluation across 15 training and 6 testing maps with repeated trials, and concrete reductions in training episodes and GPU hours. The contribution is diminished, though not necessarily invalidated, by the flawed unbiasedness argument for the cost correction, which is one of the paper's two stated technical contributions.

major comments (2)
  1. [Section 4, 'Program-Guided RL' (ActionRestriction and Cost Correction)] The importance-sampling correction is not unbiased as stated. The text says the agent resamples uniformly from the restricted set only when the Neural Policy action is non-compliant; a compliant action drawn by π_φ is executed directly. Hence the behavior-policy probability of executing a compliant action Ω' is b(Ω') = π_φ(Ω') + (1 - π_φ(C))/|C|, not 1/|C|. The paper's corrected cost ĉ_t = |C|·π_φ(Ω'_t)·c_t equals the true IS weight π_φ(Ω')/b(Ω') only when π_φ(C)=0; as training makes π concentrate on C, the formula increasingly over-weights compliant actions. This is exactly the regime in which the prior is supposed to matter, so the claimed unbiasedness of 'substituting physics-compliant actions with their unbiased reward estimates' does not follow from the described algorithm. Please either correct the IS weight, or clearly state and implement a version in which the behavior policy always samples uniformly from the compliant set, which would make 1/|C| the correct denominator.
  2. [Appendix C.1, Table 5] The ablation removes ActionRestriction and Effect Cost Correction together, so the sharp NPL decrease in 2+-NLOS from 4.62 to 2.60 conflates the effect of the SNR prior with the effect of the possibly biased IS reweighting. After fixing the IS correction, the ablation should be rerun and reported with the corrected estimator; otherwise the claim that the prior itself is responsible for the improvement is not established.
minor comments (5)
  1. [Table 7] The table heading uses 'PIRL' rather than 'PiPRL'.
  2. [Section 4, movement angle definition] The expression for ν_t should be written with arctan2(ŷ_t−ŷ_{t−1}, x̂_t−x̂_{t−1}); as printed, the unparenthesized quotient is ambiguous.
  3. [Section 4, Neural Perception] There is a duplicated 'where where' in the sentence introducing the path-estimate tuples.
  4. [Section 4, Effect declarations] The code block uses both 'Line State Estimate' and 'link state' for the same quantity; please unify the terminology.
  5. [Figure 2] The random-move Policy lists P(0.33) for each action; if this is meant literally, the probabilities sum to 0.99 rather than 1, so the figure should state that the values are rounded.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the PiPRL contributions are empirical and the physics priors are external, hand-coded inductive biases rather than quantities derived from the test outcomes.

full rationale

The paper's central claims are empirical: PiPRL is a training framework, and its zero-shot generalization is measured by normalized path length on held-out Gibson maps. The physics priors (signal reversibility, SNR decrease with distance, link-state monotonicity) are external domain knowledge encoded by hand into the RLang program; they are not fitted parameters and are not defined in terms of the reported NPL results. The importance-sampling correction in Section 4 is a mathematical derivation of an estimator, and while the reasoning may be incorrect because the behavior policy also executes already-compliant neural actions, an incorrect estimator is a correctness concern, not a circular reduction: the corrected cost is not equivalent by construction to the quantity being predicted. Self-citations (Yin et al., 2022; Li et al., 2025b; Yin et al., 2024) are used for experimental setup, pretrained modules, and baselines, but the claimed superiority of PiPRL over WAN, NPRL, and V-SLAM is established by independent test-time measurements, not by those citations. No fitted parameter is renamed as a prediction, and no result is forced by self-citation or by definition.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method rests on hand-written physics priors and external pretrained components. There are three hand-chosen hyperparameters (waypoint offset, link-state threshold, discretization step). No new physical entities are introduced. The physics priors are domain assumptions that could be wrong in real multipath environments, and the pretrained modules carry their own assumptions.

free parameters (3)
  • waypoint offset D = 2.5 m
    Hyperparameter for the reverse AoA policy's intermediate waypoint distance (Section 4, 'Physics-Informed Program').
  • link state threshold = <=2
    Meta-program switches from reverse AoA to neural policy when link state estimate exceeds 2 (Section 4, 'Meta-program').
  • angle discretization step = 10 degrees
    Action space discretization for the neural policy and the SNR prior's deviation bound (Section 4, 'Program-Guided RL').
assumptions (5)
  • domain assumption Reversibility of electromagnetic wave propagation implies following the AoA of the strongest path is a good navigation strategy
    Prior #1 in Section 4; assumed to hold in the simulation.
  • domain assumption Overall SNR decreases monotonically with distance from the transmitter
    Prior #2 in Section 4; not generally true in multipath but assumed for the navigation heuristic.
  • domain assumption An optimal navigation path has non-increasing link-state order (LOS < 1-NLOS < 2-NLOS ...)
    Prior #3 in Section 4; used to terminate episodes when link state estimate increases.
  • domain assumption Pretrained neural SLAM, wireless path estimator, and visual controller from prior works are reliable in these environments
    Section 4, 'Neural Perception'; the method inherits these modules without re-verification.
  • standard math Low-rank tensor decomposition is sufficient to extract the signal paths from the mmWave tensor
    Adapted from Zhou et al. 2017; treated as a background method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation." pith.science (2026). https://pith.science/paper/DAEGA3QJ

@misc{pith2026250622365,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DAEGA3QJ}},
  note         = {Machine review of arXiv:2506.22365}
}
read the original abstract

When using reinforcement learning (RL) to tackle physical control tasks, inductive biases that encode physics priors can help improve sample efficiency during training and enhance generalization in testing. However, the current practice of incorporating these helpful physics-informed inductive biases inevitably runs into significant manual labor and domain expertise, making them prohibitive for general users. This work explores a symbolic approach to distill physics-informed inductive biases into RL agents, where the physics priors are expressed in a domain-specific language (DSL) that is human-readable and naturally explainable. Yet, the DSL priors do not translate directly into an implementable policy due to partial and noisy observations and additional physical constraints in navigation tasks. To address this gap, we develop a physics-informed program-guided RL (PiPRL) framework with applications to indoor navigation. PiPRL adopts a hierarchical and modularized neuro-symbolic integration, where a meta symbolic program receives semantically meaningful features from a neural perception module, which form the bases for symbolic programming that encodes physics priors and guides the RL process of a low-level neural controller. Extensive experiments demonstrate that PiPRL consistently outperforms purely symbolic or neural policies and reduces training time by over 26% with the help of the program-based inductive biases.

Figures

Figures reproduced from arXiv: 2506.22365 by the authors.

Figure 1
Figure 1. The workflow of the physics-informed program-guided RL (PiPRL). The core of PiPRL is the physics-informed symbolic program, which coordinates the other modules in the wireless indoor navigation tasks. with physics constraints; and 4) Algorithmic bias points to the setup of learning algorithms, includ￾ing hyperparameters (Hessel et al., 2019), initialization (Fallah et al., 2021; Pan et al., 2023a; Li et al., 2024b),… view at source ↗
Figure 2
Figure 2. Examples of Factor, Feature, Policy, and Effect in RLang. Factor and Feature describe the agent’s observations and related features. Policy and Effect represent the navigation policy and its consequence. and the target position, the optimal policy corresponds to the minimizer of the expected cumulative cost, referred to as the value function, π ∗ ∈ arg min J(π) ≜ Eπ[ PT t=1 ct|p1]. When facing high￾dimensional state… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. In-Context Reinforcement Learning via Communicative World Models

    cs.LG 2025-08 unverdicted novelty 6.0 of 10

    CORAL trains an information agent as a world model that sends concise messages to a control agent, improving in-context reinforcement learning and zero-shot adaptation.

Reference graph

Works this paper leans on

87 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [1]

    Kakade, Jason D

    Alekh Agarwal, Sham M. Kakade, Jason D. Lee, and Gaurav Mahajan. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. Journal of Machine Learning Research, 22 0 (98): 0 1--76, 2021. URL http://jmlr.org/papers/v22/19-736.html

    2021

  2. [2]

    On evaluation of embodied navigation agents

    Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, and Amir R Zamir. On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757 , 2018. doi:10.48550/arxiv.1807.06757. [Online] Available at https://arxiv.org/abs/1807.06757

    arXiv doi:10.48550/arxiv.1807.06757 2018

  3. [3]

    Deep learning based wireless localization for indoor navigation

    Roshan Ayyalasomayajula, Aditya Arun, Chenfeng Wu, Sanatan Sharma, Abhishek Rajkumar Sethi, Deepak Vasisht, and Dinesh Bharadia. Deep learning based wireless localization for indoor navigation. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking, pp.\ 1--14, New York, NY, USA, 2020. Association for Computing Machi...

    work page arXiv 2020

  4. [4]

    Learning to understand goal specifications by modelling reward

    Dzmitry Bahdanau, Felix Hill, Jan Leike, Edward Hughes, Arian Hosseini, Pushmeet Kohli, and Edward Grefenstette. Learning to understand goal specifications by modelling reward. arXiv , 2018. doi:10.48550/arxiv.1806.01946

    arXiv doi:10.48550/arxiv.1806.01946 2018

  5. [5]

    Causality and batch reinforcement learning: Complementary approaches to planning in unknown domains

    James Bannon, Brad Windsor, Wenbo Song, and Tao Li. Causality and batch reinforcement learning: Complementary approaches to planning in unknown domains. arXiv preprint arXiv:2006.02579 , 2020. doi:10.48550/arxiv.2006.02579. [Online] Available at https://arxiv.org/pdf/2006.02579

    arXiv doi:10.48550/arxiv.2006.02579 2006

  6. [6]

    A model of inductive bias learning

    Jonathan Baxter. A model of inductive bias learning. Journal of Artificial Intelligence Research, 12: 0 149--198, 2000. doi:10.1613/jair.731

    doi:10.1613/jair.731 2000

  7. [7]

    Yolov4: Optimal speed and accuracy of object detection

    Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020. [Online] Available at https://arxiv.org/pdf/2004.10934

    arXiv 2004

  8. [8]

    Learning to explore using active neural SLAM

    Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning to explore using active neural SLAM . In International Conference on Learning Representations, 2020

    2020

Show all 87 references
  1. [9]

    Learning phrase representations using RNN rncoder--decoder for statistical machine translation

    Kyunghyun Cho, Bart van Merri \"e nboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN rncoder--decoder for statistical machine translation . In 2014 Conf. Empirical Methods in Natural Language P...

    2014

  2. [10]

    Scientific machine learning through physics–informed neural networks: Where we are and what’s next

    Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92 0 (3): 0 88, 2022. ISSN ...

    2022 doi

  3. [11]

    Interpretable and explainable logical policies via neurally guided symbolic abstraction

    Quentin Delfosse, Hikaru Shindo, Devendra Dhami, and Kristian Kersting. Interpretable and explainable logical policies via neurally guided symbolic abstraction. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Proces...

    2023

  4. [12]

    IMPALA : Scalable distributed deep- RL with importance weighted actor-learner architectures

    Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu. IMPALA : Scalable distributed deep- RL with importance weighted actor-learner architectures. In Jennifer Dy and...

    2018

  5. [13]

    Learning explanatory rules from noisy data

    Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. J. Artif. Int. Res., 61 0 (1): 0 1–64, January 2018. ISSN 1076-9757

    2018

  6. [14]

    On the convergence theory of debiased model-agnostic meta-reinforcement learning

    Alireza Fallah, Kristian Georgiev, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of debiased model-agnostic meta-reinforcement learning. In Advances in Neural Information Processing Systems, volume 24, pp.\ 3096---3107, 2021. URL https://proceedings.neurips.cc...

    2021

  7. [15]

    Speaker-follower models for vision-and-language navigation

    Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. Speaker-follower models for vision-and-language navigation. In Proceedings of the 32nd International Conference o...

    2018

  8. [16]

    Addressing function approximation error in actor-critic methods

    Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80, pp.\ 1587--1596, 2018. URL https://proce...

    2018

  9. [17]

    A theory of regularized M arkov decision processes

    Matthieu Geist, Bruno Scherrer, and Olivier Pietquin. A theory of regularized M arkov decision processes. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97, pp.\ 2160--2169, 2019. URL https://...

    2019

  10. [18]

    PDDL ---The Planing Domain Definition Language

    Malik Ghallab, Adele Howe, Craig Knoblock, Drew McDermott, Ashwin Ram, Manuela Veloso, Daniel Weld, and David Wilkins. PDDL ---The Planing Domain Definition Language . The International Conference on Artificial Intelligence Planning Systems Planning Competition Committee, 1998...

    1998

  11. [19]

    Millimeter-wave massive arrays for indoor SLAM

    Francesco Guidi, Anna Guerra, and Davide Dardari. Millimeter-wave massive arrays for indoor SLAM . In 2014 IEEE International Conference on Communications Workshops (ICC), pp.\ 114--120, 2014. doi:10.1109/ICCW.2014.6881182

    2014

  12. [20]

    Unpacking reward shaping: Understanding the benefits of reward engineering on sample complexity

    Abhishek Gupta, Aldo Pacchiano, Yuexiang Zhai, Sham Kakade, and Sergey Levine. Unpacking reward shaping: Understanding the benefits of reward engineering on sample complexity. In Advances in Neural Information Processing Systems, volume 35, pp.\ 15281--15295, 2022. URL https:/...

    2022

  13. [21]

    Adaptive security response strategies through conjectural online learning

    Kim Hammar, Tao Li, Rolf Stadler, Quanyan Zhu, and Kim Hammar. Adaptive security response strategies through conjectural online learning. IEEE Transactions on Information Forensics and Security , 20: 0 4055--4070, 2025. ISSN 1556-6013. doi:10.1109/tifs.2025.3558600

    2025

  14. [22]

    A physics-informed reinforcement learning-based strategy for local and coordinated ramp metering

    Yu Han, Meng Wang, Linghui Li, Claudio Roncoli, Jinda Gao, and Pan Liu. A physics-informed reinforcement learning-based strategy for local and coordinated ramp metering. Transportation Research Part C: Emerging Technologies, 137: 0 103584, 2022. ISSN 0968-090X. doi:10.1016/j.t...

    2022

  15. [23]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 770--778, 2016. doi:10.1109/CVPR.2016.90

    2016 doi

  16. [24]

    On inductive biases in deep reinforcement learning

    Matteo Hessel, Hado van Hasselt, Joseph Modayil, and David Silver. On inductive biases in deep reinforcement learning. arXiv , 2019. doi:10.48550/arxiv.1907.02908

    2019 doi arXiv

  17. [25]

    Social influence as intrinsic motivation for multi-agent deep reinforcement learning

    Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro A Ortega, DJ Strouse, Joel Z Leibo, and Nando de Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In Proceedings of the 36th International Conference on Mach...

    2019

  18. [26]

    Neural logic reinforcement learning

    Zhengyao Jiang and Shan Luo. Neural logic reinforcement learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97, pp.\ 3110--3119, 09--15 Jun 2019. URL https://proceedings.mlr.press/v97/jia...

    2019

  19. [27]

    A composable specification language for reinforcement learning tasks

    Kishor Jothimurugan, Rajeev Alur, and Osbert Bastani. A composable specification language for reinforcement learning tasks. In Advances in Neural Information Processing Systems, volume 32, 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/f5aa4bd09c07d8b2f65...

    2019

  20. [28]

    Policy optimization with demonstrations

    Bingyi Kang, Zequn Jie, and Jiashi Feng. Policy optimization with demonstrations. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80, pp.\ 2469--2478, 2018. URL https://proceedings.mlr.press/v80/kang18a.html

    2018

  21. [29]

    Beating atari with natural language guided reinforcement learning

    Russell Kaplan, Christopher Sauer, and Alexander Sosa. Beating atari with natural language guided reinforcement learning. arXiv , 2017. doi:10.48550/arxiv.1704.05539

    2017 doi arXiv

  22. [30]

    The Computers & Typesetting, Vol

    Donald Knuth. The Computers & Typesetting, Vol. A: The Texbook. Addison-Wesley Longman Publishing Co., Inc., MA, USA, 1986. ISBN 0201134470

    1986

  23. [31]

    Reprel: Integrating relational planning and reinforcement learning for effective abstraction

    Harsha Kokel, Arjun Manoharan, Sriraam Natarajan, Balaraman Ravindran, and Prasad Tadepalli. Reprel: Integrating relational planning and reinforcement learning for effective abstraction. Proceedings of the International Conference on Automated Planning and Scheduling, 31 0 (1)...

    2021 doi

  24. [32]

    Actor-critic--type learning algorithms for markov decision processes

    Vijaymohan R Konda and Vivek S Borkar. Actor-critic--type learning algorithms for markov decision processes. SIAM Journal on Control and Optimization , 38 0 (1): 0 94--123, 1999. ISSN 0363-0129. doi:10.1137/s036301299731669x

    1999 doi

  25. [33]

    Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation

    Tejas D Kulkarni, Karthik R Narasimhan, Ardavan Saeedi, and Joshua B Tenenbaum. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. In Advances in Neural Information Processing Systems, volume 29, 2016

    2016

  26. [34]

    Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of ltl formulas

    Yen-Ling Kuo, Boris Katz, and Andrei Barbu. Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of ltl formulas. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 5604--5610, 2020. doi:10.1109/IROS45743.2020.9341325

    2020

  27. [35]

    Bandit Algorithms

    Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. Cambridge Core. Cambridge University Press, Cambridge, 01 2020. ISBN 9781108486828. doi:10.1017/9781108571401. URL https://www.cambridge.org/core/books/bandit-algorithms/8E39FD004E6CE036680F90DD0C6F09FC

    2020 doi

  28. [36]

    Partially observable markov decision processes in robotics: A survey

    Mikko Lauri, David Hsu, and Joni Pajarinen. Partially observable markov decision processes in robotics: A survey. IEEE Transactions on Robotics , 39 0 (1): 0 21--40, 2023. ISSN 1552-3098. doi:10.1109/tro.2022.3200138

    2023

  29. [37]

    State representation learning for control: An overview

    Timoth\'ee Lesort, Natalia D\'iaz-Rodr\'iguez, Jean-Fran o is Goudou, and David Filliat. State representation learning for control: An overview. Neural Networks, 108: 0 379--392, 2018. ISSN 0893-6080. doi:10.1016/j.neunet.2018.07.006

    2018 doi

  30. [38]

    Learning neural network policies with guided policy search under unknown dynamics

    Sergey Levine and Pieter Abbeel. Learning neural network policies with guided policy search under unknown dynamics. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 27, 2014. URL https:/...

    2014

  31. [39]

    On convergence rate of adaptive multiscale value function approximation for reinforcement learning

    Tao Li and Quanyan Zhu. On convergence rate of adaptive multiscale value function approximation for reinforcement learning. 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing ( MLSP ) , pp.\ 1--6, 2019. doi:10.1109/mlsp.2019.8918816

    2019

  32. [40]

    Blackwell online learning for markov decision processes

    Tao Li, Guanze Peng, and Quanyan Zhu. Blackwell online learning for markov decision processes. In 2021 55th Annual Conference on Information Sciences and Systems (CISS), pp.\ 1--6, 2021. doi:10.1109/CISS50987.2021.9400319

    2021

  33. [41]

    The confluence of networks, games, and learning a game-theoretic framework for multiagent decision making over networks

    Tao Li, Guanze Peng, Quanyan Zhu, and Tamer Baar. The confluence of networks, games, and learning a game-theoretic framework for multiagent decision making over networks. IEEE Control Systems , 42 0 (4): 0 35--67, 2022 a . ISSN 1066-033X. doi:10.1109/mcs.2022.3171478

    2022

  34. [42]

    The role of information structures in game-theoretic multi-agent learning

    Tao Li, Yuhan Zhao, and Quanyan Zhu. The role of information structures in game-theoretic multi-agent learning. Annual Reviews in Control, 53: 0 296--314, 2022 b . ISSN 1367-5788. doi:10.1016/j.arcontrol.2022.03.003

    2022 doi

  35. [43]

    Self-adaptive driving in nonstationary environments through conjectural online lookahead adaptation

    Tao Li, Haozhe Lei, and Quanyan Zhu. Self-adaptive driving in nonstationary environments through conjectural online lookahead adaptation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 7205--7211, 2023. doi:10.1109/ICRA48891.2023.10161368

    2023

  36. [45]

    Meta stackelberg game: Robust federated learning against adaptive and mixed poisoning attacks

    Tao Li, Henger Li, Yunian Pan, Tianyi Xu, Zizhan Zheng, and Quanyan Zhu. Meta stackelberg game: Robust federated learning against adaptive and mixed poisoning attacks. arXiv preprint arXiv:2410.17431, 2024 b . [Online] Available at https://arxiv.org/pdf/2410.17431

    2024 arXiv

  37. [46]

    Self-confirming transformer for belief-conditioned adaptation in offline multi-agent reinforcement learning

    Tao Li, Juan Guevara, Xinghong Xie, and Quanyan Zhu. Self-confirming transformer for belief-conditioned adaptation in offline multi-agent reinforcement learning. In Proceedings of the Seventh Workshop on Adaptive and Learning Agents, the Twenty Fourth International Conference ...

    2025 doi arXiv

  38. [47]

    Digital twin-enhanced wireless indoor navigation: Achieving efficient environment sensing with zero-shot reinforcement learning

    Tao Li, Haozhe Lei, Hao Guo, Mingsheng Yin, Yaqi Hu, Quanyan Zhu, and Sundeep Rangan. Digital twin-enhanced wireless indoor navigation: Achieving efficient environment sensing with zero-shot reinforcement learning. IEEE Open Journal of the Communications Society , 6: 0 2356--2...

    2025

  39. [48]

    Guided exploration with proximal policy optimization using a single demonstration

    Gabriele Libardi, Gianni De Fabritiis, and Sebastian Dittert. Guided exploration with proximal policy optimization using a single demonstration. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139, pp.\ 6611--...

    2021

  40. [49]

    Understanding deep neural function approximation in reinforcement learning via epsilon-greedy exploration

    Fanghui Liu, Luca Viano, and Volkan Cevher. Understanding deep neural function approximation in reinforcement learning via epsilon-greedy exploration. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems...

    2022

  41. [50]

    Game-theoretic distributed empirical risk minimization with strategic network design

    Shutian Liu, Tao Li, and Quanyan Zhu. Game-theoretic distributed empirical risk minimization with strategic network design. IEEE Transactions on Signal and Information Processing over Networks , 9: 0 542--556, 2023. ISSN 2373-776X. doi:10.1109/tsipn.2023.3306106

    2023

  42. [51]

    A survey of reinforcement learning informed by natural language

    Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, and Tim Rocktäschel. A survey of reinforcement learning informed by natural language. In Proceedings of the Twenty-Eighth International Joint Conference on ...

    2019 doi

  43. [52]

    On the global convergence rates of softmax policy gradient methods

    Jincheng Mei, Chenjun Xiao, Csaba Szepesvari, and Dale Schuurmans. On the global convergence rates of softmax policy gradient methods. In Proceedings of the 37th International Conference on Machine Learning, volume 119, pp.\ 6820--6829, 2020. URL https://proceedings.mlr.press/...

    2020

  44. [53]

    Human-level control through deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Sha...

    2015 doi

  45. [54]

    Asynchronous methods for deep reinforcement learning

    Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd Intern...

    1928

  46. [55]

    Measuring sample efficiency and generalization in reinforcement learning benchmarks: Neurips 2020 procgen benchmark

    Sharada Mohanty, Jyotish Poonganam, Adrien Gaidon, Andrey Kolobov, Blake Wulfe, Dipam Chakraborty, Gra z vydas S emetulskis, Jo\ a o Schapke, Jonas Kubilius, Jurgis Pa\"ukonis, Linas Klimas, Matthew Hausknecht, Patrick MacAlpine, Quang Nhat Tran, Thomas Tumiel, Xiaocheng Tang,...

    2020

  47. [56]

    Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning, pp.\ 807–814, 2010. ISBN 9781605589077

    2010

  48. [57]

    Ng, Daishi Harada, and Stuart J

    Andrew Y. Ng, Daishi Harada, and Stuart J. Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In Proceedings of the Sixteenth International Conference on Machine Learning, pp.\ 278–287, 1999. ISBN 1558606122

    1999

  49. [58]

    Deep reinforcement learning for cyber security

    Thanh Thi Nguyen and Vijay Janapa Reddi. Deep reinforcement learning for cyber security. IEEE Transactions on Neural Networks and Learning Systems , 34 0 (8): 0 3779--3795, 2023. ISSN 2162-237X. doi:10.1109/tnnls.2021.3121870

    2023

  50. [59]

    Making efficient use of demonstrations to solve hard exploration problems

    Tom Le Paine, Caglar Gulcehre, Bobak Shahriari, Misha Denil, Matt Hoffman, Hubert Soyer, Richard Tanburn, Steven Kapturowski, Neil Rabinowitz, Duncan Williams, Gabriel Barth-Maron, Ziyu Wang, Nando de Freitas, and Worlds Team. Making efficient use of demonstrations to solve ha...

    2020 doi arXiv

  51. [60]

    A first order meta S tackelberg method for robust federated learning

    Yunian Pan, Tao Li, Henger Li, Tianyi Xu, Zizhan Zheng, and Quanyan Zhu. A first order meta S tackelberg method for robust federated learning. In Adversarial Machine Learning Frontiers Workshop at 40th International Conference on Machine Learning, 2023 a . doi:10.48550/arxiv.2...

    2023 doi arXiv

  52. [61]

    Is stochastic mirror descent vulnerable to adversarial delay attacks? a traffic assignment resilience study

    Yunian Pan, Tao Li, and Quanyan Zhu. Is stochastic mirror descent vulnerable to adversarial delay attacks? a traffic assignment resilience study. In 2023 62nd IEEE Conference on Decision and Control ( CDC ) , pp.\ 8328--8333, 2023 b . doi:10.1109/cdc49753.2023.10384003

    2023

  53. [62]

    On the variational interpretation of mirror play in monotone games

    Yunian Pan, Tao Li, and Quanyan Zhu. On the variational interpretation of mirror play in monotone games. In 2024 IEEE 63rd Conference on Decision and Control ( CDC ) , pp.\ 6799--6804, 2024. doi:10.1109/cdc56724.2024.10885800. URL https://ieeexplore.ieee.org/document/10885800

    2024

  54. [63]

    Model-agnostic meta-policy optimization via zeroth-order estimation: A linear quadratic regulator perspective

    Yunian Pan, Tao Li, and Quanyan Zhu. Model-agnostic meta-policy optimization via zeroth-order estimation: A linear quadratic regulator perspective. arXiv preprint arXiv:2503.00385, 2025. [Online] Available at https://arxiv.org/pdf/2503.00385

    2025 arXiv

  55. [64]

    Raissi, P

    M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378: 0 686--707, 2019. ISSN 0021-9991. doi...

    2019 doi

  56. [65]

    RLang : A declarative language for describing partial world knowledge to reinforcement learning agents

    Rafael Rodriguez-Sanchez, Benjamin Adin Spiegel, Jennifer Wang, Roma Patel, Stefanie Tellex, and George Konidaris. RLang : A declarative language for describing partial world knowledge to reinforcement learning agents. In Proceedings of the 40th International Conference on Mac...

    2023

  57. [66]

    Habitat: A platform for embodied ai research

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A platform for embodied ai research. In 2019 IEEE/CVF International Conference on Computer...

    2019

  58. [67]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37, pp.\ 1889--1897, 2015. URL https://proceed...

    2015

  59. [68]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [Online] Available at https://arxiv.org/pdf/1707.06347

    2017 arXiv

  60. [69]

    A fast marching level set method for monotonically advancing fronts

    J A Sethian. A fast marching level set method for monotonically advancing fronts . Proceedings of the National Academy of Sciences (PNAS), 93 0 (4): 0 1591--1595, Feb. 1996. ISSN 0027-8424. doi:10.1073/pnas.93.4.1591

    1996 doi

  61. [70]

    Garcia, Giuseppe Destino, Gonzalo Seco-Granados, and Henk Wymeersch

    Arash Shahmansoori, Gabriel E. Garcia, Giuseppe Destino, Gonzalo Seco-Granados, and Henk Wymeersch. Position and orientation estimation through millimeter-wave MIMO in 5g systems. IEEE Transactions on Wireless Communications , 17 0 (3): 0 1822--1835, 2018. ISSN 1536-1276. doi:...

    2018

  62. [71]

    BlendRL : A framework for merging symbolic and neural policy learning

    Hikaru Shindo, Quentin Delfosse, Devendra Singh Dhami, and Kristian Kersting. BlendRL : A framework for merging symbolic and neural policy learning. In International Conference on Learning Representations, 2025. doi:10.48550/arxiv.2410.11689

    2025 doi arXiv

  63. [72]

    Shao-Hua Sun, Te-Lin Wu, and Joseph J. Lim. Program guided agent. In International Conference on Learning Representations, 2020

    2020

  64. [73]

    Indoor point-to-point navigation with deep reinforcement learning and ultra-wideband

    Enrico Sutera, Vittorio Mazzia, Francesco Salvetti, Giovanni Fantin, and Marcello Chiaberge. Indoor point-to-point navigation with deep reinforcement learning and ultra-wideband. In 12th International Conference on Agents and Artificial Intelligence, 2020. [Online] Available a...

    2020 arXiv

  65. [74]

    Sutton, Doina Precup, and Satinder Singh

    Richard S. Sutton, Doina Precup, and Satinder Singh. Between MDPs and semi- MDPs : A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112 0 (1-2): 0 181--211, 1999. ISSN 0004-3702. doi:10.1016/s0004-3702(99)00052-1

    1999 doi

  66. [75]

    Policy gradient methods for reinforcement learning with function approximation

    Richard S Sutton, David A McAllester , Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems, pp.\ 1057---1063, 2000

    2000

  67. [76]

    Walter, Ashis Gopal Banerjee, Seth Teller, and Nicholas Roy

    Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R. Walter, Ashis Gopal Banerjee, Seth Teller, and Nicholas Roy. Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial In...

    2011

  68. [77]

    Analysis of temporal-diffference learning with function approximation

    John N Tsitsiklis and Benjamin Van Roy. Analysis of temporal-diffference learning with function approximation. IEEE Transactions on Automatic Control , 42 0 (5): 0 674--690, 1997. doi:10.1109/9.580874

    1997 doi

  69. [78]

    Python programming language

    Guido van Rossum. Python programming language. https://www.python.org, 1991. Accessed: 2025-06-06

    1991

  70. [79]

    Gibson env: Real-world perception for embodied agents

    Fei Xia, Amir R Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: Real-world perception for embodied agents. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 9068--9079, 2018

    2018

  71. [80]

    Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling

    Tengyang Xie, Yifei Ma, and Yu-Xiang Wang. Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling. In Advances in Neural Information Processing Systems, volume 32, 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/f...

    2019

  72. [81]

    Hybrid policy optimization from imperfect demonstrations

    Hanlin Yang, Chao Yu, peng sun, and Siji Chen. Hybrid policy optimization from imperfect demonstrations. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 4653--4663, 2023. URL http...

    2023

  73. [82]

    Millimeter wave wireless assisted robot navigation with link state classification

    Mingsheng Yin, Akshaj Kumar Veldanda, Amee Trivedi, Jeff Zhang, Kai Pfeiffer, Yaqi Hu, Siddharth Garg, Elza Erkip, Ludovic Righetti, and Sundeep Rangan. Millimeter wave wireless assisted robot navigation with link state classification. IEEE Open Journal of the Communications S...

    2022

  74. [83]

    Zero-shot wireless indoor navigation through physics-informed reinforcement learning

    Mingsheng Yin, Tao Li, Haozhe Lei, Yaqi Hu, Sundeep Rangan, and Quanyan Zhu. Zero-shot wireless indoor navigation through physics-informed reinforcement learning. In 2024 IEEE International Conference on Robotics and Automation ( ICRA ) , pp.\ 5111--5118, 2024. doi:10.1109/icr...

    2024

  75. [84]

    Physics informed deep reinforcement learning for aircraft conflict resolution

    Peng Zhao and Yongming Liu. Physics informed deep reinforcement learning for aircraft conflict resolution. IEEE Transactions on Intelligent Transportation Systems , 23 0 (7): 0 8288--8301, 2022. ISSN 1524-9050. doi:10.1109/tits.2021.3077572

    2022

  76. [85]

    Sim-to-real transfer in deep reinforcement learning for robotics: A survey

    Wenshuai Zhao, Jorge Peña Queralta, and Tomi Westerlund. Sim-to-real transfer in deep reinforcement learning for robotics: A survey. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp.\ 737--744, 2020. doi:10.1109/SSCI47803.2020.9308468

    2020

  77. [86]

    RTFM : Generalising to new environment dynamics via reading

    Victor Zhong, Tim Rocktaschel, and Edward Grefenstette. RTFM : Generalising to new environment dynamics via reading. In International Conference on Learning Representations, 2020. doi:10.5555/2900423.2900661

    2020

  78. [87]

    Zhou Zhou, Jun Fang, Linxiao Yang, Hongbin Li, Zhi Chen, and Rick S. Blum. Low-rank tensor decomposition-aided channel estimation for millimeter wave mimo-ofdm systems. IEEE Journal on Selected Areas in Communications, 35 0 (7): 0 1524--1538, 2017. doi:10.1109/JSAC.2017.2699338

    2017

  79. [88]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.