REVIEW 4 major objections 5 minor 104 references
Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read ICMO, an encoder built from Neural Production Systems with language-conditioned memory and memory feedback, drives zero-shot systematic generalization in BabyAI to near-perfect levels.
desk verdict A plausible and well-ablated architecture for language-conditioned RL whose headline 'significant' claim is undercut by two-seed evaluation and test-set model selection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the Neural Production System (NPS), a differentiable set of condition-action rules in which each input slot is assigned a rule and a contextual slot by Gumbel-softmax attention, so interactions are routed sparsely instead of densely. ICMO adds a language-conditioned LSTM memory whose hidden state is projected and concatenated to the attention queries, so the memory guides both rule selection and context-selection attentions. That feedback connection is the mechanism the paper credits for grounding instructions in observation processing.
What would settle it
Run ICMO on the same six BabyAI levels with observations fed as raw grids instead of entity-slot decompositions; if the average test success rate and near-zero generalization gap are not preserved, the claim that modular sparse rules plus memory feedback are responsible for the generalization collapses.
Extended reading notes
Core claim
The paper's central claim is that injecting modularity and sparsity through Neural Production Systems, and routing the instruction through memory with feedback, can produce systematic generalization in language-conditioned reinforcement learning. Its final model, ICMO, is instruction-conditioned memory plus memory feedback to context selection, and it outperforms CNN-GRU, FiLM-BabyAI, and AttentionFusion encoders on test splits defined by held-out instruction strings. It reports an average test success rate of 0.97, an average test mean return of 0.85, an average generalization gap of about 0.01, and the best sample efficiency among the models compared. The ablations show that language is most effective when it enters the memory rather than being fused early with observations, and that feedback from memory to the selection of contextual slots adds training stability as well as accuracy. The paper frames these design choices as analogous to prefrontal-cortex working-memory feedback in selective visual attention.
Load-bearing premise
The whole result depends on observations being given as pre-segmented entity slots; if the environment does not supply that factorization, the paper has not shown that the modular sparse mechanism works.
Editorial extensions
If this is right
- If correct, language in RL encoders should be injected at the memory level, not fused early with raw observations, because instruction-conditioned memory and actor-critic variants beat early FiLM-style fusion.
- Memory feedback to mid-level selection mechanisms can improve not just accuracy but training stability, lowering variance across random seeds.
- Near-zero generalization gap on held-out instruction combinations suggests zero-shot systematic generalization is achievable in symbolic instruction-following without any training on test tasks.
- The encoder is agnostic to the training algorithm, so the same ICMO encoder could be paired with other policy or value learning algorithms, not only PPO.
Reading between the lines
- The central mechanism presupposes a slot decomposition; transferring the method to raw pixels requires learning slots first, for example with Slot Attention or DINOSAUR, and the combined system has not been validated here. This is my inference, not the paper's demonstration.
- If slot selection is imperfect, the sparse-rule inductive bias could act on spurious object boundaries; a testable extension is to corrupt or shuffle slot assignments and measure the drop in test success.
- The systematic split used here is based on string matching within instructions; a harder split over unseen semantic combinations or unseen colors and types would stress whether the learned rules truly recombine rather than memorize concrete instruction strings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ICMO, a modular encoder for language-conditioned reinforcement learning built on Neural Production Systems (NPS), with a language-conditioned LSTM memory and attentional feedback from the memory to NPS's rule/context selection. The authors evaluate ICMO on six BabyAI levels under a compositional train/test split of instructions and report average test success rate 0.97 with near-zero generalization gap, claiming significant improvements in systematic generalization, sample efficiency, and training stability over baselines such as CNN-GRU, FiLM-BabyAI, and AttentionFusion. An ablation study varies the location of language injection and the presence of memory feedback.
Significance. If the empirical claims were robust, the paper would make a useful contribution by showing that modular, sparse, memory-feedback encoding is a strong inductive bias for zero-shot compositional generalization in language-informed RL, and the paper includes several strengths: it uses a publicly available benchmark, reports a detailed ablation, makes code available, and explicitly states its slot-based input assumption and its limitations. However, the headline quantitative claim currently rests on a statistically fragile foundation: only two random seeds per condition, no significance tests, and selection of the final model variant based on test-set results. These issues are load-bearing because the paper's central contribution is the empirical superiority of ICMO, not a new derivation or a provable guarantee.
major comments (4)
- [Section 4.1.1 and Table 2] The central claim that ICMO 'significantly improved systematic generalization' is not supported by the reported statistics. Each model is trained with only two random seeds, so the standard deviations and the averages reported in Tables 2-4 are based on n=2 per environment. With this sample size, a single seed failure can invert conclusions; for example, Table 4 reports PickupLoc test SR for IC-AC+FC as 0.64 ± 0.33, a range that overlaps substantially with most other entries. The paper uses the word 'significantly' repeatedly (e.g., Abstract, Section 4.2, Section 5) but reports no significance tests, confidence intervals, or paired comparisons. I request either more seeds (at least 5, ideally 10), or a proper significance test over environments with paired comparisons, before the superiority claim can be accepted.
- [Section 5 and Table 4] The final model is selected after inspecting test-set results: Section 5 states 'the final model that we propose in this paper as ICMO, is IC-M-FC' based on comparisons across the six test environments. This is test-set model selection. Since IC-M+FC and IC-M+FR have nearly identical averages in Table 4 (e.g., average test SR 0.99 ± 0.00 vs 0.99 ± 0.01), the choice of ICMO over IC-M+FR could easily be a selection artifact. This inflates apparent performance and invalidates the 'significant margin' language. The authors should either pre-register the final variant, select it on a held-out validation split, or report all variants as equally eligible hypotheses without designating one as the winner based on the same test set used for the headline numbers.
- [Section 3.2 and Section 7] The architecture assumes that observations are already provided as a set of slots that align with meaningful entities, and Section 7 acknowledges that 'slots are key to its success.' Yet Section 7 also claims that the method 'does not make any assumptions about the input structure' and that pre-trained encoders 'could be candidates' for producing such slots. These two statements are in tension. Because the entire modularity and sparsity argument depends on the slot decomposition, the paper should either restrict its claims to symbolic, pre-segmented observations or demonstrate a working pipeline that learns slots from raw observations. As it stands, the paper's relevance to realistic visual observations is unsupported.
- [Section 4.1.2 and Tables 2-4] The sample-efficiency claim is weaker than the paper's abstract suggests. The SE(α=0.9) rows contain many '−' entries indicating that baselines never reached the threshold, and the reported averages in Table 2 exclude Synth entirely. This makes it difficult to compare sample efficiency across models: ICMO's average SE is computed on a different set of environments than the baselines' averages, which are themselves mostly undefined. The paper should either define a threshold-free sample-efficiency metric that can be computed for all runs, or explicitly qualify the sample-efficiency conclusion as applying only to the environments where all models can be compared.
minor comments (5)
- [Section 5] The phrase 'compostional generalization' should be 'compositional generalization'.
- [Tables 2-4] The tables are difficult to read because the model names and some numeric entries are formatted inconsistently (e.g., '0.55 ± 0.0.2' in Table 2), and the 'Average' rows mix '−' entries with numeric values without explaining the treatment of missing values.
- [Section 4.1.3] The AttentionFusion baseline description says the original paper 'involves descriptive sentences' and that those were omitted here, but the paper does not discuss how this modification might affect the baseline's expected performance; a brief note on this limitation would be useful for a fair comparison.
- [Section 4.2] The phrase 'these results indicate that our model outperforms the baselines with a significant margin' appears before any statistical evidence is introduced; please move this claim to the discussion or qualify it until the statistical analysis is added.
- [Section 7] When mentioning Slot-Attention and DINOSAUR as candidate slot encoders, the paper does not provide any experimental evidence that these encoders would preserve the systematic generalization benefits; consider explicitly listing this as a future-work item rather than a remedy.
Circularity Check
No circularity: empirical architecture study; held-out instruction generalization is measured, not derived from fitted parameters or self-citation.
full rationale
The paper's claimed derivation chain is an architecture proposal tested empirically on BabyAI. Section 3.2 defines ICMO through NPS equations (1)-(9), language-conditioned memory, and memory feedback; no parameter is fitted to the test results, and no output quantity is defined in terms of the metric it is said to predict. The central 'prediction' is zero-shot success on instructions in Gtest, which are disjoint from Gtrain by construction (Table 1) and are never seen during training. The ablation study compares variants, and the final label ICMO is assigned to IC-M-FC after observing results; this is a model-selection concern that inflates apparent performance, but it is not circular reasoning because the architecture is not defined by or equivalent to the test outcomes. The only self-citation is Rohani, Hedayatian, and Baghshah (2022), cited alongside J.X. Wang et al. (2018) and other neuroscience references to motivate prefrontal-cortex working-memory analogies; this is contextual support, not a load-bearing uniqueness theorem or an imported ansatz. Similarly, the acknowledged reliance on pre-segmented slots is an explicit input assumption, stated in Section 7, and is not a renamed fit. All quantitative claims are empirical performance measurements on held-out episodes, so no self-definitional, fitted-input-as-prediction, or uniqueness-imported circularity is present. The reviewer-flagged weaknesses (two seeds, test-set model selection, no significance testing) are correctness and robustness risks, not circularity under the specified criteria.
Assumptions & free parameters
free parameters (2)
- number of production rules N
- LSTM hidden state size =
1024
assumptions (4)
- domain assumption Neural Production Systems provide an effective differentiable production-rule inductive bias
- domain assumption BabyAI symbolic 7x7x3 observations can be treated as pre-segmented slots
- domain assumption Instruction split by substrings yields a valid zero-shot compositional generalization test
- domain assumption PPO with two seeds provides stable enough estimates
Cite this review
Pith. "Pith review of Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning." pith.science (2026). https://pith.science/paper/S4JIFSQ4
@misc{pith2026250115270,
author = {Pith},
title = {Pith review of: Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/S4JIFSQ4}},
note = {Machine review of arXiv:2501.15270}
}
read the original abstract
Sample efficiency and systematic generalization are two long-standing challenges in reinforcement learning. Previous studies have shown that involving natural language along with other observation modalities can improve generalization and sample efficiency due to its compositional and open-ended nature. However, to transfer these properties of language to the decision-making process, it is necessary to establish a proper language grounding mechanism. One approach to this problem is applying inductive biases to extract fine-grained and informative representations from the observations, which makes them more connectable to the language units. We provide architecture-level inductive biases for modularity and sparsity mainly based on Neural Production Systems (NPS). Alongside NPS, we assign a central role to memory in our architecture. It can be seen as a high-level information aggregator which feeds policy/value heads with comprehensive information and simultaneously guides selective attention in NPS through attentional feedback. Our results in the BabyAI environment suggest that the proposed model's systematic generalization and sample efficiency are improved significantly compared to previous models. An extensive ablation study on variants of the proposed method is conducted, and the effectiveness of each employed technique on generalization, sample efficiency, and training stability is specified.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline " cite write " FUNCTION editor.postfix editor num.names #1 > "( )" "( )" if FUNCTION editor.trans.postfix editor num.names #1 > "( )" "( )" if FUNCTION trans.postfix translator num.names #1 > "( )" "( )" if FUNCTION authors.editors.reflist.apa5 'field := 'dot := field num.names 'numnames := numnames 'format.num.names := format.num.names na...
-
[2]
, " * write output.state after.block = add.period write newline
ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key keywords month note number organization pages publisher school series title type url volume year eprint archive archivePrefix primaryClass adsurl adsnote version label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.sta...
-
[3]
write newline
" write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...
-
[4]
, " * write output.state after.block = add.period write newline
ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key keywords month note number organization pages publisher school series title type url volume year archivePrefix primaryClass adsurl adsnote version label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sent...
-
[5]
write newline
" write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION ...
-
[6]
write newline
" write newline "" before.all 'output.state := FUNCTION output.doi doi empty skip "doi:" doi * "" * output if FUNCTION format.archive archivePrefix empty "" archivePrefix ":" * if FUNCTION format.primaryClass primaryClass empty "" " [" primaryClass * "] " * if FUNCTION format.eprint eprint empty "" archive empty " https://arxiv.org/abs/" eprint * " " * " ...
-
[7]
write newline
" write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTION find.integer 't := #0 'int := int not t empty not and t #1 #1 substring 's :=...
-
[8]
, " * write output.state after.block = add.period write newline
ENTRY address archive author booktitle chapter edition editor eprint howpublished institution journal key keywords month note number organization pages publisher school series title type url doi volume year archivePrefix primaryClass eid adsurl adsnote version label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.sta...
Show all 104 references
-
[9]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
-
[10]
Available from:
ENTRY address assignee author booktitle chapter cartographer day edition editor howpublished institution inventor journal key keywords month note number organization pages part publisher school series title type volume word year eprint doi url lastchecked updated archive archi...
-
[11]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
-
[12]
, Machado, M.C
agarwal_contrastive_2021 APACrefauthors Agarwal, R. , Machado, M.C. , Castro, P.S. Bellemare, M.G. APACrefauthors \ 2021 . Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning Contrastive behavioral similarity embeddings for generalization ...
2021
-
[13]
, Brohan, A
ahn2022can APACrefauthors Ahn, M. , Brohan, A. , Brown, N. , Chebotar, Y. , Cortes, O. , David, B. others APACrefauthors \ 2022 . Do as i can, not as i say: Grounding language in robotic affordances Do as i can, not as i say: Grounding language in robotic affordances . arXiv p...
2022 arXiv
-
[14]
, Colas, C
akakzia_grounding_2021 APACrefauthors Akakzia, A. , Colas, C. , Oudeyer, P Y. , CHETOUANI, M. Sigaud, O. APACrefauthors \ 2021 . Grounding Language to Autonomously-Acquired Skills via Goal Generation Grounding language to autonomously-acquired skills via goal generation . Inte...
2021
-
[15]
, Andrychowicz, M
openai_solving_2019 APACrefauthors Akkaya, I. , Andrychowicz, M. , Chociej, M. , Litwin, M. , McGrew, B. , Petron, A. others APACrefauthors \ 2019 . Solving rubik's cube with a robot hand Solving rubik's cube with a robot hand . arXiv preprint arXiv:1910.07113 ,
2019 arXiv
-
[16]
u rek, E. , Aky \
akyurek_learning_2021 APACrefauthors Aky \"u rek, E. , Aky \"u rek, A.F. Andreas, J. APACrefauthors \ 2021 . Learning to Recombine and Resample Data For Compositional Generalization Learning to recombine and resample data for compositional generalization . International Confer...
2021
-
[17]
, Didolkar, A
alias_parth_goyal_neural_2021 APACrefauthors Alias Parth Goyal, A.G. , Didolkar, A. , Ke, N.R. , Blundell, C. , Beaudoin, P. , Heess, N. Bengio, Y. APACrefauthors \ 2021 . Neural production systems Neural production systems . Advances in Neural Information Processing Systems 3...
2021
-
[18]
, Hill, F
bahdanau_learning_2018 APACrefauthors Bahdanau, D. , Hill, F. , Leike, J. , Hughes, E. , Kohli, P. Grefenstette, E. APACrefauthors \ 2019 . Learning to Understand Goal Specifications by Modelling Reward Learning to understand goal specifications by modelling reward . Internati...
2019
-
[19]
, Murty, S
bahdanau_systematic_2019 APACrefauthors Bahdanau, D. , Murty, S. , Noukhovitch, M. , Nguyen, T.H. , de Vries, H. Courville, A. APACrefauthors \ 2019 . Systematic Generalization: What Is Required and Can It Be Learned? Systematic generalization: What is required and can it be l...
2019
-
[20]
APACrefauthors \ 1958
berko1958child APACrefauthors Berko, J. APACrefauthors \ 1958 . The child's learning of English morphology The child's learning of english morphology . Word 14 2-3 150--177,
1958
-
[21]
, Fabiano, F
booch2021thinking APACrefauthors Booch, G. , Fabiano, F. , Horesh, L. , Kate, K. , Lenchner, J. , Linck, N. others APACrefauthors \ 2021 . Thinking fast and slow in AI Thinking fast and slow in ai . Proceedings of the AAAI Conference on Artificial Intelligence Proceedings of t...
2021
-
[22]
, Brown, N
brohan2022rt APACrefauthors Brohan, A. , Brown, N. , Carbajal, J. , Chebotar, Y. , Dabis, J. , Finn, C. others APACrefauthors \ 2022 . Rt-1: Robotics transformer for real-world control at scale Rt-1: Robotics transformer for real-world control at scale . arXiv preprint arXiv:2...
2022 arXiv
-
[23]
\ Parisi, D
calabretta200514evolutionary APACrefauthors Calabretta, R. \ Parisi, D. APACrefauthors \ 2005 . Evolutionary Connectionism and Mind/Brain Modularity Evolutionary connectionism and mind/brain modularity . Modularity 309,
2005
-
[24]
, Raileanu, R
campero2021learning APACrefauthors Campero, A. , Raileanu, R. , Kuttler, H. , Tenenbaum, J.B. , Rockt \"a schel, T. Grefenstette, E. APACrefauthors \ 2021 . Learning with \ AMIG \ o: Adversarially Motivated Intrinsic Goals Learning with \ amig \ o: Adversarially motivated intr...
2021
-
[25]
, Wang, J
cao2020babyai++ APACrefauthors Cao, T. , Wang, J. , Zhang, Y. Manivasagam, S. APACrefauthors \ 2020 . Babyai++: Towards grounded-language learning beyond memorization Babyai++: Towards grounded-language learning beyond memorization . arXiv preprint arXiv:2004.07200 ,
2020 arXiv
-
[26]
, Oudeyer, P Y
carta2022eager APACrefauthors Carta, T. , Oudeyer, P Y. , Sigaud, O. sylvain lamprier. APACrefauthors \ 2022 . EAGER : Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL EAGER : Asking and answering questions for automatic reward shaping in langu...
2022
-
[27]
, Gupta, A
chen_ask_2021 APACrefauthors Chen, V. , Gupta, A. Marino, K. APACrefauthors \ 2020 . Ask your humans: Using human instructions to improve generalization in reinforcement learning Ask your humans: Using human instructions to improve generalization in reinforcement learning . ar...
2020 arXiv
-
[28]
, Liang, C
chen_compositional_2020 APACrefauthors Chen, X. , Liang, C. , Yu, A.W. , Song, D. Zhou, D. APACrefauthors \ 2020 . Compositional Generalization via Neural - Symbolic Stack Machines Compositional Generalization via Neural - Symbolic Stack Machines . Advances in Neural Informati...
2020
-
[29]
, Bahdanau, D
chevalier-boisvert_babyai_2018 APACrefauthors Chevalier-Boisvert, M. , Bahdanau, D. , Lahlou, S. , Willems, L. , Saharia, C. , Nguyen, T.H. Bengio, Y. APACrefauthors \ 2019 . Baby AI : First Steps Towards Grounded Language Learning With a Human In the Loop Baby AI : First step...
2019
-
[30]
APACrefauthors \ 2014
chomsky2014aspects APACrefauthors Chomsky, N. APACrefauthors \ 2014 . Aspects of the Theory of Syntax Aspects of the theory of syntax \ ( 11). MIT press
2014
-
[31]
, Klimov, O
cobbe2019quantifying APACrefauthors Cobbe, K. , Klimov, O. , Hesse, C. , Kim, T. Schulman, J. APACrefauthors \ 2019 09--15 Jun . Quantifying Generalization in Reinforcement Learning Quantifying generalization in reinforcement learning . K. Chaudhuri\ R. Salakhutdinov\ ( ), Pro...
2019
-
[32]
, Karch, T
colas_language_2020 APACrefauthors Colas, C. , Karch, T. , Lair, N. , Dussoux, J M. , Moulin-Frier, C. , Dominey, P. Oudeyer, P Y. APACrefauthors \ 2020 . Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration Language as a Cognitive Tool to Imagine Goal...
2020
-
[33]
, Gupta, A
co-reyes_guiding_2018 APACrefauthors Co-Reyes, J.D. , Gupta, A. , Sanjeev, S. , Altieri, N. , DeNero, J. , Abbeel, P. Levine, S. APACrefauthors \ 2019 . Meta-Learning Language-Guided Policy Learning Meta-learning language-guided policy learning . International Conference on Le...
2019
-
[34]
, K \'a d \'a r, A
cote_textworld_2018 APACrefauthors C \^o t \'e , M A. , K \'a d \'a r, A. , Yuan, X. , Kybartas, B. , Barnes, T. , Fine, E. others APACrefauthors \ 2019 . Textworld: A learning environment for text-based games Textworld: A learning environment for text-based games . Computer G...
2019
-
[35]
, Shenoy, K
driscoll2022flexible APACrefauthors Driscoll, L. , Shenoy, K. Sussillo, D. APACrefauthors \ 2022 . Flexible multitask computation in recurrent networks utilizes shared dynamical motifs Flexible multitask computation in recurrent networks utilizes shared dynamical motifs . bioR...
2022
-
[36]
, Schulman, J
duan_rl2_2016 APACrefauthors Duan, Y. , Schulman, J. , Chen, X. , Bartlett, P.L. , Sutskever, I. Abbeel, P. APACrefauthors \ 2016 . Rl2: Fast reinforcement learning via slow reinforcement learning Rl2: Fast reinforcement learning via slow reinforcement learning . arXiv preprin...
2016 arXiv
-
[37]
APACrefauthors \ 2017
eichenbaum2017prefrontal APACrefauthors Eichenbaum, H. APACrefauthors \ 2017 . Prefrontal--hippocampal interactions in episodic memory Prefrontal--hippocampal interactions in episodic memory . Nature Reviews Neuroscience 18 9 547--558,
2017
-
[38]
\ Frank, M.J
franklin2020generalizing APACrefauthors Franklin, N.T. \ Frank, M.J. APACrefauthors \ 2020 . Generalizing to generalize: Humans flexibly switch between compositional and conjunctive structures during reinforcement learning Generalizing to generalize: Humans flexibly switch bet...
2020
-
[39]
, Korattikara, A
fu_language_2018 APACrefauthors Fu, J. , Korattikara, A. , Levine, S. Guadarrama, S. APACrefauthors \ 2019 . From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following From language to goals: Inverse reinforcement learning for vision-based in...
2019
-
[40]
APACrefauthors \ 2022
geffner_target_2022 APACrefauthors Geffner, H. APACrefauthors \ 2022 . Target Languages (vs. Inductive Biases ) for Learning to Act and Plan Target Languages (vs. Inductive Biases ) for Learning to Act and Plan . Proceedings of the AAAI Conference on Artificial Intelligence Pr...
2022
-
[41]
, Formica, S
gonzalez2020attentional APACrefauthors Gonz \'a lez-Garc \' a, C. , Formica, S. , Liefooghe, B. Brass, M. APACrefauthors \ 2020 . Attentional prioritization reconfigures novel instructions into action-oriented task sets Attentional prioritization reconfigures novel instruction...
2020
-
[42]
, Formica, S
gonzalez-garcia_frontoparietal_2021 APACrefauthors González-García, C. , Formica, S. , Wisniewski, D. Brass, M. APACrefauthors \ 2021 02 . Frontoparietal action-oriented codes support novel instruction implementation Frontoparietal action-oriented codes support novel instructi...
2021
-
[43]
, Niekum, S
goyal_using_2019 APACrefauthors Goyal, P. , Niekum, S. Mooney, R.J. APACrefauthors \ 2019 7 . Using Natural Language for Reward Shaping in Reinforcement Learning Using natural language for reward shaping in reinforcement learning . Proceedings of the Twenty-Eighth Internationa...
2019
-
[44]
\ Diepold, K
hein_minimal_2022 APACrefauthors Hein, A. \ Diepold, K. APACrefauthors \ 2022 12 . A Minimal Model for Compositional Generalization on gSCAN A Minimal Model for Compositional Generalization on gSCAN . Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting ...
2022
-
[45]
, Abbeel, P
hejna_iii_improving_2021 APACrefauthors Hejna, J. , Abbeel, P. Pinto, L. APACrefauthors \ 2023 . Improving long-horizon imitation through instruction prediction Improving long-horizon imitation through instruction prediction . Proceedings of the AAAI Conference on Artificial I...
2023
-
[46]
, Mokra, S
hill_human_2020 APACrefauthors Hill, F. , Mokra, S. , Wong, N. Harley, T. APACrefauthors \ 2020 . Human instruction-following with deep reinforcement learning via transfer-learning from text Human instruction-following with deep reinforcement learning via transfer-learning fro...
2020 arXiv
-
[47]
, Xia, F
huang2022inner APACrefauthors Huang, W. , Xia, F. , Xiao, T. , Chan, H. , Liang, J. , Florence, P. others APACrefauthors \ 2022 . Inner monologue: Embodied reasoning through planning with language models Inner monologue: Embodied reasoning through planning with language models...
2022 arXiv
-
[48]
, Klinger, T
ito2022compositional APACrefauthors Ito, T. , Klinger, T. , Schultz, D. , Murray, J. , Cole, M. Rigotti, M. APACrefauthors \ 2022 . Compositional generalization through abstract representations in human and artificial neural networks Compositional generalization through abstra...
2022
-
[49]
, Gu, S.S
jiang_language_2019 APACrefauthors Jiang, Y. , Gu, S.S. , Murphy, K.P. Finn, C. APACrefauthors \ 2019 . Language as an Abstraction for Hierarchical Deep Reinforcement Learning Language as an Abstraction for Hierarchical Deep Reinforcement Learning . Advances in Neural Informat...
2019
-
[50]
, Glerean, E
jaaskelainen_sparse_2022 APACrefauthors Jääskeläinen, I.P. , Glerean, E. , Klucharev, V. , Shestakova, A. Ahveninen, J. APACrefauthors \ 2022 11 . Do sparse brain activity patterns underlie human cognition? Do sparse brain activity patterns underlie human cognition? NeuroImage...
2022
-
[51]
, Sch \"a rli, N
keysers_measuring_2020 APACrefauthors Keysers, D. , Sch \"a rli, N. , Scales, N. , Buisman, H. , Furrer, D. , Kashubin, S. Bousquet, O. APACrefauthors \ 2020 . Measuring Compositional Generalization: A Comprehensive Method on Realistic Data Measuring compositional generalizati...
2020
-
[52]
, Zhang, A
kirk_survey_2023 APACrefauthors Kirk, R. , Zhang, A. , Grefenstette, E. Rockt \"a schel, T. APACrefauthors \ 2023 . A survey of zero-shot generalisation in deep reinforcement learning A survey of zero-shot generalisation in deep reinforcement learning . Journal of Artificial I...
2023
-
[53]
, Nardelli, N
kuttler_nethack_2020 APACrefauthors Küttler, H. , Nardelli, N. , Miller, A. , Raileanu, R. , Selvatici, M. , Grefenstette, E. Rocktäschel, T. APACrefauthors \ 2020 . The nethack learning environment The nethack learning environment . Advances in Neural Information Processing S...
2020
-
[54]
\ Baroni, M
lake_generalization_2018 APACrefauthors Lake, B. \ Baroni, M. APACrefauthors \ 2018 . Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks Generalization without systematicity: On the compositional skills of sequence-to-s...
2018
-
[55]
APACrefauthors \ 2019
lake_compositional_2019 APACrefauthors Lake, B.M. APACrefauthors \ 2019 . Compositional generalization through meta sequence-to-sequence learning Compositional generalization through meta sequence-to-sequence learning . Advances in Neural Information Processing Systems Advance...
2019
-
[56]
, Linzen, T
lake_human_2019 APACrefauthors Lake, B.M. , Linzen, T. Baroni, M. APACrefauthors \ 2019 . Human few-shot learning of compositional instructions Human few-shot learning of compositional instructions . arXiv preprint arXiv:1901.04587 ,
2019 arXiv
-
[57]
, Zhu, M
liu_goal-conditioned_2022 APACrefauthors Liu, M. , Zhu, M. Zhang, W. APACrefauthors \ 2022 . Goal-conditioned reinforcement learning: Problems and solutions Goal-conditioned reinforcement learning: Problems and solutions . arXiv preprint arXiv:2201.08299 ,
2022 arXiv
-
[58]
, Weissenborn, D
locatello_object-centric_2020 APACrefauthors Locatello, F. , Weissenborn, D. , Unterthiner, T. , Mahendran, A. , Heigold, G. , Uszkoreit, J. Kipf, T. APACrefauthors \ 2020 . Object-centric learning with slot attention Object-centric learning with slot attention . Advances in N...
2020
-
[59]
\ Anderson, J.R
lovett_thinking_2005 APACrefauthors Lovett, M.C. \ Anderson, J.R. APACrefauthors \ 2005 . Thinking as a production system Thinking as a production system . The Cambridge handbook of thinking and reasoning 401--429, Publisher: Cambridge University Press New York, NY
2005
-
[60]
, Fernandez, R
loynd2020working APACrefauthors Loynd, R. , Fernandez, R. , Celikyilmaz, A. , Swaminathan, A. Hausknecht, M. APACrefauthors \ 2020 . Working memory graphs Working memory graphs . International conference on machine learning International conference on machine learning \ ( \ 64...
2020
-
[61]
, Nardelli, N
luketina_survey_2019 APACrefauthors Luketina, J. , Nardelli, N. , Farquhar, G. , Foerster, J. , Andreas, J. , Grefenstette, E. Rocktäschel, T. APACrefauthors \ 2019 . A survey of reinforcement learning informed by natural language A survey of reinforcement learning informed by...
2019
-
[62]
, Ke, N.R
madan_fast_2021 APACrefauthors Madan, K. , Ke, N.R. , Goyal, A. , Sch \"o lkopf, B. Bengio, Y. APACrefauthors \ 2021 . Fast And Slow Learning Of Recurrent Independent Mechanisms Fast and slow learning of recurrent independent mechanisms . International Conference on Learning R...
2021
-
[63]
malik_when_2021 APACrefauthors Malik, D. , Li, Y. Ravikumar, P. APACrefauthors \ 2021 . When Is Generalizable Reinforcement Learning Tractable? When is generalizable reinforcement learning tractable? Advances in Neural Information Processing Systems 34 8032--8045,
2021
-
[64]
, Gagnon, L
marton2021efficient APACrefauthors M \'a rton, C.D. , Gagnon, L. , Lajoie, G. Rajan, K. APACrefauthors \ 2021 . Efficient and robust multi-task learning in the brain with modular latent primitives Efficient and robust multi-task learning in the brain with modular latent primit...
2021 arXiv
-
[65]
, Lambiotte, R
meunier_modular_2010 APACrefauthors Meunier, D. , Lambiotte, R. Bullmore, E. APACrefauthors \ 2010 . Modular and Hierarchically Modular Organization of Brain Networks Modular and Hierarchically Modular Organization of Brain Networks . Frontiers in Neuroscience 4 ,
2010
-
[66]
\ Venditto, S.J.C
miller2021multi APACrefauthors Miller, K.J. \ Venditto, S.J.C. APACrefauthors \ 2021 . Multi-step planning in the brain Multi-step planning in the brain . Current Opinion in Behavioral Sciences 38 29--39,
2021
-
[67]
, Karamcheti, S
mirchandani2021ella APACrefauthors Mirchandani, S. , Karamcheti, S. Sadigh, D. APACrefauthors \ 2021 . Ella: Exploration through learned language abstraction Ella: Exploration through learned language abstraction . Advances in Neural Information Processing Systems 34 29529--29540,
2021
-
[68]
, Rohaninejad, M
mishra_simple_2018 APACrefauthors Mishra, N. , Rohaninejad, M. , Chen, X. Abbeel, P. APACrefauthors \ 2017 . A simple neural attentive meta-learner A simple neural attentive meta-learner . arXiv preprint arXiv:1707.03141 ,
2017 arXiv
-
[69]
, Langford, J
misra_mapping_2017 APACrefauthors Misra, D. , Langford, J. Artzi, Y. APACrefauthors \ 2017 09 . Mapping Instructions and Visual Observations to Actions with Reinforcement Learning Mapping Instructions and Visual Observations to Actions with Reinforcement Learning . Proceedings...
2017
-
[70]
\ Lapata, M
mitchell_composition_2010 APACrefauthors Mitchell, J. \ Lapata, M. APACrefauthors \ 2010 . Composition in Distributional Models of Semantics Composition in Distributional Models of Semantics . Cognitive Science 34 8 1388--1429, \_eprint: https://onlinelibrary.wiley.com/doi/pdf...
2010
-
[71]
, Duncan, J
muhle-karbe_neural_2017 APACrefauthors Muhle-Karbe, P.S. , Duncan, J. , De Baene, W. , Mitchell, D.J. Brass, M. APACrefauthors \ 2017 03 . Neural Coding for Instruction - Based Task Sets in Human Frontoparietal and Visual Cortex Neural Coding for Instruction - Based Task Sets ...
2017
-
[72]
, Myers, N.E
muhle2021hierarchy APACrefauthors Muhle-Karbe, P.S. , Myers, N.E. Stokes, M.G. APACrefauthors \ 2021 . A hierarchy of functional states in working memory A hierarchy of functional states in working memory . Journal of Neuroscience 41 20 4461--4475,
2021
-
[73]
, Clavera, I
nagabandi_learning_2019 APACrefauthors Nagabandi, A. , Clavera, I. , Liu, S. , Fearing, R.S. , Abbeel, P. , Levine, S. Finn, C. APACrefauthors \ 2018 . Learning to adapt in dynamic, real-world environments through meta-reinforcement learning Learning to adapt in dynamic, real-...
2018 arXiv
-
[74]
\ Averbeck, B.B
neftci2019reinforcement APACrefauthors Neftci, E.O. \ Averbeck, B.B. APACrefauthors \ 2019 . Reinforcement learning in artificial and biological systems Reinforcement learning in artificial and biological systems . Nature Machine Intelligence 1 3 133--143,
2019
-
[75]
\ Stokes, M.G
nobre2019premembering APACrefauthors Nobre, A.C. \ Stokes, M.G. APACrefauthors \ 2019 . Premembering experience: A hierarchy of time-scales for proactive attention Premembering experience: A hierarchy of time-scales for proactive attention . Neuron 104 1 132--146,
2019
-
[76]
, Adler, T
paischer2023semantic APACrefauthors Paischer, F. , Adler, T. , Hofmarcher, M. Hochreiter, S. APACrefauthors \ 2023 . Semantic HELM: A Human-Readable Memory for Reinforcement Learning Semantic helm: A human-readable memory for reinforcement learning . Thirty-seventh Conference ...
2023
-
[77]
\ Gregoriou, G.G
paneri_top-down_2017 APACrefauthors Paneri, S. \ Gregoriou, G.G. APACrefauthors \ 2017 . Top- Down Control of Visual Attention by the Prefrontal Cortex . Functional Specialization and Long - Range Interactions Top- Down Control of Visual Attention by the Prefrontal Cortex . Fu...
2017
-
[78]
, Andrychowicz, M
peng_sim--real_2018 APACrefauthors Peng, X.B. , Andrychowicz, M. , Zaremba, W. Abbeel, P. APACrefauthors \ 2018 . Sim-to-real transfer of robotic control with dynamics randomization Sim-to-real transfer of robotic control with dynamics randomization . 2018 IEEE international c...
2018
-
[79]
, Strub, F
perez_film_2018 APACrefauthors Perez, E. , Strub, F. , De Vries, H. , Dumoulin, V. Courville, A. APACrefauthors \ 2018 . Film: Visual reasoning with a general conditioning layer Film: Visual reasoning with a general conditioning layer . Proceedings of the AAAI conference on ar...
2018
-
[80]
\ Rajan, K
perich_rethinking_2020 APACrefauthors Perich, M.G. \ Rajan, K. APACrefauthors \ 2020 12 . Rethinking brain-wide interactions through multi-region 'network of networks' models Rethinking brain-wide interactions through multi-region 'network of networks' models . Current Opinion...
2020
-
[81]
, Cohen, A.L
power_functional_2011 APACrefauthors Power, J.D. , Cohen, A.L. , Nelson, S.M. , Wig, G.S. , Barnes, K.A. , Church, J.A. Petersen, S.E. APACrefauthors \ 2011 11 . Functional network organization of the human brain Functional network organization of the human brain . Neuron 72 4...
2011
-
[82]
, Niv, Y
radulescu_holistic_2019 APACrefauthors Radulescu, A. , Niv, Y. Ballard, I. APACrefauthors \ 2019 04 . Holistic Reinforcement Learning : The Role of Structure and Attention Holistic Reinforcement Learning : The Role of Structure and Attention . Trends in Cognitive Sciences 23 4...
2019
-
[83]
\ Pouget, A
riveland2022generalization APACrefauthors Riveland, R. \ Pouget, A. APACrefauthors \ 2022 . Generalization in Sensorimotor Networks Configured with Natural Language Instructions Generalization in sensorimotor networks configured with natural language instructions . bioRxiv 2022--02,
2022
-
[84]
o der, F. , \
roder_embodied_2021 APACrefauthors R \"o der, F. , \"O zdemir, O. , Nguyen, P.D. , Wermter, S. Eppe, M. APACrefauthors \ 2021 . The embodied crossmodal self forms language and interaction: a computational cognitive review The embodied crossmodal self forms language and interac...
2021
-
[85]
, Hedayatian, S
rohani_bimrl_2022 APACrefauthors Rohani, S.R.R. , Hedayatian, S. Baghshah, M.S. APACrefauthors \ 2022 . BIMRL: Brain Inspired Meta Reinforcement Learning Bimrl: Brain inspired meta reinforcement learning . 2022 IEEE/RSJ International Conference on Intelligent Robots and System...
2022
-
[86]
, O’Reilly, R.C
russin2020deep APACrefauthors Russin, J. , O’Reilly, R.C. Bengio, Y. APACrefauthors \ 2020 . Deep learning needs a prefrontal cortex Deep learning needs a prefrontal cortex . Work Bridging AI Cogn Sci 107 603-616 1,
2020
-
[87]
, Wolski, F
schulman_proximal_2017 APACrefauthors Schulman, J. , Wolski, F. , Dhariwal, P. , Radford, A. Klimov, O. APACrefauthors \ 2017 . Proximal policy optimization algorithms Proximal policy optimization algorithms . arXiv preprint arXiv:1707.06347 ,
2017 arXiv
-
[88]
, Horn, M
seitzer2022bridging APACrefauthors Seitzer, M. , Horn, M. , Zadaianchuk, A. , Zietlow, D. , Xiao, T. , Simon-Gabriel, C J. Locatello, F. APACrefauthors \ 2023 . Bridging the Gap to Real-World Object-Centric Learning Bridging the gap to real-world object-centric learning . The ...
2023
-
[89]
, Osi \'n ski, B
shah_lm-nav_2022 APACrefauthors Shah, D. , Osi \'n ski, B. , Levine, S. \ . APACrefauthors \ 2023 . Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action...
2023
-
[90]
\ Ilin, A
spilsbury_compositional_2022 APACrefauthors Spilsbury, S. \ Ilin, A. APACrefauthors \ 2022 . Compositional Generalization in Grounded Language Learning via Induced Model Sparsity Compositional Generalization in Grounded Language Learning via Induced Model Sparsity . arXiv prep...
2022 arXiv
-
[91]
\ Betzel, R.F
sporns_modular_2016 APACrefauthors Sporns, O. \ Betzel, R.F. APACrefauthors \ 2016 . Modular Brain Networks Modular Brain Networks . Annual Review of Psychology 67 613--640,
2016
-
[92]
\ Barto, A.G
sutton_reinforcement_2018 APACrefauthors Sutton, R.S. \ Barto, A.G. APACrefauthors \ 2018 . Reinforcement learning: An introduction Reinforcement learning: An introduction . MIT press
2018
-
[93]
, Zhong, V
wang_grounding_2021 APACrefauthors Wang, H.A. , Zhong, V. , Narasimhan, K. , Reid, M. , Zhong, V. , Zhong, V. others APACrefauthors \ 2021 . Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning Grounding Language to Entities and Dynamics for...
2021
-
[94]
, Kurth-Nelson, Z
wang2018prefrontal APACrefauthors Wang, J.X. , Kurth-Nelson, Z. , Kumaran, D. , Tirumala, D. , Soyer, H. , Leibo, J.Z. Botvinick, M. APACrefauthors \ 2018 . Prefrontal cortex as a meta-reinforcement learning system Prefrontal cortex as a meta-reinforcement learning system . Na...
2018
-
[95]
, Lehman, J
wang_paired_2019 APACrefauthors Wang, R. , Lehman, J. , Clune, J. Stanley, K.O. APACrefauthors \ 2019 . Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions Paired open-ended trailblazer (poet): E...
2019 arXiv
-
[96]
\ Kennedy, H
wang_brain_2016 APACrefauthors Wang, X J. \ Kennedy, H. APACrefauthors \ 2016 04 . Brain structure and dynamics across scales: In search of rules Brain structure and dynamics across scales: In search of rules . Current opinion in neurobiology 37 92--98,
2016
-
[97]
, Joglekar, M.R
yang2019task APACrefauthors Yang, G.R. , Joglekar, M.R. , Song, H.F. , Newsome, W.T. Wang, X J. APACrefauthors \ 2019 . Task representations in neural networks trained to perform many cognitive tasks Task representations in neural networks trained to perform many cognitive tas...
2019
-
[98]
, Kostrikov, I
kostrikov_image_2021 APACrefauthors Yarats, D. , Kostrikov, I. Fergus, R. APACrefauthors \ 2021 . Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels Image augmentation is all you need: Regularizing deep reinforcement learning from pixels ....
2021
-
[99]
, Lyle, C
zhang_invariant_2020 APACrefauthors Zhang, A. , Lyle, C. , Sodhani, S. , Filos, A. , Kwiatkowska, M. , Pineau, J. Precup, D. APACrefauthors \ 2020 . Invariant causal prediction for block mdps Invariant causal prediction for block mdps . International Conference on Machine Lear...
2020
-
[100]
, McAllister, R.T
zhang_learning_2021 APACrefauthors Zhang, A. , McAllister, R.T. , Calandra, R. , Gal, Y. Levine, S. APACrefauthors \ 2021 . Learning Invariant Representations for Reinforcement Learning without Reconstruction Learning invariant representations for reinforcement learning withou...
2021
-
[101]
\ Guo, Y
zhang_generalization_2021 APACrefauthors Zhang, H. \ Guo, Y. APACrefauthors \ 2022 . Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation Generalization of reinforcement learning with policy-aware adversarial data augmentation . Decision Awa...
2022
-
[102]
, Liu, Z
zhao2021consciousness APACrefauthors Zhao, M. , Liu, Z. , Luan, S. , Zhang, S. , Precup, D. Bengio, Y. APACrefauthors \ 2021 . A consciousness-inspired planning agent for model-based reinforcement learning A consciousness-inspired planning agent for model-based reinforcement l...
2021
-
[103]
, Rocktäschel, T
zhong_rtfm_2020 APACrefauthors Zhong, V. , Rocktäschel, T. Grefenstette, E. APACrefauthors \ 2020 . RTFM : Generalising to New Environment Dynamics via Reading RTFM : Generalising to New Environment Dynamics via Reading . International Conference on Learning Representations In...
2020
-
[104]
, Schulze, S
zintgraf_varibad_2020 APACrefauthors Zintgraf, L. , Schulze, S. , Lu, C. , Feng, L. , Igl, M. , Shiarlis, K. Whiteson, S. APACrefauthors \ 2021 . Varibad: Variational bayes-adaptive deep rl via meta-learning Varibad: Variational bayes-adaptive deep rl via meta-learning . The J...
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.