AdaCred: Adaptive Causal Decision Transformers with Feature Crediting

Hemant Kumawat; Saibal Mukhopadhyay

REVIEW 6 major objections 5 minor 1 cited by

AdaCred: Adaptive Causal Decision Transformers with Feature Crediting

T0 review · 6 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AdaCred shows that a decision transformer can learn to prune 75% of spatial and temporal tokens, use 10-step contexts, and still match or beat full-context baselines on Atari and D4RL.

desk verdict A useful token-pruning recipe wrapped in an unsupported causal-credit story; the missing control and a contradictory theorem keep me from trusting the interpretation. read the letter →

arxiv 2412.15427 v1 pith:JNSKVTNM submitted 2024-12-19 cs.LG cs.RO

Hemant Kumawat , Saibal Mukhopadhyay This is my paper · ORCID

classification cs.LGcs.RO

keywords offlinereinforcementlearningdecisiontransformerfeaturecreditingcausalgraphtokenpruningGumbelSigmoidspatial-temporalattentionsequencemodeling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AdaCred recasts offline reinforcement learning as a problem of learning which parts of a trajectory matter, then discarding the rest. The paper argues that a decision transformer outfitted with a learned credit function, implemented as a GumbelSigmoid gating layer plus an efficiency loss, can prune roughly three-quarters of spatial and temporal tokens without hurting action quality. Across Atari and D4RL benchmarks, the pruned model matches or exceeds full-context decision transformers while requiring only a third of the sequence length. If the claim holds, transformer-based offline RL becomes substantially cheaper to run and better suited to datasets with suboptimal trajectories.

What carries the argument

The carrying mechanism is a two-stage transformer pipeline: a Spatial Transformer builds per-timestep state-action-reward tokens, and a Temporal Causal Transformer mixes them across time, with a credit function $\sigma: \mathbb{R}^d \to [0,1]^d$ feeding a GumbelSigmoid layer that produces binary masks $m_t$. The masks prune both spatial patches and temporal steps, and a stage-2 loss $L_{\text{action}} + \alpha L_{\text{eff}}$ drives the masks toward task-relevant features, where $L_{\text{eff}}$ is an MSE between active and target token fractions weighted by embedding dimension. The theoretical scaffold is the claim that under Markov and faithfulness assumptions, latent states without a directed path to future reward can be pruned without changing expected return.

What would settle it

Train AdaCred on the Atari benchmarks with the same 75% retention but with masks drawn randomly, or fixed to a spatial/temporal schedule, and compare returns; if random pruning matches AdaCred's numbers, the credit function is not the driver of the reported improvement. A softer check is to remove the spatial regions AdaCred's masks mark as irrelevant, such as the static background in Breakout, and verify that performance does not drop.

Watch

Extended reading notes

Core claim

The central claim is that a trajectory can be treated as a causal graph whose nodes are latent state components, and that a learned binary mask on those components can remove reward-irrelevant spatial patches and timesteps without loss of policy performance. AdaCred computes token selection probabilities via a credit function $\sigma$, applies a GumbelSigmoid draw to obtain binary masks $m_t$, and trains the masks jointly with action prediction and an efficiency loss that pushes the active-token fraction toward a target, such as 75%. The paper's theorems assert that, under Markov and faithfulness assumptions, the minimal sufficient latent states are exactly those with a directed path to future rewards, so pruning the rest preserves expected return. Empirically, AdaCred with 75% spatial and temporal pruning outperforms Decision Transformer and Elastic Decision Transformer on six Atari games and several D4RL locomotion tasks, and reaches its best behavior with 10-step contexts rather than 30-step ones.

Load-bearing premise

The learned credit function $\sigma$ is assumed to produce masks that match the true causal credit masks of Section 3.1, meaning the pruned tokens are genuinely reward-irrelevant rather than merely low-scoring; the paper does not compare against random or fixed masks at the same retention rate.

Editorial extensions

If this is right

Decision-transformer policies can operate with much shorter contexts, e.g., 10 steps instead of 30, lowering memory and compute during inference.
Spatial and temporal token pruning can be done adaptively per layer and per timestep, so retained tokens concentrate on task-relevant objects and events.
On the benchmarks tested, 75% pruning does not degrade returns; in several Atari games it improves them, including Assault, Pong, Qbert, and Seaquest.
The credit-and-prune strategy outperforms Elastic Decision Transformer's segment-stitching approach in the reported D4RL and Atari settings.
The learned masks provide interpretability: spatial masks highlight dynamic objects such as the ball and paddle in Breakout, and temporal masks highlight reward-relevant events.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the crediting mechanism could be transplanted to other transformer-based sequence models, such as language-conditioned RL, as a drop-in gating layer that only requires a scalar credit score per token and an efficiency target.
The paper does not test whether the learned masks beat random masks at equal retention; a direct comparison would isolate whether credit assignment or mere sparsity drives the gains.
Because Theorem 2 assumes latent states are observed, the faithfulness guarantee does not automatically transfer to the POMDP setting the paper motivates; a POMDP-specific identifiability argument is left implicit.
The efficiency loss with a fixed target retention rate suggests a deployment knob: the same architecture could be tuned to a given compute budget by changing the target, trading a little accuracy for more pruning.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Plain take: AdaCred is a token-pruning variant of a Starformer-style decision transformer, with learned GumbelSigmoid masks on spatial and temporal tokens and an efficiency loss. On Atari, keeping 75% of tokens often improves over the unpruned model, so there is a real if incremental result here. The causal-graph framing is the new packaging, but the operational contribution is pruning, and the paper does not show that the pruning is causal rather than just regularizing.

The main soft spot is the unsupported credit-claim. There is no random-mask or fixed-mask control at the same retention rate, so the results are equally compatible with generic sparsity benefits. The theory does not rescue it: Theorem 1 defines g_min as the variables that influence future rewards and then proves they are necessary and sufficient—close to a tautology. Theorem 2 assumes the latent states are observed, which is the opposite of the POMDP setup the paper opens with. That is an internal contradiction, not just an overstatement. The paper also claims 'consistently outperform' in the abstract, but Table 2 has a direct counterexample: Hopper Medium-Expert sparse, baseline 106.2±2.6 vs theirs 97.33±3.6. That is one case, but it is their own table. Code is not released, which makes the missing control harder to check externally.

What is good: the two-stage training and the mask visualization on Breakout are clear, the efficiency loss is simple, and the Atari gains are non-trivial. Someone building efficient transformer RL might reuse this recipe. But as a scientific claim about causal credit assignment, the paper does not yet support it.

For peer review: yes, I would send it out, not desk reject. The missing random-mask baseline is easy to demand, and the theory section can be cut down to a remark or fixed to match the actual method. If the authors add the control and soften the causal story, this becomes a modest but useful paper. As is, I would not cite it for the causal interpretation.

Referee Report

6 major / 5 minor

Summary. The paper proposes AdaCred, a transformer-based offline RL method that represents trajectories as causal graphs over latent states and learns binary masks, via a GumbelSigmoid credit function and an efficiency loss, to prune spatial and temporal tokens. The authors claim that pruning 75% of spatial and temporal tokens does not hurt performance, allows shorter sequences, and consistently outperforms Decision Transformer, Elastic Decision Transformer, and other baselines in offline RL and imitation learning. The paper includes two theorems intended to justify the pruning framework, experiments on six Atari games and D4RL locomotion tasks, and visualizations of learned masks.

Significance. If the causal-credit claim were established, AdaCred would offer a practical efficiency gain: pruning 75% of tokens while preserving or improving returns could reduce memory and compute for transformer-based offline RL, and the causal-graph framing is a plausible way to choose what to prune. The paper provides a clear two-stage training scheme, visualizations of learned masks (Figure 6), hyperparameters in the appendix, and comparisons against several standard baselines (DT, EDT, qDT, CQL, QR-DQN, REM, BEAR). However, no code or reproducibility artifacts are provided, and the theoretical results are largely definitional. The central causal-credit claim is untested as stated: the learned masks are never compared with random or fixed masks at the same retention rate, and Theorem 2 explicitly assumes an MDP while the paper's own setup is a POMDP.

major comments (6)

[Section 3.1, Theorem 1 and Definition 1] Theorem 1 is essentially definitional. The set gmin_t is defined as the latent variables that either have c^{g->r}_i = 1 or have a directed path to a future reward, and the proof then uses d-separation to assert that exactly this set is necessary and sufficient. This restates the definition rather than deriving a substantive result, and it does not establish that the masks learned by the GumbelSigmoid credit function actually identify gmin_t. The central theoretical prediction is therefore unsupported.
[Section 3.2, Stage 2 training objective] The objective L_stage2 = L_action + alpha * L_eff does not force the selected tokens to be reward-relevant. The efficiency loss L_eff only penalizes the difference between the fraction of active tokens and the target fraction, so any mask at the target retention rate, including a random mask, can achieve low L_eff. Because no random-mask or fixed-mask control at the same retention rate is reported, the observed improvements are compatible with generic sparsity or regularization benefits, and the claim that pruned tokens are genuinely reward-irrelevant is untested.
[Appendix B.2, Theorem 2] The structural identifiability theorem assumes that the underlying latent states g_t are observed, i.e., that the environment is an MDP. This is exactly what the POMDP formulation in Section 3.1 and Figure 1b deny. Under the POMDP setting, no identifiability result for the structural matrices c^{g->g}, c^{a->g}, c^{g->r}, c^{a->r} is proven, and no theorem relates the learned crediting function sigma to those matrices. The theoretical bridge from the causal graphs to the learned masks is therefore missing.
[Table 2, Hopper Medium-Expert sparse row] The abstract and Section 4.2.4 claim that AdaCred 'consistently outperforms' baselines, but Table 2 reports Ours Sparse = 97.33 ± 3.6 versus Baseline Sparse = 106.2 ± 2.6 in Hopper Medium-Expert, a direct counterexample on the paper's own numbers. In addition, the Medium Hopper comparison (54.47 ± 2.1 vs 52.11 ± 0.6) is within noise, and no significance tests are reported. The consistency claim fails even under the reported evaluation.
[Section 4.2.4 and Table 2] Section 4.2.4 is titled 'Analysis in Sparse Reward Settings' and Section 4.1.1 mentions a 2D reacher task with sparse rewards, but Table 2 reports only dense-reward Medium and Medium-Expert locomotion results; no reacher or genuinely sparse-reward results are presented. This mislabels the evaluation and does not support the sparse-reward credit-assignment claim.
[Abstract and Section 4.2.1] The abstract claims that AdaCred 'consistently outperform[s] conventional methods' in both offline reinforcement learning and imitation learning, but no quantitative imitation learning comparison is provided. Section 4.2.1 describes qualitative trends for Figure 4 and Figure 5 without a table of imitation learning scores or baseline comparisons, so the imitation learning claim is not empirically supported.

minor comments (5)

[Section A.2] The text says the hyperparameters are 'provided in Tables 3 and 3'; the second reference should be Table 4.
[Algorithm 1, step 3.10] Step 3.10 says 'Observe reward r_t and next observation o_{t+1}', which describes online interaction; since the method is offline, clarify that these values are read from the fixed dataset.
[Table 1, Qbert row] The Qbert result 4487 ± 5099 has a relative standard error over 100%, and no significance testing is provided; the 'superior performance' claim for Qbert is fragile.
[Figure 7] The discussion of pruning ratios does not specify the baseline configuration used for comparison, and no error bars are shown, making it difficult to assess whether the reported 2.8x improvement is significant.
[Section 3.2] The phrase 'without any explicit action or review' appears to contain a typo; it should likely be 'reward' rather than 'review'.

Circularity Check

2 steps flagged · score 6.0 of 10

Theorem 1 defines minimal sufficiency as reward-ancestry and then proves that definition; Theorem 2 assumes observed latent states (MDP) while making faithfulness its own identification criterion, so the causal-credit justification for the learned masks is partly circular.

self definitional [Section 3.1, Definition 1 and Theorem 1 (Appendix B.1)]
"For each latent state component 𝑔𝑖,𝑡 , the crediting mechanism outputs a binary mask𝑐𝑖,𝑡∈{ 0, 1}, where𝑐𝑖,𝑡 = 1 if𝑔𝑖,𝑡 contributes to the future reward 𝑟𝑡+𝑘 for some𝑘 > 0, and 0 otherwise. ... Under the assumption that the causal graph𝐺 is Markov and faithful to the observed data, the set of minimal latent states gmin 𝑡 ⊆ g𝑡 is defined as:𝑔𝑖,𝑡∈ gmin 𝑡 if𝑐𝑔→𝑟 𝑖 = 1 or 𝑔𝑖,𝑡 has a directed path to a future reward through other latent states This set forms a minimal and sufficient representation for policy learning."

The theorem's 'minimal sufficient set' is defined, not derived: it is the set of latent variables with a directed path to future rewards, which is exactly how Definition 1 defines credit (c_{i,t}=1 iff g_{i,t} contributes to a future reward). The d-separation proof only unpacks that definition under the Markov and faithfulness assumptions; it establishes no independent characterization of which states are needed for policy learning. Consequently, the paper's headline guarantee that pruning does not negatively impact the agent's ability to maximize future rewards is true by construction of the set, not by a substantive equivalence result.
self definitional [Section 3.1, Theorem 2 and Appendix B.2]
"Suppose the underlying latent states g𝑡 are observed, i.e., the environment follows a Markov Decision Process (MDP). Then, under the Markov condition and faithfulness assumption, the structural matrices C𝑔→𝑔, C𝑎→𝑔, C𝑔→𝑟, C𝑎→𝑟 are identifiable. ... For each entry𝑐𝑔→𝑔 𝑖,𝑗 , we have: 𝑐𝑔→𝑔 𝑖,𝑗 = 1 ⇐⇒ g𝑖,𝑡̸⊥ g𝑗,𝑡−1| g\{𝑖},𝑡−1,𝑎𝑡−1."

The proof defines each structural entry by the same conditional (in)dependence that it claims identifies it; the if-and-only-if is exactly the faithfulness assumption. So the theorem restates the assumption rather than proving identifiability. It also assumes the latent states are observed (an MDP), which contradicts the POMDP setup introduced in Section 3.1 and Figure 1 that motivates the method; hence the theorem cannot support the claim that the algorithm's learned masks recover the true causal credit in the actual partially observed setting.

full rationale

The two theorems offered as the theory for AdaCred are largely definitional: Theorem 1 defines minimal sufficiency as reward-ancestry and then proves that definition under Markov/faithfulness, and Theorem 2 assumes latent states are observed while making the faithfulness assumption the very iff used for identifiability. The algorithm in Section 3.2 trains masks with Lstage2 = Laction + αLeff, where Leff is only a retention-ratio penalty; no term or theorem connects the learned mask m_t to the true causal credit c_{i,t} of Definition 1, so the causal-credit interpretation of the pruning gains is an asserted link rather than a derived consequence. The empirical results themselves (Tables 1–2, Figure 7) are external comparisons and are not circular; however, the absence of a random-mask or fixed-mask control at the same retention rate means the pruning benefit cannot be attributed to causal credit. Also, Table 2's Medium-Expert Hopper row (Ours Sparse 97.33 ± 3.6 vs Baseline Sparse 106.2 ± 2.6) contradicts the abstract's 'consistently outperform,' a correctness risk rather than a circularity. Overall, the theoretical 'predictions' reduce by construction in part, while the main experimental comparisons retain independent content, so the score is 6.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on a causal model of latent variables, the identifiability theorem requires observed latent states, and the learned masks are assumed to match the causal masks. The only fitted free parameters reported are the retention percentages; the training loss weights alpha and lambda_reg are not given.

free parameters (4)

Spatial token retention percentage = 75%
Used in main experiments as the target activation percentage in the efficiency loss; ablations vary it, but 75% is chosen for headline results.
Temporal token retention percentage = 75%
Same as spatial retention; the (50%,100%) ablation in Breakout outperforms (75%,75%), so this choice is not justified as optimal.
alpha (efficiency loss weight) = not reported
Lstage2 = Laction + alpha * Leff; no value is given, but it controls the trade-off between action accuracy and pruning.
lambda_reg (regularization coefficient) = not reported
Defined in Jreg in Section 3.1, but never appears in the training objective in Section 3.2; either unused or unreported.

assumptions (4)

domain assumption The environment's generative process follows the binary-mask structural equations for g_t, o_t, r_t, including Markov and faithfulness assumptions.
Section 3.1 introduces causal masks and independent noise; all theory depends on this model being true, but it is not tested.
ad hoc to paper The latent states g_t are observed, as required by Theorem 2.
Appendix B.2 states 'Suppose the underlying latent states g_t are observed', which contradicts the POMDP formulation in Section 3.1; this makes the identifiability result inapplicable to the actual setting.
ad hoc to paper The learned GumbelSigmoid masks m_t correspond to the true causal credit masks c_i,t.
Section 3.2 claims the crediting mechanism outputs binary masks that gate features, but no proof or experiment shows these masks coincide with the causal masks from Theorem 1.
domain assumption Markov condition and faithfulness assumptions hold for the causal graph G.
These are standard assumptions in causal inference but are substantive; the paper invokes them for Theorems 1 and 2 without empirical validation.

invented entities (2)

Binary causal masks c^{g->g}, c^{a->g}, c^{g->r}, c^{a->r}
purpose: Represent causal structure of latent state transitions and rewards.
Introduced as theoretical constructs; Theorem 2 claims they are identifiable only when latent states are observed, which does not hold in the POMDP setting. No direct validation of estimated masks against ground-truth causal structure is provided. Visualizations of spatial and temporal masks in Figure 6 are qualitative.
Feature crediting function sigma
purpose: Computes token selection probabilities for pruning.
Core trainable component, but its architecture and training are not specified; the paper does not show it recovers causal credit beyond qualitative attention maps.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AdaCred: Adaptive Causal Decision Transformers with Feature Crediting." pith.science (2026). https://pith.science/paper/JNSKVTNM

@misc{pith2026241215427,
  author       = {Pith},
  title        = {Pith review of: AdaCred: Adaptive Causal Decision Transformers with Feature Crediting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JNSKVTNM}},
  note         = {Machine review of arXiv:2412.15427}
}

read the original abstract

Reinforcement learning (RL) can be formulated as a sequence modeling problem, where models predict future actions based on historical state-action-reward sequences. Current approaches typically require long trajectory sequences to model the environment in offline RL settings. However, these models tend to over-rely on memorizing long-term representations, which impairs their ability to effectively attribute importance to trajectories and learned representations based on task-specific relevance. In this work, we introduce AdaCred, a novel approach that represents trajectories as causal graphs built from short-term action-reward-state sequences. Our model adaptively learns control policy by crediting and pruning low-importance representations, retaining only those most relevant for the downstream task. Our experiments demonstrate that AdaCred-based policies require shorter trajectory sequences and consistently outperform conventional methods in both offline reinforcement learning and imitation learning environments.

Figures

Figures reproduced from arXiv: 2412.15427 by the authors.

**Figure 1.** POMDP view of RL at memorizing sequences, struggle to assign appropriate importance to task-relevant trajectories [15]. Consequently, DT and its variants often require extended sequences—spanning 30 to 50 past trajectories—just to generate a single action. Moreover, these methods exhibit poor adaptability in scenarios involving suboptimal trajectories. By attending to entire sequences indiscriminately, they risk l… view at source ↗

**Figure 2.** Our model design relationships among different observations across the temporal domain. The core hypothesis of our approach is that, by structuring the learned representations as a causal graph, not all representations contribute to the future reward. Consequently, low-credit representations—those that do not influence future rewards—can be pruned without negatively impacting the agent’s performance. This pruning m… view at source ↗

**Figure 3.** a.) Spatial Transformer with Crediting. Output of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Performance for 75% Spatial and Temporal Crediting [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Performance for 75% Spatial and Temporal Crediting for Offline RL [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Visualization of our spatial and temporal masks [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Effect of Spatial and Temporal Pruning proper temporal pruning had been applied, these erroneous temporal relationships could have been removed, allowing for better decision-making. This pruning ratio significantly underperforms compared to the baseline, emphasizing t…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Intelligent Sensing-to-Action for Robust Autonomy at the Edge: Opportunities and Challenges
cs.RO 2025-02 conditional novelty 3.0 of 10

Bidirectional sensing-action loops, built from generative sensing, Koopman-based control, neuromorphic processing, and multi-agent coordination, can markedly reduce energy and latency for edge autonomy, according to t...

Reference graph

Works this paper leans on

25 extracted references · 13 canonical work pages · cited by 1 Pith paper

[1]

Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. 2020. An opti- mistic perspective on offline reinforcement learning. In International conference on machine learning. PMLR, 104–114

work page 2020
[2]

Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. 2021. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems 34 (2021), 15084–15097

2021
[3]

Will Dabney, Mark Rowland, Marc Bellemare, and Rémi Munos. 2018. Distri- butional reinforcement learning with quantile regression. In Proceedings of the AAAI conference on artificial intelligence , Vol. 32

2018
[4]

Will Dabney, Mark Rowland, Marc G Bellemare, and Remi Munos. 2018. Distri- butional Reinforcement Learning with Quantile Regression. In AAAI Conference on Artificial Intelligence (AAAI)

work page 2018
[5]

Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. 2020. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219 (2020)

arXiv 2020
[6]

Michael Janner, Qiyang Li, and Sergey Levine. 2021. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems 34 (2021), 1273–1286

2021
[7]

Michael Janner, Qiyang Li, and Sergey Levine. 2021. Offline Reinforcement Learning as One Big Sequence Modeling Problem. arXiv:2106.02039 [cs.LG] https://arxiv.org/abs/2106.02039

arXiv 2021
[8]

Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. 2019. Stabilizing off-policy q-learning via bootstrapping error reduction. Advances in neural information processing systems 32 (2019)

2019

Show all 25 references

[9]

Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. 2020. Conserva- tive q-learning for offline reinforcement learning.Advances in Neural Information Processing Systems 33 (2020), 1179–1191

2020
[10]

Hemant Kumawat, Biswadeep Chakraborty, and Saibal Mukhopadhyay. 2024. RoboKoop: Efficient Control Conditioned Representations from Visual Input in Robotics using Koopman Operator. In 8th Annual Conference on Robot Learning . https://openreview.net/forum?id=NiA8hVdDS7

2024
[11]

Hemant Kumawat, Biswadeep Chakraborty, and Saibal Mukhopadhyay. 2024. STAGE Net: Spatio-Temporal Attention-based Graph Encoding for Learning Multi-Agent Interactions in the presence of Hidden Agents. https://openreview. net/forum?id=tsj6rDzI0V

2024
[12]

Hemant Kumawat, Biswadeep Chakraborty, and Saibal Mukhopadhyay. 2024. STEMFold: Stochastic Temporal Manifold for Multi-Agent Interactions in the Presence of Hidden Agents. arXiv preprint arXiv:2401.14522 (2024)

2024 arXiv
[14]

Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. 2020. Offline Re- inforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv:2005.01643 [cs.LG] https://arxiv.org/abs/2005.01643

2020 arXiv
[15]

Tianwei Ni, Michel Ma, Benjamin Eysenbach, and Pierre-Luc Bacon. 2023. When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment. In Thirty-seventh Conference on Neural Information Processing Systems . https: //openreview.net/forum?id=APGXBNkt6h

2023
[16]

Kruttidipta Samal, Hemant Kumawat, Priyabrata Saha, Marilyn Wolf, and Saibal Mukhopadhyay. 2022. Task-Driven RGB-Lidar Fusion for Object Tracking in Resource-Efficient Autonomous System. IEEE Transactions on Intelligent Vehicles 7, 1 (2022), 102–112. https://doi.org/10.1109/TI...

2022
[17]

Kruttidipta Samal, Hemant Kumawat, Marilyn Wolf, and Saibal Mukhopadhyay
[18]

Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, and Michael S Ryoo. 2022. Starformer: Transformer with state-action-reward representations for visual reinforcement learning. In European conference on computer vision . Springer, 462–479

2022
[19]

Sudarshan Sharma, Hemant Kumawat, and Saibal Mukhopadhyay. 2024. Chirp- Net: Noise-Resilient Sequential Chirp Based Radar Processing for Object Detec- tion. In 2024 IEEE/MTT-S International Microwave Symposium - IMS 2024 . 102–105. https://doi.org/10.1109/IMS40175.2024.10600387

2024
[20]

Richard S Sutton. 2018. Reinforcement learning: An introduction. A Bradford Book (2018)

2018
[21]

Yueh-Hua Wu, Xiaolong Wang, and Masashi Hamaya. 2024. Elastic decision transformer. Advances in Neural Information Processing Systems 36 (2024)

2024
[22]

Taku Yamagata, Ahmed Khalil, and Raul Santos-Rodriguez. 2023. Q-learning decision transformer: Leveraging dynamic programming for conditional sequence modelling in offline rl. In International Conference on Machine Learning . PMLR, 38989–39007

2023
[23]

Yupei Yang, Biwei Huang, Fan Feng, Xinyue Wang, Shikui Tu, and Lei Xu. 2024. To- wards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representations. arXiv preprint arXiv:2407.20651 (2024)

2024 arXiv
[24]

Kun Zhang and Peter Spirtes. 2011. Causal inference and causal explanation with background knowledge. Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence (2011), 581–588. A Experimental Details 9 A.1 Baselines . . . . . . . . . . . . . . . . ...

2011
[26]

Thus, pruning𝑔𝑖,𝑡 ∉ gmin 𝑡 does not affect the future rewards, and such states are irrelevant for policy learning

By the d-separation criterion,𝑔𝑖,𝑡 is conditionally independent of future rewards, given the minimal latent states gmin 𝑡 and the cumulative reward𝑅𝑡+1: 𝑔𝑖,𝑡⊥𝑟𝑡+𝑘| gmin 𝑡 ,𝑅𝑡+1 for all𝑘 > 0. Thus, pruning𝑔𝑖,𝑡 ∉ gmin 𝑡 does not affect the future rewards, and such states are irr...
[2022]

In 2022 International Joint Conference on Neural Networks (IJCNN)

A Methodology for Understanding the Origins of False Negatives in DNN Based Object Detectors. In 2022 International Joint Conference on Neural Networks (IJCNN). 1–8. https://doi.org/10.1109/IJCNN55064.2022.9892390

2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. 2020. An opti- mistic perspective on offline reinforcement learning. In International conference on machine learning. PMLR, 104–114

work page 2020

[2] [2]

Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. 2021. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems 34 (2021), 15084–15097

2021

[3] [3]

Will Dabney, Mark Rowland, Marc Bellemare, and Rémi Munos. 2018. Distri- butional reinforcement learning with quantile regression. In Proceedings of the AAAI conference on artificial intelligence , Vol. 32

2018

[4] [4]

Will Dabney, Mark Rowland, Marc G Bellemare, and Remi Munos. 2018. Distri- butional Reinforcement Learning with Quantile Regression. In AAAI Conference on Artificial Intelligence (AAAI)

work page 2018

[5] [5]

Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. 2020. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219 (2020)

arXiv 2020

[6] [6]

Michael Janner, Qiyang Li, and Sergey Levine. 2021. Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems 34 (2021), 1273–1286

2021

[7] [7]

Michael Janner, Qiyang Li, and Sergey Levine. 2021. Offline Reinforcement Learning as One Big Sequence Modeling Problem. arXiv:2106.02039 [cs.LG] https://arxiv.org/abs/2106.02039

arXiv 2021

[8] [8]

Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. 2019. Stabilizing off-policy q-learning via bootstrapping error reduction. Advances in neural information processing systems 32 (2019)

2019