{"id":"f183b242-ef5b-4307-a086-17fffe49de42","arxiv_id":"2412.15427","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"AdaCred trains decision transformers with learned GumbelSigmoid token masks and an efficiency loss, claiming improved offline RL and imitation learning with shorter, pruned sequences.","lead":"AdaCred adds learned token-pruning masks to decision transformers for offline reinforcement learning, keeping only the spatial and temporal features it deems most relevant. The authors report that this improves Atari and D4RL scores with shorter sequences, but the causal theory and the evidence contain important gaps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Learned masks are never compared with random/fixed masks at the same retention rate, so the causal-credit claim is untested; Theorem 2 assumes observed latent states, contradicting the paper's POMDP setup.","rationale":"The paper's central explanatory claim is that AdaCred identifies and prunes causally reward-irrelevant features, not merely that pruning helps. The load-bearing assumption is that the learned GumbelSigmoid masks recover the true causal credit masks defined in Section 3.1. The reader's weakest assumption identified exactly this. I agree. The training objective in Section 3.2 does not incentivize causal relevance beyond sparsity and action prediction, and the theoretical proof in Theorem 2 explicitly assumes observed latent states (an MDP), while the method is motivated by partial observability. No random-mask or fixed-mask baseline is reported, so the 75% pruning results cannot distinguish causal credit assignment from generic pruning or regularization. The paper does provide useful ablations over pruning ratios (Figure 7), visualizations of masks, and a detailed two-stage training scheme, and the sparse-reward experiments are a reasonable stress test; but none of this validates the mechanism claimed. The concrete random-mask control test would settle whether the causal-credit explanation is necessary. Because the current evidence does not support the central claim, the REJECT verdict is appropriate; the residual value would be an incremental pruning study requiring new baselines and artifacts.","tokens_in":16457,"tokens_out":3554,"duration_ms":32902,"concrete_test":"Re-run the Breakout (Atari) and HalfCheetah Medium-Expert (D4RL) experiments with random binary masks at exactly the same 75% spatial / 75% temporal retention, using the same two-stage training, hyperparameters, and ten-seed protocol as the paper. Compare per-seed returns of the learned-mask variant to the random-mask control. If the learned-mask mean is within one standard error of the random-mask mean, the central causal-credit claim is unsupported and AdaCred reduces to a pruning heuristic.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is not that token pruning helps, but that AdaCred's learned credit function sigma produces masks m_t corresponding to the true causal credit c_i,t of Section 3.1. That correspondence is load-bearing. Nothing in Section 3.2 establishes it: sigma is trained with L_stage2 = L_action + alpha*L_eff, and L_eff only enforces that the fraction of active tokens equals a target percentage. Any mask at the target retention, including a random one, can satisfy L_eff; the objective does not force selected tokens to be reward-relevant. The only theoretical support, Theorem 2, assumes 'the underlying latent states g_t are observed, i.e., the environment follows an MDP,' which is the opposite of the POMDP setup introduced in Section 3.1 and used to motivate the method. In the POMDP setting the structural matrices are not proven identifiable, and no proof relates sigma to them. Consequently, the reported 75% pruning results are equally compatible with generic sparsity or regularization benefits. The absence of a random-mask or fixed-mask control at identical retention rates means the causal-credit explanation is untested; the method could simply be a token-pruning variant with architectural changes. Additionally, Table 2 contradicts 'consistently outperform': in Medium-Expert Hopper, Ours Sparse (97.33 ± 3.6) is below Baseline Sparse (106.2 ± 2.6), so the consistency claim fails even on the paper's own numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaCred, a transformer-based offline RL method that represents trajectories as causal graphs over latent states and learns binary masks, via a GumbelSigmoid credit function and an efficiency loss, to prune spatial and temporal tokens. The authors claim that pruning 75% of spatial and temporal tokens does not hurt performance, allows shorter sequences, and consistently outperforms Decision Transformer, Elastic Decision Transformer, and other baselines in offline RL and imitation learning. The paper includes two theorems intended to justify the pruning framework, experiments on six Atari games and D4RL locomotion tasks, and visualizations of learned masks.","tokens_in":16787,"tokens_out":5875,"duration_ms":53646,"significance":"If the causal-credit claim were established, AdaCred would offer a practical efficiency gain: pruning 75% of tokens while preserving or improving returns could reduce memory and compute for transformer-based offline RL, and the causal-graph framing is a plausible way to choose what to prune. The paper provides a clear two-stage training scheme, visualizations of learned masks (Figure 6), hyperparameters in the appendix, and comparisons against several standard baselines (DT, EDT, qDT, CQL, QR-DQN, REM, BEAR). However, no code or reproducibility artifacts are provided, and the theoretical results are largely definitional. The central causal-credit claim is untested as stated: the learned masks are never compared with random or fixed masks at the same retention rate, and Theorem 2 explicitly assumes an MDP while the paper's own setup is a POMDP.","major_comments":[{"comment":"Theorem 1 is essentially definitional. The set gmin_t is defined as the latent variables that either have c^{g->r}_i = 1 or have a directed path to a future reward, and the proof then uses d-separation to assert that exactly this set is necessary and sufficient. This restates the definition rather than deriving a substantive result, and it does not establish that the masks learned by the GumbelSigmoid credit function actually identify gmin_t. The central theoretical prediction is therefore unsupported.","section":"Section 3.1, Theorem 1 and Definition 1"},{"comment":"The objective L_stage2 = L_action + alpha * L_eff does not force the selected tokens to be reward-relevant. The efficiency loss L_eff only penalizes the difference between the fraction of active tokens and the target fraction, so any mask at the target retention rate, including a random mask, can achieve low L_eff. Because no random-mask or fixed-mask control at the same retention rate is reported, the observed improvements are compatible with generic sparsity or regularization benefits, and the claim that pruned tokens are genuinely reward-irrelevant is untested.","section":"Section 3.2, Stage 2 training objective"},{"comment":"The structural identifiability theorem assumes that the underlying latent states g_t are observed, i.e., that the environment is an MDP. This is exactly what the POMDP formulation in Section 3.1 and Figure 1b deny. Under the POMDP setting, no identifiability result for the structural matrices c^{g->g}, c^{a->g}, c^{g->r}, c^{a->r} is proven, and no theorem relates the learned crediting function sigma to those matrices. The theoretical bridge from the causal graphs to the learned masks is therefore missing.","section":"Appendix B.2, Theorem 2"},{"comment":"The abstract and Section 4.2.4 claim that AdaCred 'consistently outperforms' baselines, but Table 2 reports Ours Sparse = 97.33 ± 3.6 versus Baseline Sparse = 106.2 ± 2.6 in Hopper Medium-Expert, a direct counterexample on the paper's own numbers. In addition, the Medium Hopper comparison (54.47 ± 2.1 vs 52.11 ± 0.6) is within noise, and no significance tests are reported. The consistency claim fails even under the reported evaluation.","section":"Table 2, Hopper Medium-Expert sparse row"},{"comment":"Section 4.2.4 is titled 'Analysis in Sparse Reward Settings' and Section 4.1.1 mentions a 2D reacher task with sparse rewards, but Table 2 reports only dense-reward Medium and Medium-Expert locomotion results; no reacher or genuinely sparse-reward results are presented. This mislabels the evaluation and does not support the sparse-reward credit-assignment claim.","section":"Section 4.2.4 and Table 2"},{"comment":"The abstract claims that AdaCred 'consistently outperform[s] conventional methods' in both offline reinforcement learning and imitation learning, but no quantitative imitation learning comparison is provided. Section 4.2.1 describes qualitative trends for Figure 4 and Figure 5 without a table of imitation learning scores or baseline comparisons, so the imitation learning claim is not empirically supported.","section":"Abstract and Section 4.2.1"}],"minor_comments":[{"comment":"The text says the hyperparameters are 'provided in Tables 3 and 3'; the second reference should be Table 4.","section":"Section A.2"},{"comment":"Step 3.10 says 'Observe reward r_t and next observation o_{t+1}', which describes online interaction; since the method is offline, clarify that these values are read from the fixed dataset.","section":"Algorithm 1, step 3.10"},{"comment":"The Qbert result 4487 ± 5099 has a relative standard error over 100%, and no significance testing is provided; the 'superior performance' claim for Qbert is fragile.","section":"Table 1, Qbert row"},{"comment":"The discussion of pruning ratios does not specify the baseline configuration used for comparison, and no error bars are shown, making it difficult to assess whether the reported 2.8x improvement is significant.","section":"Figure 7"},{"comment":"The phrase 'without any explicit action or review' appears to contain a typo; it should likely be 'reward' rather than 'review'.","section":"Section 3.2"}],"recommendation":"reject","confidential_remarks":"The central causal-credit claim is not supported by the evidence: the efficiency loss does not tie selected tokens to future rewards, Theorem 2 assumes an MDP in a paper whose motivation is POMDP, and one reported result directly contradicts the 'consistently outperform' claim. Adding random-mask controls would improve the empirical comparison, but the missing identifiability result is a fundamental gap in the paper's stated scope. I would not encourage resubmission without a substantially new theoretical and experimental treatment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plain take: AdaCred is a token-pruning variant of a Starformer-style decision transformer, with learned GumbelSigmoid masks on spatial and temporal tokens and an efficiency loss. On Atari, keeping 75% of tokens often improves over the unpruned model, so there is a real if incremental result here. The causal-graph framing is the new packaging, but the operational contribution is pruning, and the paper does not show that the pruning is causal rather than just regularizing.\n\nThe main soft spot is the unsupported credit-claim. There is no random-mask or fixed-mask control at the same retention rate, so the results are equally compatible with generic sparsity benefits. The theory does not rescue it: Theorem 1 defines g_min as the variables that influence future rewards and then proves they are necessary and sufficient—close to a tautology. Theorem 2 assumes the latent states are observed, which is the opposite of the POMDP setup the paper opens with. That is an internal contradiction, not just an overstatement. The paper also claims 'consistently outperform' in the abstract, but Table 2 has a direct counterexample: Hopper Medium-Expert sparse, baseline 106.2±2.6 vs theirs 97.33±3.6. That is one case, but it is their own table. Code is not released, which makes the missing control harder to check externally.\n\nWhat is good: the two-stage training and the mask visualization on Breakout are clear, the efficiency loss is simple, and the Atari gains are non-trivial. Someone building efficient transformer RL might reuse this recipe. But as a scientific claim about causal credit assignment, the paper does not yet support it.\n\nFor peer review: yes, I would send it out, not desk reject. The missing random-mask baseline is easy to demand, and the theory section can be cut down to a remark or fixed to match the actual method. If the authors add the control and soften the causal story, this becomes a modest but useful paper. As is, I would not cite it for the causal interpretation.","headline":"A useful token-pruning recipe wrapped in an unsupported causal-credit story; the missing control and a contradictory theorem keep me from trusting the interpretation.","tokens_in":17331,"tokens_out":4482,"would_cite":false,"duration_ms":37163,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaCred shows that a decision transformer can learn to prune 75% of spatial and temporal tokens, use 10-step contexts, and still match or beat full-context baselines on Atari and D4RL.","keywords":["offline reinforcement learning","decision transformer","feature crediting","causal graph","token pruning","GumbelSigmoid","spatial-temporal attention","sequence modeling"],"falsifier":"Train AdaCred on the Atari benchmarks with the same 75% retention but with masks drawn randomly, or fixed to a spatial/temporal schedule, and compare returns; if random pruning matches AdaCred's numbers, the credit function is not the driver of the reported improvement. A softer check is to remove the spatial regions AdaCred's masks mark as irrelevant, such as the static background in Breakout, and verify that performance does not drop.","tokens_in":16185,"feed_emoji":"🎮","tokens_out":6914,"duration_ms":57044,"temperature":0.7,"pith_summary":"AdaCred recasts offline reinforcement learning as a problem of learning which parts of a trajectory matter, then discarding the rest. The paper argues that a decision transformer outfitted with a learned credit function, implemented as a GumbelSigmoid gating layer plus an efficiency loss, can prune roughly three-quarters of spatial and temporal tokens without hurting action quality. Across Atari and D4RL benchmarks, the pruned model matches or exceeds full-context decision transformers while requiring only a third of the sequence length. If the claim holds, transformer-based offline RL becomes substantially cheaper to run and better suited to datasets with suboptimal trajectories.","feed_headline":"Pruning 75% of tokens keeps offline RL policies strong","feed_subtitle":"Adaptive credit masks let decision transformers act from 10-step contexts and beat 30-step baselines.","key_machinery":"The carrying mechanism is a two-stage transformer pipeline: a Spatial Transformer builds per-timestep state-action-reward tokens, and a Temporal Causal Transformer mixes them across time, with a credit function $\\sigma: \\mathbb{R}^d \\to [0,1]^d$ feeding a GumbelSigmoid layer that produces binary masks $m_t$. The masks prune both spatial patches and temporal steps, and a stage-2 loss $L_{\\text{action}} + \\alpha L_{\\text{eff}}$ drives the masks toward task-relevant features, where $L_{\\text{eff}}$ is an MSE between active and target token fractions weighted by embedding dimension. The theoretical scaffold is the claim that under Markov and faithfulness assumptions, latent states without a directed path to future reward can be pruned without changing expected return.","core_discovery":"The central claim is that a trajectory can be treated as a causal graph whose nodes are latent state components, and that a learned binary mask on those components can remove reward-irrelevant spatial patches and timesteps without loss of policy performance. AdaCred computes token selection probabilities via a credit function $\\sigma$, applies a GumbelSigmoid draw to obtain binary masks $m_t$, and trains the masks jointly with action prediction and an efficiency loss that pushes the active-token fraction toward a target, such as 75%. The paper's theorems assert that, under Markov and faithfulness assumptions, the minimal sufficient latent states are exactly those with a directed path to future rewards, so pruning the rest preserves expected return. Empirically, AdaCred with 75% spatial and temporal pruning outperforms Decision Transformer and Elastic Decision Transformer on six Atari games and several D4RL locomotion tasks, and reaches its best behavior with 10-step contexts rather than 30-step ones.","pith_inferences":["Beyond the paper, the crediting mechanism could be transplanted to other transformer-based sequence models, such as language-conditioned RL, as a drop-in gating layer that only requires a scalar credit score per token and an efficiency target.","The paper does not test whether the learned masks beat random masks at equal retention; a direct comparison would isolate whether credit assignment or mere sparsity drives the gains.","Because Theorem 2 assumes latent states are observed, the faithfulness guarantee does not automatically transfer to the POMDP setting the paper motivates; a POMDP-specific identifiability argument is left implicit.","The efficiency loss with a fixed target retention rate suggests a deployment knob: the same architecture could be tuned to a given compute budget by changing the target, trading a little accuracy for more pruning."],"forward_implications":["Decision-transformer policies can operate with much shorter contexts, e.g., 10 steps instead of 30, lowering memory and compute during inference.","Spatial and temporal token pruning can be done adaptively per layer and per timestep, so retained tokens concentrate on task-relevant objects and events.","On the benchmarks tested, 75% pruning does not degrade returns; in several Atari games it improves them, including Assault, Pong, Qbert, and Seaquest.","The credit-and-prune strategy outperforms Elastic Decision Transformer's segment-stitching approach in the reported D4RL and Atari settings.","The learned masks provide interpretability: spatial masks highlight dynamic objects such as the ball and paddle in Breakout, and temporal masks highlight reward-relevant events."],"supporting_citations":[{"why":"Supplies the base Decision Transformer formulation that AdaCred extends with credit assignment and pruning.","marker":"[2]"},{"why":"The Elastic Decision Transformer is the main competing approach that selectively stitches trajectory segments; AdaCred is compared against it.","marker":"[21]"},{"why":"Provides the observation-action-reward token grouping and visual transformer design that AdaCred's spatial transformer follows.","marker":"[18]"},{"why":"The D4RL benchmark provides the continuous-control offline RL datasets used for evaluation.","marker":"[5]"},{"why":"Motivates the credit-assignment difficulty of transformers in RL that AdaCred targets.","marker":"[15]"},{"why":"Supply the compact/non-compact state representation classification and the edge-minimality principle that the pruning theory adapts.","marker":"[23, 24]"}],"fun_headline_variants":["Causal graph pruning keeps offline RL strong at 10 steps","Adaptive masks shorten RL contexts to 10 steps","Prune token masks for faster, stronger offline RL","AdaCred: causal credit masks beat longer contexts","AdaCred: Adaptive causal transformers prune 75% tokens"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The learned credit function $\\sigma$ is assumed to produce masks that match the true causal credit masks of Section 3.1, meaning the pruned tokens are genuinely reward-irrelevant rather than merely low-scoring; the paper does not compare against random or fixed masks at the same retention rate.","fun_headline_variants_meta":{"raw":{"variants":["Causal graph pruning keeps offline RL strong at 10 steps","Adaptive masks shorten RL contexts to 10 steps","Prune token masks for faster, stronger offline RL","AdaCred: causal credit masks beat longer contexts","AdaCred: Adaptive causal transformers prune 75% tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001044,"raw_usage":{"total_tokens":4351,"prompt_tokens":868,"completion_tokens":3483,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":3405}},"tokens_in":484,"tokens_out":3483,"duration_ms":23996,"temperature":1.0,"reasoning_tokens":3405,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:26:32.222449+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train AdaCred on the Atari benchmarks with the same 75% retention but with masks drawn randomly, or fixed to a spatial/temporal schedule, and compare returns; if random pruning matches AdaCred's numbers, the credit function is not the driver of the reported improvement. A softer check is to remove the spatial regions AdaCred's masks mark as irrelevant, such as the static background in Breakout, and verify that performance does not drop.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the observation-action-reward token grouping and visual transformer design that AdaCred's spatial transformer follows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates the credit-assignment difficulty of transformers in RL that AdaCred targets."}],"review_version":1}