Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a deep policy gradient method called Action Value Gradient (AVG) can learn continuous-control policies from a single stream of experience, with no replay buffer, batch update, or target network, and can run on real…

desk verdict Solid empirical work on incremental deep policy gradient learning, but the printed Algorithm 2 has a 0/0 divide and inconsistent variance/standard-deviation usage, so the method as written is not reproducible. read the letter →

arxiv 2411.15370 v2 pith:Y35BVPGP submitted 2024-11-22 cs.LG cs.AIcs.ROcs.SYeess.SY

Gautham Vasan , Mohamed Elsayed , Alireza Azimi , Jiamin He , Fahim Shariar , Colin Bellinger , Martha White , A. Rupam Mahmood This is my paper · ORCID
classification cs.LGcs.AIcs.ROcs.SYeess.SY
keywords actionvaluegradientincrementalreinforcementlearningreparameterizationdeeppolicyreal-timerobotreplay-freeobservationnormalizationTDerrorscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that deep reinforcement learning for continuous control does not require the usual stabilizing machinery of large replay buffers, minibatch updates, and target networks. It introduces Action Value Gradient (AVG), an incremental actor-critic that updates from the most recent transition only, using the reparameterization gradient and three normalization and scaling techniques. On standard continuous-control benchmarks, the authors report that AVG is the only incremental method that learns effectively, often reaching final performance comparable to batch soft actor-critic. On two physical robots, they report the first effective deep reinforcement learning using only incremental updates. If true, this would make real-time, on-device deep RL feasible on memory-limited hardware.

What carries the argument

The load-bearing object is a three-part stabilization stack. Observation normalization maintains running mean and variance online with Welford's algorithm; penultimate normalization projects the network's penultimate-layer features to a unit vector; and TD-error scaling divides each temporal difference error by $\sigma_\delta$ with $\sigma_\delta^2 = \mathbb{V}[R] + \mathbb{V}[\gamma]\mathbb{E}[G^2]$, computed from online statistics. The update itself is the reparameterized, squashed-normal policy gradient with sample entropy regularization, and the same sampled action is used for both the critic and actor updates. This stack is what replaces replay buffers, batch updates, and target networks.

What would settle it

Train AVG on a standard continuous-control task after multiplying all rewards by a factor of 100, and also in a version where the reward scale shifts mid-training, while recording the critic gradient norm and episodic return; if learning diverges or collapses despite the normalization stack in either setting, the paper's central claim about stability would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that an incremental, single-sample actor-critic can be made stable enough to learn deep continuous-control policies, provided the update uses the reparameterization gradient and a specific set of normalization and scaling tricks. The actor update is the entropy-regularized action-value gradient $\nabla_\theta J(\theta) \propto \mathbb{E}_{S\sim d_{\pi,\gamma}, A\sim\pi_\theta}[\nabla_\theta f_\theta(\xi; S)|_{\xi=h_\theta(A;S)} \nabla_A (q_{\pi_\theta}(S,A) - \eta \log \pi_\theta(A|S))]$, with the same action reused for the critic update. The paper demonstrates empirically that AVG is the only incremental method tested that avoids catastrophic failure on the benchmarks, and that removing target networks does not hurt it. The convergence analysis in the appendix is for a simplified variant called RPG-TD and is not a proof for AVG itself.

Load-bearing premise

The stability of AVG rests on the unproven assumption that the three online normalization and scaling techniques do not distort the reparameterized policy gradient enough to prevent convergence; the paper's convergence theorem is proven only for a simplified variant with i.i.d. transition samples, no entropy regularization, and no normalization, so nothing in the paper guarantees AVG will learn on new tasks or reward scales.

Editorial extensions

If this is right

  • AVG learns from a single stream of experience, so memory use collapses to the network weights and a few running statistics, making onboard learning practical on devices like a Jetson Nano.
  • Batch methods SAC, PPO, and TD3 degrade catastrophically when their replay buffer is shrunk toward size 1, whereas AVG succeeds with a buffer of exactly 1, i.e., no buffer at all.
  • Target Q-networks provide no benefit for AVG; only near-instant Polyak averaging ($\tau$ close to 1) matches performance, so they can be dropped without cost.
  • The normalization and scaling stack also helps other incremental methods, with IAC+ performing comparably to AVG on several tasks, though SAC-1+ and TD3-1+ remain unreliable.
  • The robot experiments demonstrate for the first time effective deep reinforcement learning on real robots using only incremental updates, on a manipulator and a mobile robot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The results suggest that the instability commonly blamed on missing replay buffers and target networks may be addressable at the level of gradient and target scaling, a hypothesis the paper tests only for one architecture and one set of benchmarks.
  • A natural extension, flagged by the authors as future work, is to combine AVG with eligibility traces to close the sample-efficiency gap to batch methods; that would make the approach more competitive on harder tasks.
  • Because the convergence theorem applies only to RPG-TD with i.i.d. samples and no normalization, a cautious reader should treat the theorem as support for the reparameterization estimator, not as a guarantee for AVG's normalization stack.
  • The reported sensitivity to hyperparameters suggests that AVG's practical success may depend on per-task tuning; a testable extension would be an automated or adaptive scheme for the entropy coefficient and learning rates.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Action Value Gradient (AVG), an incremental deep policy gradient method that updates online from the most recent transition without a replay buffer, minibatch, or target network. AVG combines the reparameterized policy gradient with entropy regularization and three stabilizers: Welford-style observation normalization, penultimate-layer normalization, and TD-error scaling. The authors evaluate AVG against IAC, SAC-1, TD3-1, and full batch methods on MuJoCo/Gymnasium tasks, DeepMind Control Suite tasks, a visual reacher, and two physical robots, and they provide a convergence analysis for a simplified variant called RPG-TD in Appendix I. The main claims are that AVG is the only incremental method that learns effectively and that it enables the first effective real-robot incremental deep RL demonstration.

Significance. The empirical program is in many ways exemplary: 30 seeds, 10M-timestep runs, a 300-configuration random-search protocol, ablations of all three stabilizers, learning-from-pixels experiments, and real-robot demonstrations with reported computation times. The public code and videos are valuable assets. If the central claims hold, the paper makes a useful contribution to resource-limited and on-device reinforcement learning. However, two load-bearing issues currently undercut the strongest claims: the printed normalization routine is internally inconsistent, and the paper's own Figure 7 contradicts the abstract's 'only incremental method' assertion. The theoretical appendix, while transparent about its scope, analyzes a simplified variant rather than AVG itself. These problems are fixable and do not invalidate the empirical core, but they must be addressed before the paper can be accepted.

major comments (3)
  1. [§3, Algorithms 2–3] The normalization routine as printed is internally inconsistent. After the Welford update, Algorithm 2 sets sigma to the biased variance M2/n and returns Xnorm = delta2/sigma. On the first observation, n=1, delta2=0, and sigma=0, so Xnorm is 0/0 and Algorithm 1 would feed NaN into both the policy and the critic on the first timestep. In addition, Algorithm 3 treats the value returned as sigma_R as a standard deviation and squares it, while Algorithm 2 defines it as a variance; Schaul-style return scaling requires the variance itself, not its square. If the released code contains a square root and an epsilon, then the pseudocode does not describe the algorithm that produced the reported results; if it does not, the printed method is not reproducible. Because observation normalization is one of the three stabilizers on which AVG's stability claim rests, the pseudocode and text must be corrected to match the actual implementation.
  2. [Abstract and §5.2, Figure 7] The abstract's claim that AVG is 'the only incremental method that learns effectively' is contradicted by the paper's own results. In Figure 7, IAC+, which is still an incremental method after receiving the same normalization and scaling techniques, 'performs in a mostly comparable manner to AVG,' as stated in Section 5.2. If the intended uniqueness claim is only about the raw IAC baseline without the proposed stabilizers, that should be stated explicitly and the abstract reworded; as written, the claim overstates what the experiments demonstrate.
  3. [Appendix I and 'On the Theory of AVG'] The convergence analysis is proved for RPG-TD, a simplified variant that uses i.i.d. transition samples, has no entropy regularization, and does not include the normalization and scaling techniques. The authors explicitly acknowledge this does not match AVG's single-stream incremental setting. Yet the main text presents this analysis as 'On the Theory of AVG' and claims the first convergence result for model-free methods using the reparameterization gradient estimator. That theoretical claim should be qualified so that it is not read as a convergence guarantee for the actual AVG algorithm. Since the paper's primary contribution is empirical, this is not fatal, but the theory section currently promises more than the theorem delivers.
minor comments (4)
  1. [§3, Algorithm 2] The notation in Algorithm 2 uses the same symbol for the running mean and for the sum of squared differences M2, which makes the Welford update hard to follow; distinct symbols such as bar-mu and M2 should be used and defined.
  2. [§4, Figure 3 caption] The SAC reference line is trained for 1M timesteps while AVG and the other incremental curves are shown for 10M timesteps; the text saying AVG achieves performance 'comparable to SAC' should acknowledge the different training budgets.
  3. [§5.2, Figure 6 caption] The caption says the learning curves are for 'the best seed,' but the surrounding text describes a random search over hyperparameter configurations; the caption should say 'best hyperparameter configuration' to avoid confusion.
  4. [§1, Figure 1] PPO does not use a replay buffer, so the x-axis label 'Replay Buffer Size' for PPO is misleading; the paper should define what is reduced for PPO (for example, update horizon or minibatch size) so the buffer-size comparison is meaningful across all three methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: AVG is a direct reparameterization-gradient actor-critic; the stabilizers are empirical, and the convergence result is explicitly for a simplified variant.

full rationale

The paper's central update (Eq. 1) is derived from the reparameterization policy gradient theorem with an entropy-augmented value function, and Appendix A supplies a proof sketch, so the algorithm is not defined in terms of its benchmark outcomes. The three stabilizers (observation normalization, penultimate normalization, and TD-error scaling) are adopted from Welford (1962), Bjorck et al. (2022), and Schaul et al. (2021) and are evaluated through ablations rather than fitted to the target result. The convergence analysis in Appendix I explicitly studies RPG-TD, a simplified variant that 'does not have entropy regularization and normalizations' and uses i.i.d. sampling that 'do not perfectly match the real setting for AVG,' so the theory does not smuggle in the empirical conclusion. Self-citations appear (e.g., Lan et al. 2022 for the reparameterization gradient theorem and Vasan et al. 2024 for the SAC baseline), but the cited theorem is re-derived in Appendix A and the baselines are external comparisons, so the self-citations are not load-bearing in a circular way. The main empirical claim is supported by benchmark experiments against independent baselines. A separate reproducibility concern exists that is not circularity: Algorithm 2 returns Xnorm = delta2/sigma with sigma = M2/n, which is 0/0 on the first observation, and Algorithm 3 squares the quantity that Algorithm 2 defines as a variance. That internal inconsistency does not make the derivation circular and does not raise the circularity score.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The central algorithm depends on standard RL mathematics plus empirically chosen hyperparameters and normalization techniques. No new physical entities, forces, or mediators are introduced. The main unpaid-for input is the empirical assertion that the normalization and scaling combination stabilizes single-sample deep updates; the convergence analysis covers a simplified variant only.

free parameters (5)
  • Actor learning rate alpha_pi = 1.1e-05 to 0.0063 depending on task (Table 6)
    Selected per task by random search over 300 configurations; controls policy update magnitude and is central to AVG's stability.
  • Critic learning rate alpha_Q = 4.8e-05 to 0.0087 depending on task (Table 6)
    Selected per task by random search; controls critic update magnitude and interacts with TD-error scaling.
  • Entropy coefficient eta = 0.006 to 0.3 depending on task (Table 6)
    Chosen during random search; weights the sample-entropy term in the actor and critic objectives.
  • Discount factor gamma = 0.95, 0.97, or 0.99 depending on task (Table 6)
    Sampled from a discrete set in random search; changes the effective horizon and the scale of bootstrapped targets.
  • Adam beta1 = 0.0 or 0.9
    Sampled in the random search procedure; affects the momentum of the optimizer and the stability of incremental updates.
assumptions (7)
  • standard math Episodic discounted MDP with continuous state and action spaces.
    The RL problem formulation and value definitions in Section 2 are standard.
  • standard math Reparameterization gradient theorem and its entropy-augmented extension (Appendix A, Theorem 2).
    The AVG update (Equation 1) is derived from this theorem, which is a known result with a proof sketch in the appendix.
  • domain assumption Squashed Gaussian policy with tanh reparameterization.
    The policy is restricted to a tanh-transformed Gaussian, a common but nontrivial modeling choice for continuous control.
  • domain assumption Assumptions 3 to 6 in Appendix I.1: Lipschitz policy and gradient, non-singular feature matrices, compatible critic parameterization, and bounded features.
    These assumptions are required for the convergence result of RPG-TD and are not verified for the deep networks used in AVG.
  • ad hoc to paper i.i.d. sampling of transition tuples in the RPG-TD analysis.
    The authors explicitly state this does not match AVG's single-stream online setting, so the theory does not directly cover the proposed algorithm.
  • standard math Welford running mean and variance provide unbiased and stable statistics.
    The online normalization in Algorithm 2 relies on Welford's algorithm, a standard method with well-known properties.
  • domain assumption The normalization and scaling techniques do not introduce bias large enough to prevent learning.
    This is supported empirically by the ablation study, but there is no proof that these transformations preserve the policy gradient direction in the deep learning setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers." pith.science (2026). https://pith.science/paper/Y35BVPGP

@misc{pith2026241115370,
  author       = {Pith},
  title        = {Pith review of: Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y35BVPGP}},
  note         = {Machine review of arXiv:2411.15370}
}
read the original abstract

Modern deep policy gradient methods achieve effective performance on simulated robotic tasks, but they all require large replay buffers or expensive batch updates, or both, making them incompatible for real systems with resource-limited computers. We show that these methods fail catastrophically when limited to small replay buffers or during incremental learning, where updates only use the most recent sample without batch updates or a replay buffer. We propose a novel incremental deep policy gradient method -- Action Value Gradient (AVG) and a set of normalization and scaling techniques to address the challenges of instability in incremental learning. On robotic simulation benchmarks, we show that AVG is the only incremental method that learns effectively, often achieving final performance comparable to batch policy gradient methods. This advancement enabled us to show for the first time effective deep reinforcement learning with real robots using only incremental updates, employing a robotic manipulator and a mobile robot.

Figures

Figures reproduced from arXiv: 2411.15370 by the authors.

Figure 1
Figure 1. Impact of reducing replay buffer size on SAC, PPO, and TD3: Decreasing the replay buffer [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Effect of σ on en￾tropy of normal and squashed normal distribution Incremental methods can be particularly prone to issues stemming from large and noisy gradients. While off-policy batch methods such as SAC and TD3 benefit from many compute-intensive gradient updates, which effectively smooth out noisy gradients, incremen￾tal methods require alternative strategies to manage large gradient updates. Hence, we focus on… view at source ↗
Figure 3
Figure 3. AVG on Gymnasium and DeepMind Control Suite tasks. Each solid learning curve is [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: AVG and IAC on the Visual Reacher task Sparse reward environments can present additional challenges, often increasing both the difficulty and the time required for learning (Vasan et al. 2024). Hence, we also evaluate our algorithms on sparse reward environments from t…
Figure 5
Figure 5. Figure 5: The gradient norm of the critic and actor networks for AVG and SAC, along with their [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Ablation study of normalization and scaling techniques used with AVG. We plot the learning [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Impact of normalization and scaling on IAC, SAC-1 and TD3-1. Suffix “ [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Impact of target Q network on AVG for different values of [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Robot Tasks On-device learning enables mobile robots to continuously improve, adapt to new data, and handle unforeseen sit￾uations, which is crucial for tasks like autonomous navigation and object recog￾nition. Commercial robots, such as the iRobot Roomba, often use on…
Figure 10
Figure 10. Figure 10: Learning curves on Real Robot Tasks The performance of AVG and resource-constrained SAC on UR￾Reacher-2 is shown in [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Squashed Normal Distribution PDF Entropy Regularization Given that batch methods such as SAC benefit from entropy regularization, we consider variants of AVG with and without entropy regularization. There are two types of entropy terms that can be added to the actor, …
Figure 12
Figure 12. Figure 12: Hyperparameter Evaluation via Random Search. Scatter plot of the performance of the [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Comparing different neural network feature normalizations — penultimate normalization [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Performance of IAC Variants. Learning curves of the best hyper-parameter configurations [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Performance of Incremental Actor Critic (IAC) Variants. Each solid learning curve is an [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Ablation study of normalization and scaling techniques used with IAC (Algo. 6). We [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: Hyperparameter Evaluation via Random Search. Scatter plot of the performance of the [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]
Figure 18
Figure 18. Figure 18: SAC 0 2M 4M 6M 8M 10M Timesteps 1250 1000 750 500 250 0 250 500 750 Ant-v4 0 2M 4M 6M 8M 10M Timesteps 1000 0 1000 2000 3000 4000 HalfCheetah-v4 0 2M 4M 6M 8M 10M Timesteps 0 500 1000 1500 2000 Hopper-v4 0 2M 4M 6M 8M 10M Timesteps 200 400 600 800 1000 Humanoid-v4 1 1…
Figure 19
Figure 19. Figure 19: PPO 0 2M 4M 6M 8M 10M Timesteps 0 1000 2000 3000 4000 5000 6000 Ant-v4 0 2M 4M 6M 8M 10M Timesteps 0 2000 4000 6000 8000 10000 12000 14000 HalfCheetah-v4 0 2M 4M 6M 8M 10M Timesteps 0 500 1000 1500 2000 2500 3000 3500 Hopper-v4 0 2M 4M 6M 8M 10M Timesteps 0 1000 2000 …
Figure 20
Figure 20. Figure 20: TD3 27 [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]
Figure 21
Figure 21. Figure 21: Learning curves on Robots. Comparison of AVG with full PPO & SAC. Note that running [PITH_FULL_IMAGE:figures/full_fig_p028_21.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning

    cs.LG 2025-06 conditional novelty 6.0 of 10

    One-shot random pruning at initialization lets deep RL networks keep improving at model sizes where dense networks collapse in performance.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    ∇aQw(s, a)|a=f θ(s,ϵ) = ∇θf θ(s, ϵ)T w

    work page

  2. [2]

    w = w∗ ξθ minimizes the mean square error Eνθ,p ξ(s, ϵ; θ, w)T ξ(s, ϵ; θ, w) , where ξ(s, ϵ; θ, w) =∇aQw(s, a)|a=f θ(s,ϵ) −∇aQf θ (s, a)|a=f θ(s,ϵ). Given the above assumption, one can show that the reparameterization gradient is smooth (Lemma 8), and that Algorithm 7 converges (Theorem 9), the proofs of which are presented in Appendix I.2. Lemma 8. Suppo...

    work page 2022

  3. [3]

    If ˆY is an unbiased estimator of ¯Y and { ˆYj}j are i.i.d

    Let ˆY ,¯Y ∈ Rd1×d2 be matrices satisfying ˆY F ≤ CY , ¯Y F ≤ CY . If ˆY is an unbiased estimator of ¯Y and { ˆYj}j are i.i.d. estimators, then we have E 1 M M −1X j=0 ˆYj − ¯Y 2 F ≤ 4C 2 Y M . 33

    work page

  4. [4]

    If ˆy is an unbiased estimator of ¯y and {yj}j are i.i.d

    Let ˆy, ¯y ∈ Rd be vectors satisfying ∥ˆy∥ ≤Cy, ∥¯y∥ ≤Cy. If ˆy is an unbiased estimator of ¯y and {yj}j are i.i.d. estimators, then we have E 1 M M −1X j=0 ˆyj − ¯y 2 ≤ 4C 2 y M . Proof. See the proof of Lemma 4 of Xiong et al. (2022). Lemma 14. Let w∗ ξθ be defined in Proposition 7. Suppose Assumptions 3-5 hold. Then we have w∗ ξθ ≤ Cwξ , where Cwξ = Lf...

    work page 2022

  5. [5]

    Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: All our claims are verified by extensive experimental results and ablation studies in simulation and also tested on robots. Guidelines: • The answer NA means that the abstract and introduction do no...

    work page

  6. [6]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discuss the limitations of our work in the main paper. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • Th...

    work page

  7. [7]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory Assumptions and Proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] 42 Justification: We discuss the relevant theory and convergence proofs of the reparametrization gradient estimator in Appendix A and Appendix I respectively. Guidelines: • The answer NA ...

    work page

  8. [8]

    We provide pseudo- code and implementation details which are easy to follow and reproduce

    Experimental Result Reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: We propose a novel increme...

    work page

Show all 19 references
  1. [9]

    All data can be generated during training

    Open access to data and code 43 Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We share the relevant co...

  2. [10]

    We also list important hyper-parameters, neural network architectures, and other training details in the appendix

    Experimental Setting/Details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: We provide descriptions of ou...

  3. [11]

    All results are averaged over 30 runs and reported with 95% confidence interval

    Experiment Statistical Significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: We assume normally distributed errors. All results...

  4. [12]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments Compute Resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: It is listed in the app...

  5. [13]

    • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics

    Code Of Ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: Our paper conforms to the NeurIPS Code of Ethics Guidelines: • The answer NA me...

  6. [14]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: We discuss this in the main paper under the paragraph title Societal Impact. Guidelines: • The answer NA ...

  7. [15]

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: This paper does...

  8. [16]

    All our results are generated during training

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We implemented m...

  9. [17]

    • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates

    New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We provide our code and a Readme file to run experiments Guidelines: • The answer NA means that the paper does not r...

  10. [18]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and Research with Human Subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  11. [19]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.