{"id":"f94744b7-dcd6-4b8e-8227-3a641fa786dd","arxiv_id":"2501.14856","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"NEAR trains a noise-conditioned energy-based reward from expert state transitions and anneals the noise level during RL to imitate humanoid motions at AMP-comparable performance.","lead":"NEAR learns a smooth energy reward from expert pose videos and uses it to train robot motion policies by trial and error, without needing the expert's actions. It matches the adversarial imitation method AMP on humanoid walking, running, and martial arts tasks, while reporting more stable learning dynamics.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"As written, Eq. (2) trains the energy gradient to point away from expert transitions, inverting NEAR's reward sign; if implemented literally, the policy would be anti-imitative.","rationale":"The reader's containment concern is genuine: Theorem A.2 only proves smoothness inside the perturbed support P, and the Limitations and Figure 5 admit drops at noise-level switches. However, the paper already acknowledges this risk, and the annealing rule is explicitly designed to reduce support mismatch. The more fundamental problem is that the equations defining the reward have the wrong sign. In standard denoising score matching, the conditional score for x′∼N(x,σ²I) is −(x′−x)/σ² = (x−x′)/σ². Equation (2) sets the target to +(x′−x)/σ². Section 4.2's 'flip' makes higher e_θ mean closer to pD, which would require e_θ ≈ −||x′−x||²/(2σ²) and ∇_{x′}e_θ = (x−x′)/σ². Equation (2) matches neither the raw Boltzmann energy nor the flipped energy; it matches the gradient of the squared distance pointing away from the expert sample. Maximising r = e_θ(s,s′) then drives the policy away from pD. If the implementation instead uses the standard sign, the paper misdescribes its own algorithm, and the empirical results cannot be checked against the equations. Either way, the central claim is not supported by the text. This is an internal inconsistency, not a disagreement with consensus, so it is the load-bearing concern. The authors should be credited for a sensible non-adversarial framing and for releasing code, but the paper's formal specification is the issue. I would reject as written; if the authors confirm a typographical sign error and the released code reproduces the results, a corrected resubmission could be reconsidered.","tokens_in":21898,"tokens_out":9147,"duration_ms":82728,"concrete_test":"Re-derive the conditional score and compare with Eq. (2). Concretely, train e_θ on the 2D maze domain exactly as written—loss target (x′−x)/σ², reward r = e_θ(s,s′), no task reward—and record whether the policy's occupancy moves toward or away from the expert's L-shaped corridor. If it moves away, the sign error is confirmed. Alternatively, run the paper's released code on the same maze task and inspect whether e_θ is high near pD or far from it; if it is high near pD, the implemented loss must have used the opposite sign and Eq. (2) should be corrected.","verdict_should_be":"REJECT","load_bearing_attack":"Section 4.1, Eq. (2) defines the denoising score matching target as (x′−x)/σ² for perturbed samples x′∼N(x,σ²I). For that Gaussian perturbation, the conditional score is ∇_{x′} log q_σ(x′|x) = −(x′−x)/σ² = (x−x′)/σ², so Eq. (2) trains s_θ = ∇_{x′} e_θ to match the negative of the conditional score. Section 4.2 then states that the energy sign is flipped so that higher e_θ means closer to pD; that flipped energy has gradient (x−x′)/σ², again opposite to the Eq. (2) target. Thus, as written, e_θ increases with distance from the expert state transition, and since Algorithm 1 and Eq. (3) maximise r = e_θ(s,s′) directly, the reward is anti-imitative. This is not a tuning or support issue: it is a sign inconsistency in the central construction. If the released code uses the standard NCSN target (x−x′)/σ², then the paper's equations do not describe the implemented method, and the reported comparisons cannot be traced to the stated algorithm.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NEAR, an imitation-from-observation algorithm that first trains a noise-conditioned energy network on expert state transitions via denoising score matching and then uses the learned energy as a reward for reinforcement learning, with an annealing schedule that switches among noise levels according to the policy's progress. Experiments on humanoid locomotion, martial arts, and goal-conditioned tasks compare NEAR against AMP and report comparable or better performance on pose error, spectral arc length, task return, and wall-clock time, together with ablations on annealing and task-reward composition.","tokens_in":22198,"tokens_out":7848,"duration_ms":68930,"significance":"If the technical issues below are resolved, the core idea is attractive: a stationary, pretrained reward that avoids adversarial min-max training and can be combined with any RL algorithm. The manuscript has concrete strengths: code and videos are promised, each algorithm is run with five independent seeds, ablations isolate annealing and reward composition, and the limitations section is unusually candid about the failure mode at noise-level switches. However, the central construction currently contains a sign inconsistency in the score-matching objective, and the key support-containment assumption is asserted rather than proved and is contradicted by the paper's own Figure 5. These issues must be addressed before the empirical claims can be accepted.","major_comments":[{"comment":"There is a sign inconsistency in the central reward construction. For the Gaussian perturbation qσ(x′|x)=N(x′|x,σ²I), the conditional score is ∇_{x′} log qσ(x′|x) = −(x′−x)/σ², but Eq. (2) trains ∇_{x′}eθ toward +(x′−x)/σ². Combined with the statement in Section 4.2 that the energy sign is flipped so that higher eθ means closer to pD, the reward r=eθ would be maximized by moving away from expert transitions if Algorithm 1 were implemented literally. Appendix A.2.1 repeats the incorrect identity s(x′,σ)=∇x′eθ=(x′−x)/σ². If the released code uses the standard NCSN target (x−x′)/σ², then the equations do not describe the implemented method; if it uses Eq. (2), the reported imitation results would not follow from the stated objective. This must be corrected in both directions before the empirical claims can be traced to the algorithm.","section":"Section 4.1, Eq. (2); Section 4.2; Appendix A.2.1"},{"comment":"The assumption that supp(πθG) is contained in the support of the current perturbed distribution qσk is asserted but never proved. Appendix A.1 ends with 'the annealing strategy ... and a sufficiently large σ ensure that the manifold of policy-generated samples always lies in P', but no formal bound or mechanism is given. The paper's own Limitations section and Figure 5 document that at noise-level switches the energy becomes ill-defined on part of the policy manifold and the return drops. Since the central claim is that the reward is always well-defined and informative, this gap is load-bearing. A concrete test would be to measure the fraction of policy rollouts whose state transitions fall outside the estimated support of qσk before and after each switch, or to add a support-regularization term that provably keeps the policy inside P.","section":"Section 4.2, Appendix A.1, Section 6 (Limitations), Figure 5"},{"comment":"All reported metrics (DTW pose error and SAL) are computed only on the k=20 most rewarding trajectories selected from the rollouts, and policy-evaluation metrics are recorded as means over the k=20 most rewarding environments. This selection bias can inflate the scores of both algorithms and, more importantly, can change the relative ordering when one method has heavier-tailed trajectory quality. The paper should report metrics over all rollouts (or all non-terminated episodes) and justify the top-k selection independently of the algorithms being compared.","section":"Appendix B.2.3"},{"comment":"For the spin-kick task, reference states are drawn from a beta distribution (β=3.0, α=1.0) instead of uniformly, and this task-specific initial-state shaping is described in the NEAR training details. If this modification was applied only to NEAR and not to the AMP baseline, it confounds the single-clip comparison in Table 1 because the initial-state distribution strongly affects the difficulty of imitation from observation. Please clarify whether AMP was run with the same reference-state distribution, or add an ablation showing NEAR with the uniform initialisation.","section":"Appendix B.2.2"}],"minor_comments":[{"comment":"The sentence 'NEAR also outperforms AMP in stylised goal-conditioned tasks' is not supported by the Target Reaching & Punching row, where NEAR has a lower mean task return (3.6±2.64) than AMP (3.85±0.76), with large variance; please qualify the claim.","section":"Section 5.2, Table 2"},{"comment":"The annealing progress formula in Algorithm 1 ('progress = eθ(A,σk)/mean energy on switching to σk − 1') is not defined precisely; the text in Section 4.2 describes tracking average return but does not specify the window size or how the initial return is estimated. The reward transformation in Section 5.1 also introduces r′ without stating how it interacts with the annealing progress signal.","section":"Algorithm 1, Section 5.1"},{"comment":"The notation in the proof is confusing: qσ is described as supported in a closed manifold P, but a convolution with a Gaussian kernel is supported on all of R^d whenever pD has bounded support. Please clarify whether P is meant to be the effective support (e.g., high-probability region) rather than the mathematical support.","section":"Appendix A.1"},{"comment":"The sum in the definition of LDSM has the index bound 'i=i' instead of 'i=1', and the notation x′ versus x is used inconsistently between Eq. (2) and the surrounding text.","section":"Section 4.1, Eq. (2)"},{"comment":"Figure 5 lacks error bars and a precise definition of the plotted quantity ('mini-batch energy return'); it would be helpful to show the raw energy reward before the tanh transform and to indicate which noise-level switch corresponds to each drop.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The sign inconsistency and the support-containment gap are fixable in a revision, but they are central to the method's correctness; the empirical comparison would also need re-evaluation with unbiased metrics. I do not see evidence of novelty or citation problems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe thing to know: NEAR's core idea is attractive, but the paper's central equation has a sign error that makes the method as written anti-imitative. Eq. (2) trains ∇_{x'} e_θ to match (x'−x)/σ^2, which is the gradient of the squared distance away from the expert transition, not the score of the perturbed distribution (that is (x−x')/σ^2). Section 4.2 says they flip the sign so higher energies mean closer to pD, but they never change Eq. (2), and Eq. (3) uses e_θ(s,s') directly as a reward. So if you implement the paper literally, the RL agent is rewarded for moving away from expert data. If the released code uses the standard NCSN target, then the equations don't describe the implementation. Either way, the formal description is broken.\n\nThat's a shame, because the rest of the paper has real value. The idea of using noise-conditioned energy functions as stationary rewards, with an annealed curriculum over noise levels, is a sensible non-adversarial alternative to AMP and friends. The paper is honest about limitations: single-clip imitation struggles, and Figure 5 shows energy returns drop at noise-level switches. The Appendix E experiments on AMP discriminator variance and perfect discrimination are useful even outside this paper.\n\nSecondary soft spots worth naming: evaluation metrics are computed only on the k=20 most rewarding trajectories (Appendix B.2.3), which can inflate both methods' scores; the annealing threshold α is never reported; and the claim that policy-generated samples always stay inside the support of the current perturbed energy is asserted (Appendix A.1) rather than proven, with Figure 5 showing the failure mode.\n\nWho is this for: people working in imitation from observation and character animation will find the idea worth discussing, but no one should use the method as described without first checking the code and fixing the sign. The empirical comparisons to AMP are suggestive but conditional on the reward sign being correct.\n\nMy recommendation: send it to peer review if the authors can provide the code and clarify the sign; as written, it would not pass. I'd want to see the equations match the implementation, the α value reported, and the evaluation bias addressed. If the code indeed uses the standard target, the paper needs a correction note.","headline":"NEAR's non-adversarial energy-reward idea is promising, but Eq. (2)'s sign error makes the method as written anti-imitative; the paper needs code verification.","tokens_in":22727,"tokens_out":10523,"would_cite":false,"duration_ms":84631,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-10T14:55:54.289807+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}