{"id":"4b53d643-bb55-4202-aa05-85b9d7e981de","arxiv_id":"2412.05515","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Video2Reward uses keypoint trajectories extracted from videos to let a language model generate and iteratively refine reward functions, improving legged robot behavior learning over text-only methods.","lead":"This paper presents a system that writes robot reward functions from videos of the desired behavior, instead of from text descriptions. It lets legged robots learn gaits like walking and running by watching a clip and refining the reward through visual feedback.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The behavioral-mimicry claim rests on an ill-defined 2D projection and on reusing the same DTW metric for feedback and evaluation, so the reported gains are not yet independently established.","rationale":"The paper's two central contributions are video-conditioned reward generation and video-assisted iterative feedback. The feedback mechanism and the DTW evaluation both depend on the same Fsim, and the projection step in Section 4.3 is underspecified: video keypoints come from an unknown perspective camera, while simulated 3D keypoints are projected along an unstated 'direction of motion'. If that projection is not the camera projection, the FastDTW comparison cannot be interpreted as visual behavioral similarity. Because the same Fsim is fed back to the LLM in Algorithm 1 and then reported in Table 1, the DTW improvements are not independent evidence of mimicry. The Hmts-based human normalized score is also the selection criterion, so the +37.6% is at least partly an optimized training signal; this is acceptable if Eureka uses the same selector, but it does not support the stronger claim that the robot is actually imitating the video. The qualitative figures and the ablation in Table 3 provide some support: video input helps even without VAF, and adding VAF improves both metrics. Still, the projection ambiguity and feedback/evaluation coupling mean the method's mechanism and its behavioral-mimicry claim are not yet established. The existing CONDITIONAL verdict remains appropriate, but the conditions should include releasing the LLM identity and prompts, specifying the camera/projection details, reporting error bars across seeds, and validating with an independent held-out similarity metric.","tokens_in":12571,"tokens_out":7840,"duration_ms":77552,"concrete_test":"Re-run the Anymal and Humanoid final-policy comparisons (Table 1) with a camera-calibrated projection: recover the source camera pose from each input video (or use known camera parameters), project simulated 3D keypoints with that perspective camera instead of projecting 'along the direction of motion', and recompute DTW scores. In parallel, evaluate the final policies with a behavioral-similarity metric not used inside the feedback loop (e.g., human pairwise preference judgments or retargeted joint-angle distances). If Ours no longer beats Eureka and the w/o-VAF variant on this held-out metric, the behavioral-mimicry and video-feedback claims are not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the trajectory-similarity signal used in Section 4.3 is a valid, independent measure of behavioral mimicry. Two assumptions are load-bearing and currently unsupported. First, projection ambiguity: video keypoints are 2D projections under an unknown perspective camera, but the paper projects simulated 3D keypoints 'along the direction of motion' (Section 4.3). Without camera intrinsics/extrinsics or a view-independent representation, FastDTW scores can be made small or large by changing the projection direction, so they are not established as measuring behavior. Second, metric reuse: Algorithm 1 computes d = Fsim(hat-T, T) as feedback (line 12) and feeds it to the LLM for refinement, then Table 1 reports DTW scores of the resulting policies as evidence of mimicry. The LLM is explicitly prompted to reduce d, so Table 1 is not independent confirmation; it partly measures success at optimizing the feedback metric. The headline +37.6% also uses Hmts, which is the selection criterion in line 8, so the sparse-reward advantage is optimized directly; that is fair if Eureka uses the same selector, but it does not establish video fidelity. The mimicry claim should be demonstrated with a held-out, independently defined behavioral-similarity metric.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Video2Reward, a method that uses videos depicting target behaviors to automatically generate reward functions for legged robot reinforcement learning. A video-to-text module extracts 2D keypoint trajectories from the video and feeds them, together with auxiliary textual context, to an LLM, which produces executable reward code. A video-assisted iterative refinement loop then compares the simulated robot's projected 3D keypoint trajectories with the video trajectories using FastDTW, converts the similarity scores into textual feedback, and prompts the LLM to refine the rewards. Experiments in IsaacGym on Anymal (amble and run) and Humanoid (run) tasks report that the method outperforms Eureka by 37.6% and 11.1% in human-normalized score, and also achieves lower DTW scores, indicating closer behavioral mimicry. The paper further presents ablations showing the contribution of video-assisted feedback and qualitative visualizations of learned gaits.","tokens_in":12839,"tokens_out":5333,"duration_ms":54770,"significance":"If the results are reliable, this work would be a meaningful step toward reducing manual reward engineering and enabling more controllable, video-specified behavior learning for legged robots. The proposed pipeline is straightforward and reproducible in principle, and the ablation study is a positive feature. However, the paper's central claims are weakened by two interconnected issues: the metric used to evaluate behavioral mimicry (DTW) is also the feedback signal used to optimize the reward via the LLM, and the trajectory comparison itself relies on an ill-defined 2D projection and an unspecified keypoint correspondence. The lack of multiple seeds and error bars further limits the strength of the quantitative conclusions. These concerns are specific and load-bearing; they affect both the headline performance numbers and the behavioral-mimicry claim, and they need to be addressed before the contribution can be considered established.","major_comments":[{"comment":"The DTW-based trajectory similarity is used both as the feedback signal d in Algorithm 1 (line 12) and as the evaluation metric for behavioral mimicry in Table 1. Since the LLM is explicitly prompted to reduce d during the refinement iterations, the lower DTW scores of the final policies are not independent evidence of closer behavioral mimicry; they may simply reflect optimization of the same objective. The authors should evaluate the learned policies with a held-out, independently defined similarity metric (for example, a learned pose-distance, a different projection, or a manual labeling of behavioral similarity) to support the claim that the method produces behaviors closer to the target videos.","section":"Section 4.3, Algorithm 1, and Table 1"},{"comment":"The projection of the simulated robot's 3D keypoint trajectories 'along the direction of motion' is not well-defined, and it is not matched to the camera geometry of the input video. The video keypoints are 2D projections under an unknown perspective camera, while the simulated keypoints are in 3D world coordinates; projecting them along an arbitrary axis can make the FastDTW similarity arbitrarily large or small. The paper also does not specify how the pixel-scale video coordinates are reconciled with the metric-scale simulation coordinates. The authors should specify the exact projection axis and normalization procedure, and show that the reported DTW scores and the feedback are robust to reasonable choices of projection direction.","section":"Section 4.3"},{"comment":"All experimental results are reported as single numbers without error bars, multiple random seeds, or statistical significance tests. Reinforcement learning in IsaacGym is stochastic with respect to initializations and sampling, so the headline claim of a 37.6% improvement over Eureka on the Anymal task is not supported without variance estimates. Please repeat the experiments with at least 3-5 seeds and report mean and standard deviation, and perform a significance test where appropriate.","section":"Section 5.3 and 5.5"},{"comment":"The paper compares keypoint trajectories extracted from the video with keypoints of the simulated robot, but it does not specify how the correspondence between, for example, a dog's nose, knees, and ankles in the video and the corresponding body parts of the Anymal robot is established. The HRNet and RTMpose keypoints follow a specific skeleton, while the simulated robot has its own kinematic structure. Without an explicit mapping, the per-keypoint DTW scores computed in Table 1 and the feedback used in Algorithm 1 are not well defined. The authors should provide the exact keypoint correspondence or use a common skeleton representation for both the video and the simulation.","section":"Section 4.2 and Section 5.5 (Q1)"}],"minor_comments":[{"comment":"The notation d is introduced as a scalar Fsim(T_hat, T), but earlier the paper computes per-joint similarity scores S_j; please clarify whether d is a scalar average or a vector/list and how it is formatted in the textual feedback.","section":"Section 4.3 and Algorithm 1"},{"comment":"The 'Human' baseline is described only as a reward function designed by experts in IsaacGym; please specify which exact reward function is used and whether it is identical to the one used in the Eureka baseline for fair comparison.","section":"Section 5.2"},{"comment":"Human-normalized scores above 1.0 (e.g., 1.003 on Anymal and 1.180 on Humanoid) imply that the method surpasses the expert-designed reward; please comment on whether this is expected and whether the sparse-reward-based Hmts metric can be gamed by the policies.","section":"Table 3 and Section 5.3"},{"comment":"The video processing details, such as the frame-sampling interval and the normalization for moving cameras, are described only briefly and refer to supplementary material; please provide the essential details in the main text or ensure the supplementary material is included and accessible.","section":"Section 5.4"},{"comment":"The paper claims to surpass 'state-of-the-art LLM-based reward generation methods' but only compares against Eureka; consider adding other baselines such as L2R or Text2Reward, or soften the claim to say that the method outperforms the evaluated baseline.","section":"Section 5.5 and Conclusions"}],"recommendation":"major_revision","confidential_remarks":"The core idea is appealing and the paper addresses a timely problem, but the evaluation methodology has a fundamental circularity (DTW is both feedback and evaluation metric) and the trajectory comparison is under-specified in terms of projection and keypoint correspondence. These are not mere presentation issues; they affect the validity of the central claims. The authors should be asked to provide an independent similarity metric and a well-defined, camera-consistent trajectory comparison, along with multi-seed statistical results, before the contribution can be fully assessed. If addressed, the paper could be a solid contribution to the field."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Suppose we take this paper at face value. The core idea—feed the LLM a keypoint trajectory extracted from a video, then refine the reward with DTW-based feedback—is genuinely new relative to text-only methods like Eureka. That alone makes it worth reading. The ablations show the video feedback does something: removing VAF drops the normalized score from 1.003 to 0.921 on Anymal. That is a meaningful signal, not a trivial detail.\n\nThe problem is the evidence for the two headline claims. First, the 37.6% improvement over Eureka is reported without any variance or seed information. RL training is noisy; without multiple runs, we cannot tell if that is a real difference or one lucky roll. Second, the behavioral mimicry claim in Table 1 is weakened by metric reuse: the same FastDTW similarity that is fed back to the LLM in Algorithm 1 is then used to measure success. Of course the refined policies score better on that metric; the LLM is explicitly asked to reduce it. That gives you an optimization result, not independent confirmation. A held-out behavioral similarity measure, or at least a different projection and alignment scheme, would be needed.\n\nThe projection step is also under-specified. Video keypoints are 2D, and the paper says it projects simulated 3D keypoints 'along the direction of motion.' That is not a well-defined operation without camera intrinsics or a view-independent representation. Changing the projection could change the DTW scores, so the absolute values in Table 1 are not interpretable. This is a genuine soft spot, though not a fatal one—it mostly undermines the mimicry comparison, not the task-performance comparison.\n\nWhat else? The LLM is never named. The supplementary link exists but the code and data are not analyzed. These are fixable omissions. The paper also has some sloppy writing (e.g., 'Hmts' defined loosely), but nothing load-bearing.\n\nWho is this for? Anyone working on LLM-based reward design or legged locomotion. It is a useful prompt for the field even if the specific numbers are not yet trustworthy. I would send it to review, but with a strong recommendation for major revision. If the authors come back with seeds, a well-defined projection, and a feedback/evaluation metric split, it could be a solid contribution. If they do not, the core novelty still stands but the claims need to be scaled back.","headline":"A promising video-to-reward idea whose headline results are undercut by metric reuse and missing variance.","tokens_in":13376,"tokens_out":3685,"would_cite":true,"duration_ms":35863,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes Video2Reward, which turns videos of target behaviors into keypoint-coordinate texts that an LLM converts into reward functions, and shows this beats text-only reward generation by over 37.6% in human normalized score on…","keywords":["video2reward","reward function generation","large language models","legged robots","keypoint trajectories","FastDTW","reinforcement learning","behavior imitation"],"falsifier":"Run Video2Reward on a reference video that depicts a locomotion pattern the LLM has not seen in its text training, e.g., a dog performing a pace gait, and compare the learned robot behavior against the same video using an independent motion-capture similarity metric that does not rely on the projection used in the paper; if the video-conditioned reward does not yield lower distance to the reference than a text-only baseline across multiple seeds, the central claim fails.","tokens_in":12390,"feed_emoji":"🤖","tokens_out":5132,"duration_ms":48726,"temperature":0.7,"pith_summary":"This paper proposes replacing textual task descriptions in LLM-based reward generation with video demonstrations: a pose estimator extracts joint keypoint trajectories from a video of the desired behavior, and these coordinate sequences are formatted as text along with the environment code and task description for an LLM to write an executable reward function. The paper's core claim is that this video-grounded reward generation produces legged-robot behaviors that are both more successful on the task and more visually similar to the target behavior than rewards generated from text alone, with human normalized scores over 37.6% higher than the leading text-based method on quadruped and over 11% higher on humanoid tasks. A second claim is that using the video again as a feedback signal — comparing simulated keypoint trajectories to the video trajectories with FastDTW and feeding the similarity scores back to the LLM — materially improves the reward over using video only as input. A sympathetic reader would care because reward shaping is currently the main bottleneck in legged-robot reinforcement learning, and this offers a path from raw internet video to trainable reward without expert manual tuning.","feed_headline":"Video-based reward design beats text-only by 37.6%","feed_subtitle":"Keypoint trajectories from a video teach an LLM to write robot rewards that mimic the shown gait.","key_machinery":"The load-bearing mechanism is a video-to-text transforming module combined with a video-assisted feedback loop. The module uniformly samples frames, runs pose estimation (RTMpose for humans, HRNet for animals) to obtain J keypoint coordinates per frame, and formats the resulting trajectories as coordinate text; this text is concatenated with environment code, task description, and reward-design rules into the LLM prompt. For feedback, the trained policy is rolled out in simulation, its 3D keypoint trajectories are projected to 2D along the direction of motion, segmented into two-period chunks using autocorrelation, and each segment is compared with the video trajectory by FastDTW; the per-joint similarity scores become textual feedback appended to the prompt for the next round. The reward generation loop samples K candidate reward functions per round, trains each with PPO, selects the best by the maximum training success metric, and repeats for N rounds, so the video acts both as the initial specification and as the persistent performance signal.","core_discovery":"The central discovery is that the motion information in a video, once reduced to keypoint coordinate trajectories, is a sufficiently precise learning objective for an LLM to produce reward functions that drive reinforcement-learned policies to reproduce the demonstrated behavior. The paper shows this in two ways: quantitatively, the learned policies achieve higher maximum training success and lower DTW trajectory distance to the reference video than policies trained with Eureka, Eureka with a text description of the gait, or the expert-designed IsaacGym rewards; qualitatively, the robots exhibit the alternating footfalls of an amble or the extended hind-leg propulsion of a run as shown in the input videos. The authors also demonstrate through ablation that the video-assisted feedback loop — in which the robot's projected 3D keypoint trajectories are compared with the video's 2D keypoint trajectories using FastDTW, and per-joint similarity scores are returned as text to the LLM — is responsible for a substantial share of the gain. The claim is conditional on the simulator and the chosen LLM, but the mechanism is presented as domain-general for legged behavior learning.","pith_inferences":["One extension the authors do not test is sensitivity to camera viewpoint: projecting the robot's 3D trajectories to 2D along its direction of motion is a critical freedom, and a substantial change in the reference video's camera angle could alter similarity scores for the same behavior, suggesting a need for camera-aware or multi-view projection.","The same video-to-text and feedback loop could in principle apply to manipulation or whole-body tasks beyond legged locomotion, with the caveat that keypoint semantics must match between the video subject and the robot.","A testable refinement would be to normalize temporal dynamics before FastDTW comparison, since raw coordinate trajectories may conflate speed variations with postural differences; isolating phase-normalized pose could give a cleaner behavioral similarity signal.","As LLMs become increasingly multimodal, the explicit pose-extraction step might be replaced by direct video-conditioned reward generation, but the paper's evidence suggests that explicit keypoint text remains an effective and interpretable intermediate representation."],"forward_implications":["If the claim holds, switching the input video should change the learned gait without retraining the reward-design pipeline, enabling rapid re-purposing of one RL training setup across behaviors such as amble and run.","Reward functions produced from video should reduce the workload of manually designing dense rewards for new legged platforms, since the same video-to-text conversion and feedback loop can be applied whenever a reference movement video exists.","The video-assisted feedback is a stronger signal than text-only or reward-based feedback: even without feedback, video input beats text-only, and with feedback it surpasses expert-designed rewards on the tested tasks.","The DTW-score improvements imply that trajectories of individual joints, not just task-level metrics, align more closely to the reference, which is what makes the learned behavior look natural."],"supporting_citations":[{"why":"Eureka is the text-only LLM reward-generation baseline that this work extends and compares against; its feedback loop and reward-sampling design are reused and modified.","marker":"[19]"},{"why":"IsaacGym is the simulation environment where all policies are trained and the source of the expert-designed baseline rewards and tasks.","marker":"[20]"},{"why":"FastDTW is the similarity function used to compare robot and video trajectories and to generate the numeric feedback scores fed to the LLM.","marker":"[27]"},{"why":"The autocorrelation-based period detection method is used to segment trajectories into two-period chunks before similarity comparison.","marker":"[6]"},{"why":"RTMpose is the pose estimator used to extract human keypoint trajectories from the input videos.","marker":"[7]"},{"why":"HRNet is the pose estimator used to extract animal keypoint trajectories from the input videos.","marker":"[31]"},{"why":"PPO is the reinforcement learning algorithm used to train a policy for each candidate reward function.","marker":"[29]"}],"fun_headline_variants":["Video rewards beat text for legged robot learning by 37.6%","Watch a video, get a reward function: Robots learn gaits from clips","Robots learn to walk by watching, not reading: 37.6% better","Video2Reward: Teaching robots to walk via video-derived rewards"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 2D keypoint trajectories extracted from a video and the 2D projections of a simulated robot's 3D keypoints measure the same thing about a behavior, so that FastDTW scores and the resulting text feedback are meaningful.","fun_headline_variants_meta":{"raw":{"variants":["Video rewards beat text for legged robot learning by 37.6%","Watch a video, get a reward function: Robots learn gaits from clips","Robots learn to walk by watching, not reading: 37.6% better","Video2Reward: Teaching robots to walk via video-derived rewards"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000307,"raw_usage":{"total_tokens":1800,"prompt_tokens":1031,"completion_tokens":769,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":686}},"tokens_in":647,"tokens_out":769,"duration_ms":6774,"temperature":1.0,"reasoning_tokens":686,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:38:41.189580+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Video2Reward on a reference video that depicts a locomotion pattern the LLM has not seen in its text training, e.g., a dog performing a pace gait, and compare the learned robot behavior against the same video using an independent motion-capture similarity metric that does not rely on the projection used in the paper; if the video-conditioned reward does not yield lower distance to the reference than a text-only baseline across multiple seeds, the central claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Eureka is the text-only LLM reward-generation baseline that this work extends and compares against; its feedback loop and reward-sampling design are reused and modified."},{"cited_title":"Salvador and P","cited_arxiv_id":null,"evidence_quote":"FastDTW is the similarity function used to compare robot and video trajectories and to generate the numeric feedback scores fed to the LLM."},{"cited_title":"Breitenbach, B","cited_arxiv_id":null,"evidence_quote":"The autocorrelation-based period detection method is used to segment trajectories into two-period chunks before similarity comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RTMpose is the pose estimator used to extract human keypoint trajectories from the input videos."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HRNet is the pose estimator used to extract animal keypoint trajectories from the input videos."}],"review_version":1}