{"id":"3d1f71c3-6c9c-432f-b255-eb2b5f3b4fbf","arxiv_id":"2508.11049","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"GenFlowRL converts generated 2D object keypoint flows into a compact delta-flow that shapes dense RL rewards, improving robot manipulation performance on 10 simulation and real-world probe tasks.","lead":"This paper trains a video diffusion model to generate object motion flows from human and robot videos, then uses those flows to design rewards for reinforcement-learning robot policies. The result is a way to learn manipulation skills from easy-to-collect cross-embodiment data without labeling or large robot datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-world cross-embodiment transfer is not evaluated: Sec. 4.5 reports open-loop reward matching on manually aligned robot trajectories, with no learned policy rollout or task success rate.","rationale":"The reader's conditional verdict already notes that the real-world section does not evaluate a learned policy, but the reader's stated weakest assumption is about the reliability and sufficiency of generated object-centric flow. My stress-test identifies the real-world evidence gap as the single most load-bearing concern because the abstract prominently advertises 'real-world cross-embodiment evaluations' as part of the central claim. The manuscript's own Sec. 4.5 limits the experiment to open-loop robot trajectories and reward-curve visualization, making the real-world transfer claim unsupported by the reported data. This does not invalidate the simulation results, but it requires the paper's claims to be narrowed and the evaluation to be supplemented before the real-world transfer statement can be accepted. Since the reader's verdict is already CONDITIONAL and flags this issue in the rationale, my analysis does not move the verdict; it sharpens the specific condition that must be met: a real-world closed-loop policy evaluation or an explicit retraction of that part of the central claim.","tokens_in":16525,"tokens_out":8710,"duration_ms":92953,"concrete_test":"Inspect the project page and supplementary videos for a real-world closed-loop execution of the GenFlowRL learned policy on the XArm7, where the policy receives live observations and outputs actions during task execution. If no such policy rollout exists, the abstract and conclusion should be revised to claim only an open-loop real-world reward-matching demonstration, and the central claim of real-world cross-embodiment transfer should be retracted or reclassified as unverified. If such a rollout exists, report task success rates, number of trials, and whether the policy was trained online in the real world or transferred from simulation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and introduction claim that GenFlowRL is validated by 'real-world cross-embodiment evaluations' and that it 'consistently achiev[es] superior performance' including real-world cross-embodiment transfer. The only real-world experiment, Sec. 4.5, does not deploy or evaluate a learned policy. It states that the authors 'roll out open-loop robot trajectories aligned with each expert motion' and then compute the flow-matching reward between human and robot trajectories. Figure 7 shows reward curves for these open-loop rollouts, not closed-loop control, task success, or any comparison against baselines. Thus the central claim of real-world cross-embodiment transfer is unsupported by the reported evidence. The experiment demonstrates only that the proposed reward signal can increase along a hand-aligned trajectory, which is far weaker than showing that the learned RL policy transfers to a real robot. If the project page or supplementary material contains a closed-loop real-robot policy evaluation, this concern would be resolved; based on the manuscript text and Appendix F, the real-world study is explicitly a 'reward matchness probing' experiment, not a policy evaluation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GenFlowRL, a reinforcement learning framework that replaces video-based reward shaping with dense rewards derived from a generative object-centric 'delta-flow' prior. A diffusion-based flow generator is fine-tuned on cross-embodiment trajectories, the generated 2D keypoint flows are condensed into a delta-flow representation of centroid translation and average rotation, and a hybrid reward combines this flow-matching term with sparse environment feedback. The policy is trained with DrQv2 and conditioned on the current robot state, observed flow, future generated flow, and the initial 3D keypoint centroid. Experiments cover five Im2Flow2Act-style simulation tasks, five MetaWorld tasks, representation and noise ablations, and a real-world case study with human hand videos and a robot arm.","tokens_in":16783,"tokens_out":6884,"duration_ms":70512,"significance":"The central idea is plausible and potentially valuable: using generated object-centric flow as a cross-embodiment motion prior for reward shaping is lower-dimensional and more robust than video-based reward shaping, and the paper includes a useful noise-robustness analysis in Table 4 that directly probes the reliability of the flow prior. The delta-flow condensation is a sensible denoising step, and the comparison against VIPER and Diffusion Reward in Fig. 4 addresses the most relevant video-based reward-shaping line of work. However, the real-world study does not evaluate a learned policy, and the closest flow-based reward-shaping baseline, HuDor, is omitted; these gaps limit the strength of the central claims. The Gaussian tied-variance derivation in Sec. 3.2 is internally sound and does not constitute circular reasoning.","major_comments":[{"comment":"The headline real-world claim is not supported by the reported experiment. Section 4.5 states that the authors 'roll out open-loop robot trajectories aligned with each expert motion' and then compute the flow-matching reward between human and robot trajectories; no learned policy is executed, no task success rate is reported, and no baseline comparison is provided. Figure 7 therefore demonstrates only that the proposed reward signal increases along hand-aligned open-loop trajectories, and it does so using flow extracted from human videos rather than generated flow. The abstract's phrase 'real-world cross-embodiment evaluations' and the claim of 'consistently achieving superior performance' overstate this evidence. Either a closed-loop real-robot policy evaluation with success rates and baselines should be added, or the real-world claims should be explicitly limited to a reward-matchness feasibility probe.","section":"§4.5 / Abstract"},{"comment":"Table 2 reports success rates as single point values with no statement of the number of seeds or variance, unlike Figs. 4–6, which show shaded standard deviations over three seeds. On a benchmark where several entries are separated by only 5–10 percentage points (e.g., Pouring under demonstration-conditioning, 100 vs. 95), it is impossible to judge whether the claimed 'consistent superiority' is statistically meaningful. Please report multiple seeds and error bars or confidence intervals for all main results, and define the success criteria used for each task.","section":"§4.1, Table 2"},{"comment":"The closest reward-shaping prior, HuDor [11], is discussed in Related Work but is not included in any comparison. Because HuDor also derives dense shaped rewards from object-centric flow in a cross-embodiment setting, omitting it leaves unresolved whether the gains of GenFlowRL come from the proposed delta-flow and generated prior or are already obtainable with a single expert flow. Please add a HuDor-style baseline, or explain quantitatively why a direct comparison is not feasible.","section":"§2 / §4.1"},{"comment":"Equation (5) is advertised as a 'task-agnostic design' of the reward signal, but Appendix D.2 shows that the sparse component requires per-task subgoal and goal definitions: grasping versus contact rewards, task-specific final states, and task-specific success criteria (e.g., the pouring orientation range sampled from 5π/16 to 7π/16). This contradicts the task-agnostic claim. Either the sparse reward should be made genuinely task-agnostic, or the claim should be reworded to describe task-specified environment feedback rather than a task-agnostic design.","section":"§3.2 / §D.2"}],"minor_comments":[{"comment":"The Language-Conditioned column header appears to be mislabeled: 'PickNP. Pour Fold Fold Pivot' contains a duplicate Fold and omits Opening; please correct the column labels to match the five evaluated tasks.","section":"§4.1, Table 2"},{"comment":"Equation (2) uses a cross product between 2D vectors; please state explicitly that this denotes the scalar 2D cross product.","section":"§3.2, Eq. (2)"},{"comment":"The sentence that 'the robot flow mean TR serves as the target for the generated flow mean TG' is confusing, since under the tied-variance Gaussian assumption the KL divergence is symmetric in the two means and neither distribution is a target.","section":"§3.2, Eqs. (3)–(4)"},{"comment":"The text refers to 'CIIP [38]' for text embeddings; this appears to be a typo for CLIP.","section":"Appendix C.1"},{"comment":"The alignment between human demonstrations tracked at 5 Hz and robot rollouts tracked at 2.5 Hz is not described; please specify how the trajectories were temporally aligned before computing the reward.","section":"§4.5 / Appendix F"},{"comment":"The conclusion's caveat that 2D flow may struggle with out-of-plane rotations is important and should be stated earlier in the limitations context, with a discussion of which of the ten tasks are affected.","section":"§5"},{"comment":"The caption contains a typo: 'The shade area' should be 'The shaded area'.","section":"Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The paper has a sound core idea and useful simulation evidence, but the real-world validation is currently only an open-loop reward probe, and the closest baseline is absent. I would be willing to see a revision that either adds a closed-loop real-robot policy evaluation or narrows the claims to reward-matchness, while adding a HuDor-style baseline and error bars for the headline results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is plausible and the simulation results are coherent, but the real-world 'cross-embodiment evaluations' in the abstract are not supported by what Section 4.5 actually does. That is the one thing I want you to know before reading further.\n\nWhat is genuinely new: the delta-flow representation—condensing a set of tracked keypoints into a centroid plus average translation and rotation—is simple and seems to work. The hybrid reward that mixes that flow-matching dense term with a sparse state-based reward is a sensible way to keep a generated motion prior from leading the policy astray. The representation ablation (6D pose, 3D keypoints, trajectories) and the simulated noise study in Appendix E are genuinely informative; they show the authors are thinking about when the prior helps and when it does not. In simulation, the method beats flow-based IL baselines and video-based reward baselines on the tasks tested.\n\nThe soft spots are real. The biggest is the real-world section. Read Section 4.5 and Appendix F: there is no learned policy, no closed-loop rollout, no task success rate. They execute pre-written, manually aligned open-loop robot trajectories and plot the flow-matching reward along them. That is a reward-sanity check, not an evaluation of the transfer method. The stress-test note is accurate. The abstract claims 'real-world cross-embodiment evaluations' and 'superior performance'; that overclaims. Either add a real closed-loop policy experiment or cut the claims.\n\nOther issues, in decreasing order: Table 2 shows single-run success rates with no error bars, which is hard to interpret for RL; HuDor—the closest flow-reward-shaping baseline—is cited but not compared; and the 'distributional distance' framing in Section 3.2 is a hand-tuned L2 in practice. The many free parameters (alpha, beta, tau, C, filter thresholds, keypoint counts) get only partial sensitivity analysis, though the keypoint-count ablation is a start. The authors honestly note the 2D-flow limitation for out-of-plane rotations in the conclusion; I credit that.\n\nNet: this is a useful contribution to the robot-learning reward-shaping toolkit, and the audience is researchers working on reward shaping or learning from human video for manipulation. The simulation story is reasonably convincing, and delta-flow is worth knowing about. But as written, the real-world evidence does not back the abstract. A serious editor should still send it to review, expecting heavy revision: fix or reframe the real-world experiment, add error bars, add HuDor, and simplify the reward story.","headline":"The core idea is plausible and the simulation results are coherent, but the real-world 'cross-embodiment evaluations' in the abstract are not supported by what Section 4.5 actually does.","tokens_in":17314,"tokens_out":4116,"would_cite":true,"duration_ms":40635,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GenFlowRL turns generated object flow into dense rewards for robot learning, outperforming flow imitation and video-based RL on 10 manipulation tasks.","keywords":["object-centric flow","reward shaping","visual reinforcement learning","robot manipulation","cross-embodiment transfer","diffusion models","delta-flow","deformable objects"],"falsifier":"Run GenFlowRL on a task whose success requires a large out-of-plane rotation of the object, such as twisting a lid off a bottle, where 2D keypoint flow cannot represent the motion. If success rates fall to the level of the pure sparse reward baseline while a 3D-flow policy succeeds, the central claim that generated 2D flow is a sufficient reward prior fails.","tokens_in":16342,"feed_emoji":"🤖","tokens_out":5900,"duration_ms":58853,"temperature":0.7,"pith_summary":"This paper claims that a robot can learn dexterous manipulation policies by using generated object-centric flow, the tracked 2D motion of keypoints on an object, as a dense reward signal for reinforcement learning. The authors argue that this low-dimensional representation avoids the artifacts and cost of video-based reward models and bridges different robot embodiments. On ten simulation tasks spanning deformable, articulated, and contact-rich manipulation, GenFlowRL is reported to outperform flow-based imitation learning and video-guided RL baselines, and a real-world case study reports that rewards derived from human-hand demonstrations match robot rollouts. The practical stake is that easy-to-collect cross-embodiment demonstrations could replace large task-specific robot datasets for training reliable closed-loop policies.","feed_headline":"Generated object-flow rewards beat imitation on 10 robot tasks","feed_subtitle":"A dense reward built from generated object flow trains closed-loop robot policies, transferring from human hands to a real arm.","key_machinery":"The load-bearing object is delta-flow, a per-timestep summary of object keypoint motion as centroid position, average translation, and average rotation relative to the first frame. It carries the argument by converting noisy generated 2D keypoint trajectories into a low-dimensional, embodiment-agnostic motion prior that can be matched online against the robot's observed flow. The hybrid reward model combines this dense matching reward with sparse state-aware subgoal rewards, and the policy consumes k-step lookahead generated delta-flow plus initial 3D centroids as conditions.","core_discovery":"The paper's central claim is that object-centric flow generated by a fine-tuned diffusion model can be condensed into a delta-flow representation, per-step centroid, average translation, and average rotation of tracked keypoints, and used as a dense reward by matching the robot's observed delta-flow to the generated delta-flow. Under a Gaussian tied-variance assumption, the distributional alignment reduces to squared-error matching, and the hybrid reward adds sparse state-aware subgoal rewards. The paper reports that this reward model yields faster convergence and higher success rates than flow-based imitation learning and video-based reward baselines, particularly on contact-rich and deformable-object tasks, and that the same flow-derived reward transfers from human hand videos to a real robot arm.","pith_inferences":["Extending delta-flow to full 3D flow could handle out-of-plane rotations, a limitation the paper itself names, and would likely broaden the method to tasks such as twisting or unscrewing.","Because the reward model is task-agnostic and the flow generator is trained once on 12K trajectories, the approach may amortize across new tasks and embodiments with only sparse goal definitions, though the paper evaluates a fixed task set.","The dense reward relies on online keypoint tracking; deployment robustness may depend on tracking quality under occlusion and lighting, which the paper does not stress-test beyond simulated noise.","The policy consumes future generated delta-flow as input, so one testable extension is to remove that future-flow condition at inference and see whether the reward alone can reconstruct the motion, isolating the contribution of the prior."],"forward_implications":["On the five simulation tasks used for the main comparison, language-conditioned success rates rise substantially over the flow-based imitation baseline, for example from 35 to 80 on the folding task and from 45 to 85 on the pivoting task.","The dense delta-flow reward accelerates training and improves final performance over sparse reward, novelty-seeking exploration, and video-prediction-based reward methods, especially on assembly-style tasks.","Delta-flow enables reward shaping for deformable and articulated objects where 6D pose and 3D keypoint representations are unsuitable.","The real-world case study indicates that human hand demonstrations can provide matching flow rewards for a robot arm, suggesting cross-embodiment transfer without robot data."],"supporting_citations":[{"why":"Supplies the flow-based imitation learning baseline and the cross-embodiment flow dataset construction the paper adapts.","marker":"[51]"},{"why":"Supplies the heuristic policy baseline that receives ground-truth 3D flow and is compared against.","marker":"[58]"},{"why":"Supplies the video-prediction-reward baseline using next-frame likelihood as RL reward.","marker":"[6]"},{"why":"Supplies the state-of-the-art video-based reward shaping baseline using conditional entropy rewards.","marker":"[19]"},{"why":"Supplies the prior object-oriented reward method using a single expert flow that GenFlowRL extends to generated conditional flows.","marker":"[11]"},{"why":"Supplies the pretrained video diffusion model that is fine-tuned into the flow generator.","marker":"[10]"},{"why":"Supplies the keypoint tracker used to build flow datasets and to compute online execution flow for reward matching.","marker":"[22]"},{"why":"Supplies the object detector used to localize the object bounding box from which keypoints are sampled.","marker":"[33]"},{"why":"Supplies the semantic segmentation filter used to remove non-object keypoints from the flow.","marker":"[24]"},{"why":"Supplies the off-policy RL algorithm and its replay strategy used to optimize the flow-derived policy.","marker":"[54]"}],"fun_headline_variants":["Flow-shaped rewards boost robot learning","Object-flow rewards outdo imitation in 10 tasks","Flow-based rewards transfer from human to robot","From human hands to robot arms via flow rewards","Flow rewards turn demos into robot policies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that generated object-centric delta-flow is a reliable and sufficient motion prior: if the diffusion model's flow is noisy, embodiment-biased, or misses out-of-plane rotations, the dense reward will guide the policy away from good behavior.","fun_headline_variants_meta":{"raw":{"variants":["Flow-shaped rewards boost robot learning","Object-flow rewards outdo imitation in 10 tasks","Flow-based rewards transfer from human to robot","From human hands to robot arms via flow rewards","Flow rewards turn demos into robot policies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001536,"raw_usage":{"total_tokens":6097,"prompt_tokens":848,"completion_tokens":5249,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":5182}},"tokens_in":464,"tokens_out":5249,"duration_ms":36676,"temperature":1.0,"reasoning_tokens":5182,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:27:53.466292+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GenFlowRL on a task whose success requires a large out-of-plane rotation of the object, such as twisting a lid off a bottle, where 2D keypoint flow cannot represent the motion. If success rates fall to the level of the pure sparse reward baseline while a 3D-flow policy succeeds, the central claim that generated 2D flow is a sufficient reward prior fails.","supporting_citations":[{"cited_title":"Flow as the cross-domain manipulation interface, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the flow-based imitation learning baseline and the cross-embodiment flow dataset construction the paper adapts."},{"cited_title":"Video prediction models as rewards for reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the video-prediction-reward baseline using next-frame likelihood as RL reward."},{"cited_title":"Dif- fusion reward: Learning rewards via conditional video dif- fusion","cited_arxiv_id":null,"evidence_quote":"Supplies the state-of-the-art video-based reward shaping baseline using conditional entropy rewards."},{"cited_title":"Bridging the human to robot dexterity gap through object-oriented rewards, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the prior object-oriented reward method using a single expert flow that GenFlowRL extends to generated conditional flows."},{"cited_title":"Animatediff: Animate your personalized text-to- image diffusion models without specific tuning, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained video diffusion model that is fine-tuned into the flow generator."},{"cited_title":"Co- Tracker: It is better to track together","cited_arxiv_id":null,"evidence_quote":"Supplies the keypoint tracker used to build flow datasets and to compute online execution flow for reward matching."},{"cited_title":"Grounding dino: Marrying dino with grounded pre-training for open-set object detection","cited_arxiv_id":null,"evidence_quote":"Supplies the object detector used to localize the object bounding box from which keypoints are sampled."},{"cited_title":"Mastering visual continuous control: Improved data- augmented reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the off-policy RL algorithm and its replay strategy used to optimize the flow-derived policy."}],"review_version":2}