{"id":"277f4dfe-fa38-4c28-952c-0694e4e5e605","arxiv_id":"2608.13028","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Generative RGB-D handover videos with face and body context, plus a gaze-based gating module, reduce false triggers and improve intention prediction on a physical robot.","lead":"This paper presents Hand2Bot, a new RGB-D video dataset for human-to-robot handovers that includes full-body context and realistic depth noise, and PassGen, a generative pipeline that synthesizes additional handover clips. If the results hold, it points toward robots that read gaze and posture to anticipate handover intent rather than simply tracking the hand.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PassGen's transfer claim is not yet supported: the key unseen-object gain is 7/10 vs 6/10 with tau selected on the evaluation set and no significance testing; the depth-noise heuristic is secondary.","rationale":"The reader's weakest_assumption is the heuristic depth-noise model in Section 4.2. That is a legitimate concern, but I find the experimental basis for the headline claim more load-bearing. The depth-noise heuristic is explicitly self-identified as an approximation, and its failure would mainly weaken the sim-to-real story; however, even a perfect noise model would not rescue the downstream comparison because the key unseen-object result is a 6/10 vs 7/10 difference without significance testing or seed variation. Section 6.7 also describes selecting tau=0.80 through an offline sweep whose 13.6% FPR matches the evaluation-subset number in Table 4, so the threshold appears to be tuned on the evaluation data. The physical trial is small, and the abstract's comparison to 'traditional hand-centric baselines' is not directly operationalized in any reported experiment. These issues do not invalidate the dataset or the generative pipeline as a resource, but they do mean the central claim of robust zero-shot transfer and earlier anticipation is conditional on further evidence. The reader's CONDITIONAL verdict therefore remains appropriate; my read does not move it.","tokens_in":10425,"tokens_out":5093,"duration_ms":53916,"concrete_test":"Run a pre-registered physical or simulated evaluation with: a held-out calibration split for tau, at least 50 unseen-object positive trials and 50 negative trials, and 3 random seeds. Report binomial confidence intervals and a permutation test comparing Real vs Real+Syn under an identical prediction architecture. If the 7/10 vs 6/10 difference does not survive, or the confidence interval includes zero, the transfer claim should be weakened to a variance-reduction and false-positive suppression effect on the tested scene only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims that training on PassGen gives 'robust zero-shot transfer and earlier intention anticipation'. The only direct evidence is Section 6.6 / Table 6: Real+Syn raises unseen-object ISR from 6/10 to 7/10 and mean FPR from 22.8% to 13.6%, with N=10 unseen-object trials, no multiple random seeds, and no significance test. The paper itself concedes this in Section 6.6: 'without formal statistical significance across multiple random seeds.' Moreover, the activation threshold tau=0.80 used in the physical trials is chosen in Section 6.7 by an offline sweep whose selected FPR (13.6%) matches the Table 4 evaluation-subset result, implying the threshold was tuned on the same data used to report the ablations; no held-out calibration set is described. A one-trial change from 6/10 to 7/10 is within binomial noise, so the central claim that PassGen-augmented data, rather than threshold tuning or trial noise, drives the improvement is not established. Even if the Section 4.2 depth-noise heuristic were perfect, the reported experiments would not settle the transfer claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces Hand2Bot, a dataset of 5,000 RGB-D human-to-robot handover videos (2,125 real, 2,875 synthesized), and PassGen, a generative pipeline that combines Stable Video Diffusion with an Intention-Aware Temporal Face Encoder and a morphological depth-noise editing strategy. The authors also propose an Intention Gating module that fuses gaze features and object-approach velocity to trigger robot grasps. Experiments include animation-quality benchmarks, ablations on an evaluation subset of Hand2Bot-Real, and physical trials on a UR5e platform with 10 objects. The central claim is that training on PassGen-augmented data enables robust zero-shot transfer and earlier intention anticipation relative to hand-centric baselines.","tokens_in":10718,"tokens_out":5035,"duration_ms":50617,"significance":"If the transfer claim were established, the paper would make a useful contribution: it provides a full-body RGB-D handover dataset with realistic depth noise, a generative pipeline conditioned on explicit social cues, and an evaluation on a physical robot. The manuscript also contains strengths worth noting: a real-world data collection with informed consent, a clear gating formulation, and an unusually candid limitation statement in Section 6.6 conceding the lack of statistical significance. However, the central claim currently rests on a 6/10-to-7/10 unseen-object success difference and on an FPR reduction that appears to be achieved by selecting the activation threshold on the evaluation subset. As presented, the evidence does not substantiate 'robust zero-shot transfer' or the 'earlier intention anticipation' promised in the abstract.","major_comments":[{"comment":"The central claim that PassGen-augmented training yields robust zero-shot transfer is not supported by the reported statistics. In Table 6, the unseen-object ISR increases from 6/10 to 7/10 (N=10), a difference well within binomial sampling noise, and Section 6.6 explicitly concedes the absence of formal statistical significance across random seeds. Moreover, the FPR improvement from 22.8% to 13.6% is compromised by the threshold-selection procedure: Section 6.7 reports an offline sweep over tau on the evaluation subset, and the selected tau=0.80 yields a frame-level FPR of 13.6%, identical to the 'Full Module' value in Tables 4 and 6; no held-out calibration set is described. The headline FPR is therefore a fitted value rather than an independent prediction. To support the transfer claim, the authors need either a held-out threshold calibration protocol or a significance test over multiple seeds/trials, and the reporting in Tables 4 and 6 should distinguish fitted from predicted quantities.","section":"§6.6, Table 6; §6.7, Table 4"},{"comment":"The abstract claims 'earlier intention anticipation' relative to hand-centric baselines, but no temporal metric is reported anywhere in the experiments. Section 6.5 reports only ISR and FTR, and Section 6.7 reports frame-level FPR and interactive deadlocks, but there is no quantitative measurement of anticipation latency, time-to-trigger relative to presentation onset, or early-warning margin. Without such a metric, the 'earlier' component of the central claim is unsupported. The authors should add a timing analysis, e.g., the distribution of trigger times before the end of the presentation period, for the IG-enabled and baseline configurations.","section":"§5.3 and §6.5"},{"comment":"The depth-noise simulation is explicitly labeled as an 'empirical, heuristic morphology-based approximation' rather than a validated L515 sensor-noise model. This matters because Section 7 states that sensor-specific noise simulation is critical for cross-modal policy transfer, and the sim-to-real explanation in Section 4.2 depends on N0 and w(h) faithfully representing the boundary-level voids of the physical sensor. The paper should validate the heuristic quantitatively, e.g., by comparing the spatial and statistical distribution of zero-depth boundary pixels in generated depth against held-out real L515 frames, and should test whether downstream gains persist when the augmented data is evaluated with a different depth sensor or a different scene layout. As written, the depth-editing step is a plausible but unvalidated component of the transfer claim.","section":"§4.2"},{"comment":"The trial counts are internally inconsistent. The text says the authors selected 10 distinct objects (5 seen and 5 unseen) and ran positive trials per object type, which would imply 100 positive trials if 'per object' is literal. Table 5, however, lists six rows (five named seen objects plus one aggregate unseen row) with an overall 54/60, implying 60 positive trials. The table also reports only one aggregate 'Unseen-' row rather than per-object results for five unseen objects. Please clarify the number of objects, the number of trials per object, and report all per-object and per-category results so readers can reproduce the evaluation and assess variance across objects.","section":"§6.5.1 and Table 5"}],"minor_comments":[{"comment":"The text says augmented training 'significantly improves downstream tracking accuracy, reduces FPR, and enhances robustness,' but two sentences later it concedes the result is without formal statistical significance. Please align the wording with the stated limitation, or provide the statistical test that supports 'significantly.'","section":"§6.6"},{"comment":"The row header 'Unseen-7 / 10' appears to be a formatting artifact; the values should be separated clearly, e.g., 'Unseen 7/10 0/5 3/5', to avoid ambiguity.","section":"Table 5"},{"comment":"The caption says 'We mask all facial regions and replace them with the corresponding visualized depth maps,' but it is unclear what is displayed in the figure and why facial regions are masked in grasp-pose annotation; please clarify.","section":"Figure 2"},{"comment":"The animation test set is described as '250 clips from Hand2Bot-Real,' but no details are given on how these clips were split from the 2,125 real sequences, whether the split is person-disjoint or scene-disjoint, or whether the PassGen training used all other real clips. Please provide the split procedure to assess possible information leakage.","section":"§6.2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a potentially useful dataset and a sensible generative pipeline, but the central transfer claim is resting on a one-trial difference in unseen-object ISR and on a threshold that appears to be tuned on the evaluation subset. I recommend major revision rather than rejection because the deficiencies are addressable within the manuscript's scope: add a held-out calibration protocol, significance testing or confidence intervals, a timing metric, and validation of the depth-noise heuristic. If a revised version cannot supply those, the paper should be repositioned as a dataset and generation-quality contribution rather than as evidence for robust zero-shot sim-to-real transfer."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a useful dataset paper with a plausible generation pipeline, but its headline transfer claim is ahead of its evidence. The Hand2Bot dataset—2,125 real full-body RGB-D handover clips plus 2,875 generated ones, with negative intention examples and real L515 noise patterns—is genuinely useful for the handover community. The PassGen pipeline (pose-guided video diffusion plus a Temporal Face Encoder for gaze, and a morphological depth-noise trick) is a reasonable integration of existing parts, and the authors show it beats other animation models on fidelity metrics.\n\nThe soft spots are real but manageable. The central sim-to-real claim rests on Table 6: Real+Syn raises unseen-object ISR from 6/10 to 7/10 and cuts FPR from 22.8% to 13.6%. That's a one-trial difference on N=10 with no significance testing, and the paper itself concedes this in Section 6.6. On top of that, the activation threshold tau=0.80 is set in Section 6.7 by sweeping on the evaluation subset, and the same FPR (13.6%) then appears in the ablation tables—so the headline FPR is partly a fitted value. The physical robot trials (Table 5) do provide some independent evidence: 2/30 false triggers with the gate versus 25/30 without. That part is encouraging, but it doesn't isolate the contribution of the synthesized data.\n\nThe depth-noise model is honestly labeled in Section 4.2 as an empirical heuristic, not a validated sensor model. That's a limitation, not a fraud—the authors are transparent about it. I'd like to see a held-out tau calibration and a bigger trial, but the dataset itself is the main deliverable, and it stands even if the augmentation effect turns out to be modest.\n\nWho should read this: anyone working on human-robot handover or human-centric RGB-D datasets. The dataset fills a real gap, and the negative-example labeling is a thoughtful touch. I'd take it to reading group, and I'd cite it if I worked in the area.\n\nFor peer review: yes, send it out. The evaluation protocol needs work—hold out the threshold, report uncertainty, maybe soften the abstract—but the dataset and pipeline deserve referee time. The core idea is sound; the evidence just needs to catch up to the rhetoric.","headline":"Useful dataset, plausible pipeline, but the transfer claim is a 7/10 vs 6/10 difference with a threshold fit to the test set—worth reviewing, needs stronger evidence.","tokens_in":11236,"tokens_out":3692,"would_cite":true,"duration_ms":33027,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that synthesizing full-body RGB-D handover video with realistic depth noise and gaze conditioning can substitute for large real-world handover datasets when training robot intention prediction, and supports this with…","keywords":["human-robot handover","RGB-D video generation","intention anticipation","video diffusion","sim-to-real transfer","depth noise simulation","gaze conditioning","socially aware robotics"],"falsifier":"Replace the morphological depth-noise step with clean DepthCrafter output while keeping the same PassGen RGB stream and Intention Gating, then run the same negative trials on the physical robot; if the false-trigger rate stays near 2/30, the sensor-noise simulation is not the load-bearing component for sim-to-real transfer. If it rises, that step is carrying the transfer.","tokens_in":10252,"feed_emoji":"🤖","tokens_out":9264,"duration_ms":90929,"temperature":0.7,"pith_summary":"This paper tries to establish that a robot can learn to read human handover intent from generated, not just recorded, RGB-D video. It proposes a pipeline that synthesizes full-body handover clips from pose and facial-gaze cues, then corrupts the depth channel with realistic sensor-style void noise, and shows that training on these augmented clips transfers to a physical robot with far fewer false triggers. The reason to care: if synthetic data can bridge the sim-to-real gap this way, handover-capable robots no longer depend on expensive, large-scale human capture, and they can react to whole-body social signals instead of just hand position.","feed_headline":"Synthetic video cuts robot handover false triggers to 2 in 30","feed_subtitle":"Generated full-body RGB-D handover clips train the robot to spot intent early and ignore ambient motion.","key_machinery":"PassGen combines two conditioning streams on a Stable Video Diffusion backbone: a pose-guidance branch that injects per-frame skeletons, and a Temporal Face Encoder (TFE) that turns facial-embedding sequences into temporally consistent face tokens $\\tilde{F}_t$, which are injected along with pose guidance into the U-Net via cross-attention. For depth, a morphology-based erosion uses boundary-width mapping $w(h)$ and a recorded noise distribution $N_0$ to imprint realistic void patterns on an otherwise smooth DepthCrafter estimate. Downstream, the Intention Gating module fuses gaze confidence with approach velocity: $S_{\\mathrm{intent}}(t)=\\sigma(w_g f_{\\mathrm{gaze}}(\\tilde{F}_t)+w_v v_{\\mathrm{obj}}(t))$, with a monotone max constraint that latches the trigger and a threshold $\\tau=0.80$.","core_discovery":"On the paper's own terms, the central discovery is that full-body social context plus sensor-realistic depth noise, delivered through generated video, is what makes handover-intention prediction transfer from training data to a real robot. PassGen-generated clips (2,875 of the 5,000 Hand2Bot pairs) improve animation quality over prior animation models; adding them to real training raises mean intention accuracy from 87.5% to 90.0%, lowers mean FPR from 22.8% to 13.6%, and raises the unseen-object intention success rate from 6/10 to 7/10; and the full Intention-Gated system on a UR5e triggers 54/60 intended handovers while cutting false triggers from 25/30 to 2/30. The paper frames the synthetic contribution as variance reduction and false-positive suppression, not as broad manipulation superiority.","pith_inferences":["If face-gaze tokens carry most of the intention signal, the same gating could transfer to other social-robotic tasks such as robot-initiated handover, where the robot must detect willingness rather than intention.","The depth-noise recipe is a cheap, sensor-specific transfer tool: record a sensor's void distribution, fit a boundary-width erosion, and reuse it for any generative depth pipeline; a natural test is to apply it to a second depth sensor and measure downstream false-trigger rate.","Because the monotone max trigger latches on the highest score, the system is deliberately biased toward early triggering; in longer autonomous sessions a single early false positive cannot be undone within a trial, so frame-level FPR may understate long-horizon risk.","The seen/unseen gap (ISR 6/10 vs 7/10 over only ten trials) is framed by the paper as variance reduction; a stronger test would run more objects and seeds before treating synthetic augmentation as a general manipulation improvement."],"forward_implications":["Adding PassGen-Syn sequences to real training improves downstream intention accuracy from 87.5% to 90.0% and cuts mean FPR from 22.8% to 13.6% without changing the prediction architecture.","A physical UR5e with Intention Gating initiates 54/60 intended handovers and reduces false triggers from 25/30 to 2/30, evidence that gaze-plus-approach gating suppresses ambient motion.","Synthetic depth that carries realistic L515 void patterns prevents overfitting to clean geometry; replacing it with smooth DepthCrafter output is not sufficient.","Handover intention can be anticipated earlier from full-body social cues than from hand-centric views alone, which is the premise for socially aware shared-workspace behavior.","The combination of timestamped presentation labels and 6-DoF grasp annotations enables proactive control rather than reactive grasping."],"supporting_citations":[{"why":"Supplies the Stable Video Diffusion backbone that PassGen fine-tunes with LoRA.","marker":"[27]"},{"why":"Provides the initial temporally smooth depth sequence that Stage II corrupts with sensor-style noise.","marker":"[14]"},{"why":"Supplies whole-body skeleton estimates used as deterministic pose conditioning.","marker":"[29]"},{"why":"Supplies facial embeddings that the Temporal Face Encoder converts into gaze and focus tokens.","marker":"[30]"},{"why":"Supplies the appearance-encoder structure used to condition generation on reference identity.","marker":"[17]"},{"why":"Noise-free synthetic handover dataset whose sim-to-real gap motivates the depth-noise simulation.","marker":"[6]"},{"why":"Synthetic handover dataset used as a comparison baseline for dataset coverage.","marker":"[7]"},{"why":"Real-world hand-centric handover dataset whose restricted scope motivates the full-body Hand2Bot view.","marker":"[8]"},{"why":"Generates the candidate 6-DoF grasp poses used for Hand2Bot grasp annotations.","marker":"[26]"},{"why":"Defines the presentation period and handover phases used for timestamp labels and evaluation timing.","marker":"[2]"}],"fun_headline_variants":["Synthetic video cuts robot handover false triggers from 25 to 2","Generated RGB-D video raises robot handover intent to 90%","Synthetic full-body video enables zero-shot robot handover","Training on synthetic RGB-D clips cuts false handover triggers 92%","Synthetic depth video gives robots early handover intent"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the recorded L515 noise distribution and the boundary-width erosion $w(h)$ used to corrupt synthetic depth faithfully capture the artifacts that affect downstream prediction; the paper itself frames this as an empirical heuristic rather than a validated sensor model.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic video cuts robot handover false triggers from 25 to 2","Generated RGB-D video raises robot handover intent to 90%","Synthetic full-body video enables zero-shot robot handover","Training on synthetic RGB-D clips cuts false handover triggers 92%","Synthetic depth video gives robots early handover intent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000646,"raw_usage":{"total_tokens":2951,"prompt_tokens":912,"completion_tokens":2039,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":1950}},"tokens_in":528,"tokens_out":2039,"duration_ms":16837,"temperature":1.0,"reasoning_tokens":1950,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:15:10.899823+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the morphological depth-noise step with clean DepthCrafter output while keeping the same PassGen RGB stream and Intention Gating, then run the same negative trials on the physical robot; if the false-trigger rate stays near 2/30, the sensor-noise simulation is not the load-bearing component for sim-to-real transfer. If it rises, that step is carrying the transfer.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the initial temporally smooth depth sequence that Stage II corrupts with sensor-style noise."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies whole-body skeleton estimates used as deterministic pose conditioning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies facial embeddings that the Temporal Face Encoder converts into gaze and focus tokens."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the appearance-encoder structure used to condition generation on reference identity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Noise-free synthetic handover dataset whose sim-to-real gap motivates the depth-noise simulation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Synthetic handover dataset used as a comparison baseline for dataset coverage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Real-world hand-centric handover dataset whose restricted scope motivates the full-body Hand2Bot view."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Generates the candidate 6-DoF grasp poses used for Hand2Bot grasp annotations."},{"cited_title":"Ortenzi, A","cited_arxiv_id":null,"evidence_quote":"Defines the presentation period and handover phases used for timestamp labels and evaluation timing."}],"review_version":1}