{"id":"7aa7e3c5-4a7a-404f-ad94-6fca5279377a","arxiv_id":"2506.02444","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"SViMo jointly generates HOI videos and explicit 3D hand-object motion via synchronized diffusion with a closed-loop 3D interaction diffusion model.","lead":"SViMo generates hand-object interaction videos and matching 3D motion in one synchronized diffusion process, using only a reference image and text prompt. It avoids needing predefined 3D object models at generation time, which could make animation and robotic imitation easier.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative support for the central claim is internally inconsistent: the identical 'Ours' configuration reports order-of-magnitude different MPJPE, Chamfer, and FID in Table 2 vs Table 3, so the claimed SOTA superiority is not currently reproducible.","rationale":"The reader's verdict of CONDITIONAL is appropriate, but I do not think the most load-bearing weakness is the 2D projection assumption. The projection is indeed underspecified and could lose depth information, but the VID branch also receives the noised 3D motion as input, so the architecture is not forced to recover 3D purely from 2D. In contrast, the Table 2/Table 3 discrepancy is a direct, quantitative contradiction in the paper's own results. The abstract promises superiority over state-of-the-art approaches in generating dynamically plausible HOI sequences; that claim is supported mainly by Table 2. If the same method's errors change by one to two orders of magnitude between two tables when no protocol difference is described, the evidence base is not stable enough to support the claim. This is an internal inconsistency, not a disagreement with the field's consensus. It could be resolved by a careful re-evaluation, so the verdict should remain conditional rather than reject. I mark agreement as partial because the reader's rationale explicitly identified this discrepancy even though their stated weakest assumption concerns the 2D projection; my attack focuses on the numerical inconsistency as the single most decisive check on the central quantitative claim.","tokens_in":18406,"tokens_out":10060,"duration_ms":104476,"concrete_test":"Obtain the released model (or ask the authors for weights) and run the exact 'SViMo w/ VID' configuration from Table 3 on the same test split used for Table 2, using the published evaluation scripts. Recompute MPJPE, motion smoothness, Chamfer distance, and FID with a fixed seed and report per-sequence mean and standard deviation. If the recomputed numbers agree with one table but not the other (difference >10%), the conflicting table is not reproducible and the quantitative superiority claim fails; if they reconcile the two tables, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is an unexplained internal inconsistency in the quantitative evidence. The 'Ours' row of Table 2 and the 'SViMo w/ VID (Ours)' row of Table 3 describe the same full configuration, trained and evaluated on the same TACO 1:9 split with the same metrics, yet report MPJPE 0.1087 vs 0.0121, motion smoothness 0.0255 vs 0.0053, Chamfer distance 0.1577 vs 0.0019, and FID 0.1050 vs 0.0100. The Chamfer and FID values differ by roughly 80x and 10x. No normalization, subset, or protocol difference is stated in the paper. Because Table 2 is the only head-to-head motion comparison against MDM and EMDM, the claimed superiority over state-of-the-art motion generators rests on numbers that cannot both be correct under the description given. The corresponding VBench video scores also shift slightly between Table 1 and Table 3, but the motion-metric shifts are too large to be noise or seed variation. Unless the discrepancy is resolved by releasing the evaluation protocol and code, the core quantitative claim is unverifiable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SViMo, a synchronized diffusion framework that jointly generates hand-object interaction (HOI) videos and explicit 3D motion sequences from a reference image and text prompt. The method extends a pretrained video DiT by introducing tri-modal adaptive modulation for text, video, and motion tokens, a 3D full-attention mechanism, and a 'rendered motion video' representation that projects 3D hand joints and object point clouds onto 2D. A vision-aware 3D Interaction Diffusion model (VID) recovers explicit 3D interactions from the synchronized latent codes and feeds them back into the denoising loop, forming a closed-loop optimization. The framework is trained and evaluated on the TACO dataset on a 1:9 split, reporting video quality via VBench metrics and motion quality via MPJPE, motion smoothness, Chamfer distance, and a motion FID. The authors claim improved video-motion consistency and zero-shot generalization to unseen real-world scenarios.","tokens_in":18661,"tokens_out":5124,"duration_ms":49662,"significance":"If the quantitative evidence were internally consistent, this would be a noteworthy contribution: an end-to-end generative model that produces a video and a matched explicit 3D interaction sequence without predefined object models or pose guidance, leveraging large-scale video priors. The idea of a synchronized video-motion diffusion process with closed-loop vision-aware feedback is original and plausible, and the paper provides useful implementation details, including training/inference pseudocode (Algorithms 1 and 2) and a user study with a substantial number of responses. The qualitative results in Figures 4 and 6 are suggestive. However, the current quantitative support has a serious internal inconsistency between Table 2 and Table 3, and several evaluation details are insufficient to verify the claimed state-of-the-art performance. The paper is not ready for acceptance in its present form.","major_comments":[{"comment":"The identical full configuration, listed as 'Ours' in Table 2 and as 'SViMo w/ VID (Ours)' in Table 3, reports vastly different values on the same TACO 1:9 split: MPJPE 0.1087 versus 0.0121, motion smoothness 0.0255 versus 0.0053, Chamfer distance 0.1577 versus 0.0019, and FID 0.1050 versus 0.0100. These are differences of roughly 2x, 5x, 80x, and 10x, respectively, with no explanation of any normalization, unit, subset, or protocol change. Because Table 2 is the only head-to-head motion comparison against MDM and EMDM, the claimed superiority over state-of-the-art motion generators cannot be verified as reported. The authors must clarify the exact evaluation protocol for each table, state the units and normalization, and if necessary rerun the comparisons so that both tables describe the same configuration consistently.","section":"Tables 2 and 3"},{"comment":"The 'rendered motion video' is the core motion representation through which 3D hand joints and object point clouds are fed into the video model, and it is also the space from which VID recovers 3D structure. However, the paper never specifies the projection function: it does not state whether the projection is orthographic or perspective, which camera parameters are used, how hand joints and object point clouds are rasterized, or whether any depth information is retained. Since the whole closed-loop feedback (Eq. 7) depends on the information content of this 2D rendering, the missing specification makes the method not fully reproducible and leaves the central assumption that 3D cues survive the projection uncheckable. Please provide the exact projection procedure and, ideally, a quantitative or qualitative analysis of what 3D information is preserved.","section":"Section 3.3 and Eq. (7)"},{"comment":"The motion FID is computed using a feature extractor that the authors trained on the TACO dataset itself, and the proposed method is fine-tuned on TACO and evaluated on a 1:9 split of the same dataset. This same-distribution evaluation can inflate FID scores, especially since the extractor may have seen the same action and object distributions. The paper should report whether the FID extractor was trained only on the training split, whether any test sequences were used, and provide quantitative cross-dataset evaluation to support the claimed zero-shot generalization; currently, generalization to unseen real-world data is evidenced only by qualitative examples in Figure 6. Without such clarification, the generalization claim is not quantitatively substantiated.","section":"Appendix B.3 and Section 4.4"},{"comment":"No error bars, number of test samples, or random seeds are reported for any of the quantitative results. This is particularly problematic in the ablation study (Table 3), where many differences are tiny (e.g., Subject Consistency 0.9534 versus 0.9522 versus 0.9499; Overall 0.8800 versus 0.8770 versus 0.8723). Without variance estimates or significance testing, it is impossible to tell whether these differences are meaningful or within noise. Please report standard deviations over multiple seeds or at least the number of evaluation videos and the test-split size.","section":"Tables 1-3 and Section 4.5"},{"comment":"The motion generation comparison is limited to MDM and EMDM, which are general human motion generation models, not methods specifically designed for hand-object interaction such as Text2HOI, InterDiff, or CG-HOI. While the paper states that the baselines were modified to align the setup, the absence of HOI-specific state-of-the-art baselines weakens the claim of 'superiority over state-of-the-art approaches' for HOI motion generation. The authors should either include HOI-specific baselines or justify why MDM and EMDM are the appropriate comparisons.","section":"Tables 2 and 4.3"}],"minor_comments":[{"comment":"Two different project page URLs are given: 'https://github.com/Droliven/SViMo_project' in the abstract and 'https://droliven.github.io/SViMo_project/' in Section 1. Please unify them.","section":"Abstract and Section 1"},{"comment":"There is a typo: 'esulting' should be 'resulting' in the sentence about the patchified visual feature.","section":"Section 3.3, Feature Embedding"},{"comment":"The caption says 'Varients', which should be 'Variants'. Also, in Section 4.5 the text mentions a variant 'VModel w/ GT Mot. Guid.' but the table row is labeled 'VModel w/ Pred. Mot.'; please align the naming.","section":"Table 3 caption and Section 4.5"},{"comment":"The 'Training' column with entries 'w/o' and 'w/' is not defined in the caption. It appears to indicate whether each baseline was fine-tuned on TACO or used zero-shot, but this should be stated explicitly.","section":"Table 1"},{"comment":"The limitations section acknowledges that the generated object point clouds are restricted to rigid, simple objects and that the method relies on paired video-motion data. These are important caveats that affect the central claims and should be summarized in the main text rather than only in the appendix.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the proposed architecture is interesting. The main concern is the large unexplained discrepancy between the numbers for the same configuration in Table 2 and Table 3, which currently makes the primary quantitative claim unverifiable. This is not necessarily a reason to reject outright, because the issue might be resolvable by clarifying the evaluation protocol, correcting units, or rerunning experiments. I would encourage the editor to ask for a revised version with a consistent and detailed evaluation, including error bars and a clearer description of the projection and FID setup."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is worth your time: SViMo couples video diffusion with explicit 3D motion generation in a single denoising pass, using rendered motion videos as the bridge and a closed-loop interaction diffusion model to keep the two modalities consistent. That is a real architectural novelty, and the ablations support the claim that synchronization helps. The qualitative zero-shot results are also a point in its favor—no object models or pose guidance is a meaningful capability.\n\nBut the paper has a load-bearing problem in its numbers. Table 2 and Table 3 both report the same “Ours” configuration, same TACO 1:9 split, same metrics, yet MPJPE is 0.1087 in one and 0.0121 in the other, Chamfer 0.1577 versus 0.0019, FID 0.1050 versus 0.0100. That is not noise; it is either a reporting error or a protocol difference that is never explained. Since Table 2 is the only head-to-head comparison against MDM and EMDM, the claimed superiority over motion baselines currently rests on numbers that cannot both be correct. No error bars anywhere makes this worse. The video baselines are also uneven—Hunyuan and Wan are evaluated zero-shot, while CogVideoX is fine-tuned on TACO—so the video comparison is generous to the method.\n\nOne more soft spot: the FID metric uses a feature extractor the authors trained themselves on TACO. That is a step toward circularity and should at least be justified or supplemented with a standard feature extractor. The zero-shot generalization claim is only qualitative. These are not fatal to the approach, but they matter.\n\nThe central design is plausible. Projecting 3D joint and point-cloud trajectories to 2D is an odd but defensible representation choice, and the closed-loop feedback is a reasonable way to enforce consistency. The paper is honest about failure cases and limitations, and the writing is clear. This is serious work, not a sloppy toy.\n\nBottom line: the idea deserves a serious referee, but the current manuscript should not be accepted until the Table 2/Table 3 discrepancy is resolved and the evaluation protocol (and ideally code) is released. I would not cite it in its present form because the headline numbers are not trustworthy. But I would bring it to a reading group to discuss the design and the inconsistency—it is a good lesson in why internal consistency checks matter.","headline":"A genuinely new synchronized video-motion diffusion framework for HOI, but the main quantitative evidence is internally inconsistent across tables, so the SOTA claim is not yet verifiable.","tokens_in":686,"tokens_out":1034,"would_cite":false,"duration_ms":25433,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that synchronized video-motion diffusion with a closed-loop vision-aware 3D interaction model generates high-fidelity hand-object videos and explicit 3D motions together, without predefined object models or pose guidance.","keywords":["hand-object interaction","video generation","3D motion generation","diffusion models","synchronized diffusion","closed-loop feedback","multimodal generation","latent video diffusion"],"falsifier":"Run the method on annotated contact frames of the test split and measure hand-object penetration depth and contact coverage between predicted hand joints and predicted object point clouds; if these are not better than the baselines, the physical-plausibility benefit of the closed loop is not supported.","tokens_in":18180,"feed_emoji":"🤲","tokens_out":7236,"duration_ms":61611,"temperature":0.7,"pith_summary":"The paper sets out to show that video appearance and 3D motion in hand-object interaction should be generated as one synchronized diffusion event, not as separate pipelines. It proposes SViMo, which jointly denoises video latents and motion latents from a reference image and text prompt, with the motion represented as a rendered 2D motion video so a pretrained video model can ingest it. A companion vision-aware 3D interaction diffusion model (VID) reads the denoised output, predicts explicit hand trajectories and object point clouds, and feeds them back into the video denoiser in a closed loop. According to the paper, this removes the need for predefined 3D object models or pose-sequence guidance and yields higher-fidelity, physically plausible HOI sequences that generalize to unseen real-world objects. A reader should care because it is a direct test of whether visual priors and physical constraints can reinforce each other in one diffusion model rather than being traded off.","feed_headline":"One diffusion pass yields hand-object video and 3D motion together","feed_subtitle":"Video and 3D motion denoise together in one closed loop, beating baselines without object meshes or pose guidance.","key_machinery":"The load-bearing mechanism is the closed loop between two diffusion components. SViMo is the synchronized video-motion diffusion model: a pretrained video diffusion Transformer extended so its token sequence contains text, video, and motion tokens, aligned by triple-modality adaptive modulation and fused by 3D full-attention; the motion token stream is a rendered motion video obtained by projecting 3D hand joints and object point clouds onto a 2D image plane and encoding it with the same video VAE. VID is a vision-aware 3D interaction diffusion model that takes the denoised video and motion latents, fuses them in a dual-stream 3D convolutional module, and generates explicit hand poses and object point clouds, supervised by joint mean squared error and Chamfer distance. The closed loop works in two directions: VID's output is projected and encoded to become interaction guidance concatenated with the noised motion latent, and gradients from VID's loss backpropagate into SViMo, so the two models co-evolve during training.","core_discovery":"SViMo's central discovery is that the long-standing trade-off between visually realistic HOI video and physically plausible 3D motion can be dissolved by letting both modalities diffuse together in one Transformer. The paper's architecture feeds text, image-video, and rendered-motion tokens into a single 3D full-attention stack with tri-modal adaptive modulation, then closes the loop with VID, which converts the denoised video and motion latents into explicit 3D hand joint trajectories and object point clouds and re-injects them as interaction guidance plus gradient constraints. The paper reports that this synchronized design outperforms state-of-the-art video and motion baselines on video and 3D metrics, and that ablations show removing VID or splitting video and motion generation degrades both outputs. Its claim is that visual appearance and motion patterns share the same physical laws and can be modeled by a shared diffusion process without needing object models or explicit pose inputs.","pith_inferences":["Beyond the paper, the rendered-motion-video trick suggests a general recipe for grafting any explicit 3D representation onto a video diffusion foundation model: project the 3D signal to 2D, let the video model ingest it, then lift it back with a small 3D head, so the idea may transfer to full-body interactions, articulated objects, or scenes.","The closed-loop feedback is not limited to hand-object pairs; the same design of letting one modality diffuse while a predictor recovers an explicit structure that is re-encoded as guidance could synchronize video with audio, tactile, or robot-action streams where paired supervision is scarce.","A testable extension the paper does not run is to quantify physical plausibility directly with contact and penetration metrics between predicted hand joints and object point clouds, rather than only joint error and Chamfer distance, which would sharpen the claim of dynamic plausibility.","Because the paper's appendix lists rigid simple objects as a limitation, a natural stress test is to feed objects with complex or deformable geometry and see whether the recovered object point clouds remain coherent."],"forward_implications":["Holding a reference image and text prompt, SViMo produces both a video and an explicit 3D hand-and-object motion sequence from the same denoising run, with no 3D object mesh and no input pose sequence required.","The closed-loop feedback keeps video pixels and 3D motion mutually consistent: object point clouds and hand trajectories are regenerated from the video latents, and those 3D outputs refine the video denoising.","On the paper's reported evaluations, the joint scheme outperforms sole video generators on overall video score and outperforms motion diffusion baselines by a wide margin on joint error, Chamfer distance, and interaction FID.","Ablations imply that synchronized training is necessary: splitting video and motion generation, or removing VID, drops both the video overall score and the motion FID.","The same model transfers to unseen household objects in zero-shot real-world tests without retraining on those objects."],"supporting_citations":[{"why":"Supplies the paired video and 3D hand-object motion data the model is trained and evaluated on.","marker":"[34]"},{"why":"Provides the pretrained video diffusion Transformer backbone that SViMo extends into a joint video-motion denoiser.","marker":"[65]"},{"why":"Provides the diffusion Transformer block formulation underlying the joint denoising architecture.","marker":"[40]"},{"why":"Provides the video evaluation metrics used for subject consistency, background consistency, smoothness, and dynamic degree.","marker":"[19]"},{"why":"Serves as a motion diffusion baseline whose metrics SViMo compares against for 3D motion generation.","marker":"[53]"},{"why":"Serves as an improved motion diffusion baseline whose metrics SViMo compares against for 3D motion generation.","marker":"[70]"}],"fun_headline_variants":["Synchronized diffusion outputs video and 3D motion in one pass","One diffusion model generates HOI video and 3D motion together","Video and 3D motion denoised jointly without object models","Closed-loop diffusion syncs HOI video and 3D motion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that projecting 3D hand joints and object points onto a 2D rendered motion video preserves enough spatial, contact, and depth information for a video model to encode and for the 3D model to recover, and if that projection loses crucial 3D cues the video-motion consistency and generalization claims collapse.","fun_headline_variants_meta":{"raw":{"variants":["Synchronized diffusion outputs video and 3D motion in one pass","One diffusion model generates HOI video and 3D motion together","Video and 3D motion denoised jointly without object models","Closed-loop diffusion syncs HOI video and 3D motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000615,"raw_usage":{"total_tokens":2870,"prompt_tokens":968,"completion_tokens":1902,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":1826}},"tokens_in":584,"tokens_out":1902,"duration_ms":12090,"temperature":1.0,"reasoning_tokens":1826,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:23:29.273149+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the method on annotated contact frames of the test split and measure hand-object penetration depth and contact coverage between predicted hand joints and predicted object point clouds; if these are not better than the baselines, the physical-plausibility benefit of the closed loop is not supported.","supporting_citations":[{"cited_title":"Cogvideox: Text-to-video diffusion models with an expert transformer","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained video diffusion Transformer backbone that SViMo extends into a joint video-motion denoiser."},{"cited_title":"Emdm: Efficient motion diffusion model for fast and high-quality motion generation","cited_arxiv_id":null,"evidence_quote":"Serves as an improved motion diffusion baseline whose metrics SViMo compares against for 3D motion generation."}],"review_version":1}