{"id":"dc6ad0a9-6cf4-403d-a85f-18741e55b204","arxiv_id":"2603.14686","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using a 3D foundation model to produce viewpoint-aware anchors plus multi-view reference textures enables realistic human-object-interaction reenactment with large out-of-plane rotations.","lead":"MVHOI transfers a person's object-manipulation motion to a different object by first generating geometric anchor frames with a 3D vision model and then refining them with a video diffusion model. The authors report large gains over prior video-editing methods on object fidelity, motion consistency, and visual quality for complex 3D rotations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DisMo motion embeddings' object-agnosticism is asserted but untested; if false, cross-object transfer fails before Stage II.","rationale":"The paper's contribution hinges on transferring source dynamics to a novel target object. This requires the motion representation to be separable from object appearance and category. The manuscript identifies DisMo as the provider of such a descriptor and freezes it, but the claim is an assumption, not a demonstrated property. My reading of the evidence: the Objaverse experiment (Tab. 2) does not control for category similarity between source and target, and the AnchorCrafter evaluation (29 samples, three views) is too small to reveal category dependence. A linear-probe experiment would directly falsify the assumption. I considered the missing ReHold comparison and the Stage II proxy-training mismatch; both matter empirically, but they can be remedied by additional evaluation/tuning. The DisMo assumption is more load-bearing because if it fails, the framework's central mechanism (object-to-object transfer via UOA) breaks before video generation. Thus I agree with the reader's weakest_assumption; the CONDITIONAL verdict is appropriate pending this test and the other requested artifacts.","tokens_in":13476,"tokens_out":7349,"duration_ms":83239,"concrete_test":"Run a controlled identity-vs-motion experiment with DisMo: render N objects from distinct categories under the same rotation/translation trajectory; extract motion embeddings m_t. Train a linear classifier for object identity on m_t. If classification accuracy is well above chance (e.g., >33% on 10 classes), the embedding carries identity and the object-agnostic assumption in Sec. 3.2 is false. As confirmation, re-run Stage I cross-reenactment for same-category vs cross-category source-target pairs; a significant drop in coarse-anchor LPIPS/FID (e.g., >10% relative) would demonstrate the practical impact on the object-swapping claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Contribution 1) requires that the frozen DisMo motion extractor (Sec. 3.2, Eq. 1) provide a genuinely object-agnostic descriptor of dynamics, so that a source video's motion can be transferred to an arbitrary target object. The paper asserts this ('This latent representation serves as a compact descriptor of the underlying physical dynamics') but provides no dedicated evidence. The only object-transfer evaluation (Sec. 4.3, Tab. 2) uses 100 source and 100 target Objaverse objects without stratifying by category similarity, and the cross-reenactment test uses 29 self-selected AnchorCrafter training samples with only three views. If DisMo embeddings encode source-object identity (shape, texture, or category), the UOA's autoregressive anchors will be contaminated before Stage II runs, so object swapping collapses regardless of the strength of the video generator. This is not an external-consensus dispute; it is a hidden correctness condition. The omission of ReHold (the closest HOI reenactment baseline) further prevents assessing the 'transcending 2D translational models' part of the claim, but that is a separate empirical gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MVHOI, a two-stage framework for human-object interaction (HOI) video reenactment. Stage I extracts motion embeddings from a source video using a frozen DisMo motion extractor, then conditions a DepthAnything3-based Unified Object Anchor (UOA) on those embeddings and multi-view references of a target object to autoregressively predict coarse object-anchor frames. Stage II uses a DiT-based video generator (initialized from Wan2.1-T2V-14B, with a VACE-style context adapter) that ingests the coarse anchors and multi-view references, and applies an inference-time attention bias derived from intermediate UOA attention maps to reduce reference-view confusion. A cross-iterative inference scheme alternates between coarse anchor prediction and video refinement for long sequences. The paper claims to be the first HOI reenactment framework to handle complex non-planar dynamics, and reports quantitative improvements over MimicMotion, VACE, HuMo, and HunyuanCustom on self- and cross-reenactment, plus an ablation of the coarse-guidance and attention-enhancement components.","tokens_in":13753,"tokens_out":6321,"duration_ms":68371,"significance":"If the reported results are reliable, the proposed direction is significant: leveraging 3D foundation models as an intermediate anchor for transferring object dynamics is a plausible and timely alternative to explicit 6D pose estimation or 2D keypoint control, and the two-stage design is well-motivated. The paper is largely free of circularity: the performance claims are empirical, the pipeline is described in enough detail to be implemented, and the ablation in Table 4 suggests that both the coarse guidance and the attention bias contribute to the final result. The reported metric gaps over VACE (e.g., self-reenactment FID 17.9 vs 47.4) are large, which gives the central claim initial plausibility. However, the evidence is weakened by the small, self-selected public evaluation set, the proprietary training and test data, the absence of the closest baseline ReHold, and the untested assumption that DisMo motion embeddings are object-agnostic. No code or data is released, which further limits verifiability.","major_comments":[{"comment":"The cross-object transfer mechanism relies on the frozen DisMo motion extractor producing descriptors that are object-agnostic. The sentence “This latent representation serves as a compact descriptor of the underlying physical dynamics” is asserted but never probed. If m_t encodes source-object shape, texture, or category, the UOA’s predicted anchors are contaminated before Stage II begins, regardless of the strength of the video generator. I request a dedicated experiment: e.g., train a linear probe to predict source-object identity from m_t; stratify cross-reenactment results by source–target category similarity; or compare UOA outputs when the same target object is driven by motions from different source objects. Without this, the central “object-to-object” claim rests on an untested hidden condition.","section":"Sec. 3.2, Eq. (1)"},{"comment":"The main quantitative claim is supported by a weak evaluation protocol. Cross-reenactment is evaluated on 29 self-selected samples from the AnchorCrafter training split, and the self-collected 100-video set has no stated split or selection criterion. No error bars, confidence intervals, or significance tests are reported for any metric in Tables 1–4. The headline FID/FVD gaps (e.g., 17.9 vs 47.4 in self-reenactment) could be affected by sample selection or noise. Add bootstrap confidence intervals or per-sample distributions, use a held-out test set, and specify the selection protocol for both the 29 AnchorCrafter samples and the 100 self-collected videos.","section":"Sec. 4.2, Table 1"},{"comment":"ReHold [7] is identified in Related Work as the closest HOI reenactment baseline, but it is never included in the experiments. Without this comparison, the claim to be “the first HOI video reenactment framework capable of synthesizing complex non-planar dynamics” and to “transcend the limitations of 2D translational models” is not empirically substantiated relative to the most relevant prior work. Please add ReHold (and, if feasible, DreamActor-H1 [30]) to Tables 1 and 3, or explicitly justify their exclusion.","section":"Sec. 2.2 / Sec. 4.3"},{"comment":"Cross-iterative long-video inference is a stated contribution, but the experiments only compare the full model against external baselines. There is no ablation comparing the full model with and without the cross-iterative loop, so the specific claim that the strategy “effectively mitigates drift” is not directly tested. Add an ablation where long sequences are generated without re-initializing from refined video segments (e.g., naive autoregressive rollout from previous coarse anchors) to isolate the benefit of the proposed mechanism.","section":"Sec. 3.5, Table 3"}],"minor_comments":[{"comment":"The attention-bias strength α is not reported, nor is the designated UOA layer from which the attention map w is extracted. These details are needed for reproducibility. Similarly, the HOI loss weight β in Eq. (6) is not specified.","section":"Sec. 3.3, Eq. (3)"},{"comment":"The initialization O_0 for the autoregressive rollout in cross-reenactment is not specified. It should be clarified how the first target-object frame is obtained from the multi-view references and aligned with the source video’s initial hand state.","section":"Sec. 3.2"},{"comment":"The Stage I comparison against DisMo conflates the UOA architecture with the addition of multi-view reference conditioning, since DisMo does not receive multi-view references. An ablation with the same reference inputs but without the UOA mechanism would better isolate the contribution of the proposed module.","section":"Sec. 4.3, Table 2"},{"comment":"The metrics O-CLIP, MC, VQ, and HR are not defined in the main text; the paper refers to an Appendix that is not included in the submitted version. Please define them or include the appendix.","section":"Sec. 4.2"},{"comment":"There are numerous typos and formatting artifacts, including “shwon”, “structed”, “Ojbect”, “adpots”, “desin”, “conherence”, “freezen”, and “achieves”. References [17] and [18] are duplicates of the same Depth Anything 3 paper. The final “Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009” line appears to be a template artifact and should be removed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong central idea and the reported gains are directionally convincing, but the evaluation section needs substantial strengthening before the claims can be accepted. In particular, the 29-sample training-split evaluation and the absence of ReHold are issues that an editor should weigh seriously. The proprietary 100-hour training set and 100-video test set also make independent verification difficult; I would encourage the authors to release at least the evaluation protocol and feature statistics. The untested object-agnosticism of the DisMo motion extractor is a correctness risk that should be addressed with a dedicated experiment rather than a rhetorical assertion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MVHOI is a legitimate integration: it takes DisMo's motion embeddings, runs them through a DepthAnything3-based autoregressive anchor predictor with multi-view references, and uses that anchor plus an attention bias to steer a Wan2.1 DiT. The attention-bias retrieval idea is genuinely new and the ablation shows it helps (FID 46.55->41.14). The two-stage design is coherent and the training/inference split is sensible. That's real work.\n\nWhat's strong: The system targets a real gap — 2D control signals fail on non-planar rotations — and the proposed UOA plus attention-bias pipeline is a plausible fix. Self-reenactment numbers are much better than the 2D-conditioned baselines (FID 17.9 vs 47.4), and the long-video strategy of re-initializing from refined frames is a reasonable anti-drift measure. Credit where due: the novelty is in the integration, not the components, and the ablation supports both main modules.\n\nThe soft spots are mostly about evidence, not design. The 29-sample AnchorCrafter evaluation is drawn from the training set and self-selected; no error bars or significance tests anywhere. The proprietary 100-hour training set and 100-video test set mean the main claims can't be reproduced. ReHold — the closest HOI reenactment prior, and listed in the references — is not compared, which makes the 'transcending 2D translational models' claim hard to verify. And the stress-test concern is real: the paper asserts DisMo's motion embeddings are a compact descriptor of physical dynamics without checking whether they're object-agnostic. If they encode source identity, cross-object transfer fails before Stage II, and the object-transfer evaluation (Table 2) doesn't stratify by category similarity. That's an untested load-bearing assumption. Minor: α and β are introduced but never given values.\n\nThe math itself is clean — no circularity, no fitting disguised as prediction. The design choices are well-motivated.\n\nWho is this for? People building controllable video generation or HOI reenactment systems. It deserves a serious referee, but the authors should be pushed to release artifacts, compare against ReHold, report error bars, and probe the motion-descriptor transfer assumption.","headline":"A well-engineered two-stage HOI reenactment system with real gains, but the evaluation is too thin and the object-agnostic motion assumption untested to fully back the headline claim.","tokens_in":14289,"tokens_out":2020,"would_cite":true,"duration_ms":22885,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MVHOI claims to be the first HOI video reenactment framework that transfers complex 3D manipulation dynamics — flips, spins, out-of-plane reorientations — onto novel objects by anchoring motion and appearance in a single 3D latent space.","keywords":["human-object interaction","video reenactment","3D foundation model","multi-view reference","diffusion transformer","object swap","view synthesis","controllable video generation"],"falsifier":"Take a single source interaction (e.g., a hand flipping a phone 360 degrees) and run it with source objects of very different visual and physical properties (a rigid mug, a soft plush toy, a transparent glass). If the extracted motion embeddings differ measurably across source objects for identical hand kinematics, or if cross-reenactment quality on a novel rigid target drops sharply for one of these source videos in a user study, the object-agnostic motion assumption is undermined. A second probe: replace the UOA's coarse anchors with a simple per-frame bounding-box trajectory under the same","tokens_in":13366,"feed_emoji":"🎥","tokens_out":5178,"duration_ms":54025,"temperature":0.7,"pith_summary":"This paper argues that the fundamental limitation of prior HOI reenactment methods is their reliance on sparse 2D controls (bounding boxes, keypoints) and monocular references, which cannot represent the view-dependent geometry of an object during rotation. The proposed two-stage framework, MVHOI, claims to solve this by embedding a 3D foundation model as a Unified Object Anchor: the anchor consumes frozen motion embeddings from the source video and multi-view references of the target object, and autoregressively predicts coarse, view-consistent object frames. These frames then guide a diffusion-transformer video generator, which also uses an attention bias derived from the anchor's internal cross-view attention to retrieve the correct reference texture per viewpoint. If the claims hold, HOI reenactment would no longer be restricted to image-plane translations and could handle rapid rotations, severe hand-object occlusion, and long videos without cumulative drift.","feed_headline":"3D anchor transfers object flips and spins in video reenactment","feed_subtitle":"MVHOI uses a 3D foundation model as a unified anchor so generators retrieve the right view, cutting drift and view confusion.","key_machinery":"The Unified Object Anchor (UOA): a transformer backbone (initialized from a large 3D foundation model) that takes the current target-object frame plus multi-view reference images, modulates the frame tokens with scale-and-shift from a frozen motion embedding, and predicts the next object frame through a dedicated RGB head. Its role is to provide a deterministic view-querying pathway in latent space, turning stochastic synthesis into a structured anchor-following task. The same anchor's cross-view attention map is reused as a soft bias in the video generator's context adapter, directly linking geometric reasoning to appearance retrieval.","core_discovery":"The central claim is that a feed-forward 3D foundation model, used as a Unified Object Anchor (UOA), can consolidate sparse multi-view references and implicit motion latents into a view-invariant latent repository, and that querying this anchor over time yields a sequence of coarse object states that are geometrically consistent with the source interaction. This replaces explicit pose estimation and 2D proxies. The paper further claims that the UOA's internal attention map, extracted at inference time, encodes a view-selection signal that can be added as a logit-level bias to a DiT generator's self-attention, thereby steering it to attend to the correct reference view. Together, these two me","pith_inferences":["The authors freeze the motion extractor and never test whether its descriptors are truly object-agnostic; if the same motion on a rigid mug and a plush toy produces different embeddings, the cross-category transfer claim weakens. A simple invariance test would settle this.","Since the UOA is trained on rendered 3D objects, its anchor quality may degrade on transparent, reflective, or deformable objects; if so, the framework's claim of 'complex non-planar dynamics' may hold only for rigid, diffuse objects in the training distribution.","The attention-bias injection could be applied to other editing tasks — e.g., viewpoint control in object-centric video generation, or maintaining identity across regenerations — as a drop-in prior."],"forward_implications":["Out-of-plane object rotations and flips become tractable control signals, so reenactment can follow a hand twirling a bottle or flipping a phone rather than only sliding it on a table.","Object swapping across categories (e.g., replacing a mug with a stapler) can preserve motion plausibility because the anchor supplies view-consistent geometry while the generator fetches texture from the correct reference view.","Long videos avoid drift because the cross-iterative loop re-initializes each segment from the previous refined output, not from blurred intermediate anchors.","The recipe — a 3D foundation model as a latent anchor plus an attention-bias steering mechanism — offers a general pattern for injecting 3D awareness into existing video diffusion models without retraining them fully."],"fun_headline_variants":["MVHOI: 3D anchor unifies views for complex interaction video reenactment","3D foundation model anchors multi-view reenactment of complex object interactions","Unified object anchor from 3D model cuts view confusion in reenactment","MVHOI uses 3D anchor to keep object views consistent during reenactment","View-invariant anchor from 3D model steers complex interaction video"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole transfer pipeline rests on the premise that the frozen motion extractor's latent embeddings describe the physical dynamics of the interaction and are independent of the source object's identity, category, or appearance.","fun_headline_variants_meta":{"raw":{"variants":["MVHOI: 3D anchor unifies views for complex interaction video reenactment","3D foundation model anchors multi-view reenactment of complex object interactions","Unified object anchor from 3D model cuts view confusion in reenactment","MVHOI uses 3D anchor to keep object views consistent during reenactment","View-invariant anchor from 3D model steers complex interaction video"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000511,"raw_usage":{"total_tokens":2343,"prompt_tokens":782,"completion_tokens":1561,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":1452}},"tokens_in":526,"tokens_out":1561,"duration_ms":10752,"temperature":1.0,"reasoning_tokens":1452,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T05:46:36.450464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a single source interaction (e.g., a hand flipping a phone 360 degrees) and run it with source objects of very different visual and physical properties (a rigid mug, a soft plush toy, a transparent glass). If the extracted motion embeddings differ measurably across source objects for identical hand kinematics, or if cross-reenactment quality on a novel rigid target drops sharply for one of these source videos in a user study, the object-agnostic motion assumption is undermined. A second probe: replace the UOA's coarse anchors with a simple per-frame bounding-box trajectory under the same","supporting_citations":[],"review_version":1}