REVIEW 3 major objections 5 minor 16 references
Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Motion transfer can bypass fixed structural correspondence by bootstrapping cross-category pairs from abstract motion views, then internalizing transferable dynamics directly from the reference video.
desk verdict A genuinely novel two-stage bootstrapping idea for cross-category motion transfer, but the central evidence rests on self-generated pairs and an unquantified filter, so the SOTA claims are provisional until code, data, and a controllability check appear. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-stage bootstrapping loop built on multi-granularity abstract motion views. Stage I converts raw videos into five complementary representations (semantic kinematics, depth-aware global trajectories, dense point tracks, structural edges, and 6-DoF axes) and conditions a single generator with whichever view is reliably extractable for a video. That generator synthesizes target videos for conditions drawn from other categories, and the resulting triplets are filtered by four criteria into the bootstrapped dataset used in Stage II. Stage II initializes from Stage I and trains on those pairs with the raw reference video as the motion condition, replacing the explicit view. The abstract views are thus not the final control signal; they are the scaffold that makes cross-category motion equivalence learnable, and the bootstrapped pairs are the supervision that forces the model to internalize transferable dynamics rather than exploiting source-specific shortcuts.
What would settle it
Take a random sample of bootstrapped Far pairs, have motion-transfer-experienced annotators judge whether each reference-target pair shares the same motion pattern, and compare their labels with the automated four-criteria filter; if a substantial share of filtered pairs are judged not motion-equivalent, the supervision that Stage II relies on is unreliable and the claimed internalization does not follow.
Extended reading notes
Core claim
The central claim is that motion transfer can and should be learned as an adaptive coupling problem: the model keeps the motion factors that remain meaningful under the target's morphology, such as global trajectory, local deformation, and rhythm, and drops source-specific shape, articulation, and background. The authors implement this by treating abstract motion views not as the inference representation but as a supervision mechanism. Stage I grounds five complementary views (semantic keypoint skeletons, depth-aware global trajectories, dense point tracks, structural edges, and 6-DoF axes) in one video-like conditioning interface and uses them to bootstrap cross-category video pairs whose reference and target share only the intended dynamics. After filtering the generated pairs on motion fidelity, target fidelity, reference leakage, and video quality, Stage II replaces the abstract motion condition with the raw reference video and optimizes a flow-matching objective over the bootstrapped pairs, so that appearance and morphology become unreliable predictors while the shared temporal dynamics remain informative. The result is a model that, given a reference video and a target text prompt or reference image, generates the target video directly, and the paper reports that it outperforms prior methods on the Same, Near, and Far splits of the new OpenVMT-Bench.
Load-bearing premise
The whole approach rests on the bootstrapped cross-category pairs being genuine motion-equivalent pairs, even though there is no ground-truth motion equivalence to check them against.
Editorial extensions
If this is right
- At inference the model animates a target from a raw reference video plus a text prompt or reference image, with no skeleton extraction, trajectory specification, or per-video optimization.
- Because the bootstrapped pairs make appearance uninformative, the trained model suppresses source leakage: the generated video keeps the target's identity rather than inheriting the reference's shape or background.
- Motion fidelity holds as the category gap grows from Same to Far, so transfers between objects with no part-level correspondence remain faithful to the reference's coarse trajectory and local deformation.
- The released 10K motion-equivalent pairs give the research community direct supervision for training cross-category motion transfer, complementing unpaired Internet video.
- The Same, Near, and Far benchmark structure lets future methods be compared on how they handle structural gaps, not just on within-category animation quality.
Reading between the lines
- Editorial inference: because Stage I remains a separate controllable motion-conditioned generator, it could be kept as an explicit authoring channel (editing a trajectory, edge, or skeleton) alongside the direct reference-video mode, giving users both automatic transfer and fine-grained motion control in one framework.
- Editorial inference: the bootstrapping logic suggests a way to scale supervision, since any unpaired video whose extracted abstract view is shared with another video could be treated as motion-equivalent at the level of that view, yielding far more than 10K pairs if filtering becomes cheaper and more automatic.
- Editorial inference: the paper's adaptive-coupling argument implies the view-selection rule, currently fixed per video, could itself be learned per source-target pair, and such a model might push Far-gap fidelity further than the fixed sampling used here.
- Editorial inference: the human preference study compares the model against one baseline per track, so the reported advantage over the broader set of open-source and commercial systems remains to be confirmed by direct pairwise evaluation against each of them.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a two-stage framework for open-category video motion transfer. Stage I trains a diffusion model to condition video generation on multiple abstract motion views (semantic kinematics, depth-aware trajectories, dense point tracks, 6-DoF axis, and edges) via Eq. (2), then uses the trained model to synthesize cross-category pairs (Vr, ct, tilde Vt) via Eq. (4), filters them by four qualitative criteria, and forms a bootstrapped pair dataset Dboot. Stage II initializes from Stage I and trains on Dboot to generate the bootstrapped target conditioned directly on the raw reference video and target condition (Eq. (6)), so inference requires no explicit motion extraction. The paper introduces OpenVMT-Dataset (10K pairs) and OpenVMT-Bench (Same/Near/Far splits for I2V and T2V), compares with DisMo, Wan-Move, Tora, DeT, FlowMotion, and commercial systems, and reports state-of-the-art results on the proposed benchmark, ablations, and a human GSB study.
Significance. The idea of bootstrapping cross-category supervision from abstract motion views and then distilling it into direct video-conditioned generation is novel and, if validated, would remove a major limitation of correspondence-based motion transfer. The multi-granularity view design and the stagewise training scheme are technically coherent, and the paper makes a serious effort at evaluation, including human preference studies and a benchmark with category-gap splits. The main strengths are the unified conditioning interface and the explicit attempt to create instance-level motion-equivalent cross-content pairs. However, the significance is conditional: the benchmark and training pairs are generated by the same model, the filtering step is not quantitatively validated, and no external or independent benchmark is used, so the state-of-the-art claim currently rests on a self-consistent evaluation loop.
major comments (3)
- [Method: Stage I and Cross-category pair filtering] The load-bearing assumption that Dboot contains motion-equivalent pairs is not established. In Eq. (2), the Stage I model is optimized to reconstruct Vi from the motion view z_m and the first-frame condition ci of the same video. Because ci already specifies appearance and coarse layout, and the base image-to-video model has a strong video prior, the loss can be largely minimized without z_m being a dominant control signal; nothing in LS1 forces z_m to remain reliable when ci is replaced by a different-category condition in Eq. (4). The four filtering criteria (motion fidelity, target fidelity, reference leakage, video quality) are described only in qualitative terms, with no acceptance rates, no precision/recall against human labels, and no demonstration that they are sensitive to failures of motion equivalence. Since Stage II trains on Dboot via Eq. (6), any artifacts or appearance leakage that survive filtering are inherited by the final model. Please add (i) a Stage I controllability experiment that varies the motion view while holding ci fixed and measures the resulting motion differences; (ii) filter statistics and a human validation of motion equivalence on randomly sampled accepted and rejected triplets; and (iii) an ablation that trains Stage II on unfiltered versus filtered pairs.
- [Experiments: Dataset and benchmark; Table 1] The central state-of-the-art claim is weakened by the circularity of the evaluation. OpenVMT-Bench is curated from the same bootstrapped data distribution on which the model is trained, so the proposed method has a distributional advantage that no baseline shares; the reported ranking may reflect this self-consistency rather than open-category transfer. Relatedly, Table 1 reports no confidence intervals, standard errors, or significance tests, and the Gemini-based G-Mot and G-Leak metrics are presented without prompt details or validation against human judgments. Please report variance across test instances and statistical tests against the best baseline, validate G-Mot/G-Leak on a labeled subset, and add at least one external evaluation (e.g., hand-curated real cross-category pairs or an existing motion-transfer benchmark) to break the training/evaluation loop.
- [Stage II: Cross-Category Motion Internalization, Eq. (6)] The claim that Stage II internalizes transferable dynamics exceeds what Eq. (6) can guarantee. The training target tilde Vt is itself generated by Stage I via Eq. (4), so Stage II can only learn to imitate Stage I's behavior under cross-category conditions; it cannot recover transferable dynamics that Stage I failed to encode. The statement that cross-category pairs discourage reliance on morphology-specific shortcuts is plausible but not directly tested. To support the inference-time claim, please report (i) a reference-video swap experiment (fix target condition, vary reference videos, measure motion variation in outputs), (ii) a target-condition swap experiment, and (iii) a control training where Stage II is trained on pairs with no intended motion equivalence; if the control still achieves high HMF/G-Mot, the metrics are not sensitive to the proposed supervision.
minor comments (5)
- [Metrics] The exact prompts, model version, and scoring protocol for the Gemini-based G-Mot and G-Leak scores are not given; without these details the scores are not reproducible.
- [Cross-category pair filtering] The four filtering criteria are described in one sentence each; please specify the exact algorithms, thresholds, and models used so that the construction of Dboot can be reproduced.
- [Dataset and benchmark] The paper does not state whether OpenVMT-Bench test cases overlap with the Stage II training pairs; please report the overlap explicitly and, if needed, filter the benchmark to avoid train/test contamination.
- [Table 3] The ablation results are reported without error bars or multiple seeds; single-run differences such as 0.7707 versus 0.7588 are hard to assess without variance information.
- [Human GSB study] The preference study uses only 12 evaluators; please report the evaluation instructions and inter-evaluator agreement, since preference rates above 90 percent are surprising with such a small panel.
Circularity Check
Stage II is trained on Stage-I-generated 'motion-equivalent' pairs whose equivalence is certified only by unquantified filters; final external metrics keep the claim partially independent.
-
self definitional
[Method: Eq. (4) in 'Cross-category motion-pair generation', 'Cross-category pair filtering', and Eq. (6) in Stage II]
"The Stage-I model then generates ˜Vt =G θ1(M k r , ct).(4) ... sharing the motion attributes conveyed by M k r. ... Only candidates satisfying all criteria are retained, forming the filtered bootstrapped datasetDboot used in Stage II. ... For each(Vr, ct, ˜Vt)∼ Dboot,weoptimize LS2 =E[ k˜uτ −u θ2(˜zτ , τ, zr, ct)k 2 2 ],(6)."
The supervision for Stage II is produced by Stage I itself: Eq. (6) trains G_theta2 to reproduce tilde Vt, and Eq. (4) defines tilde Vt as G_theta1(M^k_r, c_t). Thus any 'transferable dynamics' internalized by Stage II are bounded by Stage I's ability to realize the abstract motion view M when the target category changes. Stage I is trained with Eq. (2) using both z_m and the first frame c_i of the same video, so the objective can be minimized without z_m being the true controller in the cross-category regime. The four filtering criteria are described only qualitatively, with no precision/recall or human-agreement numbers, so they cannot certify that accepted pairs share genuine dynamics rather than Stage-I artifacts.
full rationale
The main circularity is the bootstrapping loop: the 'motion-equivalent' training pairs used to learn direct reference-video-conditioned transfer are synthesized by the same Stage-I model that defines what motion is kept, and the filtering does not independently verify motion equivalence. However, the final OpenVMT-Bench evaluation compares the generated video to the source reference video with external metrics (HMF, Gemini-based scores, DINO-I/CLIP-T, VBench smoothness, human GSB), rather than merely checking reconstruction of Stage-I targets, so the central claim retains independent content beyond the self-generated training distribution. The paper explicitly states that OpenVMT-Dataset is 'derived from our bootstrapped data' and co-introduces OpenVMT-Bench, but it does not explicitly say the benchmark test pairs come from the same bootstrapped pipeline, so I do not count that as a separate circular step. No load-bearing self-citation chain or imported uniqueness theorem was found; the Kling-related references are background or baseline citations, not the argument's foundation. Overall the derivation is not fully circular, but the self-bootstrapped supervision makes a key part of the claimed open-category transfer capacity partially self-definitional.
Assumptions & free parameters
free parameters (3)
- View applicability rules =
not quantified
- Filtering criteria thresholds =
not quantified
- Training hyperparameters =
15K Stage I, 4K Stage II iterations, batch 128
assumptions (3)
- domain assumption The bootstrapped cross-category pairs preserve transferable dynamics and are free of artifacts that would mislead Stage II.
- domain assumption Abstract motion views (semantic kinematics, trajectories, tracks, 6-DoF axes, edges) each capture complementary and reliable aspects of motion for their applicable categories.
- domain assumption The pretrained extraction models (ViTPose++, TAPNext++, SpaTrackerV2) provide sufficiently accurate motion signals in open-category videos.
invented entities (2)
-
OpenVMT-Dataset
-
OpenVMT-Bench
Cite this review
Pith. "Pith review of Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations." pith.science (2026). https://pith.science/paper/O54ICJ5Y
@misc{pith2026260801628,
author = {Pith},
title = {Pith review of: Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations},
year = {2026},
howpublished = {\url{https://pith.science/paper/O54ICJ5Y}},
note = {Machine review of arXiv:2608.01628}
}
read the original abstract
Video motion transfer aims to animate a target object using dynamics from a reference video. Existing formulations largely rely on fixed structural correspondence, which becomes ill-defined when reference and target objects differ substantially in morphology, articulation, or deformation mechanisms. We introduce Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence, by preserving dynamics that remain meaningful across different target morphologies. To realize this, we propose a two-stage framework. Stage~I learns complementary multi-granularity abstract motion views and uses them to bootstrap cross-category video pairs that preserve transferable dynamics across diverse morphologies. Stage~II internalizes this supervision into direct reference-video-conditioned generation, removing the need for explicit motion extraction at inference. We further introduce OpenVMT-Dataset and OpenVMT-Bench for training and evaluating image- and text-conditioned motion transfer across Same, Near, and Far category gaps. Extensive experiments demonstrate state-of-the-art motion fidelity and target preservation. Project page: https://miniz233.github.io/MotionBeyondMorphology/
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Wan-Move:Motion-controllableVideoGeneration via Latent Trajectory Guidance
Chu,R.;He,Y.;Chen,Z.;Zhang,S.;Xu,X.;xia,b.;WANG, D.; Yi, H.; Liu, X.; Zhao, H.; Liu, Y.; Zhang, Y.; and Yang, Y.2025. Wan-Move:Motion-controllableVideoGeneration via Latent Trajectory Guidance. In Belgrave, D.; Zhang, C.; Lin, H.; Pascanu, R.; Koniusz, P.; Ghassemi, M.; and Chen, N., eds.,Advances in Neural Information Processing Systems, volume 38, 404–4...
2025
-
[5]
Karaev, N.; Rocco, I.; Graham, B.; Neverova, N.; Vedaldi, A.; and Rupprecht, C
TAPNext++: What’s Next for Tracking Any Point (TAP)?arXiv preprint arXiv:2604.10582. Karaev, N.; Rocco, I.; Graham, B.; Neverova, N.; Vedaldi, A.; and Rupprecht, C
-
[6]
https://kling-ai.prt
Kling AI. https://kling-ai.prt. kuaishou.com/app. Accessed: 2026-07-29. Ling,P.;Bu,J.;Zhang,P.;Dong,X.;Zang,Y.;Wu,T.;Chen, H.; Wang, J.; and Jin, Y
2026
-
[7]
In International Conference on Learning Representations
MotionClone: Training- Free Motion Cloning for Controllable Video Generation. In International Conference on Learning Representations. Ma,Y.;He,Y.;Cun,X.;Wang,X.;Chen,S.;Shan,Y.;Li,X.; andChen,Q.2024. FollowYourPose:Pose-GuidedText-to- Video Generation using Pose-Free Videos. InProceedings of the AAAI Conference on Artificial Intelligence. Oquab, M.; Darc...
2024
-
[8]
Pondaven,A.;Siarohin,A.;Tulyakov,S.;Torr,P.;andPizzati, F.2025.VideoMotionTransferwithDiffusionTransformers
Dinov2: Learning robust visual fea- tures without supervision.Transactions on Machine Learn- ing Research Journal. Pondaven,A.;Siarohin,A.;Tulyakov,S.;Torr,P.;andPizzati, F.2025.VideoMotionTransferwithDiffusionTransformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Ressler-Antal,T.;Fundel,F.;Alaya,M.B.;Baumann,S....
2025
-
[9]
Shi, Q.; Wu, J.; Bai, J.; Zhang, J.; Qi, L.; Li, X.; and Tong, Y
Seedance 2.0: Advancing video generation for world com- plexity.arXiv preprint arXiv:2604.14148. Shi, Q.; Wu, J.; Bai, J.; Zhang, J.; Qi, L.; Li, X.; and Tong, Y
-
[10]
InACM SIGGRAPH 2024 Conference Papers, 1–11
Motion-i2v:Consistentandcontrollableimage-to-videogen- eration with explicit motion modeling. InACM SIGGRAPH 2024 Conference Papers, 1–11. Team, K.; Chen, J.; Ci, Y.; Du, X.; Feng, Z.; Gai, K.; Guo, S.;Han,F.;He,J.;He,K.;etal.2025. Kling-OmniTechnical Report.arXiv preprint arXiv:2512.16776. Team, K.; Chen, J.; Ding, Y.; Fang, Z.; Gai, K.; He, K.; He, X.;H...
arXiv 2024
-
[12]
InACMSIGGRAPH 2024 Conference Papers, 1–11
Motionctrl: A unified and flexible motioncontrollerforvideogeneration. InACMSIGGRAPH 2024 Conference Papers, 1–11. Wu, J.; Li, X.; Zeng, Y.; Zhang, J.; Zhou, Q.; Li, Y.; Tong, Y.; and Chen, K. 2024a. Motionbooth: Motion-aware cus- tomized text-to-video generation.Advances in Neural Infor- mation Processing Systems, 37: 34322–34348. Wu,W.;Li,Z.;Gu,Y.;Zhao,...
2024
Show all 16 references
-
[13]
InICCV 2025 Workshop on Wild 3D: 3D Modeling, Reconstruction, and Generation in the Wild
Spa- tialtrackerv2: 3d point tracking made easy. InICCV 2025 Workshop on Wild 3D: 3D Modeling, Reconstruction, and Generation in the Wild. Xu, Y.; Zhang, J.; Zhang, Q.; and Tao, D
2025
-
[14]
Xu,Z.;Zhang,J.;Liew,J.H.;Yan,H.;Liu,J.-W.;Zhang,C.; Feng,J.;andShou,M.Z.2024
Vitpose++: Vision transformer for generic body pose estimation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2): 1212–1230. Xu,Z.;Zhang,J.;Liew,J.H.;Yan,H.;Liu,J.-W.;Zhang,C.; Feng,J.;andShou,M.Z.2024. MagicAnimate:Temporally ConsistentHumanImageAnimationu...
2024
-
[15]
Yin, S.; Wu, C.; Liang, J.; Shi, J.; Li, H.; Ming, G.; and Duan,N.2023
MotionShop: Zero-Shot Motion Transfer in Video Diffusion Models with Mixture of Score Guidance.arXiv preprint arXiv:2412.05355. Yin, S.; Wu, C.; Liang, J.; Shi, J.; Li, H.; Ming, G.; and Duan,N.2023. DragNUWA:Fine-GrainedControlinVideo GenerationbyIntegratingText,Image,andTraj...
2023 arXiv
-
[16]
InProceedings of the Computer Vision and Pattern Recognition Conference, 2063–2073
Tora: Trajectory-oriented diffu- sion transformer for video generation. InProceedings of the Computer Vision and Pattern Recognition Conference, 2063–2073. Zhou, H.; Wang, C.; Nie, R.; Liu, J.; Yu, D.; Yu, Q.; and Wang,C.2025. Trackgo:Aflexibleandefficientmethodfor controllabl...
2025
-
[2023]
InAdvances in Neural Information Processing Systems
VideoComposer: CompositionalVideoSynthesiswithMotionControllability. InAdvances in Neural Information Processing Systems. Wang,Z.;Xu,Y.;Xiao,J.;andChen,L.2026. FlowMotion: Training-Free Flow Guidance for Video Motion Transfer. arXiv preprint arXiv:2603.06289. Wang,Z.;Yuan,Z.;W...
2026
-
[2024]
InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition
AnimateAnyone:ConsistentandControllableImage- to-VideoSynthesisforCharacterAnimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition. Huang,Z.;He,Y.;Yu,J.;Zhang,F.;Si,C.;Jiang,Y.;Zhang, Y.;Wu,T.;Jin,Q.;Chanpaisit,N.;Wang,Y.;Chen,X.;Wang, ...
-
[2025]
In ProceedingsoftheComputerVisionandPatternRecognition Conference, 7191–7200
Conmo: Controllable motion disentan- glement and recomposition for zero-shot motion transfer. In ProceedingsoftheComputerVisionandPatternRecognition Conference, 7191–7200. Geng, D.; Herrmann, C.; Hur, J.; Cole, F.; Zhang, S.; Pfaff, T.; Lopez-Guevara, T.; Aytar, Y.; Rubinstein...
2025
-
[2026]
arXiv:2602.03796
3D-Aware Implicit Mo- tion Control for View-Adaptive Human Video Generation. arXiv:2602.03796. Gao, J.; Yin, Z.; Hua, C.; Peng, Y.; Liang, K.; Ma, Z.; Guo, J.; and Liu, Y
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.