Pith. sign in

REVIEW 5 major objections 5 minor 89 references

FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FinePhys claims that incorporating Euler-Lagrange physics into skeletal guidance markedly improves fine-grained human action video generation, producing more physically plausible gymnastics videos than pose-only or text-only baselines.

desk verdict A capable engineering contribution to fine-grained human video generation, but the 'explicit physics' framing is mostly window dressing: PhysNet is a learned second-order smoother, not an enforced Euler-Lagrange system. read the letter →

arxiv 2505.13437 v1 pith:ZRDVROC6 submitted 2025-05-19 cs.CV cs.AI

Dian Shao , Mingfei Shi , Shengda Xu , Haodong Chen , Yongle Huang , Binglu Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords fine-grainedhumanactiongenerationphysics-informeddeeplearningEuler-Lagrangeequationsskeletalguidancediffusionmodelsvideo2D-to-3DposeliftingFineGym
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that fine-grained human action video generation—such as gymnastics leaps, turns, and saltos—fails when skeletal guidance is purely data-driven, because the estimated poses can be noisy and physically implausible. The authors propose FinePhys, which first lifts online-detected 2D poses to 3D via in-context learning, then re-estimates the 3D motion with a PhysNet module that instantiates the Euler-Lagrange equations of rigid-body dynamics. The physics-refined and data-driven skeletons are fused, projected back to 2D, and encoded as multi-scale heatmaps that guide a latent diffusion model. On three FineGym subsets the method reports better user-study ratings and higher CLIP-SIM* scores than competitive baselines. If the claim holds, physical equations can serve as a practical inductive bias for video generation, not just a post-hoc filter.

What carries the argument

The load-bearing object is the PhysNet module plus the Euler-Lagrange equation of motion $M(q)\ddot{q} = J(q,\dot{q}) - C(q,\dot{q})$. PhysNet's encoders predict the force vector, the constraint vector, a symmetric inverse inertia matrix (estimated via a triangular matrix, symmetrized, and perturbed with Gaussian noise), and then compute accelerations $\ddot{q} = (\hat{M}^{-1} + \hat{N})(\hat{J} - \hat{C})$. A second-order central difference formula turns those accelerations into updated joint positions, applied both forward and backward in time, and the two directional estimates are averaged. The work this machinery does is to convert data-driven 3D poses into physics-constrained poses that are then fused with the original estimates to form the skeletal heatmap guidance for the diffusion model.

What would settle it

Take the PhysNet module and replace the Euler-Lagrange estimator with a same-capacity unconstrained network that receives the same $q_t$ inputs and produces the same $q_{t+1}$ updates under identical losses, then rerun the Human3.6M and FineGym pose evaluations and the generation metrics; if the unconstrained variant matches or beats the physics-structured one, the Euler-Lagrange form is not the cause of the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that explicitly encoding Lagrangian mechanics inside the pose-estimation branch is what makes fine-grained action generation work: a PhysNet module estimates the generalized forces $J$, constraint terms $C$, a symmetric inverse inertia matrix $M^{-1}$ (with a learned noise perturbation), and then computes joint accelerations $\ddot{q} = (\hat{M}^{-1} + \hat{N})(\hat{J} - \hat{C})$, integrating bidirectionally with a central-difference ODE solver to produce physics-revised 3D skeletons. These are averaged with data-driven 3D skeletons, projected to 2D, and used as multi-scale heatmap conditioning for the denoising UNet. The paper argues that this physical re-estimation not only corrects distorted or missing online poses but also yields more natural movement dynamics than skeleton conditioning alone, and it reports consistent gains over baselines on FX-JUMP, FX-TURN, and FX-SALTO.

Load-bearing premise

The Euler-Lagrange module only enforces physics if the learned neural networks actually estimate meaningful forces, constraints, and an inertia matrix; if they are just flexible curve-fitters trained against pose losses, then the 'physics' is a reparameterization and the method's gains may come from added model capacity.

Editorial extensions

If this is right

  • Physically re-estimated skeletons reduce the impact of noisy online 2D pose detection, restoring missing or distorted joints for downstream generation.
  • Fusing data-driven and physics-predicted 3D poses, rather than using either alone, gives lower 2D and 3D pose error on both Human3.6M and FineGym evaluations.
  • Replacing PhysNet with a plain MLP degrades pose estimation, indicating that the Euler-Lagrange structure, not merely extra capacity, carries the reported benefit.
  • FinePhys outperforms pose-guided, depth-guided, and text-only baselines on user study and improved CLIP-SIM* metrics across FX-JUMP, FX-TURN, and FX-SALTO.
  • High-speed salto classes that combine large body rotation with rapid deformation remain intractable for the method, as the paper reports.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Euler-Lagrange parameterization is doing real work, then the symmetry constraint on the inverse inertia and the learned noise perturbation could transfer to other physics-conditioned generators, such as motion prediction or human mesh recovery, as a lightweight regularizer.
  • A sharper test of the physical claim would check whether the learned $\ddot{q}$ values approximate independent finite-difference accelerations from motion-capture ground truth; the paper's pose-error ablations do not directly measure physical fidelity.
  • The improved CLIP-SIM* metric, which compares generated videos against multiple reference videos from the same action class, could generalize to other domains where temporal dynamics rather than frame smoothness define quality.
  • The framework's dependence on an online pose detector that sometimes outputs nothing suggests that detection robustness, not physics, is the next bottleneck; adding optical flow as an auxiliary cue could compensate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes FinePhys, a framework for fine-grained human action video generation that incorporates physics-based skeletal guidance. The pipeline first detects 2D poses online, lifts them to 3D via an in-context learning module, then refines the 3D poses with a PhysNet module that instantiates Euler-Lagrange equations through learned estimators of generalized forces, joint constraints, inverse inertia, and a stochastic noise term. The data-driven and physics-predicted 3D skeletons are fused, projected to 2D, and converted into multi-scale heatmaps that guide a latent diffusion model (Stable Diffusion with AnimateDiff motion module and LoRA). Experiments are reported on three FineGym subsets (FX-JUMP, FX-TURN, FX-SALTO), with quantitative comparisons to several baselines using user studies, CLIP-SIM*, PickScore, and FVD. The paper also introduces CLIP-SIM*, a modified version of CLIP-SIM intended to better capture fine-grained action dynamics.

Significance. If validated, the idea of combining explicit physical laws with skeletal guidance for diffusion-based video generation would be a meaningful step toward controlling fine-grained human actions. The paper is honest about its limitations, explicitly stating in the Limitations section and supplementary material that salto actions remain intractable and that initial pose detection can fail completely; this transparency is a strength. The qualitative visualizations are thorough and suggest the method produces anatomically more plausible results than several baselines. However, the central claim that gains stem from explicit physics is not yet supported by the evidence: the PhysNet estimators are unconstrained learned functions, the ablation against a simple MLP does not match model capacity, and the quantitative evaluation rests on a self-defined metric (CLIP-SIM*) that may favor the proposed method. The omission of FX-SALTO results from the main table contradicts the abstract's claim of evaluation on three subsets.

major comments (5)
  1. [Section 3.3, Eqs. (9)-(17)] The claim that PhysNet enforces Euler-Lagrange/Newtonian dynamics is not established. The estimators in Eqs. (10)-(16) depend only on q_t (position) and are trained solely with pose reconstruction losses (Eqs. 19-20); no physical units, positivity constraints, velocity dependence, or gravity terms are imposed. Consequently, Eq. (16) defines an unconstrained learned acceleration map, and Eq. (17) is a second-order smoothing update. The reported gains could therefore reflect added model capacity or temporal smoothing rather than physical laws. Please provide a matched-capacity learned baseline without the EL structure and include physical plausibility metrics (e.g., joint-angle limits, contact consistency, ground-reaction forces) to support the attribution to physics.
  2. [Table 1 and Section 4.1] The abstract and Section 4.1 state that FinePhys is evaluated on three FineGym subsets, but Table 1 only reports results for FX-JUMP and FX-TURN. The Limitations section and supplementary Fig. 14 admit that salto actions are intractable and are not calculated in Table 1. The manuscript should either report FX-SALTO results (even if unsatisfactory) or explicitly revise the abstract and claims to reflect that only two subsets are evaluated. Additionally, no train/test split is described; please specify how videos are partitioned and how many videos per class are used for evaluation.
  3. [Section 4.2, Eqs. (25)-(26) and Fig. 4] The proposed CLIP-SIM* metric samples reference videos and frames from FineGym for domain and temporal consistency. Since FinePhys is fine-tuned on FineGym, these references are in-distribution for the proposed method, which may bias the metric in its favor. The paper should demonstrate that CLIP-SIM* is not biased (e.g., by reporting human correlation on a subset where baselines are known to be strong) and should provide confidence intervals or statistical significance tests for the reported gains. The user study also reports only mean MOS scores without variance or significance testing.
  4. [Table 2] The ablation 'Sdd+SMLP' replaces PhysNet with a 'simple MLP' of unspecified architecture. It does not match PhysNet's design (global and local heads, bidirectional forward/inverse updating, symmetric inverse inertia, noise sampling) or its parameter count, so this ablation cannot isolate the effect of the Euler-Lagrange inductive bias. Please provide a controlled comparison where the only difference is the EL structure, and report parameter counts. Furthermore, no code or data are released, which prevents external verification of the physical plausibility of the generated poses; please consider releasing code and data (or a data agreement for FineGym).
  5. [Section 4.3 and Table 2 (2D evaluation)] The 2D evaluation on FineGym uses S2D_detect as the reference, yet the paper itself notes in Fig. 6 and Section 4.3 that online 2D pose detection is noisy and often fails for gymnastics. Evaluating against a noisy reference can mislead, as the proposed pipeline is designed to correct these very errors. Please also report errors against manually annotated keypoints on a small subset, or treat S2D_detect as pseudo-ground-truth with a clear disclaimer about its limitations.
minor comments (5)
  1. [Section 3.2] The notation Vout = F(Noise,D,S2D) uses D without a definition; please clarify what D denotes (likely the data-driven 3D skeletons or the input video).
  2. [Table 2] The table header uses the metric 'MPVPE' while the text defines 'MPJVE' (mean per-joint velocity error); please unify the acronym.
  3. [Acknowledgments] The sentence 'This work was founded by' should read 'funded by'.
  4. [Section 4.2, Eq. (26)] The notation V^Ref is used inconsistently between the equation and the surrounding text; please align the definition of the reference set.
  5. [Figure 1 caption] The caption states 'FinePhys demonstrates superior performance' but does not refer to the method description; consider referencing the proposed framework as described in Section 3.3 for clarity.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: PhysNet's pose 'prediction' is validated on the same FineGym and Human3.6M poses used to train it, so the pose-accuracy evidence for the physics module is a fitted objective rather than an independent physical prediction.

  1. fitted input called prediction [Section 4.1 (Experimental Setup), Section 4.3 (Tab. 2), Supplementary A.4]
    "we construct three subsets from FineGym [62]: FX-JUMP, FX-TURN, and FX-SALTO, derived from the Floor Exercise event in FineGym. These subsets possess different motion characteristics, and are used for tuning the FinePhys framework as well as for validation. ... we first fine-tune the PhysNet and 2D projection modules for 10,000 training steps using online-detected 2D skeletons from FineGym."

    Tab. 2 evaluates S_pp and S_dd+S_pp on exactly the FineGym 2D skeletons used to fine-tune PhysNet and the projection module, and on Human3.6M poses used to pretrain them. Since Eqs. 19-20 train the physical parameter estimators E_J, E_C, E_M, E_N by minimizing ||hatS3D - S3D||^2 and ||P(hatS3D)-S2D||^2, the reported pose-accuracy improvement of S_pp over S_dd is a training objective echoed back as validation, not evidence that Euler-Lagrange dynamics caused the gain. The EL parameters are unconstrained learned functions of q_t, so the 'physically predicted' poses reduce, by the paper's own losses, to a fitted refinement network. This makes the specific PhysNet validation circular, though the central generation comparison in Tab.

full rationale

The main generation comparison is not circular: FinePhys is compared against baselines that are also fine-tuned on FineGym, using user studies, CLIP-SIM*, and FVD, so the headline outperformance is externally anchored. However, the paper's causal claim that physics (Euler-Lagrange re-estimation) is responsible for the gains leans on Tab. 2, where PhysNet's outputs are evaluated on the same pose data used to train it. The paper explicitly states the FineGym subsets are used both for tuning and validation, and the supplementary confirms PhysNet is fine-tuned on online-detected FineGym skeletons before Tab. 2 reports its accuracy on FineGym. This is the fitted-input-called-prediction pattern: the 'physically predicted' poses are optimized against the same targets on which they are then said to validate the physics module. The SMLP ablation does not isolate the EL inductive bias because it is described only as a 'simple MLP' without matched capacity or bidirectional integration, so the alternative explanation of added model capacity or temporal smoothing is not excluded. Thus the paper has partial, not total, circularity; the physics attribution is not established by the pose-validation evidence, but the generation results remain an independent empirical claim.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central physics claim rests on learned estimators for all terms in the Euler-Lagrange equation, a fixed but unreported dt, a hand-set Gaussian noise variance, and a set of domain assumptions about skeleton-based rigid-body modeling, mean-pose transfer, and online pose detection that are only partially validated.

free parameters (3)
  • Learned PhysNet parameters (E_M, E_J, E_C, E_N) = network weights trained by L3D and L2D
    The inverse inertia, generalized forces, constraints, and noise in Eq. 16 are outputs of neural networks fitted to pose data; they are not measured or independently constrained.
  • dt in central difference update = not reported
    Eq. 17 uses dt to convert accelerations into displacements for q_{t+1}; its value changes the effective step size of the physics update but is not given.
  • Gaussian noise variance sigma^2 = 1
    Eq. 14 fixes variance to 1 by hand for the noise that perturbs the inverse inertia matrix; no justification or sensitivity analysis is provided.
assumptions (5)
  • domain assumption The 17-joint, 51-coordinate human skeleton obeys the multibody form of Euler-Lagrange dynamics M(q)qddot = J - C.
    Section 3.1 Eq. 3 and Section C.1; standard for articulated robots but an approximation for a gymnast with soft tissue, contacts, and high-speed aerial motion, especially without a contact model.
  • ad hoc to paper A symmetric inverse inertia matrix plus learned Gaussian noise captures the physical inertia of the body.
    Eqs. 12-15; the symmetry assumption is motivated by inertia tensors, but the learned noise has no physical interpretation and is optimized by L_noise.
  • standard math The second-order central difference approximation qddot approximately (q_{t+1}-2q_t+q_{t-1})/dt^2 is valid for the sampled frame rates.
    Eq. 17; accuracy depends on frame rate and noise, but no frame-rate validation is reported.
  • domain assumption Mean 3D poses from Human3.6M and AMASS provide a useful pseudo-3D prior for gymnastics after in-context lifting.
    Eq. 4; daily activities and MoCap data differ substantially from gymnastics, and no quantitative validation of this transfer is given beyond FineGym 2D re-projection.
  • domain assumption The online 2D pose detector gives usable skeletons for the pipeline.
    Section 3.2 and Supplementary C.4; the authors themselves document complete detector failures that remove all motion, so this premise is load-bearing and occasionally false.
invented entities (1)
  • Asymmetry noise vector N_t
    purpose: Perturb the symmetric inverse inertia estimate to model non-symmetric body configurations in Eqs. 14-15
    N_t is sampled from a learned Gaussian with variance 1 and penalized by L_noise; it is an ad hoc stochastic term introduced to make the learned M^{-1} more flexible, with no measurement or physical quantity behind it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance." pith.science (2026). https://pith.science/paper/ZRDVROC6

@misc{pith2026250513437,
  author       = {Pith},
  title        = {Pith review of: FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZRDVROC6}},
  note         = {Machine review of arXiv:2505.13437}
}
read the original abstract

Despite significant advances in video generation, synthesizing physically plausible human actions remains a persistent challenge, particularly in modeling fine-grained semantics and complex temporal dynamics. For instance, generating gymnastics routines such as "switch leap with 0.5 turn" poses substantial difficulties for current methods, often yielding unsatisfactory results. To bridge this gap, we propose FinePhys, a Fine-grained human action generation framework that incorporates Physics to obtain effective skeletal guidance. Specifically, FinePhys first estimates 2D poses in an online manner and then performs 2D-to-3D dimension lifting via in-context learning. To mitigate the instability and limited interpretability of purely data-driven 3D poses, we further introduce a physics-based motion re-estimation module governed by Euler-Lagrange equations, calculating joint accelerations via bidirectional temporal updating. The physically predicted 3D poses are then fused with data-driven ones, offering multi-scale 2D heatmap guidance for the diffusion process. Evaluated on three fine-grained action subsets from FineGym (FX-JUMP, FX-TURN, and FX-SALTO), FinePhys significantly outperforms competitive baselines. Comprehensive qualitative results further demonstrate FinePhys's ability to generate more natural and plausible fine-grained human actions.

Figures

Figures reproduced from arXiv: 2505.13437 by the authors.

Figure 1
Figure 1. Video generation results for fine-grained human action [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Finephys. FinePhys addresses the challenging task of generating fine-grained human action videos by explicitly incorporating physical equations exploiting pose modality. The pipeline begins with online extracting 2D poses, then transforms them into 3D using an in-context learning module, achieving the data-driven 3D skeleton sequence S 3D dd . To incorporate the physical laws of motion, we introduce a Ph… view at source ↗
Figure 3
Figure 3. The PhysNet Module. Given the input S 3D dd , Phys￾Net leverages both global and local temporal dynamics in a bi￾directional manner to estimate the terms of the Euler-Lagrange equations. By integrating with an ODE solver, the module can predict future and past states, thereby enhancing the original S 3D dd across both temporal directions and producing physically pre￾dicted 3D sequences, denoted as S 3D pp . S 3D, su… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Original CLIP-SIM metrics fail to evaluate the gener￾ated results (e.g., T2I-Zero produces entirely irrelevant outputs yet achieves the highest smooth score according to the original CLIP￾SIM. In contrast, our enhanced CLIP-SIM* provides a more re￾liable evaluation tha…
Figure 5
Figure 5. Figure 5: Qualitative Results. Compared to other baselines, Fine￾Phys demonstrates superior performance in understanding com￾plex, fine-grained semantics, maintaining biomechanical consis￾tency, and adhering to physical principles. observe that these strong baselines struggle to…
Figure 6
Figure 6. Figure 6: FinePhys effectively restores distorted and missing poses [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Generation results from FinePhys and Follow-Your [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Example videos from FX-JUMP, FX-TURN and FX-SALTO. Each sample video has 16 frames, and the corresponding 2D skeleton sequence is also represented. enhances the model’s comprehension of textual prompts, fa￾cilitating subsequent video generation tasks. With the dataset …
Figure 9
Figure 9. Figure 9: Limitations of semantic consistency in original CLIP￾SIM. We utilize CLIP models to obtain the embedded textual features and Probably Approximately Correct (PAC) for dimen￾sionality reduction. The distribution of embedded category labels from FX-JUMP, FX-TURN and FX-SA…
Figure 10
Figure 10. Figure 10: Domain image of original CLIP-SIM and the improved CLIP-SIM* from FX-JUMP, FX-TURN and FX￾SALTO. Reference images generated by Stable Diffusion may not accurately reflect the nuances of specific actions or their dynam￾ics (Original CLIP-SIM), while CLIP-SIM* randomly …
Figure 11
Figure 11. Figure 11: Visualization of different pose sequences on the class “switch leap with 0.5 turn” from the FX-Jump subset, demonstrating the complete transformation process within our framework. assesses temporal consistency. FVD utilizes a pre-trained video recognition model to ext…
Figure 12
Figure 12. Figure 12: Visualization of different pose sequences on the class “2 turn with free leg held upward in 180 split position throughout turn” from the FX-Turn subset, demonstrating the complete transformation process within our framework. ation and high complexity of fine-grained a…
Figure 13
Figure 13. Figure 13: Visualization of different pose sequences on the class “salto backward stretched with 2 twist” from the FX-Salto subset, demonstrating the complete transformation process within our framework. Failure Cases On Class 31: double salto backward stretched Reference Video …
Figure 14
Figure 14. Figure 14: Limitations in intractable cases. For class 31: double salto backward stretched, FinePhys fails to generate a double salto, resulting in only a single flip being observed. pipeline involves online 2D pose estimation. Due to the complexity of fine-grained human actions…
Figure 15
Figure 15. Figure 15: Negative Impact of Initial Pose Detection.. Current online pose estimators may fail completely due to the complexity of fine-grained human actions, which affects subsequent process￾ing stages in the FinePhys framework. Even when the physical structure of the human bod…
Figure 16
Figure 16. Figure 16: Display of the interface of User Study. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]
Figure 17
Figure 17. Figure 17: Qualitative Results on FX-JUMP. FX-JUMP focuses on the motion continuity of the gymnastics’ body. Compared with other baselines, our method demonstrates superior performance in understanding physical consistency. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_17.png]
Figure 18
Figure 18. Figure 18: Qualitative Results on FX-TURN. FX-TURN focuses on the minor difference of the gymnastics’ body. Compared with other baselines, our method demonstrates superior performance in understanding complex and fine-grained semantics, keeping the consistency of bio-physical ch…
Figure 19
Figure 19. Figure 19: Qualitative Results on FX-SALTO. FX-SALTO demands gymnastics’s body rotates 360° around a horizontal axis with the feet passing over the head, which is the most difficult in all of three sub-datasets in FineGym. Compared with other baselines, results in our methods ma…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 43 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Latent-shift: Latent diffu- sion with temporal shift for efficient text-to-video genera- tion

    Jie An, Songyang Zhang, Harry Yang, Sonal Gupta, Jia-Bin Huang, Jiebo Luo, and Xi Yin. Latent-shift: Latent diffu- sion with temporal shift for efficient text-to-video genera- tion. arXiv preprint arXiv:2304.08477, 2023. 2

    arXiv 2023

  3. [3]

    Learned neural physics sim- ulation for articulated 3d human pose reconstruction

    Mykhaylo Andriluka, Baruch Tabanpour, C Daniel Freeman, and Cristian Sminchisescu. Learned neural physics sim- ulation for articulated 3d human pose reconstruction. In European Conference on Computer Vision, pages 320–336. Springer, 2024. 2

    2024

  4. [4]

    Physics-informed computer vision: A review and perspectives

    Chayan Banerjee, Kien Nguyen, Clinton Fookes, and George Karniadakis. Physics-informed computer vision: A review and perspectives. arXiv preprint arXiv:2305.18035, 2023. 4

    arXiv 2023

  5. [5]

    Physics-informed computer vision: A re- view and perspectives

    Chayan Banerjee, Kien Nguyen, Clinton Fookes, and Kar- niadakis George. Physics-informed computer vision: A re- view and perspectives. ACM Computing Surveys, 57(1):1– 38, 2024. 2

    2024

  6. [6]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  7. [7]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023. 1, 7

    arXiv

  8. [8]

    Finecliper: Multi-modal fine-grained clip for dynamic facial expression recognition with adapters

    Haodong Chen, Haojian Huang, Junhao Dong, Mingzhe Zheng, and Dian Shao. Finecliper: Multi-modal fine-grained clip for dynamic facial expression recognition with adapters. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 2301–2310, 2024. 19

    2024

Show all 89 references
  1. [9]

    Gaussianvton: 3d human virtual try- on via multi-stage gaussian splatting editing with image prompting

    Haodong Chen, Yongle Huang, Haojian Huang, Xiangsheng Ge, and Dian Shao. Gaussianvton: 3d human virtual try- on via multi-stage gaussian splatting editing with image prompting. arXiv preprint arXiv:2405.07472, 2024. 20

    2024 arXiv

  2. [10]

    Omnicreator: Self-supervised unified generation with uni- versal editing

    Haodong Chen, Lan Wang, Harry Yang, and Ser-Nam Lim. Omnicreator: Self-supervised unified generation with uni- versal editing. arXiv preprint arXiv:2412.02114, 2024. 2

    2024 arXiv

  3. [11]

    Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7310– 7...

    2024

  4. [12]

    Temporal regularization makes your video generator stronger

    Harold Haodong Chen, Haojian Huang, Xianfeng Wu, Yexin Liu, Yajing Bai, Wen-Jie Shu, Harry Yang, and Ser-Nam Lim. Temporal regularization makes your video generator stronger. arXiv preprint arXiv:2503.15417, 2025. 2

    2025 arXiv

  5. [13]

    Control-a-video: Controllable text-to-video generation with diffusion models

    Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv preprint arXiv:2305.13840, 2023. 2, 7

    2023 arXiv

  6. [14]

    Disentangling structured components: Towards adaptive, interpretable and scalable time series forecasting

    Jinliang Deng, Xiusi Chen, Renhe Jiang, Du Yin, Yi Yang, Xuan Song, and Ivor W Tsang. Disentangling structured components: Towards adaptive, interpretable and scalable time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 2024. 16

    2024

  7. [15]

    Parsimony or capability? decomposition delivers both in long-term time series forecasting

    Jinliang Deng, Feiyang Ye, Du Yin, Xuan Song, Ivor Tsang, and Hui Xiong. Parsimony or capability? decomposition delivers both in long-term time series forecasting. Advances in Neural Information Processing Systems, 37:66687–66712,

  8. [16]

    Revisiting skeleton-based action recognition

    Haodong Duan, Yue Zhao, Kai Chen, Dahua Lin, and Bo Dai. Revisiting skeleton-based action recognition. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2969–2978, 2022. 13

    2022

  9. [17]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 2

    2021

  10. [18]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7346–7356, 2023. 1

    2023

  11. [19]

    Hu- manrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance

    Guian Fang, Wenbiao Yan, Yuanfan Guo, Jianhua Han, Zu- tao Jiang, Hang Xu, Shengcai Liao, and Xiaodan Liang. Hu- manrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance. Pro- ceedings of the European conference on computer v...

    2024

  12. [20]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 2

    2022 arXiv

  13. [21]

    Differentiable dynamics for articu- lated 3d human motion reconstruction

    Erik G ¨artner, Mykhaylo Andriluka, Erwin Coumans, and Cristian Sminchisescu. Differentiable dynamics for articu- lated 3d human motion reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13190–13200, 2022. 2

    2022

  14. [22]

    Trajectory optimization for physics-based re- construction of 3d human pose from monocular video

    Erik G ¨artner, Mykhaylo Andriluka, Hongyi Xu, and Cristian Sminchisescu. Trajectory optimization for physics-based re- construction of 3d human pose from monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 13106–13115...

    2022

  15. [23]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 1, 2, 3, 6, 7, 14 9

    2023 arXiv

  16. [24]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330–348. Springer, 2025. 2

    2025

  17. [25]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and Jos ´e Lezama. Photorealistic video generation with diffusion models. Pro- ceedings of the European conference on computer vision (ECCV), 2024. 2

    2024

  18. [26]

    Animate-a-story: Storytelling with retrieval-augmented video generation

    Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, et al. Animate-a-story: Storytelling with retrieval-augmented video generation. arXiv preprint arXiv:2307.06940, 2023. 2

    2023 arXiv

  19. [27]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 2, 3

    2020

  20. [28]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 13

    2021

  21. [29]

    Neural mocon: Neural motion control for phys- ically plausible human motion capture

    Buzhen Huang, Liang Pan, Yuan Yang, Jingyi Ju, and Yan- gang Wang. Neural mocon: Neural motion control for phys- ically plausible human motion capture. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6417–6426, 2022. 2

    2022

  22. [30]

    Vistadpo: Video hierarchical spatial-temporal direct prefer- ence optimization for large video models

    Haojian Huang, Haodong Chen, Shengqiong Wu, Meng Luo, Jinlan Fu, Xinya Du, Hanwang Zhang, and Hao Fei. Vistadpo: Video hierarchical spatial-temporal direct prefer- ence optimization for large video models. arXiv preprint arXiv:2504.13122, 2025. 19

    2025 arXiv

  23. [31]

    Sefar: Semi-supervised fine-grained action recognition with temporal perturbation and learning stabilization

    Yongle Huang, Haodong Chen, Zhenbang Xu, Zihan Jia, Haozhou Sun, and Dian Shao. Sefar: Semi-supervised fine-grained action recognition with temporal perturbation and learning stabilization. arXiv preprint arXiv:2501.01245,

    arXiv

  24. [32]

    Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and pre- dictive methods for 3d human sensing in natural environ- ments. IEEE transactions on pattern analysis and machine intelligence, 36(7):1325–1339, 2013. 4, 6, 13

    2013

  25. [33]

    Humansd: A native skeleton-guided diffusion model for human image generation

    Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd: A native skeleton-guided diffusion model for human image generation. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15988–15998, 2023. 2, 6, 13

    2023

  26. [34]

    Generalization in diffusion models arises from geometry-adaptive harmonic representations

    Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, and St´ephane Mallat. Generalization in diffusion models arises from geometry-adaptive harmonic representations. In The Twelfth International Conference on Learning Representa- tions, 2024. 2, 4, 5

    2024

  27. [35]

    Pix2gif: Motion-guided diffusion for gif generation

    Hitesh Kandala, Jianfeng Gao, and Jianwei Yang. Pix2gif: Motion-guided diffusion for gif generation. Proceedings of the European conference on computer vision (ECCV), 2024. 2

    2024

  28. [36]

    How far is video generation from world model: A physical law perspective,

    Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: A physical law perspective,

  29. [37]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Comput...

    2023

  30. [38]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems , 36: 36652–36663, 2023. 6, 16

    2023

  31. [39]

    Harivo: Harnessing text-to-image models for video generation

    Mingi Kwon, Seoung Wug Oh, Yang Zhou, Difan Liu, Joon-Young Lee, Haoran Cai, Baqiao Liu, Feng Liu, and Youngjung Uh. Harivo: Harnessing text-to-image models for video generation. Proceedings of the European Confer- ence on computer Vision (ECCV), 2024. 2, 6, 17

    2024

  32. [40]

    Gd-vdm: Generated depth for better diffusion-based video generation

    Ariel Lapid, Idan Achituve, Lior Bracha, and Ethan Fetaya. Gd-vdm: Generated depth for better diffusion-based video generation. arXiv preprint arXiv:2306.11173, 2023. 2

    2023 arXiv

  33. [41]

    Movideo: Motion-aware video generation with diffusion model

    Jingyun Liang, Yuchen Fan, Kai Zhang, Radu Timofte, Luc Van Gool, and Rakesh Ranjan. Movideo: Motion-aware video generation with diffusion model. Proceedings of the European conference on computer vision (ECCV), 2024. 2

    2024

  34. [42]

    A quick tutorial on multibody dynamics

    C Karen Liu and Sumit Jain. A quick tutorial on multibody dynamics. Online tutorial, June, page 7, 2012. 3

    2012

  35. [43]

    Physgen: Rigid-body physics-grounded image- to-video generation

    Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shen- long Wang. Physgen: Rigid-body physics-grounded image- to-video generation. In European Conference on Computer Vision ECCV, 2024. 2

    2024

  36. [44]

    Spe- cialist diffusion: Plug-and-play sample-efficient fine-tuning of text-to-image diffusion models to learn any unseen style

    Haoming Lu, Hazarapet Tunanyan, Kai Wang, Shant Navasardyan, Zhangyang Wang, and Humphrey Shi. Spe- cialist diffusion: Plug-and-play sample-efficient fine-tuning of text-to-image diffusion models to learn any unseen style. In Proceedings of the IEEE/CVF Conference on Com- pute...

    2023

  37. [45]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 2, 7

    2024 arXiv

  38. [46]

    Follow your pose: Pose- guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose- guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 4117–4125, 2024. 1, 2, 6, 7, 8

    2024

  39. [47]

    Amass: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019. 4, 13

    2019

  40. [48]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 2 10

    2024

  41. [49]

    A mathematical introduction to robotic manipulation

    Richard M Murray, Zexiang Li, and S Shankar Sastry. A mathematical introduction to robotic manipulation . CRC press, 2017. 3

    2017

  42. [50]

    Conditional image-to-video gener- ation with latent flow diffusion models

    Haomiao Ni, Changhao Shi, Kai Li, Sharon X Huang, and Martin Renqiang Min. Conditional image-to-video gener- ation with latent flow diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18444–18455, 2023. 2

    2023

  43. [51]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  44. [52]

    Mevg: Multi-event video generation with text-to-video models

    Gyeongrok Oh, Jaehwan Jeong, Sieun Kim, Wonmin Byeon, Jinkyu Kim, Sungwoong Kim, and Sangpil Kim. Mevg: Multi-event video generation with text-to-video models. In European Conference on Computer Vision, pages 401–418. Springer, 2024. 2

    2024

  45. [53]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  46. [54]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 2

    2020

  47. [55]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 2

    2021

  48. [56]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  49. [57]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 3, 6, 13, 14

    2022

  50. [58]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  51. [59]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  52. [60]

    Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis

    Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila. Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis. In International con- ference on machine learning , pages 30105–30118. PMLR,

  53. [61]

    Find and focus: Retrieve and localize video events with natural language queries

    Dian Shao, Yu Xiong, Yue Zhao, Qingqiu Huang, Yu Qiao, and Dahua Lin. Find and focus: Retrieve and localize video events with natural language queries. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 200–216, 2018. 19

    2018

  54. [62]

    Finegym: A hierarchical video dataset for fine-grained action under- standing

    Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. Finegym: A hierarchical video dataset for fine-grained action under- standing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2616–2625,

  55. [63]

    Intra-and inter-action understanding via temporal action parsing

    Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. Intra-and inter-action understanding via temporal action parsing. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 730–739, 2020. 19

    2020

  56. [64]

    Neural monocular 3d human motion capture with physical awareness

    Soshi Shimada, Vladislav Golyanik, Weipeng Xu, Patrick P´erez, and Christian Theobalt. Neural monocular 3d human motion capture with physical awareness. ACM Transactions on Graphics (ToG), 40(4):1–15, 2021. 2

    2021

  57. [65]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  58. [66]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 14

    2012 arXiv

  59. [67]

    Predicting human poses via recurrent attention network

    Jianwei Tang, Jieming Wang, and Jian-Fang Hu. Predicting human poses via recurrent attention network. Visual Intelli- gence, 1(1):18, 2023. 20

    2023

  60. [68]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024. 6

    2024

  61. [69]

    To- wards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 6, 16

    2018 arXiv

  62. [70]

    Recovering ac- curate 3d human pose in the wild using imus and a moving camera

    Timo V on Marcard, Roberto Henschel, Michael J Black, Bodo Rosenhahn, and Gerard Pons-Moll. Recovering ac- curate 3d human pose in the wild using imus and a moving camera. In Proceedings of the European conference on com- puter vision (ECCV), pages 601–617, 2018. 6

    2018

  63. [71]

    Skeleton-in-context: Unified skeleton sequence modeling with in-context learning

    Xinshun Wang, Zhongbin Fang, Xia Li, Xiangtai Li, Chen Chen, and Mengyuan Liu. Skeleton-in-context: Unified skeleton sequence modeling with in-context learning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2436–2446, 2024. 6

    2024

  64. [72]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Pa- pers, pages 1–11, 2024. 1

    2024

  65. [73]

    Cat: a coarse-to-fine attention tree for semantic change detection

    Xiu-Shen Wei, Yu-Yan Xu, Chen-Lin Zhang, Gui-Song Xia, and Yu-Xin Peng. Cat: a coarse-to-fine attention tree for semantic change detection. Visual Intelligence, 1(1):3, 2023. 3 11

    2023

  66. [74]

    Physics-based human motion es- timation and synthesis from videos

    Kevin Xie, Tingwu Wang, Umar Iqbal, Yunrong Guo, Sanja Fidler, and Florian Shkurti. Physics-based human motion es- timation and synthesis from videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11532–11541, 2021. 2

    2021

  67. [75]

    Icon: Implicit clothed humans obtained from nor- mals

    Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J Black. Icon: Implicit clothed humans obtained from nor- mals. In 2022 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 13286–13296. IEEE, 2022. 20

    2022

  68. [76]

    Econ: Explicit clothed humans optimized via normal integration

    Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael J Black. Econ: Explicit clothed humans optimized via normal integration. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 512–523, 2023. 20

    2023

  69. [77]

    Dialoguenerf: Towards realistic avatar face- to-face conversation video generation

    Yichao Yan, Zanwei Zhou, Zi Wang, Jingnan Gao, and Xi- aokang Yang. Dialoguenerf: Towards realistic avatar face- to-face conversation video generation. Visual Intelligence, 2 (1):24, 2024. 2

    2024

  70. [78]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023. 2

    2023 arXiv

  71. [79]

    Video probabilistic diffusion models in projected latent space

    Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. Video probabilistic diffusion models in projected latent space. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 18456–18466,

  72. [80]

    Simpoe: Simulated character control for 3d hu- man pose estimation

    Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani, and Ja- son Saragih. Simpoe: Simulated character control for 3d hu- man pose estimation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 7159–7169, 2021. 2

    2021

  73. [81]

    Physdiff: Physics-guided human motion diffusion model

    Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 16010–16021, 2023. 2

    2023

  74. [82]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2

    2023

  75. [83]

    Physics-based interaction with 3d ob- jects via video generation

    Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. Physics-based interaction with 3d ob- jects via video generation. Proceedings of the European con- ference on computer vision (ECCV), 2024. 2

    2024

  76. [84]

    Physpt: Physics-aware pretrained transformer for estimating human dynamics from monocular videos

    Yufei Zhang, Jeffrey O Kephart, Zijun Cui, and Qiang Ji. Physpt: Physics-aware pretrained transformer for estimating human dynamics from monocular videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2305–2317, 2024. 2

    2024

  77. [85]

    Incorporating physics principles for precise human motion prediction

    Yufei Zhang, Jeffrey O Kephart, and Qiang Ji. Incorporating physics principles for precise human motion prediction. In Proceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision, pages 6164–6174, 2024. 3

    2024

  78. [86]

    Pimnet: Physics-infused neural network for human motion prediction

    Zhibo Zhang, Yanjun Zhu, Rahul Rai, and David Doermann. Pimnet: Physics-infused neural network for human motion prediction. IEEE Robotics and Automation Letters , 7(4): 8949–8955, 2022. 2

    2022

  79. [87]

    Magdiff: Multi-alignment diffusion for high-fidelity video generation and editing

    Haoyu Zhao, Tianyi Lu, Jiaxi Gu, Xing Zhang, Qingping Zheng, Zuxuan Wu, Hang Xu, and Yu-Gang Jiang. Magdiff: Multi-alignment diffusion for high-fidelity video generation and editing. Proceedings of the European conference on computer vision (ECCV), 2024. 2

    2024

  80. [88]

    Motiondirector: Motion customization of text-to-video diffusion models

    Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Jun- hao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In European Conference on Computer Vision, pages 273–290. Springer, 2025. 2, 3, 6

    2025

  81. [89]

    For each gymnastics move described in the labels below, write a detailed description as if explaining to someone who is unfamiliar with gymnastics

    Wentao Zhu, Xiaoxuan Ma, Zhaoyang Liu, Libin Liu, Wayne Wu, and Yizhou Wang. Motionbert: A unified perspective on learning human motion representations. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15085–15099, 2023. 4 12 FinePhys: Fine-g...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.