DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking

Xinyuan Chen; Yaohui Wang; Yunyi Li; Yu Qiao

REVIEW 3 major objections 5 minor 36 references

DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim: steering video generators to physics-critical regions via spatial-temporal masks yields dramatically more realistic deformations such as melting and crushing.

desk verdict A well-scoped, internally consistent framework for steering video generators toward deformable regions; the 'reasoning' claim outruns the ablations, and the headline numbers come from a same-family VLM judge. read the letter →

arxiv 2607.18664 v1 pith:YI26ZHKS submitted 2026-07-21 cs.CV

Yunyi Li , Yu Qiao , Yaohui Wang , Xinyuan Chen This is my paper · ORCID

classification cs.CV

keywords physics-awarevideogenerationdeformationspatial-temporalmaskvision-languagemodelreasoningattentionmodulationflowmatchingimage-to-videolocalfidelity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DeforM is a framework for image-to-video generation that targets deformation dynamics — melting, squeezing, fracturing, stretching, and slicing — which standard video generators handle poorly. The paper's central claim is that the root cause of failure is a semantic-spatial gap: the model knows the text says 'melting' but spreads its attention across the whole frame instead of the object that should deform. To close that gap, a vision-language model identifies the deforming object and extrapolates its bounding box trajectory, producing a spatial-temporal mask; that mask is then injected into the transformer's attention modules, either training-free or via a learned dense mask modulator. Experiments on 320 deformation videos report that the masked guidance outperforms baselines on both visual quality and physical consistency, with the biggest gains in local deformation fidelity. The deeper claim, supported by ablations, is that precise localization — not just fine-tuning or scaling — is what unlocks realistic deformation.

What carries the argument

The spatial-temporal mask is the load-bearing object: a binary volume over latent frames indicating which pixels should undergo the described deformation. It is produced by DeforM-Reason, which combines VLM semantic identification of the target object, open-vocabulary detector grounding, and VLM temporal extrapolation of the bounding box. The mask does two jobs: in DeforM-Free it modulates cross-attention queries by multiplying queries inside the mask by α > 1, sharpening attention; in DeforM-Injection it drives a Dense Mask Modulator that produces additive offsets to self-attention keys/values and cross-attention queries, trained with a region-weighted flow-matching loss. The mask is the me

What would settle it

Run DeforM-Injection on a video where the mask is deliberately shifted to a wrong region (e.g., the background instead of the deforming object). If the generator still produces physically realistic deformation, the mask is not the causal driver and the central claim is falsified. Conversely, an oracle experiment with ground-truth masks should show a large quality jump; if it does not, mask precision is not the limiting factor.

Watch

Extended reading notes

Core claim

On the paper's own terms, DeforM establishes that physics-aware video generation for complex deformations reduces to a localization problem: instead of trying to encode physical laws in the generator, one can give the model an explicit spatial-temporal mask of where deformation will occur. The mask comes from a two-step cognitive pipeline: a VLM identifies the target object from the initial image and prompt, and an open-vocabulary detector grounds it; during inference the VLM extrapolates the first bounding box into a trajectory using hand-coded physical heuristics. The mask is then injected into the attention stack. Even without any training, simply scaling cross-attention queries inside th

Load-bearing premise

The entire method rests on the assumption that the VLM-based mask trajectory accurately tracks the true region of deformation throughout the video; if the predicted bounding boxes drift away from where the object actually deforms, both the training-free and training-based guidance amplify attention in the wrong place.

Editorial extensions

If this is right

If localization is the bottleneck, improving the mask tracker (better detectors, learned trajectory prediction) should directly raise physical fidelity without retraining the generator.
Training-free DeforM-Free offers a plug-and-play way to add deformation awareness to any diffusion or flow-matching video generator, since it only touches cross-attention queries.
The mask-injection principle may transfer to other local physical phenomena beyond deformation, such as water splashing or cloth rippling, wherever the effect's location is identifiable.
Because the method avoids simulators and explicit 3D reconstruction, it scales to diverse real-world scenes that simulator-based physics approaches cannot handle.
The reported dynamic degree advantage suggests that mask guidance counteracts the static-scene bias of strong video models, so masks could serve as a general 'motion prompt' for controllable dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's strongest implication is that current video generators already contain enough internal physical priors; the failure is attentional, not knowledge-based. If true, this reframes research effort toward localization and attention routing, which is cheaper and more scalable than new loss functions or simulators.
A testable extension: with oracle masks from ground-truth object tracks, DeforM-Injection should approach ceiling performance on deformation fidelity; comparing oracle-mask results to VLM-mask results would isolate the residual loss caused by mask drift.
The hand-coded physical heuristics for bounding-box evolution (melting shrinks height, fracturing spreads coverage) are a weak point; a learned trajectory predictor trained to replace the Plan step in Algorithm 1 would likely close the gap in complex scenes that the authors flag as a limitation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, the training-free variant is the honest contribution: scaling cross-attention queries inside a VLM-produced mask makes a frozen generator produce more plausible melting and squeezing. That is a concrete, non-obvious observation, and the paper's own diagnostic plots back it up. Second, the central claim that VLM physical reasoning drives the gains is not actually isolated. The only ablation of the reasoning module replaces VLM masks with random masks. That distinguishes a targeted mask from noise, but it does not test whether temporal planning or physical semantics add value over, say, a static first-frame mask or an optical-flow mask. The authors do acknowledge in Limitations that localization accuracy may produce suboptimal masks, so this is a live threat rather than a hypothetical one.

What the paper does well: the internal logic holds. The ablations go in the expected direction — random masks hurt, training-free beats SFT on physical scores, combined cross- and self-attention injection beats either alone. The writing is clear, the setup is practical, and the training-free result is reproducible in principle. The mask-conditioned LoRA recipe is a reasonable extension and the region-sensitive loss is sensible.

Where it gets soft — and this is proportionate, not fatal. All physical-consistency scores come from Qwen3-VL-32B, the same model family as the reasoning module and the data-cleaning VLM. The judge's rubric explicitly rewards the things the method injects, so the gap between 4.59 and 4.28 could partly reflect preference alignment rather than independent physical fidelity. There are no error bars or significance tests on the 160-sample evaluation, and the user study has 10 participants. These are addressable gaps, not terminal flaws. The paper would be substantially stronger with a static-mask baseline, an off-the-shelf motion-based prior, and a non-Qwen judge.

Who this is for: people working on physics-aware video generation, especially those who want a lightweight, training-free way to focus generative attention on local deformation. DeforM-Free is worth trying in practice.

Recommendation: yes, send it to peer review. The core idea is plausible, the mechanism is consistent with the evidence, and a serious referee can push for the missing ablations and an independent judge. It is not ready as-is, but it deserves referee time rather than a desk reject.

Referee Report

3 major / 5 minor

Summary. The paper proposes DeforM, a framework for image-to-video generation of deformation dynamics (melting, squeezing, fracturing, etc.). It introduces DeforM-Reason, a VLM-guided module that identifies the objects undergoing deformation and produces spatial-temporal masks, using a VLM for temporal extrapolation of bounding boxes at inference time. Two integration strategies are developed: DeforM-Free, a training-free cross-attention modulation that scales queries inside the masked regions, and DeforM-Injection, a training-based method that injects mask features into self- and cross-attention via a dense modulator with LoRA fine-tuning and a region-sensitive loss. Experiments on a filtered WISA-80K subset compare against CogVideoX1.5, Hunyuan-I2V, Wan2.2-14B, and MAGI-1, reporting superior visual quality (VBench metrics) and physical consistency (PC/SA/LDF from Qwen3-VL-32B). Ablations show that random masks degrade performance, DeforM-Free outperforms SFT, and combining cross- and self-attention injection is best. The paper includes a 10-participant user study.

Significance. If the reported results hold, the work makes a useful contribution by showing that explicitly redirecting a video generation model's attention to physics-critical regions improves deformation realism. The training-free DeforM-Free result—outperforming SFT without any training—is a noteworthy and falsifiable observation. The internal comparisons are coherent: random masks < SFT < DeforM-Free < DeforM-Injection, and combined cross+self attention > either alone. However, the headline physical-consistency scores come from a VLM judge from the same model family as the reasoning module, and the ablations do not isolate the contribution of VLM reasoning versus simple mask guidance. These are load-bearing issues for the central claim that VLM-guided physical reasoning, rather than mask injection alone, is what drives the gains.

major comments (3)

[Sec. 4.2, Tables 1–2] The physical consistency metrics PC/SA/LDF are produced by Qwen3-VL-32B, while the reasoning module uses Qwen2.5-VL-7B and the data-cleaning pipeline uses Qwen2.5-VL-7B. This same-family evaluator may systematically prefer outputs that align with the physical priors the method injects, making the reported physical-score advantages (e.g., LDF 4.16 vs. 3.65 for Wan2.2-14B) partially an artifact of judge bias. The 10-participant user study (Table 4) is too small and lacks significance testing or inter-rater agreement. To support the headline claim, the authors should provide an independent human evaluation with statistics, or at least calibrate the VLM judge against human judgments on a subset.
[Sec. 4.4, Table 2] The only ablation of DeforM-Reason (W/O DeforM-Reason) replaces VLM masks with random spatial-temporal masks. This shows that some spatially targeted mask is better than random noise, but it does not isolate the contribution of VLM physical reasoning or temporal planning. A static mask derived from the first-frame bounding box, or a mask based on optical flow/motion, would test whether the VLM's semantic reasoning and temporal extrapolation are necessary. The paper's claim that 'precise localization of physics-critical regions is essential' is therefore not yet supported by the evidence presented; the results are consistent with any focused-mask heuristic.
[Algorithm 1, lines 5–6; Limitations] Inference-time masks come from VLM-based temporal extrapolation of the initial bounding box using hand-coded physical heuristics (e.g., melting decreases bbox height). The authors acknowledge that this 'may produce suboptimal spatial-temporal masks in complex scenes,' but no experiment evaluates mask quality (e.g., IoU with ground-truth deformation regions) or its sensitivity to extrapolation error. If the extrapolated boxes drift, both DeforM-Free and DeforM-Injection amplify attention in the wrong places. The training stage uses ground-truth boxes, but the gap between training and inference masks is not quantified. Reporting mask accuracy on a held-out set with known ground truth would directly address this threat.

minor comments (5)

[Sec. 4.2] The text says 'we excludes subject consistency' — should be 'we exclude'. Also, the VBench metric selection rationale is reasonable but the six chosen metrics should be defined more precisely, since 'overall quality score' appears to be an aggregate without a formula.
[Sec. 4.1] Hyperparameters α=1.4, λ_mask=0.3, and LoRA rank 256 are given without any sensitivity analysis. While not central, a small study varying α and λ_mask would strengthen the paper.
[Fig. 3] The caption says 'No melting is observed' but the generated frame appears to show the wax cubes. Clarify that the issue is the melting effect is absent, not the object itself.
[Appendix C.1, Table 4] The user study scores are reported without error bars or p-values; with 10 participants, the 'significant margin' claim is unsupported. Provide per-participant data or at least a paired test.
[General] The paper states it is on arXiv:2607.18664, but the year seems future-dated. This may be a formatting artifact; please check.

Circularity Check

0 steps flagged · score 2.0 of 10

No definitional circularity: the central mask-injection claim is tested against random-mask and SFT controls; the same-family VLM judge is a bias risk, not an in-paper circular reduction.

full rationale

The paper's central claim is that spatial-temporal masks localized by VLM physical reasoning improve deformation realism when injected into a video generation model's attention. The derivation chain is empirical rather than definitional: the masks are produced by an external VLM (Qwen2.5-VL-7B) with an open-vocabulary detector (LLMDet) plus hand-specified physical heuristics in Algorithm 1, and the generator is Wan2.2-TI2V-5B. The claimed gains are measured against baselines, vanilla Wan2.2, SFT, and a random-mask ablation (Tab. 2), so the central result is not true by construction. No equation in the paper defines the predicted quantity in terms of the fitted quantity: the region-sensitive loss (Eq. 7) trains on ground-truth videos and masks derived from detector boxes on GT frames during training, and at inference the masks come from extrapolation, not from the evaluation scores. The one load-bearing concern is evaluator alignment: the physical-consistency metrics PC/SA/LDF are produced by Qwen3-VL-32B (Sec. 4.2, Fig. 11) while the method's reasoning and data-cleaning use Qwen2.5-VL-7B (Sec. 4.1, Sec. A.2). The judge and the method share a model family and the judge's rubric explicitly rewards the same physical priors that DeforM-Reason injects (e.g., 'Melting/Squeezing: the bbox height may decrease'). However, this is a potential bias in the evaluation instrument, not a step where the paper's prediction is identical to its input by construction. The central claim also has independent support from VBench visual-quality scores (Tab. 1), which do not use the VLM judge, and from the random-mask ablation, which is a genuine control albeit not the most informative one. The ablations do not isolate VLM reasoning from mere object localization (random vs. static masks), and the user study is small, but these are experimental-design limitations that affect the strength of the causal attribution rather than circular reasoning. There are no self-citations invoked as load-bearing evidence for the method's premises; the cited Qwen and Wan works are external resources. Accordingly, no specific circular reduction can be exhibited, and the appropriate score is low.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

DeforM introduces no new physical entities or forces. Its free parameters are hyperparameters and hand-coded VLM prompt heuristics, not fitted physical constants. The load-bearing assumptions are the reliability of the VLM/detector pipeline for mask generation and the validity of the VLM judge as an evaluation tool. The method's contribution is a steering mechanism, not a new law or entity.

free parameters (4)

Focus scale α for DeforM-Free = 1.4
Hand-set in Sec. 4.1 as a multiplier on cross-attention queries inside masked regions; no sweep or sensitivity analysis is reported.
Region-sensitive loss weight λ_mask = 0.3
Set in Sec. 4.1 to balance flow loss and mask-weighted reconstruction loss (Eq. 7); no ablation over this value is shown.
LoRA rank for DeforM-Injection = 256
Architectural capacity hyperparameter chosen in Sec. 4.1; no comparison of ranks is provided.
VLM bbox-trajectory heuristics = hand-coded rules (e.g., melting decreases bbox height)
The VLM temporal-extrapolation prompt (Appendix B) encodes hard-coded physical priors for how bounding boxes evolve (volume redistribution, expansion under fracture). These heuristics are designed by the authors and directly influence mask quality.

assumptions (5)

domain assumption Wan2.2-TI2V-5B flow-matching latents are a suitable substrate for mask injection
The whole framework is built on this pretrained model (Sec. 3.1). The dense mask modulator assumes that downsampled binary masks align with the latent tensor z∈R^{T'×H'×W'}, and that injecting offsets into attention features is a valid steering mechanism.
domain assumption Qwen2.5-VL-7B can correctly identify the target deformation object from the initial image and caption
DeforM-Reason (Algorithm 1, line 1) relies on the VLM's multimodal reasoning to map natural-language physics instructions to the correct object; if this fails, the entire mask is wrong.
domain assumption LLMDet open-vocabulary detector can ground the identified object across video frames
During training, masks are built from detector bounding boxes on ground-truth frames (Algorithm 1, line 3). Detector failures would inject noisy or missing masks into the learned modulator.
ad hoc to paper VLM temporal extrapolation of a single initial box yields valid physics-critical regions during inference
Inference masks are produced by prompting the VLM to plan a trajectory of bounding boxes (Algorithm 1, lines 5-6, and Appendix B). The prompt encodes physical heuristics written for this paper, and no evaluation of mask accuracy against ground truth is provided.
domain assumption Qwen3-VL-32B VLM judgment is a valid proxy for physical consistency
The main physical metrics PC, SA, and LDF are assigned by a VLM-as-a-judge (Sec. 4.2, Appendix C). This assumes that VLM scores correlate with actual physical plausibility, an assumption that is neither calibrated nor validated against ground-truth dynamics in the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking." pith.science (2026). https://pith.science/paper/YI26ZHKS

@misc{pith2026260718664,
  author       = {Pith},
  title        = {Pith review of: DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YI26ZHKS}},
  note         = {Machine review of arXiv:2607.18664}
}

read the original abstract

Video generation models achieve high visual quality but often struggle to generate physics-aware videos. Unlike rigid-body motion, which can be described by explicit trajectories or formulas, complex deformation dynamics remain challenging to synthesize. We observe that a lack of physical reasoning for localizing dynamic areas allows irrelevant regions to dilute the model's attention, leading to generation failure. In this paper, we propose DeforM, a reasoning-guided image-to-video generation framework that directs the model's focus toward physics-critical regions. To reason about and localize these critical regions, we introduce a VLM-guided physical reasoning module, DeforM-Reason, to identify target objects and generate spatial-temporal masks. For physical guidance, we develop two alternative strategies: DeforM-Free for training-free mechanism analysis and DeforM-Injection as a powerful training-based generator. Experimental results demonstrate that DeforM improves the realism of generated deformation scenarios, outperforming baseline models in both visual quality and physical consistency.

Figures

Figures reproduced from arXiv: 2607.18664 by the authors.

**Figure 1.** DeforM is an image-to-video physics-aware generation framework. It generates realistic deformation dynamics by leveraging VLM-guided physical reasoning to focus on physics-critical regions. Abstract. Video generation models achieve high visual quality but often struggle to generate physics-aware videos. Unlike rigid-body motion, which can be described by explicit trajectories or formulas, complex deformation dynam… view at source ↗

**Figure 2.** Illustration of DeforM. DeforM takes a text prompt and an initial image as inputs. The DeforM-Reason module first generates spatial-temporal masks by localizing physics-critical regions via VLM-guided reasoning. For physical guidance, we provide two alternative strategies: (1) DeforM-Free, a training-free method for mechanism analysis and observation validation. (2) DeforM-Injection, a powerful training-based gene… view at source ↗

**Figure 3.** Motivation and Analysis of DeforM-Free.(Top) Base models exhibit a semantic-spatial gap, where cross-attention for the keyword melting is spatially diffused, leading to physical inconsistency. (Bottom) By anchoring attention via DeforMFree, the attention map becomes more active on the target region, resulting in physically plausible deformation dynamics. where j is the token index and α > 1 is the focus scale. By… view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Visualization results. Visualization of diverse deformation scenarios generated by DeforM-Injection. dynamics achieved by our work. Furthermore, [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

**Figure 5.** Figure 5: Qualitative comparisons. DeforM presents superior physical plausibility and vivid local deformation dynamics. The red bounding boxes indicate the physics-critical regions [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 6.** Figure 6: Example of Extracted WISA-80K Data. Visualization of raw WISA-80K data, including initial images, raw videos, labels and captions. A.2 Dataset Filtering and Refinement To align the dataset with our focus on complex deformation dynamics, we selected the most suitable s…

**Figure 7.** Figure 7: Example of Video Artifacts There are three main types of video artifacts: visual impairments, subtle physical dynamics and label noise. To address the issue, we implemented a VLM-assisted data cleaning pipeline complemented by human verification. Specifically, we lever…

**Figure 8.** Figure 8: Prompt Template for VLM-assisted Evaluation. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

**Figure 9.** Figure 9: Characteristics of Our Dataset. The word cloud figure of core objects and the donut chart of dynamics verbs. # Role You are a Physical Dynamics Analyzer specialized in grounding textual descriptions into visual scenes. # Task Given an initial image and a caption descri…

**Figure 10.** Figure 10: Prompt Template. The prompt template for identifying deformation objects [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]

**Figure 11.** Figure 11: Prompt Template. The prompt template for physical consistency evaluation [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 8 linked inside Pith

[1]

In: The Eleventh International Conference on Learning Representations (2023)

Albergo, M.S., Vanden-Eijnden, E.: Building normalizing flows with stochastic in- terpolants. In: The Eleventh International Conference on Learning Representations (2023)

2023
[2]

arXiv preprint arXiv:2309.16609 (2023)

Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., Hui, B., Ji, L., Li, M., Lin, J., Lin, R., Liu, D., Liu, G., Lu, C., Lu, K., Ma, J., Men, R., Ren, X., Ren, X., Tan, C., Tan, S., Tu, J., Wang, P., Wang, S., Wang, W., Wu, S., Xu, B., Xu, J., Yang, A., Yang, H., Yang, J., Yang, S., Yao, Y., Yu, B., Yuan, H., Yuan,...

arXiv 2023
[3]

In: The Thirty- ninth Annual Conference on Neural Information Processing Systems (2025)

Chen, H.H., Huang, H., Chen, Q., Yang, H., Lim, S.N.: Hierarchical fine-grained preference optimization for physically plausible video generation. In: The Thirty- ninth Annual Conference on Neural Information Processing Systems (2025)

2025
[4]

Advances in neural information processing systems31(2018)

Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary dif- ferential equations. Advances in neural information processing systems31(2018)

2018
[5]

arXiv preprint arXiv:2512.16906 (2025)

Cong, X., Yang, H., Wang, A., Wang, Y., Yang, Y., Zhang, C., Ma, C.: Viva: Vlm- guided instruction-based video editing with reward optimization. arXiv preprint arXiv:2512.16906 (2025)

arXiv 2025
[6]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Davtyan, A., Sameni, S., Favaro, P.: Efficient video prediction via sparsely condi- tioned flow matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 23263–23274 (2023)

2023
[7]

In: Proceedings of the Computer Vision and Pattern Recognition Conference

Fu, S., Yang, Q., Mo, Q., Yan, J., Wei, X., Meng, J., Xie, X., Zheng, W.S.: Llmdet: Learning strong open-vocabulary object detectors under the supervision of large language models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14987–14997 (2025)

2025
[8]

In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

Gillman, N., Herrmann, C., Freeman, M., Aggarwal, D., Luo, E., Sun, D., Sun, C.: Force prompting: Video generation models can learn and generalize physics-based control signals. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

2025

Show all 36 references

[9]

In: ICLR 2025 Workshop on World Models: Understanding, Modelling and Scaling (2025)

He, H., Zhang, Y., Lin, L., Xu, Z., Pan, L.: Pre-trained video generative models as world simulators. In: ICLR 2025 Workshop on World Models: Understanding, Modelling and Scaling (2025)

2025
[10]

arXiv preprint arXiv:2207.12598 (2022)

Ho, J., Salimans, T.: Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 (2022)

2022 arXiv
[11]

In: International Conference on Learning Representations (2022)

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022)

2022
[12]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024) 16 Y

Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, L., Lin, D., Qiao, Y., Liu, Z.: VBench: Comprehensive benchmark suite for video generative models. In: Proceedings of the IEEE/CVF Conference on Compu...

2024
[13]

In: The Thirteenth International Conference on Learning Representations (2025)

Jin, Y., Sun, Z., Li, N., Xu, K., Xu, K., Jiang, H., Zhuang, N., Huang, Q., Song, Y., MU, Y., Lin, Z.: Pyramidal flow matching for efficient video generative modeling. In: The Thirteenth International Conference on Learning Representations (2025)

2025
[14]

arXiv preprint arXiv:2412.03603 (2024)

Kong, W., Tian, Q., Zhang, Z., Min, R., Dai, Z., Zhou, J., Xiong, J., Li, X., Wu, B., Zhang, J., et al.: Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 (2024)

2024 arXiv
[15]

In: Forty-second International Conference on Machine Learning (2025)

Li, C., Michel, O., Pan, X., Liu, S., Roberts, M., Xie, S.: PISA experiments: Ex- ploring physics post-training for video diffusion models by watching stuff drop. In: Forty-second International Conference on Machine Learning (2025)

2025
[16]

In: Proceedings of the IEEE/CVF international conference on computer vision (2025)

Li, Z., Yu, H.X., Liu, W., Yang, Y., Herrmann, C., Wetzstein, G., Wu, J.: Wonder- play: Dynamic 3d scene generation from a single image and actions. In: Proceedings of the IEEE/CVF international conference on computer vision (2025)

2025
[17]

In: The Eleventh International Conference on Learning Representations (2023)

Lipman, Y., Chen, R.T.Q., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. In: The Eleventh International Conference on Learning Representations (2023)

2023
[18]

In: European Conference on Computer Vision ECCV (2024)

Liu, S., Ren, Z., Gupta, S., Wang, S.: Physgen: Rigid-body physics-grounded image-to-video generation. In: European Conference on Computer Vision ECCV (2024)

2024
[19]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)

2022
[20]

In: Thirteenth International Con- ference on 3D Vision (2026)

Tan, X., Jiang, Y., Li, X., Xie, T., Zong, Z., Yang, Y., Jiang, C.: Physmotion: Physics-grounded dynamics from a single image. In: Thirteenth International Con- ference on 3D Vision (2026)

2026
[21]

arXiv preprint arXiv:2505.13211 (2025)

Teng, H., Jia, H., Sun, L., Li, L., Li, M., Tang, M., Han, S., Zhang, T., Zhang, W., Luo, W., et al.: Magi-1: Autoregressive video generation at scale. arXiv preprint arXiv:2505.13211 (2025)

2025 arXiv
[22]

Advances in neural information pro- cessing systems30(2017)

Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information pro- cessing systems30(2017)

2017
[23]

arXiv preprint arXiv:2503.20314 (2025)

Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., Zeng, J., Wang, J., Zhang, J., Zhou, J., Wang, J., Chen, J., Zhu, K., Zhao, K., Yan, K., Huang, L., Feng, M., Zhang, N., Li, P., Wu, P., Chu, R., Feng, R., Zhang, S., Sun, S., Fang, T...

2025 arXiv
[24]

In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

Wang, J., Ma, A., Cao, K., Zheng, J., Feng, J., Zhang, Z., Pang, W., Liang, X.: WISA: World simulator assistant for physics-aware text-to-video generation. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

2025
[25]

arXiv preprint arXiv:2401.09985 (2024)

Wang, X., Zhu, Z., Huang, G., Wang, B., Chen, X., Lu, J.: Worlddreamer: Towards general world models for video generation via predicting masked tokens. arXiv preprint arXiv:2401.09985 (2024)

2024 arXiv
[26]

International Journal of Computer Vision (2024) DeforM 17

Wang, Y., Chen, X., Ma, X., Zhou, S., Huang, Z., Wang, Y., Yang, C., He, Y., Yu, J., Yang, P., et al.: Lavie: High-quality video generation with cascaded latent diffusion models. International Journal of Computer Vision (2024) DeforM 17

2024
[27]

In: Proceedings of the Computer Vision and Pattern Recognition Conference

Wu, J., Tang, C., Wang, J., Zeng, Y., Li, X., Tong, Y.: Diffsensei: Bridging multi- modal llms and diffusion models for customized manga generation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 28684–28693 (2025)

2025
[28]

arXiv preprint arXiv:2505.09388 (2025)

Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., ...

2025 arXiv
[29]

arXiv preprint arXiv:2412.15115 (2024)

Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, ...

2024 arXiv
[30]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

Yang, X., Li, B., Zhang, Y., Yin, Z., Bai, L., Ma, L., Wang, Z., Cai, J., Wong, T.T., Lu, H., Jia, X.: Vlipp: Towards physically plausible video generation with vision and language informed physical prior. In: Proceedings of the IEEE/CVF International Conference on Computer Vi...

2025
[31]

In: The Thirteenth International Conference on Learning Represen- tations (2025)

Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., Yin, D., Yuxuan.Zhang, Wang, W., Cheng, Y., Xu, B., Gu, X., Dong, Y., Tang, J.: Cogvideox: Text-to-video diffusion models with an expert transformer. In: The Thirteenth Interna...

2025
[32]

In: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V

Ye, W., Liu, Z., Yuwei, G., Yuan, T., Su, Y., Fang, B., Zhao, C., Liu, Q., Wang, L.: GenPilot: A multi-agent system for test-time prompt optimization in image generation. In: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V. (eds.) Findings of the Association for Com...

2025
[33]

Zhang, D., He, L., Yan, R., Shen, F., Tang, J.: R-genie: Reasoning-guided genera- tive image editing (2025)

2025
[34]

arXiv preprint arXiv:2601.11087 (2025)

Zhang,Q.,Gong,B.,Tan,S.,Zhang,Z.,Shen,Y.,Zhu,X.,Li,Y.,Yao,K.,Shen,C., Zou, C.: Physrvg: Physics-aware unified reinforcement learning for video generative models. arXiv preprint arXiv:2601.11087 (2025)

2025
[35]

A soft rubber ball is being squeezed by a hydraulic press

Zhao, Y., Li, H., He, X., Wu, B.: Phyrpr: Training-free physics-constrained video generation (2026) 18 Y. Li et al. A Details of Dataset Construction A.1 Foundational Dataset: WISA-80K We adopt WISA-80K as the foundational dataset for constructing our high- quality dataset for...

2026
[958]

Association for Computational Linguistics, Suzhou, China (Nov 2025)

2025

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

In: The Eleventh International Conference on Learning Representations (2023)

Albergo, M.S., Vanden-Eijnden, E.: Building normalizing flows with stochastic in- terpolants. In: The Eleventh International Conference on Learning Representations (2023)

2023

[2] [2]

arXiv preprint arXiv:2309.16609 (2023)

Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., Hui, B., Ji, L., Li, M., Lin, J., Lin, R., Liu, D., Liu, G., Lu, C., Lu, K., Ma, J., Men, R., Ren, X., Ren, X., Tan, C., Tan, S., Tu, J., Wang, P., Wang, S., Wang, W., Wu, S., Xu, B., Xu, J., Yang, A., Yang, H., Yang, J., Yang, S., Yao, Y., Yu, B., Yuan, H., Yuan,...

arXiv 2023

[3] [3]

In: The Thirty- ninth Annual Conference on Neural Information Processing Systems (2025)

Chen, H.H., Huang, H., Chen, Q., Yang, H., Lim, S.N.: Hierarchical fine-grained preference optimization for physically plausible video generation. In: The Thirty- ninth Annual Conference on Neural Information Processing Systems (2025)

2025

[4] [4]

Advances in neural information processing systems31(2018)

Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary dif- ferential equations. Advances in neural information processing systems31(2018)

2018

[5] [5]

arXiv preprint arXiv:2512.16906 (2025)

Cong, X., Yang, H., Wang, A., Wang, Y., Yang, Y., Zhang, C., Ma, C.: Viva: Vlm- guided instruction-based video editing with reward optimization. arXiv preprint arXiv:2512.16906 (2025)

arXiv 2025

[6] [6]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision

Davtyan, A., Sameni, S., Favaro, P.: Efficient video prediction via sparsely condi- tioned flow matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 23263–23274 (2023)

2023

[7] [7]

In: Proceedings of the Computer Vision and Pattern Recognition Conference

Fu, S., Yang, Q., Mo, Q., Yan, J., Wei, X., Meng, J., Xie, X., Zheng, W.S.: Llmdet: Learning strong open-vocabulary object detectors under the supervision of large language models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14987–14997 (2025)

2025

[8] [8]

In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

Gillman, N., Herrmann, C., Freeman, M., Aggarwal, D., Luo, E., Sun, D., Sun, C.: Force prompting: Video generation models can learn and generalize physics-based control signals. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

2025