Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Mesh

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EX-4D claims that a watertight depth mesh that explicitly marks visible and occluded faces lets a frozen video diffusion model synthesize consistent extreme-viewpoint video from monocular input, without multi-view training data.

desk verdict An interesting, efficient pipeline with a genuine train/inference mismatch and no direct geometric evaluation; worth refereeing but needs major revision. read the letter →

arxiv 2506.05554 v1 pith:YF5SHOKR submitted 2025-06-05 cs.CV

Tao Hu , Haoyang Peng , Xiao Liu , Yuewen Ma This is my paper
classification cs.CV
keywords 4DvideogenerationextremeviewpointsynthesisdepthwatertightmeshocclusionmodelingdiffusionLoRAadaptermonocularnovelview
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes EX-4D, a framework that turns a single monocular video into a camera-controllable 4D video that stays coherent as the viewpoint swings to ±90 degrees. The central idea is a Depth Watertight Mesh (DW-Mesh): a closed 3D surface built from per-frame depth estimates that keeps both visible surfaces and currently hidden regions, so rendered occlusion masks expose what the camera would newly reveal. To train without expensive multi-view captures, the paper synthesizes occlusion masks by rendering the mesh under synthetic camera trajectories and by tracking points across frames. A small LoRA adapter (about 1% of a 14B video diffusion model) injects the mesh's rendered color and mask into the frozen diffusion backbone. The paper reports consistent gains over recent geometry- and camera-based baselines on FID, FVD, VBench, and a user study, with the gap growing at extreme angles.

What carries the argument

The Depth Watertight Mesh (DW-Mesh) is the load-bearing object. Each frame's depth map is unprojected into vertex positions in canonical camera space; adjacent pixels become two triangles per 2x2 grid, border pixels get depth Dmax, and two extra faces close the mesh. Faces are flagged occluded when their minimum angle is below delta_angle or their depth discontinuity exceeds delta_depth, and occluded faces carry black texture. Rasterizing this closed surface produces the mask and color videos that condition the diffusion adapter.

What would settle it

Take a dynamic scene captured simultaneously by multiple calibrated cameras so ground-truth novel views exist at extreme angles, run EX-4D on one camera's video, and compare the disoccluded boundary regions in the synthesized extreme views against the actual captured frames. If the regions the DW-Mesh marks as occluded do not align with the true newly-visible content—as would be expected on reflective, transparent, or very thin structures where monocular depth fails—the geometric-consistency claim collapses to a hallucination artifact.

Watch

Extended reading notes

Core claim

EX-4D's central claim is that explicit modeling of occluded geometry, not just visible surfaces, is what makes extreme-viewpoint video synthesis geometrically consistent. The DW-Mesh unprojects per-frame depth into a closed triangular mesh in canonical camera space, paints occluded faces black, and marks degenerate faces (shallow angles or depth jumps) as occluded. Rasterizing this mesh from a target camera yields a color video plus a binary visibility mask that tells the diffusion model where new content must be invented and where the original surface is still trustworthy. The paper argues that this prior prevents the ghosting, warping, and wrong-occlusion artifacts that geometry-based baselines show at large angles, and that the simulated masking strategy makes this training possible from monocular data alone.

Load-bearing premise

The load-bearing premise is that the simulated occlusion masks—produced by rendering the DW-Mesh under synthetic camera trajectories and by tracking points across frames—faithfully mimic the real occlusion and disocclusion patterns that occur at extreme viewpoints; if they do not, the model learns to invent plausible-looking content rather than recover the geometry the mesh claims to provide.

Editorial extensions

If this is right

  • Extreme-viewpoint control from a monocular video no longer requires paired multi-view datasets or camera calibration, since occlusion supervision is simulated from the mesh.
  • The performance advantage over baselines grows as camera angle increases up to ±90 degrees, indicating the watertight prior matters most where disocclusion is largest.
  • The adapter's memory footprint (1% trainable parameters of the backbone) makes geometry-conditioned 4D generation practical on current GPUs, whereas a ControlNet-style baseline ran out of memory.
  • Removing either the rendering masks or the tracking masks degrades fidelity, so both simulated-occlusion realism and temporal consistency of masks contribute to the final quality.
  • Ablations show that removing the DW-Mesh causes the largest single performance drop, and replacing structured masks with random masks loses most of the benefit, confirming that the geometric prior is the main driver of the results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because training relies on monocular depth and synthetic masks, the method's upper bound is set by depth quality; pairing the same simulated-mask recipe with real multi-view dynamic captures would likely sharpen occlusion boundaries further than the paper's single-day training run shows.
  • The watertight representation could transfer to other 3D-aware generative tasks, such as rotating objects or camera control in text-to-video, wherever a closed-surface prior and a visibility mask can be rendered.
  • A testable extension would be to apply the same adapter to a model trained on real multi-view dynamic videos and compare disoccluded regions against ground truth, isolating how much of the extreme-view quality comes from the geometric prior versus the diffusion model's hallucination of newly visible content.
  • The tracking-mask component suggests that temporal consistency of visibility is itself a learnable prior; one could investigate replacing explicit point tracking with learned visibility propagation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes EX-4D, a framework for extreme-viewpoint (up to ±90°) 4D video synthesis from a monocular video. The method builds a Depth Watertight Mesh (DW-Mesh) per frame by unprojecting depth maps with padded boundary vertices and two back faces to enforce closedness, then renders color and occlusion masks from target cameras. To avoid paired multi-view training data, the authors generate simulated occlusion masks from the DW-Mesh and from point tracking, and train a LoRA-based adapter on the frozen Wan 2.1 video diffusion model, conditioning on the masked original video, rendered color/mask videos, and the first frame. Experiments compare against TrajectoryCrafter, TrajectoryAttention, and ReCamMaster on FID, FVD, VBench, and a user study, with ablations on the mesh, masks, backbone, and LoRA rank. The central claims are that DW-Mesh explicitly models visible and occluded regions, enabling geometric consistency at extreme poses, and that the simulated masking strategy removes the need for multi-view data.

Significance. If the central claims were fully supported, the paper would make a practical contribution: an efficient (≈1% trainable parameters, 1-day training) adapter that turns a frozen video diffusion model into a camera-controllable extreme-view synthesizer without multi-view training data. The DW-Mesh idea of explicitly modeling occluded regions via a watertight surface is reasonable and distinct from prior visible-only geometry guidance. The paper also ships substantial engineering: a 14B-parameter backbone, a large monocular training set, detailed ablation studies, and a user study. However, the evaluation does not directly measure geometric consistency against ground-truth novel views, and the training/inference conditioning mismatch leaves the mechanism underlying the claimed advantages untested. The reported numbers may reflect the strength of the frozen diffusion prior more than the DW-Mesh guidance.

major comments (4)
  1. [Sec. 3.2, Sec. 3.1.2, Eq. (6)] The training and inference conditioning inputs are different. In training (Sec. 3.2), V_T is produced by applying the simulated mask V_O to the original monocular frames: the network learns to inpaint holes in the original view. At inference (Sec. 3.1.2), V_T is the DW-Mesh rasterized color video rendered from the target camera trajectory, which contains parallax, texture stretching, and mesh artifacts. The denoising objective (Eq. 6) is trained only on the former, so the adapter has never learned to correct or complete mesh renders. Consequently, the 'simulated masking strategy' teaches single-view hole-filling, not novel-view synthesis. Even if the simulated masks perfectly matched true disocclusion patterns, there is no training signal that maps a target-camera mesh render to a physically consistent novel view; any success at extreme angles could be attributed to the frozen Wan2.1 prior rather than to the DW-Mesh geometric guidance. This mismatch directly undermines the paper's central claim that DW-Mesh ensures geometric consistency.
  2. [Sec. 4.1, Tables 1 and 2] The quantitative evaluation does not measure geometric accuracy. The metrics used (FID, FVD, VBench) are distributional and perceptual; they compare generated videos to the source video's statistics, not to ground-truth novel views from the target camera. Since no multi-view ground truth is used in evaluation, the stated conclusion that EX-4D 'outperforms state-of-the-art methods in terms of physical consistency' is unsupported by the reported numbers. The claim that geometry is more consistent at extreme angles is never directly verified. A geometric evaluation (e.g., depth consistency between the synthesized novel view and the DW-Mesh projection, or chamfer distance to a reconstructed surface, or a small set of real multi-view captures) is needed before the central claim can be accepted.
  3. [Sec. 3.2] The simulated masking strategy is potentially circular. The training masks are generated by rendering the DW-Mesh of the training video under synthetic camera trajectories, and the same DW-Mesh construction (same depth estimator, same thresholds, same watertight padding) is used at inference to render the conditioning color and mask videos. The model is therefore trained to fill regions whose occlusion patterns are produced by its own geometric prior, not by actual novel-view occlusions in the real world. If the DW-Mesh systematically mispredicts which regions are visible from a new viewpoint (e.g., due to depth errors, missing thin structures, or the ad-hoc back-face construction), the model will learn to hallucinate content consistent with that bias rather than with true geometry. The comparison to 'Random Masks' in Table 3 shows only that structured masks help, not that the structure corresponds to true disocclusion geometry.
  4. [Sec. 3.1.1, Sec. 4.5, Table 3] The ablation 'w/o DW-Mesh' is underspecified. Removing the DW-Mesh presumably also removes the rendered color and mask videos, but the conditioning input V_T then becomes what exactly? If the model receives only the masked original video with a random or empty mask, the 34% FID drop conflates the removal of geometric guidance with the removal of the appearance prior provided by the mesh-rendered color. A controlled ablation should keep the same conditioning channels and only replace the DW-Mesh-derived content with a trivial baseline (e.g., a flat plane at median depth or a constant-color mesh) to isolate the contribution of the watertight structure. As written, the ablation does not pin down which component of the DW-Mesh (the watertight back faces, the occlusion attribute, or the rendered color video) is responsible for the improvement.
minor comments (5)
  1. [Sec. 3.1.1] The definition of the two additional faces for watertightness appears to be a typo: the vertices are given as (0,0),(0,W),(H,0) and (H,0),(H,W),(0,W). With typical image dimensions H×W, these faces would cover the entire plane in two triangles rather than closing the boundary of the pixel grid. Please clarify the intended construction.
  2. [Sec. 3.2] The text says the rendering mask generation 'remove[s] boundary mesh faces with identical thresholds as defined in Sec. 3.1.1', but Sec. 3.1.1 describes padding boundary pixels to D_max, not removing faces. The relationship between these operations should be clarified, since removing boundary faces would open the very watertight surface the paper relies on.
  3. [Sec. 4.1] The sentence 'We evaluate geometric consistency across four increasingly challenging angular ranges' is misleading because none of the reported metrics in Tables 1 and 2 is a geometric consistency metric. Consider rewording to 'evaluate across viewpoint ranges'.
  4. [Sec. 3.3] Equation (6) writes the loss as E_{?} without specifying the distribution of the noise and noisy latents; please write E_{?, t} and define the notation for the masked color video V_T as used in training versus the mesh-rendered V_T used at inference.
  5. [Sec. 4.1] The implementation details state the adapter has 140M trainable parameters, which is about 1% of 14B. Earlier contributions claim 'only 1% trainable parameters'; please ensure the numbers are consistent throughout (including the supplementary code, where the PriorEncoding class is defined with hidden_channels=1024 and out_channels=5120, which may not match the described architecture).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the EX-4D training and inference chain uses an external geometric prior, and the reported concerns are train/test mismatch and validation limitations rather than circular reductions.

full rationale

The derivation chain is: monocular video to per-frame depth (external DepthCrafter), DW-Mesh construction with degenerate-face occlusion flags (Eqs. 1-5), rendered color and mask videos, LoRA adapter conditioning, and denoised output via Eq. (6). None of these steps returns a fitted parameter as the predicted quantity. The simulated masking strategy (Sec. 3.2) does use DW-Mesh to generate training masks, but the training target is the original monocular frame, not a novel view rendered from the same mesh; the model is trained to reconstruct an external input, so it is not learning to reproduce its own conditioning by construction. The occlusion attribute O is defined in Eq. (4) by geometric degeneracy, so saying the mesh 'marks occluded faces' is true by definition, but the paper's empirical claims about extreme-view video quality do not reduce to that definition. The real weaknesses are correctness risks, not circularity: at inference the color condition V_T is a mesh-rendered novel-view video, while during training V_T is a masked original frame, and the FID/FVD evaluation has no ground-truth novel views. These limit what the experiments can establish, but they are not instances of a prediction equaling its input by construction. The only self-citation, X-Ray [18], appears in related work and is not load-bearing. Accordingly, no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no physical entities (no new particles, forces, or conserved quantities). The DW-Mesh is a data structure, not an invented entity in the physics sense. Free parameters are hyperparameters for mesh construction and adapter design. The key axioms are the reliability of monocular depth estimation, the sufficiency of canonical camera space, and the validity of the simulated masking as a proxy for true occlusion.

free parameters (4)
  • delta_depth = 0.013 * (Max(D0) - Min(D0))
    Threshold for depth discontinuity used to mark faces as occluded (Eq. 4). Set based on experiments, reported in Sec. 4.1.
  • Dmax = 100
    Depth value assigned to border pixels for watertight closure. Chosen by hand in Sec. 4.1.
  • delta_angle = not specified
    Minimum face angle threshold in Eq. 4; no numerical value is given anywhere in the paper.
  • LoRA rank = 16
    Rank for the LoRA adapter. Ablation with rank 64 shows only marginal improvement, so 16 is a design choice.
assumptions (4)
  • domain assumption The pretrained depth estimator DepthCrafter provides sufficiently accurate per-frame depth maps for the watertight mesh prior to be reliable.
    Used throughout Sec. 3.1; paper acknowledges failures on reflective/transparent surfaces in Sec. 5 and App. A.4.
  • domain assumption Canonical camera space with fixed intrinsics suffices to bypass camera calibration.
    Invoked in Sec. 3.1.1 to unproject pixels without calibration, citing Splatter-a-Video [30].
  • ad hoc to paper Padding border pixels to Dmax and adding two back faces yields a mesh whose rendered visibility masks are realistic proxies for novel-view occlusions.
    Core design of DW-Mesh (Sec. 3.1.1); the backplane is not grounded in actual scene geometry.
  • ad hoc to paper Simulated masks generated from the DW-Mesh of the training video generalize to real novel-view occlusion patterns at inference.
    Training strategy in Sec. 3.2; without this, the self-supervised task would not transfer to extreme views.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Mesh." pith.science (2026). https://pith.science/paper/YF5SHOKR

@misc{pith2026250605554,
  author       = {Pith},
  title        = {Pith review of: EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Mesh},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YF5SHOKR}},
  note         = {Machine review of arXiv:2506.05554}
}
read the original abstract

Generating high-quality camera-controllable videos from monocular input is a challenging task, particularly under extreme viewpoint. Existing methods often struggle with geometric inconsistencies and occlusion artifacts in boundaries, leading to degraded visual quality. In this paper, we introduce EX-4D, a novel framework that addresses these challenges through a Depth Watertight Mesh representation. The representation serves as a robust geometric prior by explicitly modeling both visible and occluded regions, ensuring geometric consistency in extreme camera pose. To overcome the lack of paired multi-view datasets, we propose a simulated masking strategy that generates effective training data only from monocular videos. Additionally, a lightweight LoRA-based video diffusion adapter is employed to synthesize high-quality, physically consistent, and temporally coherent videos. Extensive experiments demonstrate that EX-4D outperforms state-of-the-art methods in terms of physical consistency and extreme-view quality, enabling practical 4D video generation.

Figures

Figures reproduced from arXiv: 2506.05554 by the authors.

Figure 1
Figure 1. Our EX-4D framework takes a monocular video as input and generates high-quality 4D [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of DW-Mesh construction. (a) Ground Truth: The original scene with complete [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of our mask generation methods. Top Row: Input Monocular Video; Middle [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Overview of the EX-4D framework. Our approach transforms monocular videos into [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of our EX-4D against state-of-the-art approaches under extreme [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: User study results comparing our EX-4D method against baselines. Participants evaluated [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Detailed visualization of our mask generation methods. (a) Rendering masks are created [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Principle of tracking mask generation. Points are tracked across frames to create consistent [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Representative failure cases of EX-4D. (a) Depth estimation errors causing visible distor [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: User study interface. Participants were presented with four methods (labeled A-D in [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Comparison of EX-4D with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Comparison of EX-4D with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Comparison of EX-4D with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Comparison of EX-4D with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Comparison of EX-4D with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]
Figure 16
Figure 16. Figure 16: Comparison of EX-4D with state-of-the-art methods. [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CameraAnything: Refilming Videos with Arbitrary Camera Control

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A video diffusion editor jointly controls extrinsic pose, multi-shot cuts, focal length, and native resolution via Plücker rays in resolution-aware 3D RoPE, trained on synthetic multi-camera pairs.

Reference graph

Works this paper leans on

47 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    Lindell, and Sergey Tulyakov

    Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion transformers.Proc. CVPR, 2025

    work page 2025

  2. [2]

    Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints, 2024

    Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Xiao Fu, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di Zhang. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints, 2024

    work page 2024

  3. [3]

    Recammaster: Camera-controlled generative rendering from a single video, 2025

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, and Di Zhang. Recammaster: Camera-controlled generative rendering from a single video, 2025

    work page 2025

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.CoRR, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models to large datasets.CoRR, 2023

    work page 2023

  5. [5]

    Videocrafter1: Open diffusion models for high-quality video generation, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation, 2023

    work page 2023

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models, 2024

    2024

  7. [7]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model, 2025

    Guoqing Ma et al. Step-video-t2v technical report: The practice, challenges, and future of video foundation model, 2025

    work page 2025

  8. [8]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    WanTeam et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    arXiv 2025

Show all 47 references
  1. [9]

    Hunyuanvideo: A systematic framework for large video generative models, 2024

    Weijie Kong et al. Hunyuanvideo: A systematic framework for large video generative models, 2024

    2024

  2. [10]

    Diffusion as shader: 3d-aware video diffusion for versatile video generation control.arXiv preprint arXiv:2501.03847, 2025

    Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, Wenping Wang, and Yuan Liu. Diffusion as shader: 3d-aware video diffusion for versatile video generation control.arXiv preprint arXiv:2501.03847, 2025

    2025 arXiv

  3. [11]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models.arXiv preprint arXiv:2311.16933, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models.arXiv preprint arXiv:2311.16933, 2023

    2023 arXiv

  4. [12]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.International Conference on Learning Representations, 2024

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.International Conference on Learning Representations, 2024

    2024

  5. [13]

    Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    2024 arXiv

  6. [14]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. InAdvances in Neural Information Processing Systems. Curran Associates, Inc., 2017

    2017

  7. [15]

    Gans trained by a two time-scale update rule converge to a nash equilibrium.CoRR, abs/1706.08500, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Günter Klambauer, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a nash equilibrium.CoRR, abs/1706.08500, 2017

    2017 arXiv

  8. [16]

    W. V . D. Hodge and D. Pedoe.Methods of Algebraic Geometry. Cambridge University Press, 1994 edition, 1994

    1994

  9. [17]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022

    2022

  10. [18]

    X-ray: A sequential 3d representation for generation

    Tao Hu, Wenhang Ge, Yuyang Zhao, and Gim Hee Lee. X-ray: A sequential 3d representation for generation. InAdvances in Neural Information Processing Systems, pages 136193–136219. Curran Associates, Inc., 2024

    2024

  11. [19]

    Depthcrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. InCVPR, 2025. 10

    2025

  12. [20]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. I...

    2024

  13. [21]

    Cotracker3: Simpler and better point tracking by pseudo-labelling real videos

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. InProc. arXiv:2410.11831, 2024

    2024 arXiv

  14. [22]

    3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), 2023

    2023

  15. [23]

    Modular primitives for high-performance differentiable rendering.ACM Transactions on Graphics, 39(6), 2020

    Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for high-performance differentiable rendering.ACM Transactions on Graphics, 39(6), 2020

    2020

  16. [24]

    Reconx: Reconstruct any scene from sparse views with video diffusion model, 2024

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Reconx: Reconstruct any scene from sparse views with video diffusion model, 2024

    2024

  17. [25]

    Zero-1-to-3: Zero-shot one image to 3d object, 2023

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object, 2023

    2023

  18. [26]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InECCV, 2020

    2020

  19. [27]

    Openvid-1m: A large-scale high-quality dataset for text-to-video generation.arXiv preprint arXiv:2407.02371, 2024

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation.arXiv preprint arXiv:2407.02371, 2024

    2024 arXiv

  20. [28]

    Gen-1: The next step forward for generative ai, 2023

    RunwayML. Gen-1: The next step forward for generative ai, 2023. Accessed May 7, 2025

    2023

  21. [29]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. InThe Eleventh International Conference on Lear...

    2023

  22. [30]

    Splatter a video: Video gaussian representation for versatile processing.arXiv preprint arXiv:2406.13870, 2024

    Yang-Tian Sun, Yi-Hua Huang, Lin Ma, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Splatter a video: Video gaussian representation for versatile processing.arXiv preprint arXiv:2406.13870, 2024

    2024 arXiv

  23. [31]

    Generative camera dolly: Extreme monocular dynamic novel view synthesis.European Conference on Computer Vision (ECCV), 2024

    Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sargent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl V ondrick. Generative camera dolly: Extreme monocular dynamic novel view synthesis.European Conference on Computer Vision (ECCV), 2024

    2024

  24. [32]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  25. [33]

    Shape of motion: 4d reconstruction from a single video, 2024

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video, 2024

    2024

  26. [34]

    Continuous 3d perception model with persistent state.arXiv preprint arXiv:2501.12387, 2025

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state.arXiv preprint arXiv:2501.12387, 2025

    2025 arXiv

  27. [35]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20697–20709, 2024

    2024

  28. [36]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  29. [37]

    Trajectory attention for fine-grained video motion control

    Zeqi Xiao, Wenqi Ouyang, Yifan Zhou, Shuai Yang, Lei Yang, Jianlou Si, and Xingang Pan. Trajectory attention for fine-grained video motion control. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  30. [38]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024. 11

    2024 arXiv

  31. [39]

    Mvsnet: Depth inference for unstructured multi-view stereo

    Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: Depth inference for unstructured multi-view stereo. InComputer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VIII, page 785–801, Berlin, Heidelberg, 2018. ...

    2018

  32. [40]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory, 2023

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory, 2023

    2023

  33. [41]

    Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models, 2025

    Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models, 2025

    2025

  34. [42]

    Monst3r: A simple approach for estimating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arxiv:2410.03825, 2024

    arXiv

  35. [43]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023

    2023

  36. [44]

    Genxd: Generating any 3d and 4d scenes

    Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, and Lijuan Wang. Genxd: Generating any 3d and 4d scenes. InICLR, 2025

    2025

  37. [45]

    " " 6A VAE model for encoding camera i n f o r m a t i o n and video features . 7

    Guangcong Zheng, Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Xi Li. Cami2v: Camera-controlled image-to-video diffusion model.arXiv preprint arXiv:2410.15957, 2024. 12 A Supplementary Material A.1 Network Structure The EX-4D Adapter consists of four main modules: Prior Encoding, ...

    2024 arXiv

  38. [46]

    __init__ () 16 17self

    -> None : 15super () . __init__ () 16 17self . l a t e n t _ e n c o d e r = torch . nn . S e q u e n t i a l ( 18torch . nn . Conv3d ( i n _ c h a n n e l s * 2 , hidden_channels , k e r n e l _ s i z e =1 , stride =1 , padding =0) , 19torch . nn . SiLU () , 20torch . nn . Co...

  39. [47]

    q ,k ,v ,o , ffn .0 , ffn .2

    -> torch . Tensor : 47p r i o r _ l a t e n t = p r i o r _ e n c o d i n g ( video , mask , vae ) 48return p r i o r _ l a t e n t Listing 1: EX-4D Adapter: Prior Encoding and Linear Projection. Feature Integrationfuses the projected geometric priors with the noise latent fea...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.