Pith. sign in

REVIEW 11 cited by

VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.12781 v3 pith:JMEMFZED submitted 2024-07-17 cs.CV

Sherwin Bahmani , Ivan Skorokhodov , Aliaksandr Siarohin , Willi Menapace , Guocheng Qian , Michael Vasilkovsky , Hsin-Ying Lee , Chaoyang Wang
show 4 more authors
Jiaxu Zou Andrea Tagliasacchi David B. Lindell Sergey Tulyakov
This is my paper
classification cs.CV
keywords cameracontrolmodelsvideodiffusiongenerationapproachcontrollable
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Modern text-to-video synthesis models demonstrate coherent, photorealistic generation of complex videos from a text description. However, most existing models lack fine-grained control over camera movement, which is critical for downstream applications related to content creation, visual effects, and 3D vision. Recently, new methods demonstrate the ability to generate videos with controllable camera poses these techniques leverage pre-trained U-Net-based diffusion models that explicitly disentangle spatial and temporal generation. Still, no existing approach enables camera control for new, transformer-based video diffusion models that process spatial and temporal information jointly. Here, we propose to tame video transformers for 3D camera control using a ControlNet-like conditioning mechanism that incorporates spatiotemporal camera embeddings based on Pl\"ucker coordinates. The approach demonstrates state-of-the-art performance for controllable video generation after fine-tuning on the RealEstate10K dataset. To the best of our knowledge, our work is the first to enable camera control for transformer-based video diffusion models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UniGeo: Unifying Geometric Guidance for Camera-Controllable Image Editing via Video Models

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    UniGeo unifies geometric guidance across three levels in video models to reduce geometric drift and improve consistency in camera-controllable image editing.

  2. SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    SymphoMotion jointly controls camera trajectories and depth-aware object dynamics inside one video diffusion model, supported by the new RealCOD-25K real-world paired-motion dataset.

  3. SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation

    cs.RO 2026-02 conditional novelty 6.0 of 10

    SoMA couples robot joint actions, environmental forces, and learned Gaussian-splat dynamics into a single neural simulator, improving resimulation and generalization on real robot soft-body manipulation by about 20% o...

  4. InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problem

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A training-free, near-zero-overhead inverse solver for novel-view video generation and inpainting that projects masks into continuous multi-channel latent masks and applies DDS with conjugate gradient in latent space.

  5. PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attention

    cs.CV 2025-11 conditional novelty 6.0 of 10

    PostCam generates new videos from a reference video along user-specified camera trajectories using a query-shared cross-attention that fuses pose data and rendered frames, improving control precision and detail preservation.

  6. OmniNWM: Omniscient Driving Navigation World Models

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniNWM jointly generates long panoramic multi-modal driving videos, controls them precisely via normalized Plücker ray-maps, and derives dense driving rewards from generated 3D occupancy.

  7. Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion

    cs.CV 2025-08 conditional novelty 6.0 of 10

    Single-image novel view synthesis is decomposed into panorama outpainting plus keyframe-conditioned video diffusion, producing loop-consistent scene tours.

  8. Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Vid-CamEdit re-synthesizes monocular videos along user-defined camera paths by conditioning a video diffusion model on 2D flows derived from estimated 3D geometry, without training on multi-view video data.

  9. MoWorld: A Flash World Model

    cs.CV 2026-07 conditional novelty 5.0 of 10

    MoWorld achieves up to 50 FPS real-time interactive world simulation on NPUs by combining a 3D-native data engine, curriculum cross-frame pretraining, autoregressive distillation, and mixed-precision parallel inference.

  10. LangScene-X: Reconstruct Generalizable 3D Language-Embedded Scenes with TriMap Video Diffusion

    cs.CV 2025-07 conditional novelty 5.0 of 10

    LangScene-X generates RGB, normal, and semantic videos from sparse views to reconstruct 3D language-embedded Gaussian fields that support open-ended text queries.

  11. From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence

    cs.RO 2026-07 conditional novelty 4.0 of 10

    Physical intelligence needs an embodied brain that reasons over interventions and emits capability requests, grounded by a physical harness and shared experience contracts rather than direct actuator policies.

Pith tools