Pith. sign in

REVIEW 3 major objections 5 minor 64 references

Voyaging into Perpetual Dynamic Scenes from a Single View

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single fixed-view video can grow into a perpetual dynamic 3D scene by repeatedly outpainting unseen regions with ray-depth and ray-distance guidance.

desk verdict A clever and plausible method for unbounded dynamic scene generation from a single view, but the central claim of 3D-consistent motion is not demonstrated: unseen-region depths are unanchored, and no drift or geometric consistency is measured. read the letter →

arxiv 2507.04183 v2 pith:OS3IVEMI submitted 2025-07-05 cs.CV

Fengrui Tian , Tianjiao Ding , Jinqi Luo , Hancheng Min , René Vidal This is my paper
classification cs.CV
keywords dynamicscenegenerationoutpainting4Dpointcloudray-conditioneddiffusionperpetualviewcamerafly-throughvideomodelsingle-view3D
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DynamicVoyager claims that a perpetual dynamic 3D scene can be grown from a single fixed-view video by repeatedly treating the missing parts as an outpainting problem and guiding that outpainting with 3D ray information. The method reconstructs a dynamic 4D point cloud from estimated video depths, renders a partial video at a new camera pose, and uses ray depth in observed regions plus ray-to-point-cloud distance in unseen regions to condition a video diffusion model. The outpainted video is then backprojected and merged into the point cloud, so the next pose starts from a larger consistent scene. If the claim holds, dynamic scenes can be explored along arbitrarily long fly-through trajectories instead of merely interpolating between training views.

What carries the argument

The load-bearing object is a 4D point cloud $P = \{p = (x, t, c)\}$, with each point carrying a 3D position, a timestamp, and a color, split into foreground and background layers. The iteration is: rasterize the point cloud at a new pose (Eq. 5), compute per-pixel ray depth in observed regions and the minimal ray-to-point distance $\text{dist}_{\mathrm{r2p}}(r, p) = \sqrt{\|p\|^2 - (r^\top p)^2}$ in unseen regions (Eqs. 6-7), outpaint with those ray maps as conditioning, then estimate depth of the outpainted video, align it to the rendered ray depth, and merge new foreground and background points into the cloud (Eq. 9). The ray-to-point distance is the signal that lets the diffusion model infer how far unseen content should be from the existing geometry.

What would settle it

Take a scene with known 3D geometry or a captured multi-view sequence, run the loop for, say, ten posed steps, and compare the outpainted regions against held-out ground-truth views; if reprojection or chamfer distance grows linearly with step count, the perpetual claim fails. A simpler check: at each step, measure the consistency between the new outpainted video and the previously built point cloud in the observed region; large boundary inconsistencies after a few steps would show that the ray-conditioned outpainting is not actually anchoring to geometry.

Watch

Extended reading notes

Core claim

The central discovery is that 3D motion consistency in outpainted regions can be achieved by conditioning a video outpainting diffusion model on per-pixel ray geometry rather than on 2D image content alone. The loop of rasterization, ray-conditioned outpainting, and point-cloud merging produces a growing 4D point cloud whose newly generated regions are anchored to the geometry and motion already reconstructed.

Load-bearing premise

The loop assumes the monocular depth estimates, after alignment, are accurate enough that geometric errors do not accumulate across repeated fly-through steps, and that consecutive views overlap enough to leave a usable anchor for outpainting.

Editorial extensions

If this is right

  • A user can start from one image or fixed camera video and move the camera backward, forward, or rotate to explore dynamic content that is not present in the input view.
  • Text prompts can steer the generated content in newly outpainted regions, so the same start point can yield different worlds such as a waterfall, a lake, glacier, or magma.
  • Because each step merges new points into a shared 4D cloud, the scene representation grows monotonically, and later views are rendered from a larger body of previously generated geometry rather than from the original view alone.
  • The approach positions perpetual dynamic generation as outpainting, which means advances in video outpainting and depth estimation can be transferred directly to longer fly-through trajectories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ray-distance conditioning is a general geometric prior: other generative models, including image or Gaussian-splatting generators, could use the same ray-to-point-cloud distance to keep hallucinated content grounded in known geometry.
  • A natural stress test is drift measurement: run many loop iterations on a scene with known or captured geometry and measure how fast reprojection error grows; the paper does not report such a curve.
  • The moderate-overlap assumption implies the trajectory step size is bounded by the outpainting model's mask tolerance; adaptive step sizing based on predicted ray distance could extend the reachable trajectory.
  • Because the background is modeled with time-invariant depth and time-varying color, the method should handle dynamic textures such as water and clouds, but may struggle with genuinely moving background geometry, a boundary the authors acknowledge in their limitations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript introduces DynamicVoyager, an iterative video-generation pipeline that represents a fixed-viewpoint input video as a 4D point cloud, renders partial videos at novel camera poses, and finetunes a video outpainting model conditioned on ray depth and ray-point distance maps to fill unobserved regions. The resulting outpainted video is unprojected back into 3D and merged with the existing point cloud, allowing repeated traversal along a specified camera path; a scene-prompt control mechanism is also included. The central claim is that iterative application yields a growing 4D point cloud with 3D-consistent motions in newly generated regions, enabling perpetual exploration of dynamic scenes from a single view.

Significance. If the central claim is established, the paper offers a novel and practical decomposition of perpetual dynamic scene generation into outpainting with ray-conditioned 3D priors, and the system-level demonstrations are visually strong. The method is clearly described, the code and pretrained models are promised publicly, and the ablation design is reasonable. However, the evidence supporting the central claim is indirect: the quantitative evaluations rely on learned 2D video metrics that cannot measure 3D consistency or drift. The contribution is significant but currently under-validated.

major comments (3)
  1. [Section 3.3, Eq. (9)] The update loop has no mechanism that prevents geometric error from accumulating in newly generated regions. In Eq. (7), the distance map is a single scalar per ray; it tells the model how far the nearest existing point is from the ray, but not where along the ray the new surface should be placed. The depth used for unprojection at update time is the monocular depth estimate D(1), and the described alignment with the rendered ray depth D-hat(1) is applied only inside the observed mask M-hat(1) (Section 3.3). For pixels in the outpainted region, depth is therefore free to place geometry at any distance along the ray, and Eq. (9) inserts that geometry into the cloud with no pruning, weighting, or conflict resolution. A misplaced point then affects every future rasterization (Eq. (5)) and ray-distance computation (Eq. (7)). Because Tables 2 and 3 report only 2D video metrics (VQ, TC, TA, FC, CLIP-SIM), the reported quantities cannot detect whether this drift has occurred. I ask for a direct geometric evaluation (e.g., multi-view re-projection error, Chamfer distance between scene subsets, drift versus number of steps) and, if drift is found, a mechanism that anchors unseen-region depth or prunes inconsistent points.
  2. [Section 3.2, Eq. (8)] The ray information is injected via ControlNet and trained with a standard denoising objective, so it is a soft prior rather than a hard consistency constraint. This does not invalidate the approach, since learning-based priors can be effective, but it means the central claim that consistent motions are generated in unseen regions is not implied by the construction. The paper needs a quantitative test of 3D consistency that is independent of the trained outpainting model: for example, render the same generated scene region from two camera poses and compare projected geometry or optical flow in their overlap, and report consistency as a function of trajectory length. The current comparisons to MOTIA and MotionCtrl (Table 2, Figure 8) establish visual superiority but not the claimed 3D consistency.
  3. [Section 3.2 and Section 4.1] The pipeline assumes a moderate overlap between consecutive views, yet the camera and trajectory parameters are reported only as '0.0005 units' and '0.45 radians' without specifying the scene scale, the effective overlap fraction at each step, or the distance at which outpainting degenerates. Perpetual generation means many iterations; if overlap falls below the training regime, the partial video supplies too little anchor and the error-propagation issue in Eq. (9) is aggravated. Please quantify the overlap percentage per step for the tested trajectories and include a failure analysis for long paths or larger camera steps.
minor comments (5)
  1. [Figure 3] The symbols for ray depth and ray distance maps appear as garbled glyphs in the caption; please replace them with D-hat and D-bar to match Eq. (5)-(7).
  2. [Section 3.2, Eq. (8)] The expectation is written as E_{z0,I,...}; since z0 is not defined before the equation, clarify whether it denotes the clean video latent, a noised target, or a VAE-encoded input.
  3. [Section 4.1] The statement 'one unit corresponds to the normalized 3D coordinate defined in PyTorch3D' does not specify the relationship between a unit and scene depth; please provide camera intrinsics or the equivalent scene-scale calibration so the step size is reproducible.
  4. [Tables 1-3] The sample sizes are not reported for Tables 1 and 3; please state the number of scenes, videos, and sampled views used for each quantitative table.
  5. [Section 3.3] The depth-alignment fine-tuning is described only as 'following common practices'; please provide the loss function, the optimization budget, and a comparison of aligned versus unaligned depth errors in the observed region.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the outpainting model is trained on independent video data, and the update loop's consistency claims are empirical rather than by construction.

full rationale

The claimed contribution is an iterative outpainting loop: rasterization (Eq. 5), ray-depth and ray-distance maps (Eq. 7), outpainting (Eq. 8), and point-cloud merging (Eq. 9). No equation defines the target output in terms of the fitted parameters. The ray information used as outpainting conditioning is computed from the current point cloud, but the outpainted video is produced by a video diffusion model trained to reconstruct full videos from partial videos and ray maps on the OpenVid dataset, so the unseen-region content is not forced by construction. The update step in Sec. 3.3 aligns depth estimates with rendered ray depths only in the observed mask regions; in unseen regions the depth is a free monocular estimate. This is a geometric anchoring and drift-accumulation limitation, not a circular reduction. The self-citations [43,44] are related-work mentions and are not load-bearing for the central claim. The stated limitations (reflections, shadows, fine structures, multi-view rendering, depth discontinuities) and the absence of direct 3D drift measurement in Tables 2 and 3 are correctness and evaluation concerns, not circularity. The ablation in Sec. 4.4 samples views from the model's own point cloud and uses learned metrics; this weakens the evaluation but does not show that the derivation reduces to its own inputs. Overall, the derivation is self-contained with respect to the input video and independently trained base models.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The method rests on several domain assumptions about depth, segmentation, and the sufficiency of ray distance conditioning; no derivation is offered for the geometric consistency, and the main empirical anchor is a learned outpainting model trained on a curated subset of OpenVid.

free parameters (4)
  • LoRA rank = 256
    LoRA rank for fine-tuning CogVideoX-5B-I2V; chosen empirically, no sensitivity analysis.
  • training subset size = 5000 videos
    Number of OpenVid videos used to fine-tune the outpainting model; no scaling study, performance dependence unknown.
  • camera translation per step = 0.0005 normalized units
    Camera translation between consecutive poses; follows prior work but controls the amount of new content to outpaint per iteration.
  • camera rotation per step = 0.45 radians
    Camera rotation between consecutive poses; follows prior work, affects overlap and potential drift.
assumptions (6)
  • domain assumption Consecutive novel views overlap moderately with the previous view.
    Stated in §3.2 following prior work; without moderate overlap the partial video leaves too little observed geometry to guide outpainting.
  • domain assumption Monocular depth estimates are geometrically accurate enough for point cloud reconstruction and merging.
    Backprojection Eq. 1-3 and depth alignment in §3.3 assume the depth model produces consistent scene geometry; no ground-truth depth evaluation is provided.
  • domain assumption The pretrained image-to-video diffusion model can synthesize a plausible fixed-view dynamic video from a single image.
    Used in §3.1 as the entry point for image inputs; failures would propagate to all subsequent steps.
  • domain assumption Foreground/background segmentation is correct enough to separate dynamic and static content.
    Equations (2)-(4) rely on binary masks to build separate layers; mask errors corrupt both layers.
  • ad hoc to paper Ray depth and ray-point distance maps are sufficient conditioning for a video outpainting model to produce 3D-consistent generated motion.
    This is the core premise of §3.2; it is supported only by ablations with learned metrics, not by geometric verification.
  • domain assumption Background has constant depth and time-varying color; foreground has independent dynamics.
    Modeling choice in §3.1; helps resolve ambiguity but may fail for moving backgrounds with depth changes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Voyaging into Perpetual Dynamic Scenes from a Single View." pith.science (2026). https://pith.science/paper/OS3IVEMI

@misc{pith2026250704183,
  author       = {Pith},
  title        = {Pith review of: Voyaging into Perpetual Dynamic Scenes from a Single View},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OS3IVEMI}},
  note         = {Machine review of arXiv:2507.04183}
}
read the original abstract

The problem of generating a perpetual dynamic scene from a single view is an important problem with widespread applications in augmented and virtual reality, and robotics. However, since dynamic scenes regularly change over time, a key challenge is to ensure that different generated views be consistent with the underlying 3D motions. Prior work learns such consistency by training on multiple views, but the generated scene regions often interpolate between training views and fail to generate perpetual views. To address this issue, we propose DynamicVoyager, which reformulates dynamic scene generation as a scene outpainting problem with new dynamic content. As 2D outpainting models struggle at generating 3D consistent motions from a single 2D view, we enrich 2D pixels with information from their 3D rays that facilitates learning of 3D motion consistency. More specifically, we first map the single-view video input to a dynamic point cloud using the estimated video depths. We then render a partial video of the point cloud from a novel view and outpaint the missing regions using ray information (e.g., the distance from a ray to the point cloud) to generate 3D consistent motions. Next, we use the outpainted video to update the point cloud, which is used for outpainting the scene from future novel views. Moreover, we can control the generated content with the input text prompt. Experiments show that our model can generate perpetual scenes with consistent motions along fly-through cameras. Project page: https://tianfr.github.io/DynamicVoyager.

Figures

Figures reproduced from arXiv: 2507.04183 by the authors.

Figure 1
Figure 1. DynamicVoyager generates 4D point clouds of unbounded dynamic scenes by our dynamic scene outpainting process. Given a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Failure examples of previous dynamic scene genera [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The overview of DynamicVoyager. First, for an initial camera pose, we build the dynamic point clouds from the input image [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Qualitative ablation studies of the proposed foreground [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Ablation studies of scene outpainting with ray informa [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Voyaging into perpetual dynamic scenes with long camera trajectories from a single image or fixed viewpoint video. We [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: View extrapolation results. While Wonderjourney [ [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Visual comparisons with video diffusion model Mo [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Controllable scene generation from input images with [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Space-time interpolation comparisons between DynamicVoyager and WonderJourney [ [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Controllable scene generation from input images with scene prompts. Given an image as input, DynamicVoyager successfully [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Detailed results of MotionCtrl [48] with large camera translation inputs. It can be seen that the model fails to generate videos with large camera motions [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 53 canonical work pages

  1. [1]

    VD3D: Taming large video diffusion transformers for 3D camera control

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siaro- hin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. VD3D: Taming large video diffusion transformers for 3D camera control. In ICLR, 2025. 3

    work page 2025

  2. [2]

    Gaudi: A neural architect for immersive 3D scene genera- tion

    Miguel Angel Bautista, Pengsheng Guo, Samira Abnar, Wal- ter Talbott, Alexander Toshev, Zhuoyuan Chen, Laurent Dinh, Shuangfei Zhai, Hanlin Goh, Daniel Ulbricht, et al. Gaudi: A neural architect for immersive 3D scene genera- tion. In NeurIPS, 2022. 2

    work page 2022

  3. [3]

    Immersive light field video with a layered mesh representation

    Michael Broxton, John Flynn, Ryan Overbeck, Daniel Er- ickson, Peter Hedman, Matthew Duvall, Jason Dourgarian, Jay Busch, Matt Whalen, and Paul Debevec. Immersive light field video with a layered mesh representation. InToG, 2020. 2

    work page 2020

  4. [4]

    High-quality streamable free-viewpoint video

    Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Den- nis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. High-quality streamable free-viewpoint video. In ToG, 2015. 2

    work page 2015

  5. [5]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In CVPR, 2023. 3

    work page 2023

  6. [6]

    Unconstrained scene generation with locally conditioned radiance fields

    Terrance DeVries, Miguel Angel Bautista, Nitish Srivastava, Graham W Taylor, and Joshua M Susskind. Unconstrained scene generation with locally conditioned radiance fields. In ICCV, 2021. 2

    work page 2021

  7. [7]

    ThreeDWorld: A platform for interactive multi-modal phys- ical simulation

    C Gan, J Schwartz, S Alter, M Schrimpf, J Traer, J De Fre- itas, J Kubilius, A Bhandwaldar, N Haber, M Sano, et al. ThreeDWorld: A platform for interactive multi-modal phys- ical simulation. In NeurIPS, 2021. 2

    work page 2021

  8. [8]

    Srinivasan, Jonathan T

    Ruiqi Gao*, Aleksander Holynski*, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul P. Srinivasan, Jonathan T. Barron, and Ben Poole*. CAT3D: Create any- thing in 3D with multi-view diffusion models. In NeurIPS,

    work page

Show all 64 references
  1. [9]

    ManiSkill2: A unified benchmark for generalizable manipulation skills

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yun- chao Yao, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, and Hao Su. ManiSkill2: A unified benchmark for generalizable manipulation skills. In ICLR, 2023. 2

    2023

  2. [10]

    AnimateDiff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. AnimateDiff: Animate your personalized text-to- image diffusion models without specific tuning. In ICLR,

  3. [11]

    CameraCtrl: En- abling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. CameraCtrl: En- abling camera control for text-to-video generation. In ICLR,

  4. [12]

    VideoScore: Building auto- matic metrics to simulate fine-grained human feedback for video generation

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bo- han Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Yuchen Lin, and Wenhu Chen. VideoScore: Building auto- m...

    2024

  5. [13]

    Text2Room: Extracting textured 3D meshes from 2D text-to-image models

    Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2Room: Extracting textured 3D meshes from 2D text-to-image models. In ICCV, 2023. 2

    2023

  6. [14]

    CogVideo: Large-scale pretraining for text-to- video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. CogVideo: Large-scale pretraining for text-to- video generation via transformers. In ICLR, 2022. 2, 3

    2022

  7. [15]

    Training-free camera control for video generation

    Chen Hou, Guoqiang Wei, Yan Zeng, and Zhibo Chen. Training-free camera control for video generation. In ICLR,

  8. [16]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 7

    2022

  9. [17]

    Worldsheet: Wrapping the world in a 3d sheet for view synthesis from a single image

    Ronghang Hu, Nikhila Ravi, Alexander C Berg, and Deepak Pathak. Worldsheet: Wrapping the world in a 3d sheet for view synthesis from a single image. In ICCV, 2021. 2

    2021

  10. [18]

    OneFormer: One transformer to rule universal image segmentation

    Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. OneFormer: One transformer to rule universal image segmentation. In CVPR, 2023. 2, 3, 4

    2023

  11. [19]

    3D gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3D gaussian splatting for real-time radiance field rendering. In TOG, 2023. 2

    2023

  12. [20]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In ICCV, 2023. 2, 3, 12

    2023

  13. [21]

    DreamDrone: Text-to-image diffusion models are zero-shot perpetual view generators

    Hanyang Kong, Dongze Lian, Michael Bi Mi, and Xinchao Wang. DreamDrone: Text-to-image diffusion models are zero-shot perpetual view generators. In ECCV, 2024. 5, 7, 8

    2024

  14. [22]

    Collab- orative video diffusion: Consistent multi-video generation with camera control

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hong- sheng Li, Leonidas Guibas, and Gordon Wetzstein. Collab- orative video diffusion: Consistent multi-video generation with camera control. In NeurIPS, 2024. 3

    2024

  15. [23]

    Black Forest Labs. Flux. https://github.com/black-forest- labs/flux, 2024. 2

    2024

  16. [24]

    Flux.1 kontext: Flow matching for in-context image generation and editing in latent space,

    Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Ku- lal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas M¨uller, Dustin Podell, Robin Rombach, Harry S...

  17. [25]

    RGBD2: Generative scene synthesis via incremental view inpainting using RGBD diffusion models

    Jiabao Lei, Jiapeng Tang, and Kui Jia. RGBD2: Generative scene synthesis via incremental view inpainting using RGBD diffusion models. In CVPR, 2023. 2

    2023

  18. [26]

    4K4DGen: Panoramic 4D generation at 4K resolution

    Renjie Li, Panwang Pan, Bangbang Yang, Dejia Xu, Shi- jie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, Zhengzhong Tu, et al. 4K4DGen: Panoramic 4D generation at 4K resolution. In ICLR, 2025. 2, 3

    2025

  19. [27]

    Infinitenature-Zero: Learning perpetual view generation of natural scenes from single images

    Zhengqi Li, Qianqian Wang, Noah Snavely, and Angjoo Kanazawa. Infinitenature-Zero: Learning perpetual view generation of natural scenes from single images. In ECCV,

  20. [28]

    Align your gaussians: Text-to-4D with dynamic 3D gaussians and composed diffusion models

    Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fi- dler, and Karsten Kreis. Align your gaussians: Text-to-4D with dynamic 3D gaussians and composed diffusion models. In CVPR, 2024. 2, 3

    2024

  21. [29]

    DL3DV-10K: A large-scale scene dataset for deep learning-based 3D vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. DL3DV-10K: A large-scale scene dataset for deep learning-based 3D vision. In CVPR, 2024. 3

    2024

  22. [30]

    Infinite Na- ture: Perpetual view generation of natural scenes from a sin- gle image

    Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. Infinite Na- ture: Perpetual view generation of natural scenes from a sin- gle image. In ICCV, 2021. 1, 2

    2021

  23. [31]

    SyncDreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. SyncDreamer: Gen- erating multiview-consistent images from a single-view im- age. In ICLR, 2024. 2

    2024

  24. [32]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 2

    2020

  25. [33]

    Openvid-1M: A large-scale high-quality dataset for text-to- video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1M: A large-scale high-quality dataset for text-to- video generation. In ICLR, 2025. 7, 12

    2025

  26. [34]

    Effi- cient4D: Fast dynamic 3D object generation from a single- view video

    Zijie Pan, Zeyu Yang, Xiatian Zhu, and Li Zhang. Effi- cient4D: Fast dynamic 3D object generation from a single- view video. arXiv preprint arXiv 2401.08742, 2024. 3

    2024

  27. [35]

    Freeman, and Michael Rubinstein

    Stefan Popov, Amit Raj, Michael Krainin, Yuanzhen Li, William T. Freeman, and Michael Rubinstein. CamCtrl3D: Single-image scene exploration with precise 3D camera con- trol. In 3DV, 2025. 2, 7

    2025

  28. [36]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021. 7, 8

    2021

  29. [37]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. In IEEE TPAMI, 2020. 2, 3, 4, 6, 7, 12

    2020

  30. [38]

    DreamGaussian4D: Genera- tive 4D gaussian splatting.arXiv preprint arXiv:2312.17142,

    Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. DreamGaussian4D: Genera- tive 4D gaussian splatting.arXiv preprint arXiv:2312.17142,

    arXiv

  31. [39]

    GEN3C: 3d-informed world-consistent video generation with precise camera con- trol

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas M ¨uller, Alexander Keller, Sanja Fidler, and Jun Gao. GEN3C: 3d-informed world-consistent video generation with precise camera con- trol. In CVPR, 2025. 3

    2025

  32. [40]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2

    2022

  33. [41]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2

    2022

  34. [42]

    DimensionX: Create any 3D and 4D scenes from a single image with controllable video diffusion

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. DimensionX: Create any 3D and 4D scenes from a single image with controllable video diffusion. arXiv preprint arXiv:2411.04928, 2024. 2, 3

    2024 arXiv

  35. [43]

    MonoN- eRF: Learning a generalizable dynamic radiance field from monocular videos

    Fengrui Tian, Shaoyi Du, and Yueqi Duan. MonoN- eRF: Learning a generalizable dynamic radiance field from monocular videos. In ICCV, 2023. 2

    2023

  36. [44]

    Semantic Flow: Learning semantic fields of dy- namic scenes from monocular videos

    Fengrui Tian, Yueqi Duan, Angtian Wang, Jianfei Guo, and Shaoyi Du. Semantic Flow: Learning semantic fields of dy- namic scenes from monocular videos. In ICLR, 2024. 2

    2024

  37. [45]

    Generative camera dolly: Ex- treme monocular dynamic novel view synthesis

    Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sar- gent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl V ondrick. Generative camera dolly: Ex- treme monocular dynamic novel view synthesis. In ECCV,

  38. [46]

    Be-your-outpainter: Mastering video outpainting through input-specific adaptation

    Fu-Yun Wang, Xiaoshi Wu, Zhaoyang Huang, Xiaoyu Shi, Dazhong Shen, Guanglu Song, Yu Liu, and Hongsheng Li. Be-your-outpainter: Mastering video outpainting through input-specific adaptation. In ECCV, 2024. 5, 7, 8

    2024

  39. [47]

    DUSt3R: Geometric 3D vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. In CVPR, 2024. 2

    2024

  40. [48]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH, 2023. 7, 8, 12, 15

    2023

  41. [49]

    Cat4D: Create anything in 4D with multi-view video diffu- sion models

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Aleksander Holynski. Cat4D: Create anything in 4D with multi-view video diffu- sion models. arXiv preprint arXiv:2411.18613, 2024. 2, 3, 8

    2024 arXiv

  42. [50]

    SV4D: Dynamic 3D content generation with multi-frame and multi-view consistency

    Yiming Xie, Chun-Han Yao, Vikram V oleti, Huaizu Jiang, and Varun Jampani. SV4D: Dynamic 3D content generation with multi-frame and multi-view consistency. arXiv preprint arXiv:2407.17470, 2024. 2, 3

    2024 arXiv

  43. [51]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In ICLR,

  44. [52]

    Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie

    Jihan Yang, Shusheng Yang, Anjali W. Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in Space: How mul- timodal large language models see, remember and recall spaces. arXiv preprint arXiv:2412.14171, 2024. 2

    2024 arXiv

  45. [53]

    Direct-a-Video: Customized video generation with user- directed camera movement and object motion

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-Video: Customized video generation with user- directed camera movement and object motion. In SIG- GRAPH, 2024. 3

    2024

  46. [54]

    CogVideoX: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. CogVideoX: Text-to-video diffusion models with an expert transformer. In ICLR, 2024. 2, 3, 4, 7

    2024

  47. [55]

    4DGen: Grounded 4D content gener- ation with spatial-temporal consistency

    Yuyang Yin, Dejia Xu, Zhangyang Wang, Yao Zhao, and Yunchao Wei. 4DGen: Grounded 4D content gener- ation with spatial-temporal consistency. arXiv preprint arXiv:2312.17225, 2023. 3

    2023 arXiv

  48. [56]

    4Real: Towards photorealistic 4D scene generation via video diffusion models

    Heng Yu, Chaoyang Wang, Peiye Zhuang, Willi Mena- pace, Aliaksandr Siarohin, Junli Cao, Laszlo A Jeni, Sergey Tulyakov, and Hsin-Ying Lee. 4Real: Towards photorealistic 4D scene generation via video diffusion models. InNeurIPS,

  49. [57]

    WonderWorld: Interactive 3D scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. WonderWorld: Interactive 3D scene generation from a single image. arXiv preprint arXiv:2406.09394, 2024. 2, 5, 6

    2024 arXiv

  50. [58]

    WonderJourney: Going from anywhere to everywhere

    Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, De- qing Sun, Noah Snavely, Jiajun Wu, et al. WonderJourney: Going from anywhere to everywhere. In CVPR, 2024. 1, 2, 5, 6, 7, 8, 12, 14

    2024

  51. [59]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 6

    2023

  52. [60]

    GenXD: Generating any 3D and 4D scenes

    Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, and Lijuan Wang. GenXD: Generating any 3D and 4D scenes. In ICLR, 2025. 3 We organize the supplementary material as follows. • §6 provides an additional video for better...

    2025

  53. [61]

    Moreover, we provide the implementation code in the sup- plementary material; our code and pretrained models will be publicly available

    Video, Code, and Notation Table We encourage readers to watch the video in the supplemen- tary material to better understand our visualization results. Moreover, we provide the implementation code in the sup- plementary material; our code and pretrained models will be publicly...

  54. [62]

    Scene generation details

    Implementation Details We present further implementation details and the dataset filtering process here. Scene generation details. Inspired by WonderJourney [58], we adjust the depth maps with SAM [20] to promote spatio-temporal consistency of the depth for each object in the ...

  55. [63]

    Background Completion As described in the method section of the main paper, in practice, we find that background regions occluded by the dynamic foregrounds can become exposed when render- ing the dynamic scene with a fly-through camera trajectory. Since these regions are full...

  56. [64]

    Figure 10 presents visualiza- tions of space-time interpolation results

    More Dynamic Scene Generation Results Space-time interpolation. Figure 10 presents visualiza- tions of space-time interpolation results. In this figure, a cartoon cat is playing guitar with moving clouds. It can be seen that our model successfully renders dynamic scenes with f...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.