Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Vid-CamEdit re-synthesizes a monocular video along a user-defined camera trajectory by grounding a video diffusion model on estimated 3D pointmaps and projected 2D flows, eliminating the need for large-scale 4D training data.

desk verdict Good engineering and a reusable factorization trick, but the decisive moving-camera regime is untested quantitatively; revise, don't desk reject. read the letter →

arxiv 2506.13697 v1 pith:CRK3OIJ2 submitted 2025-06-16 cs.CV

Junyoung Seo , Jisang Han , Jaewoo Jung , Siyoon Jin , Joungbin Lee , Takuya Narihira , Kazumi Fukuda , Takashi Shibuya
show 4 more authors
Donghoon Ahn Shoukang Hu Seungryong Kim Yuki Mitsufuji
This is my paper
classification cs.CV
keywords videocameratrajectoryeditingdynamicnovelviewsynthesisdiffusionmodelsgeometry-groundedgenerationpointmapprojection2Dflowconditioningfactorizedfine-tuningmonocularre-synthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a monocular video can be re-shot from any user-chosen camera trajectory—swinging left, zooming out, moving forward—by pairing an off-the-shelf video-geometry estimator with a video diffusion model, instead of training on expensive multi-view video (4D) data. The geometry model supplies the 3D structure while the generative model fills in the regions the geometry cannot see, so the diffusion model never has to learn full 4D dynamics from scratch. A factorized fine-tuning scheme trains the spatial blocks of the model on multi-view image data and the temporal blocks on ordinary video data, which together remove the need for 4D training data. The paper presents experiments on multi-view video benchmarks and uncurated real-world footage showing that the approach produces more plausible results than reconstruction-based and fully generative baselines, especially for large camera extrapolations.

What carries the argument

The load-bearing object is the projected 2D flow field $f_{\mathrm{rel}}$, computed as $f_{\mathrm{rel}}(u, v, t) = \Pi(C_{\mathrm{rel}}(t) \cdot G(u, v, t), K) - (u, v)$, where $G$ is the estimated pointmap, $C_{\mathrm{rel}}$ the per-frame relative camera transformation, and $\Pi$ the perspective projection. These flow fields encode the camera trajectory, the intrinsics, and the 3D geometry all in one 2D map, so the diffusion model never has to parse 3D structure directly. The flows are injected through a fully differentiable re-alignment of the sinusoidal positional encodings of the input video's feature tokens, following the GenWarp approach: each token attends at the location it would occupy in the target view, teaching the model to read the flow condition naturally. The second mechanism is the factorized fine-tuning protocol, in which spatial blocks (augmented into multi-view blocks by concatenating ReferenceNet-style feature tokens) are trained on multi-view image data while temporal blocks are trained on video data with the other block type frozen.

What would settle it

A concrete test: take synthetic or multi-view-captured videos with known ground-truth geometry and camera poses, deliberately corrupt the geometry estimate (for example, using a heavily mispredicted pointmap), and measure whether output quality falls with flow accuracy; the claim that geometry grounding is doing the work predicts a strong correlation between pointmap error and generated-video error. Conversely, on real videos, one could compare the generated video's optical flow against the flows predicted by MonST3R for a large trajectory change, since any large divergence marks locations where the model chooses appearance over correspondence and would bound the regime in which the grounding claim holds.

Watch

Extended reading notes

Core claim

The central discovery is that dynamic novel-view synthesis from a single video can be decomposed into geometry estimation followed by geometry-grounded generation, and that this decomposition avoids the need for 4D training data. Given a monocular video, the framework first estimates a temporally consistent pointmap sequence (using MonST3R) that places every pixel in a global 3D space. Projecting these points onto the desired camera path yields per-pixel 2D flow fields $f_{\mathrm{rel}}$, which are injected into a video diffusion model by warping the positional encodings of the input video's feature tokens; the flow tells the model exactly how each input pixel should move, so the model's job is reduced to synthesizing plausible appearance where the warped information is incomplete. Because the generative model is grounded this way, the spatial (multi-view) blocks can be trained on multi-view image pairs and the temporal blocks on ordinary videos, with alternating freezing during fine-tuning. The paper claims this yields a framework that re-renders arbitrary user trajectories, including extreme extrapolation on in-the-wild footage, more plausibly than Generative Camera Dolly, Pseudo-DVS, and MonST3R reprojection baselines.

Load-bearing premise

The load-bearing premise is that the flow fields obtained by projecting the estimated pointmaps onto the target camera path faithfully describe how pixels should move; if the estimated geometry is wrong, the generative model is guided by wrong correspondences, and the paper itself acknowledges that performance degrades when geometry is severely mispredicted.

Editorial extensions

If this is right

  • Video camera trajectory editing becomes possible without collecting multi-view video (4D) training data; multi-view image corpora plus ordinary video corpora suffice for fine-tuning.
  • The framework is agnostic to the geometry estimator: ablations with MonST3R, DepthCrafter, Depth-Anything 2, and DepthAnyVideo show only small quality differences, so improvements in off-the-shelf geometry estimation should transfer directly.
  • Generated novel-view videos can serve as extra supervision for per-scene 4D reconstruction methods, improving reconstruction quality on the DyCheck dataset, particularly in occluded regions.
  • Extreme camera extrapolation (large viewpoint changes) is where the approach gains most over baselines; on Neu3D and ST-NeRF it reports the best LPIPS and frame-consistency scores among the compared methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The flow-realignment conditioning is a general recipe: the same positional-encoding warping could steer a video diffusion model with any dense correspondence field, such as optical flow for motion editing or point tracks for drag-style manipulation, not just camera-derived flows.
  • Because multi-view aggregation happens implicitly inside the video model rather than through explicit dynamic-region masks, the approach tolerates subtle dynamics that masks miss, but it also offers no guarantee that a moving object's trajectory is preserved rather than gently hallucinated; whether small-object motion survives large camera motion is a testable open question.
  • The factorized training scheme implies a separable data trade-off—extrapolation quality should scale with the diversity of the multi-view image data, and temporal coherence with the video data—which offers a principled way to allocate future data collection.
  • The framework inherits the failure envelope of its geometry estimator: the authors acknowledge in Appendix E that severely mispredicted geometry limits performance, so the method's ceiling is tied to continued progress in video geometry estimation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper presents Vid-CamEdit, a generative framework for re-synthesizing a monocular input video along a user-specified camera trajectory. Geometry is first estimated as temporally consistent pointmaps with MonST3R and converted into per-pixel 2D flow fields that map source pixels to target-view locations; these flows are injected into a video diffusion model by warping positional encodings of a ReferenceNet-style video encoder. The diffusion model is fine-tuned in a factorized manner, training multi-view (spatial) blocks on multi-view image datasets and temporal blocks on conventional video data, thereby avoiding 4D training data. Experiments compare against Generative Camera Dolly, Pseudo-DVS, and MonST3R reprojection on Neu3D and ST-NeRF, with additional qualitative in-the-wild results, a user study, VBench scores, and an application to per-scene 4D reconstruction.

Significance. The idea of grounding a video diffusion model with explicit 2D flow from estimated geometry is a practical and sensible way to sidestep the scarcity of 4D training data, and the factorized fine-tuning protocol is well-motivated. The paper is generally clearly written, includes ablations of the main design choices (geometry grounding, choice of geometry estimator), and reports code and weight release. If the method is validated in the claimed moving-camera, real-world extrapolation regime, it would be a useful contribution to video camera trajectory editing and dynamic novel view synthesis. However, the current evidence does not yet substantiate the abstract's flagship claim.

major comments (5)
  1. [§4.3, Table 1] Table 1 is the only quantitative comparison against the main generative and reconstruction baselines, and the paper explicitly states that it uses stationary-camera datasets because Generative Camera Dolly does not accept moving-camera inputs such as DyCheck. Yet the abstract's central claim is superiority 'especially in extreme extrapolation scenarios on real-world footage.' The moving-camera real-world regime is supported only by qualitative examples (Figs. 5-6), a user study, and a VBench comparison against GCD alone. Please add a quantitative evaluation on moving-camera real-world videos (e.g., DyCheck or an equivalent benchmark) for all applicable baselines, or explicitly scope the claim to stationary-camera source videos.
  2. [Table 1] On ST-NeRF, Ours reports PSNR 14.89 vs. 15.33 and SSIM 0.381 vs. 0.415 relative to Pseudo-DVS, while LPIPS and frame consistency favor Ours. The text states 'superior performance across all the datasets' based on frame consistency, but the broader claim 'outperforms baselines' is not supported by all metrics. Report per-sequence numbers with error bars or confidence intervals and a significance test, and discuss why reconstruction-quality metrics favor the baseline if the evaluation is intended to assess perceived plausibility.
  3. [§3.3, Eq. (3)] In §3.3, G_t is defined as a pointmap in global 3D space. Eq. (3) then computes f_rel by projecting C_rel(t)·G(u,v,t). For a moving source camera, the correct target projection is K·C_Y(t)·[G;1], which equals K·C_rel(t)·C_X(t)·[G;1] if C_Y = C_rel·C_X, not K·C_rel(t)·[G;1]. If the pipeline first converts the global pointmap into the t-th source camera coordinate frame, that conversion is not mentioned in the main text; if it is not performed, the flow conditioning is incorrect for moving-camera inputs. Since the paper explicitly supports moving-camera inputs, this equation must be fixed or clarified.
  4. [§4.4, Fig. 9] The difficulty-versus-distortion analysis is presented with a plot but with no protocol details: no dataset name, number of videos, trajectory generation procedure, or error bars. As the only quantitative evidence for behavior under increasingly difficult trajectories, this needs to be specified; otherwise the claim that 'ours consistently achieves best performance' cannot be verified.
  5. [Appendix E] Appendix E acknowledges that performance degrades when the estimated geometry is severely mispredicted. Since extreme extrapolation on in-the-wild video is precisely the setting where geometry estimators are most likely to fail due to occlusions and dynamic objects, this limitation should be reflected in the abstract and conclusion, or the method should be evaluated specifically in those corner cases.
minor comments (4)
  1. [§4.1 vs. Appendix B] The probability with which the video encoder is dropped during video training is given as 0.3 in the Implementation Details section but as 0.5 in the Training paragraph of Appendix B. Please make these numbers consistent.
  2. [§4.3, Fig. 7] The user study reports preference percentages (74.7%, 89.8%, 86.9%) but does not provide confidence intervals, the number of trials per participant, or a statistical significance test. Please add these details.
  3. [Table 2] In the occluded-region columns, 'Shape-of-Motion*' and 'Shape-of-Motion + Ours' share SSIM 0.856 while LPIPS and PSNR differ; please explain this result, since identical SSIM with different PSNR is unusual and may indicate a reporting artefact.
  4. [§4.3, Fig. 8] The VBench comparison is reported only against Generative Camera Dolly; please clarify whether the other baselines (Pseudo-DVS, MonST3R reprojection) could not be included for practical reasons, and consider reporting all baselines that can be evaluated.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity: the geometry-to-flow conditioning is a deterministic projection, and the main empirical claim is tested against external baselines; the minor same-author GenWarp component is not the argument's foundation.

full rationale

The central derivation is not circular. Eq. (3) defines the conditioning flow by deterministic perspective projection of the estimated pointmap: f_rel(u,v,t) = Π(C_rel(t)·G(u,v,t), K) − (u,v). This quantity is computed from MonST3R geometry and the user trajectory; it is not fitted to the generated video, and the generative model is fine-tuned on held-out multi-view image and video corpora (RealEstate10K, Mannequin, MegaScene, ScanNet, TikTok) plus initialized from AnimateDiff. The headline claim is supported by quantitative comparisons against external baselines (MonST3R, Pseudo-DVS, Generative Camera Dolly) on Neu3D and ST-NeRF, a user study, and VBench, so the result is not equivalent to its inputs by construction. The reuse of the authors' GenWarp positional-encoding warping (Eq. 5) is openly attributed and is an architectural component, not an imported uniqueness theorem; it does not carry the empirical claim by itself. The quantitative evaluation's restriction to stationary-camera datasets (noted under Table 1: 'datasets containing videos from a stationary camera are used, as our primary baseline, Camera Dolly, does not officially accept input videos from moving cameras') and Appendix E's admission that severely mispredicted geometry limits performance are evidence gaps, not circular steps. No self-definition, fitted-input renaming, or author-imported uniqueness appears. Score 1 reflects only minor same-author component reuse and the non-decisive evaluation regime for the extreme-extrapolation claim.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The ledger is light because this is an empirical ML system composed of existing components. The load-bearing external assumptions are the accuracy of MonST3R geometry and the sufficiency of factorized fine-tuning; no new physical or conceptual entities are introduced.

free parameters (4)
  • null-condition probability = 0.3
    Chosen by hand in video training to drop the video encoder condition; reported in Appendix B.
  • video-encoder freeze probability = 0.5
    Chosen by hand to prevent overfitting when training temporal blocks on video data; reported in Appendix B.
  • generated video length = T=12 frames
    AnimateDiff-based model generates 12 frames per inference; a design choice tied to the backbone.
  • random novel views per reconstruction epoch = 24 (2 per iteration)
    Used in the per-scene 4D reconstruction integration; a heuristic for iterative dataset update.
assumptions (6)
  • domain assumption MonST3R pointmaps provide temporally consistent geometry accurate enough for flow conditioning.
    Section 3.3 builds the pipeline on MonST3R pointmaps; Appendix E acknowledges severe geometry errors break the method.
  • domain assumption Pre-trained video diffusion models carry temporal priors sufficient for trajectory editing after factorized fine-tuning.
    Section 3.4 relies on temporal blocks to learn 1D dynamics from video data; no formal guarantee is offered.
  • ad hoc to paper Factorized fine-tuning with alternating block freezing preserves both multi-view and temporal capabilities.
    Section 3.4 states this strategy; its validity rests on the reported ablations rather than on a derivation.
  • ad hoc to paper Temporal blocks implicitly aggregate multi-view information across frames without explicit dynamic/static masking.
    Appendix C chooses per-frame conditioning and claims the video model performs aggregation implicitly; supported only qualitatively.
  • domain assumption Source and synthesized videos share the same camera intrinsics K.
    Stated in Section 3.1 Eq. (1); limits the method when intrinsic changes are needed.
  • domain assumption Results on stationary-camera datasets Neu3D and ST-NeRF are representative of the overall trajectory-editing task.
    Quantitative evaluation in Table 1 uses only stationary-camera datasets; in-the-wild and moving-camera evaluation is qualitative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry." pith.science (2026). https://pith.science/paper/CRK3OIJ2

@misc{pith2026250613697,
  author       = {Pith},
  title        = {Pith review of: Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRK3OIJ2}},
  note         = {Machine review of arXiv:2506.13697}
}
read the original abstract

We introduce Vid-CamEdit, a novel framework for video camera trajectory editing, enabling the re-synthesis of monocular videos along user-defined camera paths. This task is challenging due to its ill-posed nature and the limited multi-view video data for training. Traditional reconstruction methods struggle with extreme trajectory changes, and existing generative models for dynamic novel view synthesis cannot handle in-the-wild videos. Our approach consists of two steps: estimating temporally consistent geometry, and generative rendering guided by this geometry. By integrating geometric priors, the generative model focuses on synthesizing realistic details where the estimated geometry is uncertain. We eliminate the need for extensive 4D training data through a factorized fine-tuning framework that separately trains spatial and temporal components using multi-view image and video data. Our method outperforms baselines in producing plausible videos from novel camera trajectories, especially in extreme extrapolation scenarios on real-world footage.

Figures

Figures reproduced from arXiv: 2506.13697 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Motivation. To edit camera trajectories in monocular videos, we embed knowledge from video geometry prediction models, e.g., MonST3R [107], into video generative models [4, 23], allowing the model to synthesize realistic novel views by filling occluded regions the geometry model cannot infer. By incorporating geometrical cues for generation, our approach demonstrates superior performance on novel view video synthesi… view at source ↗
Figure 3
Figure 3. Overview of our framework. Given a video and a tar￾get camera trajectory, we first extract video feature tokens using a Video Encoder and obtain the dynamic scene’s temporally con￾sistent geometry through Temporal Geometry Estimation. We then ground the video generative model on this estimated geometry by re-aligning the video feature tokens according to the 2D flow be￾tween the source and target camera trajectories… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results. Given a user-provided monocular video, our method can synthesize high-quality videos along desired camera trajectories. The frames from the original videos are depicted in the yellow box of the top right of each image. our framework can leverage an…
Figure 6
Figure 6. Figure 6: Qualitative comparisons with MonST3R [107], video inpainting [112] w/MonST3R, GCD (Generative Camera Dolly) [87], and Pseudo-DVS [110]. Ours is best in synthesizing visually realistic images while maintaining the original geometry. Methods Neu3D [47] ST-NeRF dataset [1…
Figure 8
Figure 8. Figure 8: Quantitative comparisons on Vbench [38] with Cam￾era Dolly [87] on uncurated in-the-wild videos. Our approach sub￾stantially outperforms Camera Dolly in both aesthetic and imaging quality. The large performance gap comes from Camera Dolly’s synthetic-looking outputs. 0…
Figure 7
Figure 7. Figure 7: User study. The user study is conducted by surveying 59 participants to evaluate (a) consistency to input videos, (b) video realness, and (c) faithfulness on camera trajectories. Ablation on design choices. We provide an ablation study on various design choices in our …
Figure 9
Figure 9. Figure 9: Comparison on various camera trajectory. Follow￾ing [71], we measure LPIPS between generated videos and target videos (Distortion) over LPIPS between input videos and target videos (Difficulty). Ours consistently achieves best performance. Baseline LPIPS ↓ SSIM ↑ w/o G…
Figure 10
Figure 10. Figure 10: More qualitative results. We rendered the video using two different camera trajectories for each input video. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: More qualitative results. We rendered the video using two different camera trajectories for each input video. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Architecture. We provide a more detailed illustration of our architecture. In the video diffusion model, which consists of spatial and temporal blocks, we enhance the spatial block by integrating tokens provided by the video encoder, thereby transforming it into a mul…
Figure 13
Figure 13. Figure 13: Qualitative results of per-scene reconstruction with our method on DyCheck [20]. As an application, the outputs generated by our method can be integrated into optimization-based frameworks for per-scene reconstruction. This helps reduce artifacts in challenging novel …
Figure 14
Figure 14. Figure 14: Two approaches to condition the novel view video generation on an input view video. (a) With estimated mask sequence for dynamic regions, generating each frame of novel view video is conditioned on all the frames following the dynamic region masking. (b) Generating ea…
Figure 15
Figure 15. Figure 15: Failure case of dynamic region masking. The dynamic region estimation used in MonST3R [107] effectively captures major dynamic objects but fails to detect more subtle dynamic elements. In the video above, the movements of other people besides the person running on the…
Figure 16
Figure 16. Figure 16: Example of the screen shown to participants. The order of the results for our method and the baseline methods in each question was randomly shuffled and thoroughly anonymized. dicates smoother motions. • Imaging Quality: Evaluates the quality of the individual frames …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CameraAnything: Refilming Videos with Arbitrary Camera Control

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A video diffusion editor jointly controls extrinsic pose, multi-shot cuts, focal length, and native resolution via Plücker rays in resolution-aware 3D RoPE, trained on synthetic multi-camera pairs.

Reference graph

Works this paper leans on

113 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vd3d: Taming large video diffu- sion transformers for 3d camera control.arXiv preprint arXiv:2407.12781, 2024

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffu- sion transformers for 3d camera control.arXiv preprint arXiv:2407.12781, 2024. 3

    arXiv 2024

  2. [2]

    Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints.arXiv preprint arXiv:2412.07760, 2024

    Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Xiao Fu, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di Zhang. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints.arXiv preprint arXiv:2412.07760, 2024. 3

    arXiv 2024

  3. [3]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G ¨ul Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 1728–1738,

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2, 3, 5, 6

    arXiv 2023

  5. [5]

    Align your latents: High-resolution video synthe- sis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthe- sis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 1

    2023

  6. [6]

    Generative rendering: Controllable 4d-guided video generation with 2d diffusion models

    Shengqu Cai, Duygu Ceylan, Matheus Gadelha, Chun- Hao Paul Huang, Tuanfeng Yang Wang, and Gordon Wet- zstein. Generative rendering: Controllable 4d-guided video generation with 2d diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7611–7620, 2024. 5

    2024

  7. [7]

    Hexplane: A fast representa- tion for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representa- tion for dynamic scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2023. 3

    2023

  8. [8]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. 19

    2021

Show all 113 references
  1. [9]

    Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation.arXiv preprint arXiv:2310.19512, 2023. 6

    arXiv

  2. [10]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion.The International Journal of Robotics Re- search, page 02783649241273668, 2023

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via ac- tion diffusion.The International Journal of Robotics Re- search, page 02783649241273668, 2023. 2

    2023

  3. [11]

    Cats: Cost aggregation transformers for visual correspondence

    Seokju Cho, Sunghwan Hong, Sangryul Jeon, Yunsung Lee, Kwanghoon Sohn, and Seungryong Kim. Cats: Cost aggregation transformers for visual correspondence. Advances in Neural Information Processing Systems, 34: 9011–9023, 2021. 3

    2021

  4. [12]

    Cats++: Boosting cost aggregation with convolutions and transformers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7174–7194, 2022

    Seokju Cho, Sunghwan Hong, and Seungryong Kim. Cats++: Boosting cost aggregation with convolutions and transformers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7174–7194, 2022. 3

    2022

  5. [13]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 3, 6, 14

    2017

  6. [14]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2023

  7. [15]

    4d gaussian splatting: To- wards efficient novel view synthesis for dynamic scenes

    Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wen- zheng Chen, and Baoquan Chen. 4d gaussian splatting: To- wards efficient novel view synthesis for dynamic scenes. arXiv preprint arXiv:2402.03307, 2024. 2, 3

    2024 arXiv

  8. [16]

    Roma: Robust dense feature matching

    Johan Edstedt, Qiyu Sun, Georg B ¨okman, M ˚arten Wadenb¨ack, and Michael Felsberg. Roma: Robust dense feature matching. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 19790–19800, 2024. 17

    2024

  9. [17]

    Vivid: Video virtual try-on using diffusion models.arXiv preprint arXiv:2405.11794, 2024

    Zixun Fang, Wei Zhai, Aimin Su, Hongliang Song, Kai Zhu, Mao Wang, Yu Chen, Zhiheng Liu, Yang Cao, and Zheng-Jun Zha. Vivid: Video virtual try-on using diffusion models.arXiv preprint arXiv:2405.11794, 2024. 1

    2024 arXiv

  10. [18]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communica- tions of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communica- tions of the ACM, 24(6):381–395, 1981. 17

    1981

  11. [19]

    K- planes: Explicit radiance fields in space, time, and appear- ance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K- planes: Explicit radiance fields in space, time, and appear- ance. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 12479–12488,

  12. [20]

    Monocular dynamic view synthe- sis: A reality check.Advances in Neural Information Pro- cessing Systems, 35:33768–33780, 2022

    Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthe- sis: A reality check.Advances in Neural Information Pro- cessing Systems, 35:33768–33780, 2022. 7, 8, 18

    2022

  13. [21]

    Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024. 2, 3

    2024 arXiv

  14. [22]

    Kubric: A scalable dataset generator

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapra- gasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. InProceedings of the IEEE/CVF conference on computer vision and pattern rec...

    2022

  15. [23]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, 9 and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023. 2, 3, 5, 6, 14

    2023 arXiv

  16. [24]

    Instruct-nerf2nerf: Edit- ing 3d scenes with instructions

    Ayaan Haque, Matthew Tancik, Alexei A Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerf: Edit- ing 3d scenes with instructions. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 19740–19750, 2023. 7, 14

    2023

  17. [25]

    Cambridge university press,

    Richard Hartley and Andrew Zisserman.Multiple view ge- ometry in computer vision. Cambridge university press,

  18. [26]

    Cameractrl: En- abling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: En- abling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024. 3

    2024 arXiv

  19. [27]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 5, 14

    2022 arXiv

  20. [28]

    Denoising dif- fusion probabilistic models.Advances in neural informa- tion processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural informa- tion processing systems, 33:6840–6851, 2020. 2, 4

    2020

  21. [29]

    Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022. 1

    arXiv

  22. [30]

    Deep matching prior: Test-time optimization for dense correspondence

    Sunghwan Hong and Seungryong Kim. Deep matching prior: Test-time optimization for dense correspondence. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9907–9917, 2021. 3

    2021

  23. [31]

    Cost aggregation with 4d convolutional swin transformer for few-shot segmentation

    Sunghwan Hong, Seokju Cho, Jisu Nam, Stephen Lin, and Seungryong Kim. Cost aggregation with 4d convolutional swin transformer for few-shot segmentation. InEuropean Conference on Computer Vision, pages 108–126. Springer, 2022

    2022

  24. [32]

    Neural matching fields: Implicit representation of matching fields for visual correspondence.Advances in Neural Information Processing Systems, 35:13512–13526, 2022

    Sunghwan Hong, Jisu Nam, Seokju Cho, Susung Hong, Sangryul Jeon, Dongbo Min, and Seungryong Kim. Neural matching fields: Implicit representation of matching fields for visual correspondence.Advances in Neural Information Processing Systems, 35:13512–13526, 2022

    2022

  25. [33]

    Unifying feature and cost aggregation with transformers for semantic and visual correspondence.arXiv preprint arXiv:2403.11120, 2024

    Sunghwan Hong, Seokju Cho, Seungryong Kim, and Stephen Lin. Unifying feature and cost aggregation with transformers for semantic and visual correspondence.arXiv preprint arXiv:2403.11120, 2024. 3

    2024 arXiv

  26. [34]

    Pf3plat: Pose-free feed-forward 3d gaussian splatting.arXiv preprint arXiv:2410.22128, 2024

    Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jisang Han, Jiaolong Yang, Chong Luo, and Seungryong Kim. Pf3plat: Pose-free feed-forward 3d gaussian splatting.arXiv preprint arXiv:2410.22128, 2024. 3

    2024 arXiv

  27. [35]

    Unifying cor- respondence pose and nerf for generalized pose-free novel view synthesis

    Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jiaolong Yang, Seungryong Kim, and Chong Luo. Unifying cor- respondence pose and nerf for generalized pose-free novel view synthesis. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 20196–2...

    2024

  28. [36]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 8153–8163, 2024. 5, 14, 20

    2024

  29. [37]

    Depthcrafter: Generating consistent long depth sequences for open-world videos.arXiv preprint arXiv:2409.02095,

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xi- aodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos.arXiv preprint arXiv:2409.02095,

    arXiv

  30. [38]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  31. [39]

    Høeg, Yilun Du, and Olav Egeland

    Sigmund H. Høeg, Yilun Du, and Olav Egeland. Streaming diffusion policy: Fast policy synthesis with variable noise diffusion models, 2024. 2

    2024

  32. [40]

    Learning high fi- delity depths of dressed humans by watching social media dance videos

    Yasamin Jafarian and Hyun Soo Park. Learning high fi- delity depths of dressed humans by watching social media dance videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12753– 12762, 2021. 6, 14

    2021

  33. [41]

    Dreammotion: Space-time self-similar score distillation for zero-shot video editing.arXiv preprint arXiv:2403.12002, 2024

    Hyeonho Jeong, Jinho Chang, Geon Yeong Park, and Jong Chul Ye. Dreammotion: Space-time self-similar score distillation for zero-shot video editing.arXiv preprint arXiv:2403.12002, 2024. 6

    2024 arXiv

  34. [42]

    Re- purposing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Re- purposing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 9492...

    2024

  35. [43]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. InProceedings of the IEEE/CVF international conference on computer vision, pages 5148–5157, 2021. 20

    2021

  36. [44]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1,

  37. [45]

    Collaborative video diffusion: Consistent multi- video generation with camera control.arXiv preprint arXiv:2405.17414, 2024

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hongsheng Li, Leonidas Guibas, and Gordon Wet- zstein. Collaborative video diffusion: Consistent multi- video generation with camera control.arXiv preprint arXiv:2405.17414, 2024. 3

    2024 arXiv

  38. [46]

    Ep n p: An accurate o (n) solution to the p n p problem.In- ternational journal of computer vision, 81:155–166, 2009

    Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. Ep n p: An accurate o (n) solution to the p n p problem.In- ternational journal of computer vision, 81:155–166, 2009. 17

    2009

  39. [47]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vi- si...

    2022

  40. [48]

    Learning the depths of moving people by watching frozen people

    Zhengqi Li, Tali Dekel, Forrester Cole, Richard Tucker, Noah Snavely, Ce Liu, and William T Freeman. Learning the depths of moving people by watching frozen people. In Proceedings of the IEEE/CVF conference on computer vi- 10 sion and pattern recognition, pages 4521–4530, 2019...

    2019

  41. [49]

    Dynibar: Neural dynamic image-based rendering

    Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic image-based rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4273–4284, 2023. 2, 6, 8

    2023

  42. [50]

    Amt: All-pairs multi-field transforms for efficient frame interpolation

    Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun- Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9801–9810, 2023. 19

    2023

  43. [51]

    Lightglue: Local feature matching at light speed

    Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Pollefeys. Lightglue: Local feature matching at light speed. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17627–17638, 2023. 17

    2023

  44. [52]

    Infinite nature: Perpetual view generation of natural scenes from a single image

    Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. Infinite nature: Perpetual view generation of natural scenes from a single image. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 14458– 14467, 2021. 3

    2021

  45. [53]

    Re- conx: Reconstruct any scene from sparse views with video diffusion model.arXiv preprint arXiv:2408.16767, 2024

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Re- conx: Reconstruct any scene from sparse views with video diffusion model.arXiv preprint arXiv:2408.16767, 2024. 3

    2024 arXiv

  46. [54]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. InProceedings of the IEEE/CVF international conference on computer vi- sion, pages 9298–9309, 2023. 2, 3, 4

    2023

  47. [55]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age.arXiv preprint arXiv:2309.03453, 2023

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age.arXiv preprint arXiv:2309.03453, 2023. 3

    2023 arXiv

  48. [56]

    Consistent video depth estimation

    Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. ACM Transactions on Graphics (ToG), 39(4):71–1, 2020. 3

    2020

  49. [57]

    Mimo: Controllable character video synthesis with spatial decomposed modeling.arXiv preprint arXiv:2409.16160,

    Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video synthesis with spatial decomposed modeling.arXiv preprint arXiv:2409.16160,

    arXiv

  50. [58]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis.Communications of the ACM, 65(1):99–106, 2021. 3

    2021

  51. [59]

    T2i-adapter: Learn- ing adapters to dig out more controllable ability for text-to- image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learn- ing adapters to dig out more controllable ability for text-to- image diffusion models. InProceedings of the AAAI Con- ference on Artificial Intelligence, pages 4296–4304...

    2024

  52. [60]

    Multidiff: Consistent novel view synthesis from a single image

    Norman M ¨uller, Katja Schwarz, Barbara R ¨ossle, Lorenzo Porzi, Samuel Rota Bul `o, Matthias Nießner, and Peter Kontschieder. Multidiff: Consistent novel view synthesis from a single image. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...

    2024

  53. [61]

    Mofa-video: Control- lable image animation via generative motion field adaptions in frozen image-to-video diffusion model.arXiv preprint arXiv:2405.20222, 2024

    Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, and Yinqiang Zheng. Mofa-video: Control- lable image animation via generative motion field adaptions in frozen image-to-video diffusion model.arXiv preprint arXiv:2405.20222, 2024. 5

    2024 arXiv

  54. [62]

    Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields.arXiv preprint arXiv:2106.13228, 2021

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin- Brualla, and Steven M Seitz. Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields.arXiv preprint arXiv:2106.13228, 2021. 6, 8

    2021 arXiv

  55. [63]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10106–10116, 2024. 17

    2024

  56. [64]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15932–15942, 2023. 1

    2023

  57. [65]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational conference on machine learning,...

    2021

  58. [66]

    Blirf: Bandlimited radiance fields for dynamic scene modeling

    Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, and Anton Van Den Hengel. Blirf: Bandlimited radiance fields for dynamic scene modeling. InProceedings of the AAAI Conference on Artificial Intelligence, pages 4641– 4649, 2024. 3

    2024

  59. [67]

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocu- lar depth estimation: Mixing datasets for zero-shot cross- dataset transfer.IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 3

    2020

  60. [68]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. InProceedings of the IEEE/CVF international conference on computer vi- sion, pages 12179–12188, 2021. 3

    2021

  61. [69]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024. 17

    2024 arXiv

  62. [70]

    Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. InProceedings of the IEEE/CVF international conference on computer vis...

    2021

  63. [71]

    Geometry-free view synthesis: Transformers and no 3d pri- 11 ors

    Robin Rombach, Patrick Esser, and Bj ¨orn Ommer. Geometry-free view synthesis: Transformers and no 3d pri- 11 ors. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 14356–14366, 2021. 7, 8

    2021

  64. [72]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 6, 14

    2022

  65. [73]

    Zeronvs: Zero-shot 360-degree view synthesis from a single real im- age.arXiv preprint arXiv:2310.17994, 2023

    Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Her- rmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, et al. Zeronvs: Zero-shot 360-degree view synthesis from a single real im- age.arXiv preprint arXiv:2310.17994, 2023. 2, 3, 4

    arXiv

  66. [74]

    Superglue: Learning feature matching with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947, 2020. 17

    2020

  67. [75]

    Laion-5b: An open large-scale dataset for train- ing next generation image-text models.Advances in Neural Information Processing Systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for train- ing next generation image-text models.Advances in Neural Info...

    2022

  68. [76]

    Genwarp: Single image to novel views with semantic-preserving generative warp- ing.arXiv preprint arXiv:2405.17251, 2024

    Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Murata, Shoukang Hu, Chieh-Hsin Lai, Se- ungryong Kim, and Yuki Mitsufuji. Genwarp: Single image to novel views with semantic-preserving generative warp- ing.arXiv preprint arXiv:2405.17251, 2024. 2, 3, 5, 14

    2024 arXiv

  69. [77]

    Learning tem- porally consistent video depth from video diffusion priors

    Jiahao Shao, Yuanbo Yang, Hongyu Zhou, Youmin Zhang, Yujun Shen, Matteo Poggi, and Yiyi Liao. Learning tem- porally consistent video depth from video diffusion priors. arXiv preprint arXiv:2406.01493, 2024. 3, 17

    2024 arXiv

  70. [78]

    Human4dit: 360-degree human video gen- eration with 4d diffusion transformer.ACM Transactions on Graphics (TOG), 43(6), 2024

    Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu. Human4dit: 360-degree human video gen- eration with 4d diffusion transformer.ACM Transactions on Graphics (TOG), 43(6), 2024. 5

    2024

  71. [79]

    Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023

    Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration.arXiv preprint arXiv:2308.16512, 2023. 3

    2023 arXiv

  72. [80]

    Light field networks: Neu- ral scene representations with single-evaluation rendering

    Vincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum, and Fredo Durand. Light field networks: Neu- ral scene representations with single-evaluation rendering. Advances in Neural Information Processing Systems, 34: 19313–19325, 2021. 8

    2021

  73. [81]

    De- noising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. De- noising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020. 4, 14

    2010 arXiv

  74. [82]

    Ldm3d-vr: Latent diffusion model for 3d vr

    Gabriela Ben Melech Stan, Diana Wofk, Estelle Aflalo, Shao-Yen Tseng, Zhipeng Cai, Michael Paulitsch, and Va- sudev Lal. Ldm3d-vr: Latent diffusion model for 3d vr. arXiv preprint arXiv:2311.03226, 2023. 1

    2023 arXiv

  75. [83]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. InComputer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020. 19

    2020

  76. [84]

    Drag-a-video: Non-rigid video editing with point-based interaction.arXiv preprint arXiv:2312.02936,

    Yao Teng, Enze Xie, Yue Wu, Haoyu Han, Zhenguo Li, and Xihui Liu. Drag-a-video: Non-rigid video editing with point-based interaction.arXiv preprint arXiv:2312.02936,

    arXiv

  77. [85]

    Sparf: Neural radiance fields from sparse and noisy poses

    Prune Truong, Marie-Julie Rakotosaona, Fabian Manhardt, and Federico Tombari. Sparf: Neural radiance fields from sparse and noisy poses. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4190–4200, 2023. 3

    2023

  78. [86]

    Megascenes: Scene-level view synthesis at scale

    Joseph Tung, Gene Chou, Ruojin Cai, Guandao Yang, Kai Zhang, Gordon Wetzstein, Bharath Hariharan, and Noah Snavely. Megascenes: Scene-level view synthesis at scale. InEuropean Conference on Computer Vision, pages 197–

  79. [87]

    Generative camera dolly: Ex- treme monocular dynamic novel view synthesis.arXiv preprint arXiv:2405.14868, 2024

    Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sar- gent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, and Carl V ondrick. Generative camera dolly: Ex- treme monocular dynamic novel view synthesis.arXiv preprint arXiv:2405.14868, 2024. 2, 3, 5, 6, 7, 8, 18, 19, 20

    2024 arXiv

  80. [88]

    Diffusion priors for dynamic view synthesis from monocu- lar videos.arXiv preprint arXiv:2401.05583, 2024

    Chaoyang Wang, Peiye Zhuang, Aliaksandr Siarohin, Junli Cao, Guocheng Qian, Hsin-Ying Lee, and Sergey Tulyakov. Diffusion priors for dynamic view synthesis from monocu- lar videos.arXiv preprint arXiv:2401.05583, 2024. 3

    2024 arXiv

  81. [89]

    Imagedream: Image-prompt multi-view diffusion for 3d generation.arXiv preprint arXiv:2312.02201, 2023

    Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation.arXiv preprint arXiv:2312.02201, 2023. 3

    2023 arXiv

  82. [90]

    Shape of motion: 4d reconstruction from a single video.arXiv preprint arXiv:2407.13764, 2024

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video.arXiv preprint arXiv:2407.13764, 2024. 2, 3, 6, 7, 8, 14

    2024

  83. [91]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 20697–20709, 2024. 3, 17

    2024

  84. [92]

    Deep spatial gradient and temporal depth learning for face anti-spoofing

    Zezheng Wang, Zitong Yu, Chenxu Zhao, Xiangyu Zhu, Yunxiao Qin, Qiusheng Zhou, Feng Zhou, and Zhen Lei. Deep spatial gradient and temporal depth learning for face anti-spoofing. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5042– 50...

    2020

  85. [93]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 3

    2024

  86. [94]

    Controlling space and time with dif- fusion models.arXiv preprint arXiv:2407.07860, 2024

    Daniel Watson, Saurabh Saxena, Lala Li, Andrea Tagliasac- chi, and David J Fleet. Controlling space and time with dif- fusion models.arXiv preprint arXiv:2407.07860, 2024. 3, 5

    2024 arXiv

  87. [95]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xi- aopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xing- gang Wang. 4d gaussian splatting for real-time dynamic scene rendering. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 20310–2...

    2024

  88. [96]

    Cat4d: Create anything in 4d with multi-view video diffu- sion models.arXiv preprint arXiv:2411.18613, 2024

    Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T Barron, and Aleksander Holynski. Cat4d: Create anything in 4d with multi-view video diffu- sion models.arXiv preprint arXiv:2411.18613, 2024. 3

    2024 arXiv

  89. [97]

    Camco: Camera- controllable 3d-consistent image-to-video generation

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. Camco: Camera- controllable 3d-consistent image-to-video generation. arXiv preprint arXiv:2406.02509, 2024. 3

    2024 arXiv

  90. [98]

    Depth any video with scalable synthetic data.arXiv preprint arXiv:2410.10815, 2024

    Honghui Yang, Di Huang, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin, Wanli Ouyang, and Tong He. Depth any video with scalable synthetic data.arXiv preprint arXiv:2410.10815, 2024. 2, 3, 4, 8, 17

    2024 arXiv

  91. [99]

    Depth anything: Un- leashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Ji- ashi Feng, and Hengshuang Zhao. Depth anything: Un- leashing the power of large-scale unlabeled data. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 3

    2024

  92. [100]

    Depth any- thing v2.arXiv preprint arXiv:2406.09414, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2.arXiv preprint arXiv:2406.09414, 2024. 3, 4, 8, 17

    2024 arXiv

  93. [101]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting.arXiv preprint arXiv:2310.10642, 2023

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting.arXiv preprint arXiv:2310.10642, 2023. 3

    2023 arXiv

  94. [102]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 20331–20341, 2024. 2, 3

    2024

  95. [103]

    Cogvideox: Text-to- video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to- video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024. 3

    2024 arXiv

  96. [104]

    Searching central difference convolutional networks for face anti-spoofing

    Zitong Yu, Chenxu Zhao, Zezheng Wang, Yunxiao Qin, Zhuo Su, Xiaobai Li, Feng Zhou, and Guoying Zhao. Searching central difference convolutional networks for face anti-spoofing. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 5295–53...

    2020

  97. [105]

    Recapture: Generative video camera controls for user- provided videos using masked video fine-tuning.arXiv preprint arXiv:2411.05003, 2024

    David Junhao Zhang, Roni Paiss, Shiran Zada, Nikhil Karnad, David E Jacobs, Yael Pritch, Inbar Mosseri, Mike Zheng Shou, Neal Wadhwa, and Nataniel Ruiz. Recapture: Generative video camera controls for user- provided videos using masked video fine-tuning.arXiv preprint arXiv:24...

    2024 arXiv

  98. [106]

    Editable free-viewpoint video using a layered neu- ral representation

    Jiakai Zhang, Xinhang Liu, Xinyi Ye, Fuqiang Zhao, Yan- shun Zhang, Minye Wu, Yingliang Zhang, Jingyi Yu, and Lan Xu. Editable free-viewpoint video using a layered neu- ral representation. 2021. 2, 6, 7, 18, 20

    2021

  99. [107]

    Monst3r: A simple approach for esti- mating geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jam- pani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for esti- mating geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024. 2, 3, 4, 5, 6, 7, 8, 17, 19, 20

    arXiv

  100. [108]

    Consistent depth of moving objects in video.ACM Transactions on Graphics (ToG), 40(4):1–12, 2021

    Zhoutong Zhang, Forrester Cole, Richard Tucker, William T Freeman, and Tali Dekel. Consistent depth of moving objects in video.ACM Transactions on Graphics (ToG), 40(4):1–12, 2021. 3

    2021

  101. [109]

    Tora: Trajectory-oriented diffu- sion transformer for video generation.arXiv preprint arXiv:2407.21705, 2024

    Zhenghao Zhang, Junchao Liao, Menghao Li, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffu- sion transformer for video generation.arXiv preprint arXiv:2407.21705, 2024. 5

    2024 arXiv

  102. [110]

    Pseudo-generalized dynamic view synthesis from a video

    Xiaoming Zhao, R Alex Colburn, Fangchang Ma, Miguel ´Angel Bautista, Joshua M Susskind, and Alex Schwing. Pseudo-generalized dynamic view synthesis from a video. InThe Twelfth International Conference on Learn- ing Representations, 2024. 2, 3, 6, 7, 20

    2024

  103. [111]

    Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018. 2, 3, 6, 14, 17

    2018 arXiv

  104. [112]

    Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility.arXiv preprint arXiv:2403.12035, 2024

    Bojia Zi, Shihao Zhao, Xianbiao Qi, Jianan Wang, Yukai Shi, Qianyu Chen, Bin Liang, Kam-Fai Wong, and Lei Zhang. Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility.arXiv preprint arXiv:2403.12035, 2024. 6, 7, 8 13 Vid-CamEd...

    2024 arXiv

  105. [214]

    6, 14, 17

    Springer, 2025. 6, 14, 17

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.