Pith. sign in

REVIEW 1 major objections 5 minor 71 references

CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion

T0 review · 1 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that fine-tuning InstructPix2Pix on a single edited reference image, followed by two-stage optimization of deformable 3D Gaussians, yields consistent and precise local edits in dynamic scenes without tracking the editing…

desk verdict CTRL-D is a credible pipeline paper: one-shot IP2P personalization plus two-stage deformable 3DGS editing is new and demonstrably works on the shown scenes, but the evaluation is too thin to support the 'significantly outperforms' claim. read the letter →

arxiv 2412.01792 v1 pith:MKDAZQSK submitted 2024-12-02 cs.CV cs.GR

Kai He , Chin-Hsuan Wu , Igor Gilitschenski This is my paper · ORCID
classification cs.CVcs.GR
keywords dynamicsceneediting3DGaussiansplattingInstructPix2Pixpersonalizationdiffusionmodelfine-tuningdeformableGaussianseditedimagebufferlocaltemporalconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a dynamic 3D scene can be edited consistently and locally by first fine-tuning InstructPix2Pix on a single edited reference image, then optimizing deformable 3D Gaussians in two stages with an edited-image buffer. If true, users could edit any dynamic scene—monocular or multi-camera—by editing just one frame with any 2D tool, and the same edit would propagate across time and viewpoints without tracking the edited region. The authors show qualitative and quantitative comparisons against Instruct 4D-to-4D, reporting higher CLIP alignment, better temporal consistency, and roughly half the runtime. The paper's own limitation section concedes that the personalized editor still generates 3D-inconsistent images when editing in empty regions.

What carries the argument

The load-bearing object is the personalized InstructPix2Pix model: the original IP2P U-Net fine-tuned with a single edited reference image, a DreamBooth-style prior-preservation loss, and affine data augmentation. This model converts the 3D/4D editing task into repeated 2D editing. The second mechanism is the two-stage deformable-3D-Gaussian optimization: stage one freezes the deformation field, optimizes canonical Gaussians against the edited keyframe, and densifies Gaussians in edited regions; stage two unfreezes the deformation field and trains both it and the Gaussians on images from the edited image buffer, with a temporal consistency loss. The edited image buffer accelerates convergence and improves temporal consistency by training only on frames that have already been edited.

What would settle it

Render an edit that inserts a new object (e.g., a bag) into a region of empty space in a dynamic scene, then measure multi-view or cross-frame consistency of the rendered object; the paper's own failure case (Fig. 8) shows the bag appears inconsistently across views, so this specific experiment would falsify the general claim of consistent editing without region tracking.

Watch

Extended reading notes

Core claim

The central discovery is that a single source–edited image pair is enough to transform InstructPix2Pix into a personalized editor for a specific scene edit, and that this personalized editor can then drive a deformable-3D-Gaussian optimization to update the whole dynamic scene. The personalization uses a DreamBooth-style prior-preservation loss and affine augmentation of the single pair to avoid collapse. The scene optimization proceeds in two stages: first the canonical space is optimized with the edited keyframe while the deformation field is frozen and Gaussians are densified; then both the deformation field and Gaussians are trained on an edited image buffer, with a temporal loss between adjacent frames. The method is compatible with text-driven, image-driven, and style-transfer 2D editing, and it generalizes a learned edit (e.g., adding sunglasses to a cat) to other domains such as portraits and full-body scenes.

Load-bearing premise

The method assumes that fine-tuning InstructPix2Pix on a single edited image pair, with affine augmentation and a prior-preservation loss, teaches the model an edit that generalizes to every frame and viewpoint of the dynamic scene; the paper's own experiments show this generalization fails when the edit adds content to empty regions.

Editorial extensions

If this is right

  • Users can edit a dynamic 3D scene by editing a single frame with any 2D editing tool, including text prompts, reference images, or style transfer.
  • The personalized editor's learned edit transfers across time and viewpoints, and can even generalize to scenes outside the reference domain.
  • The two-stage optimization with an edited image buffer converges roughly twice as fast as the Instruct 4D-to-4D baseline while improving temporal consistency.
  • Local edits are preserved without modifying unrelated regions, avoiding the global changes seen in the baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's dependence on the base IP2P's ability to add content to empty regions suggests that as base 2D editors improve, the same pipeline would inherit those improvements; conversely, edits that require hallucinating new geometry remain the hard frontier.
  • Because the personalization is trained on a single pair, the approach implicitly assumes the edit can be described by a consistent prompt or style; edits that are inherently view-dependent (e.g., reflective or anisotropic materials) would likely need extra conditioning.
  • The reported runtime reduction comes largely from the edited-image buffer; a natural ablation is to measure how buffer size trades off against temporal consistency and convergence speed.
  • The generalization result (cat sunglasses to human scenes) suggests one trained personalized editor could serve as a reusable edit operator across many scenes, which could be tested as a lightweight fine-tuning service.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper proposes CTRL-D, a framework for editing dynamic 3D scenes by first fine-tuning InstructPix2Pix (IP2P) on a single source/edited image pair, then optimizing deformable 3D Gaussians in two stages while using an edited image buffer to feed the personalized editor's outputs into the scene optimization. The central claims are that this personalization lets the model learn the editing region and style from one reference image, enabling consistent and precise local edits without tracking the target region, and that the resulting method significantly outperforms the Instruct 4D-to-4D baseline. The paper presents qualitative results on monocular and multi-camera scenes, a small quantitative comparison (Table 1) on three scenes against one baseline, ablations of the image buffer and data augmentation, and a generalization experiment.

Significance. If the central claims hold, the practical value is high: turning dynamic 3D editing into a single 2D edit and propagating it across frames and views could make 4D editing accessible to non-experts and compatible with any 2D tool. The proposed personalization idea (one-shot IP2P fine-tuning with augmentation and prior preservation) and the two-stage Gaussian optimization with an edited image buffer are plausible engineering contributions that go beyond simple prompt-based baselines. The paper also provides abundant qualitative evidence and an honest limitation section. However, the significance of the claimed advantage over prior work is not yet established: the quantitative evaluation is thin, and the paper's own failure case directly touches the 'no tracking' premise. The strength of the paper is its clear pipeline and the fact that the method can be deployed with diverse 2D editing tools, but more rigorous evaluation of the editor's per-frame/per-view consistency and of the 3D optimization's behavior under inconsistent supervision is needed before the claims can be accepted.

major comments (1)
  1. [Sec. 3.3, Eq. (6)] The temporal loss L_temp is introduced after the warm-up phase with a fixed weight lambda_t = 0.001, but the paper does not report any sensitivity analysis for lambda_t, lambda_d, the edit interval (every 50 iterations), or the number of warm-up iterations. Since these hyperparameters directly control the balance between consistency and preserving the edit, the absence of an ablation leaves the robustness of the method unverified. I ask the authors to at least show the effect of varying lambda_t by one order of magnitude and to report how the edit interval affects both convergence time and final consistency, since these are central to the claimed efficiency and consistency advantages.
minor comments (5)
  1. [Related Work, Sec. 2] The citation for Imagic is incorrect: the text says 'Imagic [52]' but reference [52] is DreamBooth; Imagic is reference [25] (Kawar et al., CVPR 2023). Please fix the citation and also check the sentence 'Imagic [52] refines a single image...' against the reference list.
  2. [Sec. 3.1, Eq. (1)] The formula for alpha_i uses P' in the exponent; from the 3DGS formulation this should be the 2D covariance matrix Sigma' (or its inverse). The symbol P' is not defined in the text and is likely a typo. Please clarify the notation.
  3. [Sec. 3.3, Eq. (5)] The variable z*_t is introduced as 'the corresponding variables used in prior preservation training,' but it would be clearer to explicitly state that z*_t is the noisy latent of I*_edited at timestep t, and to define how timestep t is sampled for the prior preservation term (the expectation notation suggests it is sampled independently).
  4. [Sec. 4.1, Fig. 4] The qualitative comparison with IN4D would be more informative if the exact same camera poses and frame indices were used for both methods, and if the reference edited image were shown for each row. The current figure places the reference at the bottom-left but the viewing order is not always obvious from the text.
  5. [Sec. 4.4, Fig. 7] The generalization experiment only shows a single example (cat sunglasses to portrait and full-body scenes). To support the claim of robust generalization, the paper should include at least one more edit type and scene domain, and ideally measure the success rate of the personalized IP2P across different reference edits.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the method is an empirical optimization pipeline with external baselines and a held-out generalization test.

full rationale

The paper does not derive its central claim from a quantity defined in terms of the claim. Eq. (5) is a standard diffusion fine-tuning objective (reconstruction on the user-edited reference plus DreamBooth-style prior preservation); the target edited image is an external input, not an output of the 3D pipeline. Eq. (6) trains deformable Gaussians with reconstruction and temporal-smoothness losses against the personalized editor's outputs; the final video is therefore a reconstruction of those outputs, which is the method's intended design rather than a self-validating reduction. The VBench/DINO consistency metric is not the training loss, and the qualitative comparisons to IN4D and AnyV2V are external. The generalization test in Sec. 4.4 applies a cat-trained editor to portrait and full-body scenes, a genuine held-out test. The self-citations (Refs. [40,41] share an author) appear in related-work comparisons and are not load-bearing. The paper's own limitation in Sec. 4.5, 'our personalized IP2P still generates 3D-inconsistent images when editing in empty regions,' is an honest boundary on the generalization claim, but it is a correctness and robustness gap, not a circular step. One mild caveat: Tab. 1 CLIP scores are partly privileged because the editor is trained to imitate a text-aligned reference and the video is optimized to match that editor; this weakens the metric's independence but is not a by-construction equivalence. Overall, no circularity is exhibited.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on several untested domain assumptions: that one edited image can teach the editing operation, that deformable 3DGS reconstruction is accurate enough to carry the edit, that the edited-image buffer and temporal loss enforce consistency, and that CLIP/VBench metrics capture editing quality. These are natural assumptions for a method paper, but they are not independently validated here, and the paper's limitation section documents cases where they fail. The explicit hyperparameters (λ_d, λ_t, iteration counts) are hand-set; several others are not reported.

free parameters (6)
  • λ_d (D-SSIM loss weight) = 0.2
    Chosen hyperparameter in Eq. 6 for the 3DGS color loss; not fitted to data, but hand-set and central to optimization.
  • λ_t (temporal loss weight) = 0.001
    Chosen hyperparameter in Eq. 6; controls temporal consistency regularization.
  • λ (prior preservation loss weight) = not specified in text
    Weight of the prior preservation term in Eq. 5; value is not reported, so exact reproduction requires guessing.
  • Stage-1 iteration count = 300 (monocular), 100 (multi-camera)
    Number of iterations for keyframe-guided densification before Stage 2; chosen by hand.
  • Edit interval = every 50 iterations
    Frequency of re-editing frames with the personalized IP2P during scene optimization; chosen by hand.
  • Data augmentation transform ranges = unspecified
    Rotation, translation, shear ranges for augmenting the single training pair are not given; critical for the claimed stability of fine-tuning.
assumptions (6)
  • domain assumption Deformable 3D Gaussian Splatting with a canonical space and a deformation MLP can represent the dynamic scenes used in the experiments.
    Inherited from [63, 67]; if the reconstruction backbone is inaccurate, edits are blurry (as the paper's own Sec 4.5 shows).
  • domain assumption A single source-edited image pair, plus affine augmentation and prior preservation loss, is sufficient to fine-tune InstructPix2Pix so that it generalizes the editing operation to other frames and views.
    This is the paper's core premise (Sec 3.2); the limitations in Sec 4.5 show it fails for edits in empty regions, so it is load-bearing and not always satisfied.
  • domain assumption Gaussian densification guided by positional gradients in Stage 1 provides sufficient geometry in edited regions.
    Proposed in Sec 3.3 without a quantitative ablation of densification quality.
  • domain assumption Training only on the edited image buffer, rather than all frames, accelerates convergence and improves temporal consistency.
    Ablation in Sec 4.3 shows the buffer matters, but the mechanism is not analyzed; the description of which frame is selected is contradictory.
  • domain assumption CLIP score and VBench DINO-consistency are valid proxies for editing fidelity and temporal consistency.
    Used in Sec 4.2 for quantitative evaluation; these are standard but not ground-truth measures.
  • domain assumption GPT-4V-generated text instructions correctly describe the intended edit.
    Used in Sec 3.2 to form the fine-tuning prompt; no validation of prompt accuracy is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion." pith.science (2026). https://pith.science/paper/MKDAZQSK

@misc{pith2026241201792,
  author       = {Pith},
  title        = {Pith review of: CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MKDAZQSK}},
  note         = {Machine review of arXiv:2412.01792}
}
read the original abstract

Recent advances in 3D representations, such as Neural Radiance Fields and 3D Gaussian Splatting, have greatly improved realistic scene modeling and novel-view synthesis. However, achieving controllable and consistent editing in dynamic 3D scenes remains a significant challenge. Previous work is largely constrained by its editing backbones, resulting in inconsistent edits and limited controllability. In our work, we introduce a novel framework that first fine-tunes the InstructPix2Pix model, followed by a two-stage optimization of the scene based on deformable 3D Gaussians. Our fine-tuning enables the model to "learn" the editing ability from a single edited reference image, transforming the complex task of dynamic scene editing into a simple 2D image editing process. By directly learning editing regions and styles from the reference, our approach enables consistent and precise local edits without the need for tracking desired editing regions, effectively addressing key challenges in dynamic scene editing. Then, our two-stage optimization progressively edits the trained dynamic scene, using a designed edited image buffer to accelerate convergence and improve temporal consistency. Compared to state-of-the-art methods, our approach offers more flexible and controllable local scene editing, achieving high-quality and consistent results.

Figures

Figures reproduced from arXiv: 2412.01792 by the authors.

Figure 1
Figure 1. We present CTRL-D, a dynamic 3D scene editing framework that enables controllable, high-quality, consistent scene edits by editing only a single image using any 2D editing approach. Our framework is also compatible with both monocular and multi-camera scenes. Please refer to our project page for dynamic visualizations. Abstract Recent advances in 3D representations, such as Neural Radiance Fields and 3D Gaussian Spl… view at source ↗
Figure 2
Figure 2. Our pipeline for controllable dynamic scene editing. Given a dynamic 3D scene, our method (a) first edits one frame as a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results on both monocular and multi-camera scenes. For each scene, we show two edited versions based on the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison with Instruct 4D-to-4D (IN4D) [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison with AnyV2V [30] on monocular scenes. The leftmost image is the edited first frame. The prompt for our fine-tuned IP2P is “Put him in a <V> suit”. Our results demonstrate higher quality and greater consistency. 4.2. Quantitative Evaluation We pre…
Figure 8
Figure 8. Figure 8: Examples of failure cases. The reconstruction backbone [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 7
Figure 7. Figure 7: Example of editing ability generalization. We demon [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 45 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  2. [2]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In CVPR, 2022

    2022

  3. [3]

    Sine: Semantic-driven image-based nerf editing with prior-guided editing field

    Chong Bao, Yinda Zhang, Bangbang Yang, Tianxing Fan, Zesong Yang, Hujun Bao, Guofeng Zhang, and Zhaopeng Cui. Sine: Semantic-driven image-based nerf editing with prior-guided editing field. In CVPR, 2023

    work page 2023

  4. [4]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In CVPR, 2023

    2023

  5. [5]

    Hexplane: A fast representa- tion for dynamic scenes

    Ang Cao and Justin Johnson. Hexplane: A fast representa- tion for dynamic scenes. In CVPR, 2023

    work page 2023

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In ICCV, 2021

    work page 2021

  7. [7]

    Duygu Ceylan, Chun-Hao Huang, and Niloy J. Mitra. Pix2video: Video editing using image diffusion. In ICCV, 2023

    work page 2023

  8. [8]

    NeuralEd- itor: Editing neural radiance fields via manipulating point clouds

    Jun-Kun Chen, Jipeng Lyu, and Yu-Xiong Wang. NeuralEd- itor: Editing neural radiance fields via manipulating point clouds. In CVPR, 2023

    work page 2023

Show all 71 references
  1. [9]

    Zero-shot image editing with reference imitation

    Xi Chen, Yutong Feng, Mengting Chen, Yiyang Wang, Shi- long Zhang, Yu Liu, Yujun Shen, and Hengshuang Zhao. Zero-shot image editing with reference imitation. arXiv preprint arXiv:2406.07547, 2024

    2024 arXiv

  2. [10]

    Unictrl: Improv- ing the spatiotemporal consistency of text-to-video diffu- sion models via training-free unified attention control

    Xuweiyi Chen, Tian Xia, and Sihan Xu. Unictrl: Improv- ing the spatiotemporal consistency of text-to-video diffu- sion models via training-free unified attention control. arXiv preprint arXiv:2403.02332, 2024

    2024 arXiv

  3. [11]

    Neumesh: Learning disentangled neural mesh-based implicit field for geometry and texture editing

    Chong Bao and Bangbang Yang, Zeng Junyi, Bao Hujun, Zhang Yinda, Cui Zhaopeng, and Zhang Guofeng. Neumesh: Learning disentangled neural mesh-based implicit field for geometry and texture editing. In ECCV, 2022

    2022

  4. [12]

    Diffedit: Diffusion-based seman- tic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based seman- tic image editing with mask guidance. arXiv preprint arXiv:2210.11427, 2022

    2022 arXiv

  5. [13]

    Drea- martist: Towards controllable one-shot text-to-image gen- eration via positive-negative prompt-tuning

    Ziyi Dong, Pengxu Wei, and Liang Lin. Drea- martist: Towards controllable one-shot text-to-image gen- eration via positive-negative prompt-tuning. arXiv preprint arXiv:2211.11337, 2022

    2022 arXiv

  6. [14]

    Gaussianeditor: Editing 3d gaussians delicately with text instructions

    Jiemin Fang, Junjie Wang, Xiaopeng Zhang, Lingxi Xie, and Qi Tian. Gaussianeditor: Editing 3d gaussians delicately with text instructions. In CVPR, 2024

    2024

  7. [15]

    An image is worth one word: Personalizing text-to-image gen- eration using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image gen- eration using textual inversion. In ICLR, 2023

    2023

  8. [16]

    Dynamic view synthesis from dynamic monocular video

    Chen Gao, Ayush Saraf, Johannes Kopf, and Jia-Bin Huang. Dynamic view synthesis from dynamic monocular video. In ICCV, 2021

    2021

  9. [17]

    Monocular dynamic view synthesis: A reality check

    Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: A reality check. In NeurIPS, 2022

    2022

  10. [18]

    Textdeformer: Geometry manipu- lation using text guidance

    William Gao, Noam Aigerman, Groueix Thibault, Vladimir Kim, and Rana Hanocka. Textdeformer: Geometry manipu- lation using text guidance. In ACM ToG, 2023

    2023

  11. [19]

    Videoswap: Customized video subject swapping with interactive semantic point cor- respondence

    Yuchao Gu, Yipin Zhou, Bichen Wu, Licheng Yu, Jia-Wei Liu, Rui Zhao, Jay Zhangjie Wu, David Junhao Zhang, Mike Zheng Shou, and Kevin Tang. Videoswap: Customized video subject swapping with interactive semantic point cor- respondence. In CVPR, 2024

    2024

  12. [20]

    Instruct-nerf2nerf: Edit- ing 3d scenes with instructions

    Ayaan Haque, Matthew Tancik, Alexei Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerf: Edit- ing 3d scenes with instructions. In ICCV, 2023

    2023

  13. [21]

    Cus- tomize your nerf: Adaptive source driven 3d scene editing via local-global iterative training

    Runze He, Shaofei Huang, Xuecheng Nie, Tianrui Hui, Lu- oqi Liu, Jiao Dai, Jizhong Han, Guanbin Li, and Si Liu. Cus- tomize your nerf: Adaptive source driven 3d scene editing via local-global iterative training. In CVPR, 2024

    2024

  14. [22]

    Prompt-to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. In ICLR, 2023

    2023

  15. [23]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020

    2020

  16. [24]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models....

    2024

  17. [25]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, 2023

    2023

  18. [26]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. In ACM ToG, 2023

    2023

  19. [27]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  20. [28]

    Decomposing nerf for editing via feature field dis- tillation

    Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitz- mann. Decomposing nerf for editing via feature field dis- tillation. In NeurIPS, 2022

    2022

  21. [29]

    Neural neighbor style transfer

    Nicholas Kolkin, Michal Kucera, Sylvain Paris, Daniel Sykora, Eli Shechtman, and Greg Shakhnarovich. Neural neighbor style transfer. arXiv preprint arXiv:2203.13215 , 2022

    2022 arXiv

  22. [30]

    Anyv2v: A tuning-free framework for any video-to- video editing tasks

    Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free framework for any video-to- video editing tasks. arXiv preprint arXiv:2403.14468, 2024

    2024 arXiv

  23. [31]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Miroslava Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, S Lovegrove, Michael Goesele, R Newcombe, et al. Neural 3d video synthesis from multi-view video. In CVPR, 2021

    2021

  24. [32]

    Neural 3d video synthesis from multi-view video

    Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, 9 Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural 3d video synthesis from multi-view video. In CVPR, 2022

    2022

  25. [33]

    Editing conditional radiance fields

    Steven Liu, Xiuming Zhang, Zhoutong Zhang, Richard Zhang, Jun-Yan Zhu, and Bryan Russell. Editing conditional radiance fields. In ICCV, 2021

    2021

  26. [34]

    Video-p2p: Video editing with cross-attention control

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In CVPR, 2024

    2024

  27. [35]

    Object-driven one- shot fine-tuning of text-to-image diffusion with prototypical embedding

    Jianxiang Lu, Cong Xie, and Hui Guo. Object-driven one- shot fine-tuning of text-to-image diffusion with prototypical embedding. arXiv preprint arXiv:2401.15708, 2024

    2024 arXiv

  28. [36]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR, 2022

    2022

  29. [37]

    Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis. In 3DV, 2024

    2024

  30. [38]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. In ICLR, 2022

    2022

  31. [39]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020

    2020

  32. [40]

    Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Konstanti- nos G Derpanis, Jonathan Kelly, Marcus A Brubaker, Igor Gilitschenski, and Alex Levinshtein. Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields. In CVPR, 2023

    2023

  33. [41]

    Watch your steps: Local image and scene editing by text instructions

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G Derpanis, and Igor Gilitschenski. Watch your steps: Local image and scene editing by text instructions. InECCV, 2024

    2024

  34. [42]

    Instruct 4d-to-4d: Editing 4d scenes as pseudo-3d scenes using 2d diffusion

    Linzhan Mou, Jun-Kun Chen, and Yu-Xiong Wang. Instruct 4d-to-4d: Editing 4d scenes as pseudo-3d scenes using 2d diffusion. In CVPR, 2024

    2024

  35. [43]

    Glide: Towards photorealistic image genera- tion and editing with text-guided diffusion models

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image genera- tion and editing with text-guided diffusion models. InICML, 2022

    2022

  36. [44]

    Neural articulated radiance field

    Atsuhiro Noguchi, Xiao Sun, Stephen Lin, and Tatsuya Harada. Neural articulated radiance field. In ICCV, 2021

    2021

  37. [45]

    Barron, Sofien Bouaziz, Dan B Goldman, Steven M

    Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. In ICCV, 2021

    2021

  38. [46]

    Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin- Brualla, and Steven M

    Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin- Brualla, and Steven M. Seitz. Hypernerf: A higher- dimensional representation for topologically varying neural radiance fields. In ACM ToG, 2021

    2021

  39. [47]

    Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans

    Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In CVPR, 2021

    2021

  40. [48]

    D-NeRF: Neural Radiance Fields for Dynamic Scenes

    Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural Radiance Fields for Dynamic Scenes. In CVPR, 2020

    2020

  41. [49]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021

    2021

  42. [50]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  43. [51]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022

    2022

  44. [52]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2023

    2023

  45. [53]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. In NeurIPS, 2022

    2022

  46. [54]

    V ox-e: Text-guided voxel editing of 3d ob- jects

    Etai Sella, Gal Fiebelman, Peter Hedman, and Hadar Averbuch-Elor. V ox-e: Text-guided voxel editing of 3d ob- jects. In ICCV, 2023

    2023

  47. [55]

    Control4d: Efficient 4d portrait editing with text

    Ruizhi Shao, Jingxiang Sun, Cheng Peng, Zerong Zheng, Boyao Zhou, Hongwen Zhang, and Yebin Liu. Control4d: Efficient 4d portrait editing with text. In CVPR, 2024

    2024

  48. [56]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015

    2015

  49. [57]

    Nerf- player: A streamable dynamic scene representation with de- composed neural radiance fields

    Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. Nerf- player: A streamable dynamic scene representation with de- composed neural radiance fields. TVCG, 2023

    2023

  50. [58]

    Generative modeling by es- timating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by es- timating gradients of the data distribution. InNeurIPS, 2019

    2019

  51. [59]

    Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video

    Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollh¨ofer, Christoph Lassner, and Christian Theobalt. Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In ICCV, 2021

    2021

  52. [60]

    Neural trajectory fields for dynamic novel view syn- thesis

    Chaoyang Wang, Ben Eckart, Simon Lucey, and Orazio Gallo. Neural trajectory fields for dynamic novel view syn- thesis. arXiv preprint arXiv:2105.05994, 2021

    2021 arXiv

  53. [61]

    Clip-nerf: Text-and-image driven manipula- tion of neural radiance fields

    Can Wang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. Clip-nerf: Text-and-image driven manipula- tion of neural radiance fields. In CVPR, 2022

    2022

  54. [62]

    Vˆ 3: View- ing volumetric videos on mobiles via streamable 2d dynamic gaussians

    Penghao Wang, Zhirui Zhang, Liao Wang, Kaixin Yao, Siyuan Xie, Jingyi Yu, Minye Wu, and Lan Xu. Vˆ 3: View- ing volumetric videos on mobiles via streamable 2d dynamic gaussians. arXiv preprint arXiv:2409.13648, 2024. 10

    2024 arXiv

  55. [63]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR, 2024

    2024

  56. [64]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, 2023

    2023

  57. [65]

    Space-time neural irradiance fields for free-viewpoint video

    Wenqi Xian, Jia-Bin Huang, Johannes Kopf, and Changil Kim. Space-time neural irradiance fields for free-viewpoint video. In CVPR, 2021

    2021

  58. [66]

    Deforming radiance fields with cages

    Tianhan Xu and Tatsuya Harada. Deforming radiance fields with cages. In ECCV, 2022

    2022

  59. [67]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. In CVPR, 2024

    2024

  60. [68]

    Real- time photorealistic dynamic scene representation and render- ing with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real- time photorealistic dynamic scene representation and render- ing with 4d gaussian splatting. In ICLR, 2024

    2024

  61. [69]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023

    2023

  62. [70]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. In ICLR, 2024

    2024

  63. [71]

    Dreameditor: Text-driven 3d scene editing with neural fields

    Jingyu Zhuang, Chen Wang, Liang Lin, Lingjie Liu, and Guanbin Li. Dreameditor: Text-driven 3d scene editing with neural fields. In SIGGRAPH Asia, 2023. 11

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.