Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A text prompt and a bounding box can now animate any captured 3D scene, without retraining or optimization.

desk verdict A clever training-free pipeline for animating existing 3DGS scenes from text, but the 3D lifting rests on an untested depth-consistency assumption and the evaluation is mostly qualitative. read the letter →

arxiv 2411.19233 v2 pith:SUREYYPA submitted 2024-11-28 cs.CV

Thomas Wimmer , Michael Oechsle , Michael Niemeyer , Federico Tombari This is my paper · ORCID
classification cs.CV
keywords 3DGaussianSplattingtext-drivenanimationvideodiffusionmodelsmulti-viewconsistencypointtrackingdepth-based2D-to-3Dliftingdeformationtransfernovelviewsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Gaussians2Life aims to make static 3D scenes movable: given a 3D Gaussian Splatting reconstruction, a text prompt describing a motion, and a bounding box around the object to animate, it produces a short animation of that object in place. The paper's central claim is that a general-purpose video diffusion model can serve as the motion generator for arbitrary scenes, provided its 2D outputs are lifted to 3D through point tracking and depth estimation rather than through score-distillation optimization. If this works, any captured 3D scene becomes animatable from a sentence, with no per-scene training and without sacrificing the original visual quality. The paper demonstrates the approach on real-world captures and reports that animating a scene takes about ten minutes.

What carries the argument

The paper's central mechanism is the 3D anchor trajectory: a sparse set of point paths in world coordinates obtained by tracking 2D points through a generated guidance video, reading their per-frame metric depth, and unprojecting with the known camera. A depth-ratio check (rejecting frames where $\max\{d_t,d_{t+1}\}/\min\{d_t,d_{t+1}\}$ exceeds $1.2$) removes tracker failures, and a per-point depth ratio at the static frame $t_0$ aligns estimated depth to the true scene depth. Multi-view consistency of the guidance itself comes from latent interpolation $z=\lambda_{\mathrm{prev}}\mathrm{enc}(v_{s-1})+(1-\lambda_{\mathrm{prev}})\mathrm{enc}(g(f)_s)$, which blends the previous video's latent with the current static rendering before decoding. Finally, weighted linear blending or Kabsch-based rigid estimation transfers anchor displacements to each 3D Gaussian, so the scene deforms coherently while its appearance stays fixed.

What would settle it

Generate two guidance videos for one scene from nearby viewpoints using the paper's latent-interpolation scheme, lift both sets of 2D tracks into 3D with the same depth alignment, and compare the 3D anchor trajectories of object points visible in both; large median per-point trajectory divergence would show that the approximate multi-view consistency the method relies on is not being delivered.

Watch

Extended reading notes

Core claim

Gaussians2Life establishes that a static Gaussian Splatting scene can be animated from a text prompt and a bounding box without per-scene optimization or diffusion-model fine-tuning. The pipeline generates a short guidance video with a text- and image-conditioned video diffusion model, makes successive viewpoint generations approximately consistent by interpolating latents between the previous video and the current static scene rendering, and then lifts the 2D motion into 3D anchor trajectories via point tracking and metric depth estimation. These trajectories are transferred to individual Gaussians with weighted linear or rigid-body motion estimation, preserving the scene's appearance while producing deformations. The paper shows this on real-world scenes and argues that, unlike prior work restricted to single objects or character priors, the approach generalizes to arbitrary object classes inside larger scenes.

Load-bearing premise

The whole pipeline depends on the idea that re-running the video generator from a nearby viewpoint will produce the same motion as the first video; the paper concedes that when the first video is poor, later videos may not fix it.

Editorial extensions

If this is right

  • Any existing 3D Gaussian Splatting reconstruction can be animated from a text prompt plus a bounding box, with no per-scene optimization and no diffusion-model fine-tuning.
  • Because only Gaussian positions, scales, and rotations are changed while appearance is preserved, the original photorealistic quality of the capture is retained in the animation.
  • The method avoids class-specific priors, so it can animate a wide variety of objects in their original scene context rather than only isolated characters or assets.
  • Each additional guidance viewpoint contributes more anchor trajectories, so the estimated 3D motion is progressively refined as more views are processed.
  • The output remains a Gaussian Splatting scene with time-dependent attributes, so the animated result can be rendered in real time from any viewpoint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that this optimization-free recipe could be added as a post-processing stage to any 3D Gaussian Splatting capture pipeline, turning every static reconstruction into an editable asset.
  • Because the quality ceiling is set by the open video diffusion model, improvements in camera control and multi-view consistency of those models should flow directly into the same anchor-trajectory pipeline without architectural changes.
  • The depth-alignment trick, using the ratio of estimated to ground-truth depth at the static frame, suggests a general recipe for lifting monocular generative 2D motion into metric 3D whenever a static reference depth is available.
  • The paper's suggested follow-up of adding an SDS refinement stage could close the known hole-filling problem, making the method suitable for scenes where moving objects vacate space.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Gaussians2Life, a training-free pipeline for animating user-selected parts of a static 3D Gaussian Splatting scene from a text prompt. The method uses an image- and text-conditioned video diffusion model (DynamiCrafter) to generate guidance videos, and improves cross-view consistency by linearly interpolating, in latent space, the previous video with the current static rendering (Eq. 1). It then lifts the 2D motion to 3D by tracking sparse points, estimating per-frame metric depth, correcting the depth of each tracked point by a single ratio computed at the static frame t0 (Eq. 2), and unprojecting the tracks to 3D (Eq. 3). The resulting anchor trajectories are propagated to all Gaussians through k-NN weighted linear or rigid (Kabsch-based) motion estimation (Eqs. 4-6). Experiments are reported on MipNeRF360 and Instruct-NeRF2NeRF scenes, with a qualitative comparison to an adapted DreamGaussian4D baseline and a single-scene quantitative ablation (Table S1).

Significance. If validated, the method would be a practical contribution to text-driven animation of existing 3DGS scenes, because it avoids per-scene optimization of a deformation field and preserves the original scene appearance. The modular use of off-the-shelf components (video diffusion, point tracking, metric depth) and the released code are strengths, as is the explicit analysis of motion-transfer options (linear vs. rigid). The main claims, however, currently rest on qualitative examples and one-scene ablations whose own metrics the authors acknowledge are insufficient; the central depth-lifting assumption is not validated directly. The paper is honest about its limitations (text conditioning failures, diffusion inconsistency, depth/tracking error compensation), but these admitted limitations directly qualify the "arbitrary scenes" claim.

major comments (4)
  1. [§3.3, Eq. (2)] The depth alignment step divides all per-frame depth values d_i,t by a single ratio d_i,t0 / d^GT_i measured at the static frame t0. This is valid only if the multiplicative error of UniDepth is temporally constant for each tracked point. Under deformation, occlusion, and the domain shift between reconstructed static scenes and video-diffusion training data (which the paper itself notes in Sec. 5 and S3), this assumption is not guaranteed; a per-frame depth error directly becomes a per-frame 3D trajectory error because the lifting is one-shot with no cross-view fusion. The paper's Limitations state that "making up for wrong depth or tracking estimations is not possible," so this assumption is load-bearing and currently unvalidated. Please add a controlled experiment with synthetic scenes of known ground-truth motion (or a real scene with known 3D motion) that quantifies the resulting 3D trajectory error, or an ablation that replaces the t0-ratio correction with per-frame alignment using a temporally consistent depth model.
  2. [§4.4 / Table S1] The quantitative evaluation is limited to a single scene (the LEGO bulldozer) and the authors explicitly state that the metrics are insufficient: the qualitatively best method does not outperform its ablations on any single metric and only ranks best on average across categories. This does not provide strong evidence for the paper's central claims of "realistic," "consistent," and "arbitrary scenes." Please add quantitative results on at least a few more scenes (even with the same caveats), or conduct a user study that compares the realism of the generated animations, or provide a downstream quantitative test (e.g., novel-view temporal consistency measured by optical-flow error against the guidance videos) that can substantiate the qualitative comparisons.
  3. [§4.1 / S1.4] Only one baseline (an adapted DreamGaussian4D without its second stage) is used, and no comparison to Animate3D is provided because its code is unavailable at the time of submission. This restricts the strength of the claim that the method "significantly outperforms" prior work. Please either clarify in the main text the specific effect of omitting the second stage of DreamGaussian4D (e.g., whether it would improve temporal consistency), or add comparisons on single-object scenes where Animate3D results are available, or at least discuss what the missing comparison means for the generalizability of the claimed advantage.
  4. [§3.2, Eq. (1)] The latent interpolation mechanism is heuristic, and the authors state that "there is no guarantee of improvements" when the first generated video is not realistic. Since this is a key component of the proposed multi-view consistency strategy, the paper should quantify how often the latent interpolation actually improves consistency across viewpoints. For example, measure the optical-flow or feature-based alignment error between generated videos from adjacent viewpoints with and without latent interpolation, or report the CLIPtemporal metric separately for different viewpoints, rather than relying only on the qualitative Fig. S6.
minor comments (4)
  1. [§2] There is a typo in the paragraph on dynamic Gaussian splatting: "attribute changes at the respective momentt for the 3D Gaussians" should read "moment".
  2. [§3] The paragraph beginning "scenario, these two components will improve each other" appears to be missing an introductory phrase such as "In this scenario."
  3. [§4.2] The optical flow visualization uses the color coding of Baker et al. [5], but this is only mentioned in the supplementary Fig. S4; please refer to the colormap in the main text as well for self-containedness.
  4. [Table S1] The "Rank over all categories" uses average ranks across categories; please clarify whether ties are broken and how the three category groups (motion amount, geometry/physics, appearance) are weighted before averaging, since the current presentation is not entirely transparent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the pipeline is training-free and self-contained, and no predicted quantity reduces to a fitted input or a self-citation chain.

full rationale

The derivation chain is self-contained. The central 3D motion is lifted from generated 2D videos by off-the-shelf point tracking and metric depth estimation, unprojected with known camera poses (Eq. 3), then transferred to Gaussians by weighted linear interpolation or Kabsch rigid alignment (Eqs. 4-6). Nothing in this chain fits a parameter to the final animation: the only alignment with scene ground truth is Eq. (2), a per-point multiplicative normalization of monocular depth to the known static depth at t0; this anchors trajectories to the input geometry but does not determine their motion, which comes from the generated video. The latent interpolation (Eq. 1) is a heuristic for multi-view consistency and is explicitly acknowledged as offering no guarantee of improvement; it is not a fitted prediction. Self-citations (e.g., RadSplat for reconstruction) are not load-bearing: the method is agnostic to the 3DGS implementation, and related-work citations do not supply a uniqueness theorem or ansatz. The stated limitations (no compensation for wrong depth/tracking, no hole filling, domain mismatch) are robustness caveats, not evidence that an output reduces to an input. Evaluation uses external CLIP and geometric regularity metrics with no fitted target; the paper even notes the qualitatively best result does not top any single metric.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters (lambda_prev, noise schedule, tracking threshold, k-NN, tau, frame count), plus assumptions about the reliability of off-the-shelf video diffusion, tracking, depth, and the proposed latent interpolation. No new physical entities are introduced.

free parameters (7)
  • Latent interpolation weight lambda_prev = decreased from 0.6 to 0.0
    Hyperparameter controlling fusion of previous video latent with static scene latent in Eq. 1; chosen by hand, affects multi-view consistency.
  • Diffusion noise level = decreased from 0.75 to 0.2
    Amount of noise added to video diffusion inputs; controls motion magnitude and alignment; chosen by hand.
  • Tracking correction depth threshold = 1.2
    Manually chosen threshold for detecting tracking errors via depth ratio between consecutive frames (Sec 3.3).
  • Number of anchor trajectories K (k-NN) = n in [50,150], increased over time
    Number of nearest anchor trajectories used in deformation transfer (Eq. 4); higher K smooths motion, lower K increases displacement; tuned.
  • Temperature tau in weighting = high (not precisely specified)
    Temperature in softmax weighting of anchor contributions (Eq. 5); chosen by hand.
  • Number of guidance video frames = 8
    Limited by memory; restricts the amount of motion that can be distilled.
  • Number of tracked points per video = 1600 (reduced to ~600 valid)
    Point tracking density for anchor trajectories.
assumptions (5)
  • domain assumption DynamiCrafter video diffusion model produces realistic motion aligned with the image condition
    The method relies on an off-the-shelf VDM to generate guidance videos with correct motion; the paper shows failure examples in Fig. S5 and states that text conditioning often does not succeed.
  • ad hoc to paper Latent interpolation (Eq. 1) yields approximately multi-view consistent videos from small viewpoint changes
    This is the paper's proposed technique; it assumes warping and latent interpolation preserve motion across views. The paper admits 'there is no guarantee of improvements' (Sec 3.2).
  • domain assumption Off-the-shelf point tracking and metric depth estimation are accurate enough, with depth ratio threshold 1.2 for error correction
    Lifting 2D to 3D relies on CoTracker and UniDepth; the correction heuristic is manual and cannot fix all errors, as noted in Limitations.
  • ad hoc to paper Deformation transfer via k-NN weighting of anchor trajectories yields plausible motion
    Motion estimation uses linear blend skinning or a rigid Kabsch fit; it assumes nearby anchors define local rigid motion and that this transfers plausibly to all Gaussians.
  • domain assumption Small baseline viewpoint sampling preserves motion consistency
    Sec S1.1 constrains viewpoint changes; optical flow warping approximates new views only for small changes, which is a stated limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes." pith.science (2026). https://pith.science/paper/SUREYYPA

@misc{pith2026241119233,
  author       = {Pith},
  title        = {Pith review of: Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SUREYYPA}},
  note         = {Machine review of arXiv:2411.19233}
}
read the original abstract

State-of-the-art novel view synthesis methods achieve impressive results for multi-view captures of static 3D scenes. However, the reconstructed scenes still lack "liveliness," a key component for creating engaging 3D experiences. Recently, novel video diffusion models generate realistic videos with complex motion and enable animations of 2D images, however they cannot naively be used to animate 3D scenes as they lack multi-view consistency. To breathe life into the static world, we propose Gaussians2Life, a method for animating parts of high-quality 3D scenes in a Gaussian Splatting representation. Our key idea is to leverage powerful video diffusion models as the generative component of our model and to combine these with a robust technique to lift 2D videos into meaningful 3D motion. We find that, in contrast to prior work, this enables realistic animations of complex, pre-existing 3D scenes and further enables the animation of a large variety of object classes, while related work is mostly focused on prior-based character animation, or single 3D objects. Our model enables the creation of consistent, immersive 3D experiences for arbitrary scenes.

Figures

Figures reproduced from arXiv: 2411.19233 by the authors.

Figure 1
Figure 1. Our proposed method Gaussians2Life preserves high visual quality of scenes while animating them according to a text prompt. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Improvement of multi-view consistency of generated videos through latent interpolation. In addition to the rendering of the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Pipeline for lifting 2D dynamics into 3D. Pre-trained [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of linear and rigid motion estimation. The [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison against ablations on the LEGO bulldozer scene for the prompt “toy bulldozer lifting its shovel.” [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Linear vs. rigid motion estimation with limited observa￾tions. When transferring mo￾tion from anchor trajecto￾ries to 3D Gaussians, linear motion estimation surpris￾ingly often results in more rigid deformation than rigid motion estimation, as seen at the back ends of …
Figure 7
Figure 7. Figure 7: Qualitative results of our method on different 3D scenes with the optical flow (see Fig. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative Physical AI in Vision: A Survey

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A structured review that categorizes physics-aware generative models in vision into explicit-simulation and implicit-learning families and proposes six integration paradigms.

Reference graph

Works this paper leans on

63 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gaussian shell maps for efficient 3d human generation

    Rameen Abdal, Wang Yifan, Zifan Shi, Yinghao Xu, Ryan Po, Zhengfei Kuang, Qifeng Chen, Dit-Yan Yeung, and Gor- don Wetzstein. Gaussian shell maps for efficient 3d human generation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page 9441–9451. IEEE, 2024. 4

    work page 2024

  2. [2]

    Sherwin Bahmani, Xian Liu, Wang Yifan, Ivan Sko- rokhodov, Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, Andrea Tagliasacchi, and David B. Lindell. TC4D: Trajectory- Conditioned Text-to-4D Generation, page 53–72. Springer Nature Switzerland, 2024. 2, 8

    work page 2024

  3. [3]

    Lin- dell

    Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B. Lin- dell. 4d-fy: Text-to-4d generation using hybrid score distilla- tion sampling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page 7996–8006. IEEE, 2024. 1, 2

    work page 2024

  4. [4]

    Vd3d: Taming large video diffu- sion transformers for 3d camera control

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siaro- hin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffu- sion transformers for 3d camera control. arXiv preprint arXiv:2407.12781, 2024. 2

    arXiv 2024

  5. [5]

    Black, J.P

    Simon Baker, Stefan Roth, Daniel Scharstein, Michael J. Black, J.P. Lewis, and Richard Szeliski. A database and eval- uation methodology for optical flow. In 2007 IEEE 11th International Conference on Computer Vision , page 1–8. IEEE, 2007. 6, 2

    work page 2007

  6. [6]

    Barron, Ben Mildenhall, Dor Verbin, Pratul P

    Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-NeRF 360: Unbounded anti-aliased neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022. 2, 6

    work page 2022

  7. [7]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 1, 2

    arXiv 2023

  8. [8]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page 22563–22575. IEEE, 2023. 1, 2

    work page 2023

Show all 63 references
  1. [9]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In 2023 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR) , page 18392–18402. IEEE, 2023. 3

    2023

  2. [10]

    Neural parametric gaussians for monocular non-rigid object reconstruction

    Devikalyan Das, Christopher Wewer, Raza Yunus, Eddy Ilg, and Jan Eric Lenssen. Neural parametric gaussians for monocular non-rigid object reconstruction. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 10715–10725. IEEE, 2024. 2

    2024

  3. [11]

    MD-splatting: Learning metric deformation from 4d gaussians in highly deformable scenes

    Bardienus P Duisterhof, Zhao Mandi, Yunchao Yao, Jia- Wei Liu, Mike Zheng Shou, Shuran Song, and Jeffrey Ich- nowski. MD-splatting: Learning metric deformation from 4d gaussians in highly deformable scenes. arXiv preprint arXiv:2312.00583, 2023. 2, 3

    2023 arXiv

  4. [12]

    Gaussianflow: Splatting gaussian dynamics for 4d content creation

    Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wen- chao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Gaussianflow: Splatting gaussian dynamics for 4d content creation. arXiv preprint arXiv:2403.12365, 2024. 2

    2024 arXiv

  5. [13]

    Efros, Aleksander Holynski, and Angjoo Kanazawa

    Ayaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerf: Edit- ing 3d scenes with instructions. In 2023 IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , page 19683–19693. IEEE, 2023. 2, 3, 6

    2023

  6. [14]

    CameraCtrl: En- abling camera control for text-to-video generation, 2024

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. CameraCtrl: En- abling camera control for text-to-video generation, 2024. 2

    2024

  7. [15]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  8. [16]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 1, 2

    arXiv

  9. [17]

    DreamPhysics: Learning physical prop- erties of dynamic 3d gaussians with video diffusion priors

    Tianyu Huang, Yihan Zeng, Hui Li, Wangmeng Zuo, and Rynson WH Lau. DreamPhysics: Learning physical prop- erties of dynamic 3d gaussians with video diffusion priors. arXiv preprint arXiv:2406.01476, 2024. 3

    2024 arXiv

  10. [18]

    SC-GS: Sparse-controlled gaussian splatting for editable dynamic scenes

    Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. SC-GS: Sparse-controlled gaussian splatting for editable dynamic scenes. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 4220–4230. IEEE, 2024. 2

    2024

  11. [19]

    Animate3d: Animating any 3d model with multi-view video diffusion

    Yanqin Jiang, Chaohui Yu, Chenjie Cao, Fan Wang, Weim- ing Hu, and Jin Gao. Animate3d: Animating any 3d model with multi-view video diffusion. arXiv preprint arXiv:2407.11398, 2024. 1, 3, 6

    2024 arXiv

  12. [20]

    Deformable 3d gaussian splatting for animat- able human avatars

    HyunJun Jung, Nikolas Brasch, Jifei Song, Eduardo Perez- Pellitero, Yiren Zhou, Zhihao Li, Nassir Navab, and Ben- jamin Busam. Deformable 3d gaussian splatting for animat- able human avatars. arXiv preprint arXiv:2312.15059, 2023. 4

    2023 arXiv

  13. [21]

    W. Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A , 32(5): 922–923, 1976. 6

    1976

  14. [22]

    Co- tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker: It is better to track together. InEuropean Conference on Computer Vision, 2024. 4

    2024

  15. [23]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4):1–14, 2023. 1, 2, 6

    2023

  16. [24]

    Col- laborative video diffusion: Consistent multi-video genera- 9 tion with camera control

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hong- sheng Li, Leonidas Guibas, and Gordon Wetzstein. Col- laborative video diffusion: Consistent multi-video genera- 9 tion with camera control. arXiv preprint arXiv:2405.17414,

    arXiv

  17. [25]

    MoSca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. MoSca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv preprint arXiv:2405.17421, 2024. 2, 4

    2024 arXiv

  18. [26]

    Gaussian-flow: 4d reconstruction with dynamic 3d gaussian particle

    Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flow: 4d reconstruction with dynamic 3d gaussian particle. arXiv preprint arXiv:2312.03431, 2023. 2

    2023 arXiv

  19. [27]

    Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models

    Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fi- dler, and Karsten Kreis. Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models. In 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), page 8576–8588. IEEE, 2024...

    2024

  20. [28]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In 2023 IEEE/CVF In- ternational Conference on Computer Vision (ICCV) , page 9264–9275. IEEE, 2023. 2, 6

    2023

  21. [29]

    Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis. In 2024 International Con- ference on 3D Vision (3DV), page 800–809. IEEE, 2024. 2, 3

    2024

  22. [30]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng.NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, page 405–421. Springer International Publishing,

  23. [31]

    Instant neural graphics primitives with a multires- olution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a multires- olution hash encoding. ACM Transactions on Graphics, 41 (4):1–15, 2022. 2

    2022

  24. [32]

    RadSplat: Radiance field-informed gaussian splat- ting for robust real-time rendering with 900+ FPS

    Michael Niemeyer, Fabian Manhardt, Marie-Julie Rakoto- saona, Michael Oechsle, Daniel Duckworth, Rama Gosula, Keisuke Tateno, John Bates, Dominik Kaeser, and Federico Tombari. RadSplat: Radiance field-informed gaussian splat- ting for robust real-time rendering with 900+ FPS. ...

    2024 arXiv

  25. [33]

    ASH: Animatable gaussian splats for efficient and photoreal human rendering

    Haokai Pang, Heming Zhu, Adam Kortylewski, Christian Theobalt, and Marc Habermann. ASH: Animatable gaussian splats for efficient and photoreal human rendering. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 4

    2024

  26. [34]

    UniDepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 10106–10116. IEEE, 2024. 4

    2024

  27. [35]

    DreamFusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. DreamFusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference on Learning Representa- tions, 2022. 2

    2022

  28. [36]

    LangSplat: 3d language gaussian splat- ting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. LangSplat: 3d language gaussian splat- ting. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 20051–20060. IEEE,

    2024

  29. [37]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  30. [38]

    Dreamgaussian4d: Genera- tive 4d gaussian splatting

    Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. Dreamgaussian4d: Genera- tive 4d gaussian splatting. arXiv preprint arXiv:2312.17142,

    arXiv

  31. [39]

    L4gm: Large 4d gaus- sian reconstruction model

    Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xi- aohui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, et al. L4gm: Large 4d gaus- sian reconstruction model. arXiv preprint arXiv:2406.10324,

    arXiv

  32. [40]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. High-resolution image synthesis with latent diffusion models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 10674–10685. IEEE, 2022. 2

    2022

  33. [41]

    SplattingAvatar: Realistic real-time human avatars with mesh-embedded gaussian splatting

    Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, and Zeyu Wang. SplattingAvatar: Realistic real-time human avatars with mesh-embedded gaussian splatting. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pag...

    2024

  34. [42]

    MVDream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3d gen- eration. In The Twelfth International Conference on Learn- ing Representations, 2023. 2

    2023

  35. [43]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  36. [44]

    Text-to-4d dy- namic scene generation

    Uriel Singer, Shelly Sheynin, Adam Polyak, Oron Ashual, Iurii Makarov, Filippos Kokkinos, Naman Goyal, Andrea Vedaldi, Devi Parikh, Justin Johnson, et al. Text-to-4d dy- namic scene generation. In Proceedings of the 40th Inter- national Conference on Machine Learning , pages 3...

    2023

  37. [45]

    Dynamic gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. arXiv preprint arXiv:2406.18717, 2024. 2, 4

    2024 arXiv

  38. [46]

    Fourier features let networks learn high frequency functions in low dimen- sional domains

    Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ra- mamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimen- sional domains. Advances in Neural Information ...

    2020

  39. [47]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024. 2 10

    2024 arXiv

  40. [48]

    RAFT: Recurrent all-pairs field transforms for optical flow (extended abstract)

    Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow (extended abstract). In Proceed- ings of the Thirtieth International Joint Conference on Arti- ficial Intelligence, page 4839–4843. International Joint Con- ferences on Artificial Intelligen...

    2021

  41. [49]

    S. Umeyama. Least-squares estimation of transformation pa- rameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence , 13(4):376–380,

  42. [50]

    Yeh, and Greg Shakhnarovich

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A. Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lift- ing pretrained 2d diffusion models for 3d generation. In 2023 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR) , page 12619–12629. IEEE, 2023. 2, 3

    2023

  43. [51]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 2

    2023 arXiv

  44. [52]

    VideoComposer: Compositional video synthe- sis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. VideoComposer: Compositional video synthe- sis with motion controllability. arXiv e-prints, pages arXiv– 2306, 2023. 2

    2023

  45. [53]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36, 2024. 2, 3

    2024

  46. [54]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR) , page 20310–20320. IEEE...

    2024

  47. [55]

    Srinivasan, Dor Verbin, Jonathan T

    Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P. Srinivasan, Dor Verbin, Jonathan T. Barron, Ben Poole, and Aleksander Hoły´nski. ReconFusion: 3d reconstruction with diffusion priors. In 2024 IEEE/CVF Conference on Computer Vision a...

    2024

  48. [56]

    PhysGaussian: Physics- integrated 3d gaussians for generative dynamics

    Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. PhysGaussian: Physics- integrated 3d gaussians for generative dynamics. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 4389–4398. IEEE, 2024. 3

    2024

  49. [57]

    Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency

    Yiming Xie, Chun-Han Yao, Vikram V oleti, Huaizu Jiang, and Varun Jampani. Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency. arXiv preprint arXiv:2407.17470, 2024. 2, 4

    2024 arXiv

  50. [58]

    DynamiCrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Xintao Wang, Tien-Tsin Wong, and Ying Shan. DynamiCrafter: Animating open-domain images with video diffusion priors. In European Conference on Computer Vision, 2024. 2, 3, 4

    2024

  51. [59]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR) , page 20331–20341. IEEE, 2024. 2

    2024

  52. [60]

    4real: Towards photorealis- tic 4d scene generation via video diffusion models

    Heng Yu, Chaoyang Wang, Peiye Zhuang, Willi Mena- pace, Aliaksandr Siarohin, Junli Cao, Laszlo A Jeni, Sergey Tulyakov, and Hsin-Ying Lee. 4real: Towards photorealis- tic 4d scene generation via video diffusion models. arXiv preprint arXiv:2406.07472, 2024. 1, 2

    2024 arXiv

  53. [61]

    Text-to-3d with classifier score distillation

    Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song- Hai Zhang, and Xiaojuan Qi. Text-to-3d with classifier score distillation. In The Thirteenth International Conference on Learning Representations, 2024. 2, 3

    2024

  54. [62]

    PhysDreamer: Physics-based interac- tion with 3d objects via video generation

    Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. PhysDreamer: Physics-based interac- tion with 3d objects via video generation. arXiv preprint arXiv:2404.13026, 2024. 3

    2024 arXiv

  55. [63]

    Bike falls over onto the green grass in the foreground

    Zhizhuo Zhou and Shubham Tulsiani. SparseFusion: Dis- tilling view-conditioned diffusion for 3d reconstruction. In 2023 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR) , page 12588–12597. IEEE, 2023. 2 11 Gaussians-to-Life: Text-Driven Animation of 3D G...

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.