pith. sign in

arxiv: 2303.13508 · v2 · pith:7MH6BZ2Unew · submitted 2023-03-23 · 💻 cs.CV · cs.AI· cs.GR

DreamBooth3D: Subject-Driven Text-to-3D Generation

Amit Raj , Srinivas Kaza , Ben Poole , Michael Niemeyer , Nataniel Ruiz , Ben Mildenhall , Shiran Zada , Kfir Aberman
show 4 more authors
Michael Rubinstein Jonathan Barron Yuanzhen Li Varun Jampani
This is my paper
classification 💻 cs.CV cs.AIcs.GR
keywords modelssubjecttext-to-3dtext-to-imageapproachassetsdreambooth3dgeneration
0
0 comments X
read the original abstract

We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject. Our approach combines recent advances in personalizing text-to-image models (DreamBooth) with text-to-3D generation (DreamFusion). We find that naively combining these methods fails to yield satisfactory subject-specific 3D assets due to personalized text-to-image models overfitting to the input viewpoints of the subject. We overcome this through a 3-stage optimization strategy where we jointly leverage the 3D consistency of neural radiance fields together with the personalization capability of text-to-image models. Our method can produce high-quality, subject-specific 3D assets with text-driven modifications such as novel poses, colors and attributes that are not seen in any of the input images of the subject.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation

    cs.CV 2023-09 unverdicted novelty 7.0

    DreamGaussian creates high-quality textured 3D meshes from single-view images in 2 minutes via generative Gaussian Splatting with mesh extraction and UV refinement.

  2. SyncDreamer: Generating Multiview-consistent Images from a Single-view Image

    cs.CV 2023-09 unverdicted novelty 6.0

    SyncDreamer produces multiview-consistent images from a single input image by jointly modeling their distribution and synchronizing intermediate diffusion states via 3D-aware attention.

  3. MVDream: Multi-view Diffusion for 3D Generation

    cs.CV 2023-08 conditional novelty 6.0

    MVDream is a multi-view diffusion model that functions as a generalizable 3D prior, enabling more consistent text-to-3D generation and few-shot 3D concept learning from 2D examples.