Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Autoregressive 'next-part' prediction generates 3D assets from text and images.

desk verdict TAR3D is a credible new triplane-token autoregressive 3D generation system, but the text-to-3D superiority claim is not quantitatively supported and the image-to-3D evidence needs hardening. read the letter →

arxiv 2412.16919 v3 pith:WVURJKX6 submitted 2024-12-22 cs.CV

classification cs.CV
keywords 3DgenerationautoregressivetransformerVQ-VAEtriplanerepresentationnext-tokenpredictiontext-to-3Dimage-to-3Dpositionalencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TAR3D proposes that 3D object generation can be treated as next-token prediction over a discrete codebook of geometric parts. The paper builds a triplane-based VQ-VAE that converts any mesh into a fixed-length sequence of codebook indices, then trains a decoder-only transformer to predict that sequence left-to-right, conditioned on an image or text prompt. It claims this autoregressive 'next-part' approach produces better geometry than multiview-reconstruction and 3D-diffusion baselines, reporting the largest margins on Chamfer Distance and F-Score. The practical payoff is that sequence length no longer grows with polygon count, making autoregressive modeling feasible for high-complexity assets.

What carries the argument

The load-bearing object is the triplane VQ-VAE: three axis-aligned 2D feature maps (XY, YZ, XZ) that compress a mesh into fixed-length discrete codes. The quantizer maps each of the $3\times32\times32$ latent vectors to one of $16{,}384$ entries, and the GPT models the index sequence in a raster-scan-within-plane, plane-adjacent order. TriPE fuses 2D and 1D rotary position embeddings so the transformer knows both where a token sits in its plane and which plane it belongs to; the decoder then upsamples to $256\times256$ and predicts occupancy, with cross-plane attention carrying the information exchange between planes.

What would settle it

Render a set of thin or highly structured objects, such as chair backs, bicycle spokes, or ropes, through the 3D VQ-VAE and compare occupancy F-Score against the continuous VAE counterpart; a large drop on those shapes would show the tokenizer is the bottleneck. An independent benchmark in which multiview or diffusion methods beat TAR3D on geometric fidelity for the same prompts would also contradict the claimed margin.

Watch

Extended reading notes

Core claim

The central claim is that a 3D shape can be encoded as a $3\times32\times32$ grid of triplane features, quantized with a $16{,}384$-entry codebook, and then generated part by part by a GPT predicting the resulting $3{,}072$-token index sequence. To keep spatial structure during autoregression, the paper introduces TriPE, which adds 2D rotary positions within each plane and 1D rotary positions across the three planes. The decoder adds feature deformation and plane-interaction attention to reconstruct fine geometry, upsampling to $256\times256$ and predicting occupancy at query points. On ShapeNet and Objaverse, the method reports Chamfer Distance $0.066$ versus $0.117$ for the best baseline and F-Score $0.303$ versus $0.226$, with visual results showing more coherent geometry than multiview and diffusion competitors.

Load-bearing premise

The whole approach depends on the fixed $3\times32\times32$ triplane codebook preserving enough geometric detail; if thin structures or fine topology are lost in quantization, no amount of sequence modeling can recover them.

Editorial extensions

If this is right

  • Sequence length for a 3D asset becomes a fixed $3{,}072$ tokens regardless of mesh complexity, so autoregressive generation can scale to industrial assets without the polygon-count blowup of mesh-face methods.
  • The same GPT with prefilling tokens handles both image and text conditioning, pointing toward a single autoregressive model for multimodal 3D generation.
  • TriPE's fused 2D and 1D positions preserves geometry that plain 1D rotary encoding loses, making position encoding a first-class factor in autoregressive 3D quality.
  • Plane-interaction attention raises reconstruction F-Score from $0.661$ to $0.822$, showing cross-plane information exchange is needed for fine-grained occupancy prediction.
  • Classifier-free guidance at scale 7.5 improves prompt alignment, so inference-time guidance transfers from image and text autoregression to 3D generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Quality gains from larger triplane grids are small ($0.066$ to $0.062$ Chamfer Distance from $3\times32\times32$ to $3\times48\times48$) while inference time doubles, suggesting the tokenizer and decoder, not sequence length, are the current quality ceiling.
  • The sequence ordering is a design choice; alternative orderings that better respect 3D locality could yield further gains, since autoregressive models are sensitive to token order.
  • Sharing a codebook with 2D image tokenizers and predicting image and 3D tokens in one sequence is a natural extension that could unify 2D and 3D generation in a single next-token model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes TAR3D, a two-stage framework for conditional 3D object generation. A 3D VQ-VAE encodes point clouds into a fixed-size triplane latent, quantizes it with a trainable codebook, and reconstructs neural occupancy fields through a decoder with a plane-interaction attention (PII) design. A decoder-only transformer, augmented with a custom triplane positional encoding (TriPE), autoregressively predicts the codebook index sequence conditioned on image or text prompts. Image-to-3D experiments on ShapeNet, Objaverse, and Google Scanned Objects report large margins over multiview-based and 3D-diffusion baselines; text-to-3D results are presented through qualitative comparisons only.

Significance. If the quantitative claims survive a more rigorous evaluation, TAR3D is a solid autoregressive alternative to diffusion-based 3D generation. The fixed-length discrete tokenization of arbitrary meshes is practically valuable, and the TriPE and PII design choices are clearly motivated and supported by ablations (Table 3, Fig. 6). The paper ships detailed implementation settings and a clear architectural description, which strengthens reproducibility. The principal weakness is that the abstract's equally prominent text-to-3D superiority claim is not quantitatively supported, and the image-to-3D comparison lacks statistical grounding.

major comments (3)
  1. [Abstract; §4.3] The abstract claims superiority in both text-to-3D and image-to-3D, but §4.3 only quantifies image-to-3D (Table 1). The text-to-3D evidence is a small set of selected qualitative examples (Fig. 5) with no metric, user study, or error bars. Because the paper's stated contribution is a unified next-token-prediction framework across both modalities, the missing quantitative text-to-3D evaluation is load-bearing; please add a quantitative comparison (e.g., CLIP-Score, user study, or a standard text-to-3D benchmark) or revise the central claim to image-to-3D only.
  2. [§4.3, Table 1] The quantitative comparison lacks statistical and protocol detail. Only single-run numbers are reported, with no standard deviation, significance test, or description of how baselines were configured (checkpoints, mesh extraction threshold, post-processing, normalization). Large margins can arise from evaluation-protocol differences, particularly when comparing normal maps rendered from meshes produced by different pipelines. Please report multiple runs or provide the evaluation code and baseline settings to rule out protocol artifacts.
  3. [§3.1, Tables 2 and 4] The paper does not analyze the relationship between tokenizer capacity and generation quality. Reconstruction F-Score is 0.822, while generation F-Score is only 0.303, and increasing triplane size from 3×32×32 to 3×48×48 changes Chamfer Distance only from 0.066 to 0.062. This suggests the autoregressive stage, not merely tokenizer capacity, is the current bottleneck; the paper should discuss this gap to calibrate the claim that the fixed triplane codebook preserves sufficient geometric detail for high-quality generation.
minor comments (6)
  1. [§3.3, Eq. (4)] The second term of the codebook loss should read ||\tilde z − sg[z_q]||_2^2; as printed it lacks the square and is dimensionally inconsistent.
  2. [§4.1] The prompt construction is described only loosely ('we adopt the rendered images and textual descriptions from ULIP'); please clarify how the 4 images and top-1 captions are selected and whether the same captions are used for all views.
  3. [Table 2] The header '3D V AE' should read '3D VAE'.
  4. [§4.2] The text says 'three multiview-based methods' but lists four (SyncDreamer, InstantMesh, OpenLRM, LGM); additionally, OpenLRM is discussed but does not appear in Table 1.
  5. [Fig. 5] The baseline is labeled 'DiffSDF' while the reference list names the method 'Diffusion-sdf'; please unify the naming.
  6. [Table 4] The heading uses inconsistent spacing ('3 ×16×16' vs '3×32×32'); please standardize and explicitly state that 3×32×32 is the default setting.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: TAR3D's generation pipeline is trained and evaluated on held-out data with independent metrics; the only mild issue is that text-to-3D superiority is asserted without quantitative benchmarks.

full rationale

The paper's central claims are empirical, not derived analytically. The 3D VQ-VAE (Section 3.1) is trained with BCE occupancy loss (Eq. 3) and codebook loss (Eq. 4); the 3D GPT (Section 3.2) is trained with a next-index cross-entropy loss (Eq. 6). Neither objective contains or is fitted to the evaluation metrics (Chamfer Distance, F-Score, PSNR, SSIM, LPIPS, CLIP-Score, Table 1). The generation results in Table 1 are measured on held-out ShapeNet/Objaverse mixed evaluation sets against ground-truth meshes, using aligned sampling in [-1,1]^3; this is an external benchmark, not a re-statement of a training target. The ablations (Tables 2-4) compare model variants under the same protocol, so improvements from PII, TriPE, and triplane size are genuine internal comparisons rather than construction identities. The only self-citation-adjacent elements are citations of the authors' prior work (e.g., AR-1-to-3 [85] in Related Work), which are contextual and not load-bearing for any derivation. A separate weakness, not circularity, is that the Abstract claims superior generation quality in text-to-3D and image-to-3D tasks, but Section 4.3 explicitly states experiments are conducted on image-to-3D tasks to ensure an accurate comparison, and the text-to-3D evidence is qualitative only (Fig. 5). This is an unverified claim, not a circular one. Similarly, the fixed 3x32x32 triplane tokenizer is a design assumption whose capacity ceilings generation performance, but this is a limitation, not a reduction of the prediction to its input. Score 1 reflects one minor over-claim (text-to-3D superiority under-quantified), which is not circular. Steps list is empty because no circular step meets the hard-evidence bar: no equation equals another by construction, no fitted parameter is renamed as a prediction, and no self-citation is used to justify the central generation claim.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard sequence-modeling assumptions, a triplane representation assumption, a data-curation assumption, and metric validity. The main free parameters are architecture and training choices, not fitted physical constants. No new physical entities are postulated.

free parameters (5)
  • Codebook size K = 16,384
    Controls the vocabulary of geometric parts; chosen by the authors, no derivation or sweep reported in Section 4.1.
  • Triplane feature map size h and w = 32x32
    Fixed latent resolution; ablated in Table 4 and selected for efficiency/quality, not derived from first principles.
  • Codebook embedding dimension dq = 8
    Dimension of each codebook embedding; chosen by hand and directly affects reconstruction fidelity.
  • Loss weights beta, lambda_rec, lambda_cb = 0.25, 1, 0.1
    Balance reconstruction and codebook losses; set by convention in Section 3.3, no sensitivity analysis reported.
  • Classifier-free guidance scale = 7.5
    Inference-time guidance scale chosen without a reported sweep; affects generation quality and prompt alignment.
assumptions (5)
  • standard math Autoregressive factorization p(s) = product p(s_t | s_<t, c) captures the joint distribution of triplane index sequences.
    The GPT loss in Eq. 6 assumes next-index prediction fully defines the sequence model; standard for transformer sequence generators.
  • domain assumption A triplane of three 2D feature maps is an expressive enough 3D representation for high-quality geometry.
    Section 3.1 relies on triplane features to encode all shape information; if the fixed 3x32x32 grid loses geometric detail, generation quality is capped.
  • domain assumption Occupancy supervision on query points is sufficient to define and reconstruct 3D geometry.
    Eq. 3 uses BCE on occupancy only; the method cannot capture geometry that is not reflected in the occupancy field.
  • domain assumption The filtered 100k Objaverse subset and ULIP prompts provide a training distribution that generalizes to the out-of-domain GSO dataset.
    Section 4.1 describes scoring normal maps and selecting about 100k objects; the validity of this curation is not independently established.
  • domain assumption Chamfer distance, F-score, PSNR, LPIPS, SSIM, and CLIP score are valid measures of 3D generation quality.
    Section 4.1 adopts these metrics without validating that they align with perceptual or functional quality of 3D assets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction." pith.science (2026). https://pith.science/paper/WVURJKX6

@misc{pith2026241216919,
  author       = {Pith},
  title        = {Pith review of: TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WVURJKX6}},
  note         = {Machine review of arXiv:2412.16919}
}
read the original abstract

We present TAR3D, a novel framework that consists of a 3D-aware Vector Quantized-Variational AutoEncoder (VQ-VAE) and a Generative Pre-trained Transformer (GPT) to generate high-quality 3D assets. The core insight of this work is to migrate the multimodal unification and promising learning capabilities of the next-token prediction paradigm to conditional 3D object generation. To achieve this, the 3D VQ-VAE first encodes a wide range of 3D shapes into a compact triplane latent space and utilizes a set of discrete representations from a trainable codebook to reconstruct fine-grained geometries under the supervision of query point occupancy. Then, the 3D GPT, equipped with a custom triplane position embedding called TriPE, predicts the codebook index sequence with prefilling prompt tokens in an autoregressive manner so that the composition of 3D geometries can be modeled part by part. Extensive experiments on ShapeNet and Objaverse demonstrate that TAR3D can achieve superior generation quality over existing methods in text-to-3D and image-to-3D tasks

Figures

Figures reproduced from arXiv: 2412.16919 by the authors.

Figure 1
Figure 1. 3D asset gallery generated by the proposed TAR3D model. We employ SyncMVD [ [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall architecture of the proposed TAR3D framework. (a) 3D VQ-VAE first encodes the point cloud uniformly sampled from [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Diagrammatic details of our TriPE designed for the posi [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visual comparisons of the 3D meshes generated by our TAR3D and recent multiview-based models, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons of our TAR3D with recent cutting-edge methods for the text-to-3D object generation task. The 3D mesh [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Ablation experiments on the effectiveness of our TriPE. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Hallo4D uses vision-language models to detect and correct spatial and temporal mistakes in AI-generated 3D and 4D content, improving consistency without retraining the base generators.

Reference graph

Works this paper leans on

88 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

  2. [2]

    Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchical neural embeddings

    Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen, and Libin Liu. Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchical neural embeddings. ACM Transactions on Graphics (TOG), 41(6):1–19, 2022. 3

  3. [3]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 1, 3

  4. [4]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 5

  5. [5]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 2

  6. [6]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 2, 5

  7. [7]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22246–22256, 2023. 1, 2, 6

  8. [8]

    Meshxl: Neural coordinate field for generative 3d foundation models

    Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Yanru Wang, Zhibin Wang, Chi Zhang, et al. Meshxl: Neural coordinate field for generative 3d foundation models. arXiv preprint arXiv:2405.20853 ,

Show all 88 references
  1. [9]

    Learn- ing to predict 3d objects with an interpolation-based differ- entiable renderer

    Wenzheng Chen, Huan Ling, Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. Learn- ing to predict 3d objects with an interpolation-based differ- entiable renderer. Advances in neural information processing systems, 32, 2019. 2

  2. [10]

    Cascade-zero123: One im- age to highly consistent 3d with self-prompted nearby views

    Yabo Chen, Jiemin Fang, Yuyang Huang, Taoran Yi, Xi- aopeng Zhang, Lingxi Xie, Xinggang Wang, Wenrui Dai, Hongkai Xiong, and Qi Tian. Cascade-zero123: One im- age to highly consistent 3d with self-prompted nearby views. arXiv preprint arXiv:2312.04424, 2023. 2

  3. [11]

    Meshanything: Artist-created mesh generation with au- toregressive transformers

    Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Ji- axiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, et al. Meshanything: Artist-created mesh generation with au- toregressive transformers. arXiv preprint arXiv:2406.10163,

  4. [12]

    Region-aware text-to-image generation via hard binding and soft refinement

    Zhennan Chen, Yajie Li, Haofan Wang, Zhibo Chen, Zhengkai Jiang, Jun Li, Qian Wang, Jian Yang, and Ying Tai. Region-aware text-to-image generation via hard binding and soft refinement. arXiv preprint arXiv:2411.06558, 2024. 2

  5. [13]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer...

  6. [14]

    3dtopia-xl: Scaling high- quality 3d asset generation via primitive diffusion

    Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, et al. 3dtopia-xl: Scaling high- quality 3d asset generation via primitive diffusion. In Pro- ceedings of the Computer Vision and Pattern Recognition...

  7. [15]

    Sdfusion: Multimodal 3d shape completion, reconstruction, and generation

    Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexan- der G Schwing, and Liang-Yan Gui. Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4456–4465, 2023. 6

  8. [16]

    Scaling instruction- finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction- finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024. 6

  9. [17]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

  10. [18]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 8

  11. [19]

    Jukebox: A gen- erative model for music

    Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A gen- erative model for music. arXiv preprint arXiv:2005.00341,

  12. [20]

    The challenge of realistic music generation: mod- elling raw audio at scale

    Sander Dieleman, Aaron Van Den Oord, and Karen Si- monyan. The challenge of realistic music generation: mod- elling raw audio at scale. Advances in neural information processing systems, 31, 2018. 3

  13. [21]

    Google scanned objects: A high- quality dataset of 3d scanned household items

    Laura Downs, Anthony Francis, Nate Koenig, Brandon Kin- man, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high- quality dataset of 3d scanned household items. In 2022 In- ternational Conference on Robotics and Automation (ICRA...

  14. [22]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3

  15. [23]

    3d-future: 3d fur- niture shape with texture

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture. International Journal of Computer Vision, 129:3313–3337, 2021. 8

  16. [24]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 4

  17. [25]

    Openlrm: Open-source large reconstruction models

    Zexin He and Tengfei Wang. Openlrm: Open-source large reconstruction models. https://github.com/ 3DTopia/OpenLRM, 2023. 2, 6

  18. [26]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 6

  19. [27]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2

  20. [28]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 4

  21. [29]

    Perceiver: General perception with iterative attention

    Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In International confer- ence on machine learning, pages 4651–4664. PMLR, 2021. 4

  22. [30]

    Jm3d & jm3d-llm: Elevating 3d rep- resentation with joint multi-modal cues

    Jiayi Ji, Haowei Wang, Changli Wu, Yiwei Ma, Xiaoshuai Sun, and Rongrong Ji. Jm3d & jm3d-llm: Elevating 3d rep- resentation with joint multi-modal cues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 2

  23. [31]

    Shap-e: Generat- ing conditional 3d implicit functions

    Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 6, 7

  24. [32]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

  25. [33]

    Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation

    Yushi Lan, Fangzhou Hong, Shuai Yang, Shangchen Zhou, Xuyi Meng, Bo Dai, Xingang Pan, and Chen Change Loy. Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation. In European Conference on Computer Vision , pages 112–130. Springer, 2025. 1, 2, 6

  26. [34]

    Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model

    Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 5

  27. [35]

    Diffusion- sdf: Text-to-shape via voxelized diffusion

    Muheng Li, Yueqi Duan, Jie Zhou, and Jiwen Lu. Diffusion- sdf: Text-to-shape via voxelized diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12642–12651, 2023. 6

  28. [36]

    Promptkd: Unsupervised prompt distillation for vision-language models

    Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, and Jian Yang. Promptkd: Unsupervised prompt distillation for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26617–26626, 2024. 3

  29. [37]

    Consistent123: One image to highly consistent 3d asset using case-aware diffusion priors

    Yukang Lin, Haonan Han, Chaoqun Gong, Zunnan Xu, Yachao Zhang, and Xiu Li. Consistent123: One image to highly consistent 3d asset using case-aware diffusion priors. arXiv preprint arXiv:2309.17261, 2023. 2

  30. [38]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems , 36, 2024. 2

  31. [39]

    Meshformer: High-quality mesh generation with 3d-guided reconstruction model.arXiv preprint arXiv:2408.10198, 2024

    Minghua Liu, Chong Zeng, Xinyue Wei, Ruoxi Shi, Ling- hao Chen, Chao Xu, Mengqi Zhang, Zhaoning Wang, Xi- aoshuai Zhang, Isabella Liu, et al. Meshformer: High-quality mesh generation with 3d-guided reconstruction model.arXiv preprint arXiv:2408.10198, 2024. 5

  32. [40]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 1, 2

  33. [41]

    Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. arXiv preprint arXiv:2309.03453, 2023. 2, 6, 7

  34. [42]

    Text-guided texturing by synchronized multi-view diffusion

    Yuxin Liu, Minshan Xie, Hanyuan Liu, and Tien-Tsin Wong. Text-guided texturing by synchronized multi-view diffusion. In SIGGRAPH Asia 2024 Conference Papers , pages 1–11,

  35. [43]

    Sgdr: Stochas- tic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochas- tic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. 5

  36. [44]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 6

  37. [45]

    Mono-internvl: Push- ing the boundaries of monolithic multimodal large language models with endogenous visual pre-training

    Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jifeng Dai, Yu Qiao, and Xizhou Zhu. Mono-internvl: Push- ing the boundaries of monolithic multimodal large language models with endogenous visual pre-training. arXiv preprint arXiv:2410.08202, 2024. 1

  38. [46]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 2

  39. [47]

    Autosdf: Shape priors for 3d comple- tion, reconstruction and generation

    Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shub- ham Tulsiani. Autosdf: Shape priors for 3d comple- tion, reconstruction and generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 306–315, 2022. 3

  40. [48]

    Instant neural graphics primitives with a mul- tiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a mul- tiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 2

  41. [49]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 1, 2

  42. [50]

    Improving language understanding by gener- ative pre-training, 2018

    Alec Radford. Improving language understanding by gener- ative pre-training, 2018. 3

  43. [51]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

  44. [52]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 3

  45. [53]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

  46. [54]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

  47. [55]

    Zero123++: a single image to consistent multi-view dif- fusion base model

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model. arXiv preprint arXiv:2310.15110, 2023. 1, 2

  48. [56]

    Meshgpt: Generating triangle meshes with decoder-only transformers

    Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Ta- tiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

  49. [57]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,

  50. [58]

    Recent advances in im- plicit representation-based 3d shape generation

    Jia-Mu Sun, Tong Wu, and Lin Gao. Recent advances in im- plicit representation-based 3d shape generation. Visual Intel- ligence, 2(1):9, 2024. 2

  51. [59]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 2, 3, 6

  52. [60]

    Auto-3d-house design from structured user requirements

    Minkui Tan, Qi Chen, Zixiong Huang, Qi Wu, Yuanqing Li, and Jiaqiu Zhou. Auto-3d-house design from structured user requirements. Machine Intelligence Research , 22(2):368– 385, 2025. 2

  53. [61]

    Fourier features let networks learn high frequency functions in low dimen- sional domains

    Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ra- mamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimen- sional domains. Advances in Neural Information ...

  54. [62]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024. 2, 6, 7

  55. [63]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3

  56. [64]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2

  57. [65]

    Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation

    Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12619–12629, 2023. 2

  58. [66]

    Nice: improving panoptic narrative detection and segmentation with cascading collab- orative learning

    Haowei Wang, Jiayi Ji, Tianyu Guo, Yilong Yang, Yiyi Zhou, Xiaoshuai Sun, and Rongrong Ji. Nice: improving panoptic narrative detection and segmentation with cascading collab- orative learning. arXiv preprint arXiv:2310.10975, 2023. 2

  59. [67]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021. 2

  60. [68]

    Rodin: A generative model for sculpting 3d digital avatars using diffusion

    Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. In Proceedings of the IEEE/CVF conference on computer vision and...

  61. [69]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 3

  62. [70]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 5

  63. [71]

    Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling

    Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Ad- vances in neural information processing systems , 29, 2016. 2

  64. [72]

    Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer

    Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. arXiv preprint arXiv:2405.14832, 2024. 1, 2, 4

  65. [73]

    Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,

  66. [74]

    A survey of deep learning-based 3d shape generation

    Qun-Ce Xu, Tai-Jiang Mu, and Yong-Liang Yang. A survey of deep learning-based 3d shape generation. Computational Visual Media, 9(3):407–442, 2023. 2

  67. [75]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference o...

  68. [76]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 3

  69. [77]

    3dilg: Ir- regular latent grids for 3d generative modeling

    Biao Zhang, Matthias Nießner, and Peter Wonka. 3dilg: Ir- regular latent grids for 3d generative modeling. Advances in Neural Information Processing Systems, 35:21871–21885,

  70. [78]

    3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023

    Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023. 2, 4

  71. [79]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 6

  72. [80]

    Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets

    Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 1, 2, 5

  73. [81]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 5

  74. [82]

    Fast- pci: Motion-structure guided fast point cloud frame interpo- lation

    Tianyu Zhang, Guocheng Qian, Jin Xie, and Yang Jian. Fast- pci: Motion-structure guided fast point cloud frame interpo- lation. In ECCV, 2024. 2

  75. [83]

    Rstnet: Captioning with adaptive attention on visual and non-visual words

    Xuying Zhang, Xiaoshuai Sun, Yunpeng Luo, Jiayi Ji, Yiyi Zhou, Yongjian Wu, Feiyue Huang, and Rongrong Ji. Rstnet: Captioning with adaptive attention on visual and non-visual words. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages ...

  76. [84]

    Temo: Towards text-driven 3d stylization for multi-object meshes

    Xuying Zhang, Bo-Wen Yin, Yuming Chen, Zheng Lin, Yun- heng Li, Qibin Hou, and Ming-Ming Cheng. Temo: Towards text-driven 3d stylization for multi-object meshes. In Pro- ceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 19531–19540, 2024. 2

  77. [85]

    Ar-1-to-3: Single image to consistent 3d object generation via next-view prediction

    Xuying Zhang, Yupeng Zhou, Kai Wang, Yikai Wang, Zhen Li, Shaohui Jiao, Daquan Zhou, Qibin Hou, and Ming- Ming Cheng. Ar-1-to-3: Single image to consistent 3d object generation via next-view prediction. arXiv preprint arXiv:2503.12929, 2025. 2

  78. [86]

    Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation

    Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in Neural Information Processing Systems, 36, 2024. 2, 6, 7

  79. [87]

    GS-ROR: 3D Gaussian splatting for reflective object relighting via sdf pri- ors

    Zuo-Liang Zhu, Beibei Wang, and Jian Yang. GS-ROR: 3D Gaussian splatting for reflective object relighting via sdf pri- ors. arXiv preprint arXiv:2406.18544, 2024. 2

  80. [88]

    Gaussian splatting with discretized sdf for relightable assets

    Zuo-Liang Zhu, Jian Yang, and Beibei Wang. Gaussian splatting with discretized sdf for relightable assets. In Pro- ceedings of IEEE International Conference on Computer Vi- sion (ICCV), 2025. 2

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.