REVIEW 3 major objections 6 minor 1 cited by
TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction
T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Autoregressive 'next-part' prediction generates 3D assets from text and images.
desk verdict TAR3D is a credible new triplane-token autoregressive 3D generation system, but the text-to-3D superiority claim is not quantitatively supported and the image-to-3D evidence needs hardening. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the triplane VQ-VAE: three axis-aligned 2D feature maps (XY, YZ, XZ) that compress a mesh into fixed-length discrete codes. The quantizer maps each of the $3\times32\times32$ latent vectors to one of $16{,}384$ entries, and the GPT models the index sequence in a raster-scan-within-plane, plane-adjacent order. TriPE fuses 2D and 1D rotary position embeddings so the transformer knows both where a token sits in its plane and which plane it belongs to; the decoder then upsamples to $256\times256$ and predicts occupancy, with cross-plane attention carrying the information exchange between planes.
What would settle it
Render a set of thin or highly structured objects, such as chair backs, bicycle spokes, or ropes, through the 3D VQ-VAE and compare occupancy F-Score against the continuous VAE counterpart; a large drop on those shapes would show the tokenizer is the bottleneck. An independent benchmark in which multiview or diffusion methods beat TAR3D on geometric fidelity for the same prompts would also contradict the claimed margin.
Extended reading notes
Core claim
The central claim is that a 3D shape can be encoded as a $3\times32\times32$ grid of triplane features, quantized with a $16{,}384$-entry codebook, and then generated part by part by a GPT predicting the resulting $3{,}072$-token index sequence. To keep spatial structure during autoregression, the paper introduces TriPE, which adds 2D rotary positions within each plane and 1D rotary positions across the three planes. The decoder adds feature deformation and plane-interaction attention to reconstruct fine geometry, upsampling to $256\times256$ and predicting occupancy at query points. On ShapeNet and Objaverse, the method reports Chamfer Distance $0.066$ versus $0.117$ for the best baseline and F-Score $0.303$ versus $0.226$, with visual results showing more coherent geometry than multiview and diffusion competitors.
Load-bearing premise
The whole approach depends on the fixed $3\times32\times32$ triplane codebook preserving enough geometric detail; if thin structures or fine topology are lost in quantization, no amount of sequence modeling can recover them.
Editorial extensions
If this is right
- Sequence length for a 3D asset becomes a fixed $3{,}072$ tokens regardless of mesh complexity, so autoregressive generation can scale to industrial assets without the polygon-count blowup of mesh-face methods.
- The same GPT with prefilling tokens handles both image and text conditioning, pointing toward a single autoregressive model for multimodal 3D generation.
- TriPE's fused 2D and 1D positions preserves geometry that plain 1D rotary encoding loses, making position encoding a first-class factor in autoregressive 3D quality.
- Plane-interaction attention raises reconstruction F-Score from $0.661$ to $0.822$, showing cross-plane information exchange is needed for fine-grained occupancy prediction.
- Classifier-free guidance at scale 7.5 improves prompt alignment, so inference-time guidance transfers from image and text autoregression to 3D generation.
Reading between the lines
- Quality gains from larger triplane grids are small ($0.066$ to $0.062$ Chamfer Distance from $3\times32\times32$ to $3\times48\times48$) while inference time doubles, suggesting the tokenizer and decoder, not sequence length, are the current quality ceiling.
- The sequence ordering is a design choice; alternative orderings that better respect 3D locality could yield further gains, since autoregressive models are sensitive to token order.
- Sharing a codebook with 2D image tokenizers and predicting image and 3D tokens in one sequence is a natural extension that could unify 2D and 3D generation in a single next-token model.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TAR3D, a two-stage framework for conditional 3D object generation. A 3D VQ-VAE encodes point clouds into a fixed-size triplane latent, quantizes it with a trainable codebook, and reconstructs neural occupancy fields through a decoder with a plane-interaction attention (PII) design. A decoder-only transformer, augmented with a custom triplane positional encoding (TriPE), autoregressively predicts the codebook index sequence conditioned on image or text prompts. Image-to-3D experiments on ShapeNet, Objaverse, and Google Scanned Objects report large margins over multiview-based and 3D-diffusion baselines; text-to-3D results are presented through qualitative comparisons only.
Significance. If the quantitative claims survive a more rigorous evaluation, TAR3D is a solid autoregressive alternative to diffusion-based 3D generation. The fixed-length discrete tokenization of arbitrary meshes is practically valuable, and the TriPE and PII design choices are clearly motivated and supported by ablations (Table 3, Fig. 6). The paper ships detailed implementation settings and a clear architectural description, which strengthens reproducibility. The principal weakness is that the abstract's equally prominent text-to-3D superiority claim is not quantitatively supported, and the image-to-3D comparison lacks statistical grounding.
major comments (3)
- [Abstract; §4.3] The abstract claims superiority in both text-to-3D and image-to-3D, but §4.3 only quantifies image-to-3D (Table 1). The text-to-3D evidence is a small set of selected qualitative examples (Fig. 5) with no metric, user study, or error bars. Because the paper's stated contribution is a unified next-token-prediction framework across both modalities, the missing quantitative text-to-3D evaluation is load-bearing; please add a quantitative comparison (e.g., CLIP-Score, user study, or a standard text-to-3D benchmark) or revise the central claim to image-to-3D only.
- [§4.3, Table 1] The quantitative comparison lacks statistical and protocol detail. Only single-run numbers are reported, with no standard deviation, significance test, or description of how baselines were configured (checkpoints, mesh extraction threshold, post-processing, normalization). Large margins can arise from evaluation-protocol differences, particularly when comparing normal maps rendered from meshes produced by different pipelines. Please report multiple runs or provide the evaluation code and baseline settings to rule out protocol artifacts.
- [§3.1, Tables 2 and 4] The paper does not analyze the relationship between tokenizer capacity and generation quality. Reconstruction F-Score is 0.822, while generation F-Score is only 0.303, and increasing triplane size from 3×32×32 to 3×48×48 changes Chamfer Distance only from 0.066 to 0.062. This suggests the autoregressive stage, not merely tokenizer capacity, is the current bottleneck; the paper should discuss this gap to calibrate the claim that the fixed triplane codebook preserves sufficient geometric detail for high-quality generation.
minor comments (6)
- [§3.3, Eq. (4)] The second term of the codebook loss should read ||\tilde z − sg[z_q]||_2^2; as printed it lacks the square and is dimensionally inconsistent.
- [§4.1] The prompt construction is described only loosely ('we adopt the rendered images and textual descriptions from ULIP'); please clarify how the 4 images and top-1 captions are selected and whether the same captions are used for all views.
- [Table 2] The header '3D V AE' should read '3D VAE'.
- [§4.2] The text says 'three multiview-based methods' but lists four (SyncDreamer, InstantMesh, OpenLRM, LGM); additionally, OpenLRM is discussed but does not appear in Table 1.
- [Fig. 5] The baseline is labeled 'DiffSDF' while the reference list names the method 'Diffusion-sdf'; please unify the naming.
- [Table 4] The heading uses inconsistent spacing ('3 ×16×16' vs '3×32×32'); please standardize and explicitly state that 3×32×32 is the default setting.
Circularity Check
No significant circularity: TAR3D's generation pipeline is trained and evaluated on held-out data with independent metrics; the only mild issue is that text-to-3D superiority is asserted without quantitative benchmarks.
full rationale
The paper's central claims are empirical, not derived analytically. The 3D VQ-VAE (Section 3.1) is trained with BCE occupancy loss (Eq. 3) and codebook loss (Eq. 4); the 3D GPT (Section 3.2) is trained with a next-index cross-entropy loss (Eq. 6). Neither objective contains or is fitted to the evaluation metrics (Chamfer Distance, F-Score, PSNR, SSIM, LPIPS, CLIP-Score, Table 1). The generation results in Table 1 are measured on held-out ShapeNet/Objaverse mixed evaluation sets against ground-truth meshes, using aligned sampling in [-1,1]^3; this is an external benchmark, not a re-statement of a training target. The ablations (Tables 2-4) compare model variants under the same protocol, so improvements from PII, TriPE, and triplane size are genuine internal comparisons rather than construction identities. The only self-citation-adjacent elements are citations of the authors' prior work (e.g., AR-1-to-3 [85] in Related Work), which are contextual and not load-bearing for any derivation. A separate weakness, not circularity, is that the Abstract claims superior generation quality in text-to-3D and image-to-3D tasks, but Section 4.3 explicitly states experiments are conducted on image-to-3D tasks to ensure an accurate comparison, and the text-to-3D evidence is qualitative only (Fig. 5). This is an unverified claim, not a circular one. Similarly, the fixed 3x32x32 triplane tokenizer is a design assumption whose capacity ceilings generation performance, but this is a limitation, not a reduction of the prediction to its input. Score 1 reflects one minor over-claim (text-to-3D superiority under-quantified), which is not circular. Steps list is empty because no circular step meets the hard-evidence bar: no equation equals another by construction, no fitted parameter is renamed as a prediction, and no self-citation is used to justify the central generation claim.
Assumptions & free parameters
free parameters (5)
- Codebook size K =
16,384
- Triplane feature map size h and w =
32x32
- Codebook embedding dimension dq =
8
- Loss weights beta, lambda_rec, lambda_cb =
0.25, 1, 0.1
- Classifier-free guidance scale =
7.5
assumptions (5)
- standard math Autoregressive factorization p(s) = product p(s_t | s_<t, c) captures the joint distribution of triplane index sequences.
- domain assumption A triplane of three 2D feature maps is an expressive enough 3D representation for high-quality geometry.
- domain assumption Occupancy supervision on query points is sufficient to define and reconstruct 3D geometry.
- domain assumption The filtered 100k Objaverse subset and ULIP prompts provide a training distribution that generalizes to the out-of-domain GSO dataset.
- domain assumption Chamfer distance, F-score, PSNR, LPIPS, SSIM, and CLIP score are valid measures of 3D generation quality.
Cite this review
Pith. "Pith review of TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction." pith.science (2026). https://pith.science/paper/WVURJKX6
@misc{pith2026241216919,
author = {Pith},
title = {Pith review of: TAR3D: Creating High-Quality 3D Assets via Next-Part Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/WVURJKX6}},
note = {Machine review of arXiv:2412.16919}
}
read the original abstract
We present TAR3D, a novel framework that consists of a 3D-aware Vector Quantized-Variational AutoEncoder (VQ-VAE) and a Generative Pre-trained Transformer (GPT) to generate high-quality 3D assets. The core insight of this work is to migrate the multimodal unification and promising learning capabilities of the next-token prediction paradigm to conditional 3D object generation. To achieve this, the 3D VQ-VAE first encodes a wide range of 3D shapes into a compact triplane latent space and utilizes a set of discrete representations from a trainable codebook to reconstruct fine-grained geometries under the supervision of query point occupancy. Then, the 3D GPT, equipped with a custom triplane position embedding called TriPE, predicts the codebook index sequence with prefilling prompt tokens in an autoregressive manner so that the composition of 3D geometries can be modeled part by part. Extensive experiments on ShapeNet and Objaverse demonstrate that TAR3D can achieve superior generation quality over existing methods in text-to-3D and image-to-3D tasks
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation
Hallo4D uses vision-language models to detect and correct spatial and temporal mistakes in AI-generated 3D and 4D content, improving consistency without retraining the base generators.
Reference graph
Works this paper leans on
-
[1]
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,
-
[2]
Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchical neural embeddings
Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen, and Libin Liu. Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchical neural embeddings. ACM Transactions on Graphics (TOG), 41(6):1–19, 2022. 3
2022
-
[3]
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 1, 3
arXiv 2023
-
[4]
Estimating or propagating gradients through stochastic neurons for conditional computation
Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 5
arXiv 2013
-
[5]
Efficient geometry-aware 3d generative adversarial networks
Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 2
2022
-
[6]
Shapenet: An information-rich 3d model repository
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015. 2, 5
arXiv 2015
-
[7]
Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22246–22256, 2023. 1, 2, 6
2023
-
[8]
Meshxl: Neural coordinate field for generative 3d foundation models
Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Yanru Wang, Zhibin Wang, Chi Zhang, et al. Meshxl: Neural coordinate field for generative 3d foundation models. arXiv preprint arXiv:2405.20853 ,
Show all 88 references
-
[9]
Learn- ing to predict 3d objects with an interpolation-based differ- entiable renderer
Wenzheng Chen, Huan Ling, Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. Learn- ing to predict 3d objects with an interpolation-based differ- entiable renderer. Advances in neural information processing systems, 32, 2019. 2
2019
-
[10]
Cascade-zero123: One im- age to highly consistent 3d with self-prompted nearby views
Yabo Chen, Jiemin Fang, Yuyang Huang, Taoran Yi, Xi- aopeng Zhang, Lingxi Xie, Xinggang Wang, Wenrui Dai, Hongkai Xiong, and Qi Tian. Cascade-zero123: One im- age to highly consistent 3d with self-prompted nearby views. arXiv preprint arXiv:2312.04424, 2023. 2
2023 arXiv
-
[11]
Meshanything: Artist-created mesh generation with au- toregressive transformers
Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Ji- axiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, et al. Meshanything: Artist-created mesh generation with au- toregressive transformers. arXiv preprint arXiv:2406.10163,
-
[12]
Region-aware text-to-image generation via hard binding and soft refinement
Zhennan Chen, Yajie Li, Haofan Wang, Zhibo Chen, Zhengkai Jiang, Jun Li, Qian Wang, Jian Yang, and Ying Tai. Region-aware text-to-image generation via hard binding and soft refinement. arXiv preprint arXiv:2411.06558, 2024. 2
2024 arXiv
-
[13]
Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer...
2024
-
[14]
3dtopia-xl: Scaling high- quality 3d asset generation via primitive diffusion
Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, et al. 3dtopia-xl: Scaling high- quality 3d asset generation via primitive diffusion. In Pro- ceedings of the Computer Vision and Pattern Recognition...
2025
-
[15]
Sdfusion: Multimodal 3d shape completion, reconstruction, and generation
Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexan- der G Schwing, and Liang-Yan Gui. Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4456–4465, 2023. 6
2023
-
[16]
Scaling instruction- finetuned language models
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction- finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024. 6
2024
-
[17]
Objaverse: A universe of annotated 3d objects
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...
2023
-
[18]
Objaverse-xl: A universe of 10m+ 3d objects
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 8
2024
-
[19]
Jukebox: A gen- erative model for music
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A gen- erative model for music. arXiv preprint arXiv:2005.00341,
2005 arXiv
-
[20]
The challenge of realistic music generation: mod- elling raw audio at scale
Sander Dieleman, Aaron Van Den Oord, and Karen Si- monyan. The challenge of realistic music generation: mod- elling raw audio at scale. Advances in neural information processing systems, 31, 2018. 3
2018
-
[21]
Google scanned objects: A high- quality dataset of 3d scanned household items
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kin- man, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high- quality dataset of 3d scanned household items. In 2022 In- ternational Conference on Robotics and Automation (ICRA...
2022
-
[22]
Taming transformers for high-resolution image synthesis
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3
2021
-
[23]
3d-future: 3d fur- niture shape with texture
Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture. International Journal of Computer Vision, 129:3313–3337, 2021. 8
2021
-
[24]
Animatediff: Animate your personalized text- to-image diffusion models without specific tuning
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 4
2023 arXiv
-
[25]
Openlrm: Open-source large reconstruction models
Zexin He and Tengfei Wang. Openlrm: Open-source large reconstruction models. https://github.com/ 3DTopia/OpenLRM, 2023. 2, 6
2023
-
[26]
Classifier-free diffusion guidance
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 6
2022 arXiv
-
[27]
Lrm: Large reconstruction model for single image to 3d
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 2
2023 arXiv
-
[28]
Animate anyone: Consistent and controllable image- to-video synthesis for character animation
Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 4
2024
-
[29]
Perceiver: General perception with iterative attention
Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In International confer- ence on machine learning, pages 4651–4664. PMLR, 2021. 4
2021
-
[30]
Jm3d & jm3d-llm: Elevating 3d rep- resentation with joint multi-modal cues
Jiayi Ji, Haowei Wang, Changli Wu, Yiwei Ma, Xiaoshuai Sun, and Rongrong Ji. Jm3d & jm3d-llm: Elevating 3d rep- resentation with joint multi-modal cues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 2
2024
-
[31]
Shap-e: Generat- ing conditional 3d implicit functions
Heewoo Jun and Alex Nichol. Shap-e: Generat- ing conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023. 6, 7
2023 arXiv
-
[32]
Auto-encoding variational bayes
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2
2013 arXiv
-
[33]
Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation
Yushi Lan, Fangzhou Hong, Shuai Yang, Shangchen Zhou, Xuyi Meng, Bo Dai, Xingang Pan, and Chen Change Loy. Ln3diff: Scalable latent neural fields diffusion for speedy 3d generation. In European Conference on Computer Vision , pages 112–130. Springer, 2025. 1, 2, 6
2025
-
[34]
Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 5
2023 arXiv
-
[35]
Diffusion- sdf: Text-to-shape via voxelized diffusion
Muheng Li, Yueqi Duan, Jie Zhou, and Jiwen Lu. Diffusion- sdf: Text-to-shape via voxelized diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12642–12651, 2023. 6
2023
-
[36]
Promptkd: Unsupervised prompt distillation for vision-language models
Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, and Jian Yang. Promptkd: Unsupervised prompt distillation for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26617–26626, 2024. 3
2024
-
[37]
Consistent123: One image to highly consistent 3d asset using case-aware diffusion priors
Yukang Lin, Haonan Han, Chaoqun Gong, Zunnan Xu, Yachao Zhang, and Xiu Li. Consistent123: One image to highly consistent 3d asset using case-aware diffusion priors. arXiv preprint arXiv:2309.17261, 2023. 2
2023 arXiv
-
[38]
One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion
Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. Advances in Neural Information Processing Systems , 36, 2024. 2
2024
-
[39]
Meshformer: High-quality mesh generation with 3d-guided reconstruction model.arXiv preprint arXiv:2408.10198, 2024
Minghua Liu, Chong Zeng, Xinyue Wei, Ruoxi Shi, Ling- hao Chen, Chao Xu, Mengqi Zhang, Zhaoning Wang, Xi- aoshuai Zhang, Isabella Liu, et al. Meshformer: High-quality mesh generation with 3d-guided reconstruction model.arXiv preprint arXiv:2408.10198, 2024. 5
2024 arXiv
-
[40]
Zero-1-to- 3: Zero-shot one image to 3d object
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023. 1, 2
2023
-
[41]
Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age. arXiv preprint arXiv:2309.03453, 2023. 2, 6, 7
2023 arXiv
-
[42]
Text-guided texturing by synchronized multi-view diffusion
Yuxin Liu, Minshan Xie, Hanyuan Liu, and Tien-Tsin Wong. Text-guided texturing by synchronized multi-view diffusion. In SIGGRAPH Asia 2024 Conference Papers , pages 1–11,
2024
-
[43]
Sgdr: Stochas- tic gradient descent with warm restarts
Ilya Loshchilov and Frank Hutter. Sgdr: Stochas- tic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. 5
2016 arXiv
-
[44]
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 6
2019
-
[45]
Mono-internvl: Push- ing the boundaries of monolithic multimodal large language models with endogenous visual pre-training
Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jifeng Dai, Yu Qiao, and Xizhou Zhu. Mono-internvl: Push- ing the boundaries of monolithic multimodal large language models with endogenous visual pre-training. arXiv preprint arXiv:2410.08202, 2024. 1
-
[46]
Nerf: Representing scenes as neural radiance fields for view syn- thesis
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 2
2021
-
[47]
Autosdf: Shape priors for 3d comple- tion, reconstruction and generation
Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shub- ham Tulsiani. Autosdf: Shape priors for 3d comple- tion, reconstruction and generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 306–315, 2022. 3
2022
-
[48]
Instant neural graphics primitives with a mul- tiresolution hash encoding
Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a mul- tiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 2
2022
-
[49]
Dreamfusion: Text-to-3d using 2d diffusion
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 1, 2
2022 arXiv
-
[50]
Improving language understanding by gener- ative pre-training, 2018
Alec Radford. Improving language understanding by gener- ative pre-training, 2018. 3
2018
-
[51]
Learn- ing transferable visual models from natural language super- vision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...
2021
-
[52]
Zero-shot text-to-image generation
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 3
2021
-
[53]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2
2022
-
[54]
Photorealistic text-to-image diffusion models with deep language understanding
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...
2022
-
[55]
Zero123++: a single image to consistent multi-view dif- fusion base model
Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model. arXiv preprint arXiv:2310.15110, 2023. 1, 2
-
[56]
Meshgpt: Generating triangle meshes with decoder-only transformers
Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Ta- tiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...
2024
-
[57]
Roformer: Enhanced transformer with rotary position embedding
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063,
-
[58]
Recent advances in im- plicit representation-based 3d shape generation
Jia-Mu Sun, Tong Wu, and Lin Gao. Recent advances in im- plicit representation-based 3d shape generation. Visual Intel- ligence, 2(1):9, 2024. 2
2024
-
[59]
Autoregressive model beats diffusion: Llama for scalable image generation
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 2, 3, 6
2024 arXiv
-
[60]
Auto-3d-house design from structured user requirements
Minkui Tan, Qi Chen, Zixiong Huang, Qi Wu, Yuanqing Li, and Jiaqiu Zhou. Auto-3d-house design from structured user requirements. Machine Intelligence Research , 22(2):368– 385, 2025. 2
2025
-
[61]
Fourier features let networks learn high frequency functions in low dimen- sional domains
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ra- mamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimen- sional domains. Advances in Neural Information ...
2020
-
[62]
Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation.arXiv preprint arXiv:2402.05054, 2024. 2, 6, 7
2024 arXiv
-
[63]
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3
2023 arXiv
-
[64]
Neural discrete representation learning
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2
2017
-
[65]
Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12619–12629, 2023. 2
2023
-
[66]
Nice: improving panoptic narrative detection and segmentation with cascading collab- orative learning
Haowei Wang, Jiayi Ji, Tianyu Guo, Yilong Yang, Yiyi Zhou, Xiaoshuai Sun, and Rongrong Ji. Nice: improving panoptic narrative detection and segmentation with cascading collab- orative learning. arXiv preprint arXiv:2310.10975, 2023. 2
-
[67]
Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021. 2
2021 arXiv
-
[68]
Rodin: A generative model for sculpting 3d digital avatars using diffusion
Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. In Proceedings of the IEEE/CVF conference on computer vision and...
2023
-
[69]
Emu3: Next-token prediction is all you need
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 3
2024 arXiv
-
[70]
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 5
2004
-
[71]
Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling
Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Ad- vances in neural information processing systems , 29, 2016. 2
2016
-
[72]
Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer
Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. arXiv preprint arXiv:2405.14832, 2024. 1, 2, 4
2024 arXiv
-
[73]
Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,
-
[74]
A survey of deep learning-based 3d shape generation
Qun-Ce Xu, Tai-Jiang Mu, and Yong-Liang Yang. A survey of deep learning-based 3d shape generation. Computational Visual Media, 9(3):407–442, 2023. 2
2023
-
[75]
Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding
Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference o...
2023
-
[76]
Scaling autoregres- sive models for content-rich text-to-image generation
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 3
2022 arXiv
-
[77]
3dilg: Ir- regular latent grids for 3d generative modeling
Biao Zhang, Matthias Nießner, and Peter Wonka. 3dilg: Ir- regular latent grids for 3d generative modeling. Advances in Neural Information Processing Systems, 35:21871–21885,
-
[78]
3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neu- ral fields and generative diffusion models.ACM Transactions on Graphics (TOG), 42(4):1–16, 2023. 2, 4
2023
-
[79]
Dino: Detr with improved denoising anchor boxes for end-to-end object detection
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 6
2022 arXiv
-
[80]
Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creat- ing high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1–20, 2024. 1, 2, 5
2024
-
[81]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 5
2018
-
[82]
Fast- pci: Motion-structure guided fast point cloud frame interpo- lation
Tianyu Zhang, Guocheng Qian, Jin Xie, and Yang Jian. Fast- pci: Motion-structure guided fast point cloud frame interpo- lation. In ECCV, 2024. 2
2024
-
[83]
Rstnet: Captioning with adaptive attention on visual and non-visual words
Xuying Zhang, Xiaoshuai Sun, Yunpeng Luo, Jiayi Ji, Yiyi Zhou, Yongjian Wu, Feiyue Huang, and Rongrong Ji. Rstnet: Captioning with adaptive attention on visual and non-visual words. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages ...
-
[84]
Temo: Towards text-driven 3d stylization for multi-object meshes
Xuying Zhang, Bo-Wen Yin, Yuming Chen, Zheng Lin, Yun- heng Li, Qibin Hou, and Ming-Ming Cheng. Temo: Towards text-driven 3d stylization for multi-object meshes. In Pro- ceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 19531–19540, 2024. 2
2024
-
[85]
Ar-1-to-3: Single image to consistent 3d object generation via next-view prediction
Xuying Zhang, Yupeng Zhou, Kai Wang, Yikai Wang, Zhen Li, Shaohui Jiao, Daquan Zhou, Qibin Hou, and Ming- Ming Cheng. Ar-1-to-3: Single image to consistent 3d object generation via next-view prediction. arXiv preprint arXiv:2503.12929, 2025. 2
2025 arXiv
-
[86]
Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation
Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in Neural Information Processing Systems, 36, 2024. 2, 6, 7
2024
-
[87]
GS-ROR: 3D Gaussian splatting for reflective object relighting via sdf pri- ors
Zuo-Liang Zhu, Beibei Wang, and Jian Yang. GS-ROR: 3D Gaussian splatting for reflective object relighting via sdf pri- ors. arXiv preprint arXiv:2406.18544, 2024. 2
2024 arXiv
-
[88]
Gaussian splatting with discretized sdf for relightable assets
Zuo-Liang Zhu, Jian Yang, and Beibei Wang. Gaussian splatting with discretized sdf for relightable assets. In Pro- ceedings of IEEE International Conference on Computer Vi- sion (ICCV), 2025. 2
2025
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.