pith. sign in

arxiv: 2605.26137 · v1 · pith:SIDB4JC6new · submitted 2026-05-22 · 💻 cs.GR · cs.AI· cs.CV

AssetGen: Deployable 3D Asset Generation at Interactive Speed

Dilin Wang , Xiaoyu Xiang , Kihyuk Sohn , Tom Monnier , Yu-Ying Yeh , Thu Nguyen-Phuoc , Jiawen Zhang , Yuchen Fan
show 9 more authors
Antoine Toisoul Hyunyoung Jung Prithviraj Dhar Michael Bunnell Nikolaos Sarafianos Chuhang Zou Roman Shapovalov Andrea Vedaldi Rakesh Ranjan
This is my paper

Pith reviewed 2026-06-30 15:03 UTC · model grok-4.3

classification 💻 cs.GR cs.AIcs.CV
keywords 3D asset generationsingle image inputreal-time renderingmesh generationtexture synthesisGPU optimizationinteractive workflowsdeployable 3D models
0
0 comments X

The pith

Given one reference image, AssetGen produces a polygon-controlled 3D mesh with baked normals and texture in 30 seconds for real-time rendering including on mobile devices.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a 3D generation system that prioritizes deployability and interactive speed over maximum resolution. It takes a single reference image as input and outputs a complete asset consisting of a mesh, baked normals, a color texture, and a user-controlled polygon budget that supports real-time rendering on consumer hardware. The pipeline achieves this through GPU-resident mesh operations and a series of accelerations that finish in 30 seconds, or 14 seconds for the Flash variant. Automated and blind human evaluations indicate the results match the visual quality of leading commercial tools under the same constraints. A sympathetic reader would care because the work targets practical workflows where assets must run immediately without further editing.

Core claim

AssetGen generates object geometry with a coarse-to-refine VecSet framework that implements mesh simplification, cleaning, and normal baking on the GPU together with fast parallel UV unwrapping. Textures are produced through multi-view generation followed by backprojection and 3D inpainting. The full pipeline is accelerated end-to-end by model distillation, kernel optimization, and pipeline parallelization, yielding assets that satisfy polygon budgets for real-time use while matching commercial visual quality in evaluations.

What carries the argument

coarse-to-refine VecSet framework for geometry that performs GPU-based simplification, cleaning, and normal baking, combined with multi-view texturing and end-to-end pipeline optimizations for speed.

If this is right

  • Assets can be dropped directly into real-time applications on mobile devices because polygon counts are explicitly controlled.
  • The 14-second Flash variant enables iterative and agentic creation loops without long waits.
  • No additional post-processing is required to reach deployable quality, supporting AI-assisted 3D content creation in production pipelines.
  • Competitive quality holds under both automated metrics and blind human comparison to commercial baselines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same GPU-first design pattern could be applied to other single-image generative tasks that must also respect runtime constraints.
  • Integration into existing game engines or AR toolkits would likely reduce the manual cleanup step that currently follows most AI 3D generators.
  • Extending the input to short video clips might improve geometric consistency while preserving the reported latency if the multi-view stage is adapted accordingly.

Load-bearing premise

The automated and blind human evaluations correctly establish that the generated assets achieve competitive visual quality against leading commercial solutions while satisfying the polygon budget and real-time rendering constraints.

What would settle it

A controlled experiment in which blind raters consistently prefer commercial assets over AssetGen outputs on the same reference images, or where the generated meshes exceed the stated polygon budget when loaded on mobile hardware.

read the original abstract

While 3D generation is progressing rapidly, recent work has often focused on obtaining high-resolution assets, leaving user experience and deployability as afterthoughts. We present AssetGen, a 3D generator that focuses instead on these two aspects. Given one reference image, in 30 seconds it produces a high-quality mesh with baked normals, a color texture, and a controlled polygon budget suitable for real-time rendering, including mobile use cases. The AssetGen Flash variant further reduces latency to 14 seconds for interactive and agentic creation loops. Our model generates the object geometry with a coarse-to-refine VecSet framework, which implements mesh simplification, cleaning, and normal baking on the GPU, and a fast parallel UV unwrapping. It then generates textures in a multi-view fashion, followed by backprojection and 3D inpainting. Model distillation, kernel optimization, and pipeline parallelization are co-designed to accelerate the system end-to-end. We introduce numerous automated and blind human evaluations and demonstrate competitive visual quality against leading commercial solutions in 30 seconds and preview-quality results in less than 15 seconds. The final result is a system that supports AI-assisted, deployable 3D content creation in interactive workflows.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper presents AssetGen, a system for single-image 3D asset generation that outputs a mesh with baked normals, color texture, and controlled polygon budget in 30 seconds (14 seconds for the Flash variant), optimized for real-time and mobile rendering. Geometry is produced via a coarse-to-refine VecSet framework with GPU mesh simplification, cleaning, normal baking, and parallel UV unwrapping; textures are generated multi-view, followed by backprojection and 3D inpainting. End-to-end acceleration uses model distillation, kernel optimization, and pipeline parallelization. Numerous automated and blind human evaluations are claimed to show competitive visual quality versus leading commercial solutions while meeting polygon and latency constraints.

Significance. If the performance and quality claims are substantiated, the work would be significant for shifting 3D generation toward deployable, interactive use cases rather than high-resolution offline assets, enabling practical AI-assisted content creation pipelines.

major comments (1)
  1. [Abstract] Abstract: the central claims of 'competitive visual quality' and satisfaction of polygon budget/real-time constraints rest entirely on 'numerous automated and blind human evaluations' whose protocols, metrics, baselines, quantitative scores, or statistical analysis are not described or shown; without these data the load-bearing evidence for the primary contribution cannot be assessed.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and for identifying the need for clearer substantiation of the evaluation claims. We respond to the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claims of 'competitive visual quality' and satisfaction of polygon budget/real-time constraints rest entirely on 'numerous automated and blind human evaluations' whose protocols, metrics, baselines, quantitative scores, or statistical analysis are not described or shown; without these data the load-bearing evidence for the primary contribution cannot be assessed.

    Authors: We agree with the referee that the abstract's claims rest on evaluations whose protocols, metrics, baselines, quantitative scores, and statistical analysis are not described or shown in the current manuscript. We will revise the manuscript to add a dedicated experiments subsection that fully details the automated metrics and their computation, the specific baselines and commercial systems compared, all quantitative scores, the blind human evaluation protocol (including participant count, rating interface, questions, and statistical tests), and the corresponding results in tables and figures. This will make the supporting evidence explicit and allow assessment of the primary contributions. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper describes an engineering system for single-image 3D asset generation with emphasis on speed, mesh quality, and deployability. No mathematical derivations, equations, fitted parameters presented as predictions, or self-citation chains appear in the abstract or description. Central claims rest on empirical evaluations of the implemented pipeline rather than any reduction of outputs to inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract contains no technical derivations, parameters, or assumptions that can be extracted; ledger remains empty due to lack of detail.

pith-pipeline@v0.9.1-grok · 5821 in / 1019 out tokens · 47977 ms · 2026-06-30T15:03:25.667156+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 27 canonical work pages · 15 internal anchors

  1. [1]

    Meta 3d texturegen: Fast and consistent texture generation for 3d objects.arXiv preprint arXiv:2407.02430,

    Raphael Bensadoun, Yanir Kleiman, Idan Azuri, Omri Harosh, Andrea Vedaldi, Natalia Neverova, and Oran Gafni. Meta 3d texturegen: Fast and consistent texture generation for 3d objects.arXiv preprint arXiv:2407.02430,

    work page arXiv

  2. [2]

    SAM 3: Segment Anything with Concepts

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, et al. Sam 3: Segment anything with concepts.arXiv preprint arXiv:2511.16719,

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    Real-time mesh simplification using the gpu

    Christopher DeCoro and Natalya Tatarchuk. Real-time mesh simplification using the gpu. InProceedings of the 2007 symposium on Interactive 3D graphics and games, pages 161–166,

    2007

  4. [4]

    Yeh, and Ziwei Liu

    Weichen Fan, Amber Yijia Zheng, Raymond A. Yeh, and Ziwei Liu. CFG-Zero*: Improved classifier-free guidance for flow matching models.arXiv preprint arXiv:2503.18886,

    work page arXiv

  5. [5]

    George Hart

    Zekun Hao, David W. Romero, Tsung-Yi Lin, and Ming-Yu Liu. Meshtron: High-fidelity, artist-like 3D mesh generation at scale.arXiv preprint arXiv:2412.09548,

    work page arXiv

  6. [6]

    VideoMatGen: PBR materials through joint generative modeling.arXiv preprint arXiv:2603.16566,

    Jon Hasselgren, Zheng Zeng, Milos Hasan, and Jacob Munkberg. VideoMatGen: PBR materials through joint generative modeling.arXiv preprint arXiv:2603.16566,

    work page arXiv

  7. [7]

    Classifier-Free Diffusion Guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Manifoldplus: A robust and scalable watertight manifold surface generation method for triangle soups.arXiv preprint arXiv:2005.11621,

    Jingwei Huang, Yichao Zhou, and Leonidas Guibas. Manifoldplus: A robust and scalable watertight manifold surface generation method for triangle soups.arXiv preprint arXiv:2005.11621,

    work page arXiv 2005

  9. [9]

    Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Material

    Xin Huang, Tengfei Wang, Ziwei Liu, and Qing Wang. Material anything: Generating materials for any 3d object via diffusion. InProc. CVPR, pages 26556–26565, 2025a. Zixuan Huang, Mark Boss, Aaryaman Vasishta, James M Rehg, and Varun Jampani. Spar3d: Stable point-aware reconstruction of 3d objects from single images. InProc. CVPR, 2025b. Team Hunyuan3D, Shu...

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    Lattice: Democratize high-fidelity 3d generation at scale.arXiv preprint arXiv:2512.03052, 2025a

    Zeqiang Lai, Yunfei Zhao, Zibo Zhao, Haolin Liu, Qingxiang Lin, Jingwei Huang, Chunchao Guo, and Xiangyu Yue. Lattice: Democratize high-fidelity 3d generation at scale.arXiv preprint arXiv:2512.03052, 2025a. Zeqiang Lai, Yunfei Zhao, Zibo Zhao, Haolin Liu, Fuyun Wang, Huiwen Shi, Xianghui Yang, Qingxiang Lin, Jingwei Huang, Yuhong Liu, et al. Unleashing v...

    work page arXiv

  11. [11]

    Hunyuan3d studio: End-to-end ai pipeline for game-ready 3d asset generation.arXiv preprint arXiv:2509.12815,

    Biwen Lei, Yang Li, Xinhai Liu, Shuhui Yang, Lixin Xu, Jingwei Huang, Ruining Tang, Haohan Weng, Jian Liu, Jing Xu, et al. Hunyuan3d studio: End-to-end ai pipeline for game-ready 3d asset generation.arXiv preprint arXiv:2509.12815,

    work page arXiv

  12. [12]

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3d object. InProc. ICCV, pages 9298–9309, 2023a. Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Topomesh: High-fidelity mesh autoencoding via topological unification.arXiv preprint arXiv:2603.24278,

    Guan Luo, Xiu Li, Rui Chen, Xuanyu Yi, Jing Lin, Chia-Hao Chen, Jiahang Liu, Song-Hai Zhang, and Jianfeng Zhang. Topomesh: High-fidelity mesh autoencoding via topological unification.arXiv preprint arXiv:2603.24278,

    work page arXiv

  15. [15]

    DINOv2: Learning Robust Visual Features without Supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    Texture: Text-guided texturing of 3d shapes

    Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing of 3d shapes. InACM SIGGRAPH 2023 conference proceedings, pages 1–11,

    2023

  17. [17]

    Wavelet latent diffusion (wala): Billion-parameter 3d generative model with compact wavelet encodings.arXiv preprint arXiv:2411.08017,

    Aditya Sanghi, Aliasghar Khani, Pradyumna Reddy, Arianna Rampini, Derek Cheung, Kamal Rahimi Malekshan, Kanika Madan, and Hooman Shayani. Wavelet latent diffusion (wala): Billion-parameter 3d generative model with compact wavelet encodings.arXiv preprint arXiv:2411.08017,

    work page arXiv

  18. [18]

    Mvpainter: Accurate and detailed 3d texture generation via multi-view diffusion with geometric control.arXiv preprint arXiv:2505.12635,

    Mingqi Shao, Feng Xiong, Zhaoxu Sun, and Mu Xu. Mvpainter: Accurate and detailed 3d texture generation via multi-view diffusion with geometric control.arXiv preprint arXiv:2505.12635,

    work page arXiv

  19. [19]

    GLU Variants Improve Transformer

    Noam Shazeer. Glu variants improve transformer.arXiv preprint arXiv:2002.05202,

    work page internal anchor Pith review Pith/arXiv arXiv 2002

  20. [20]

    TripoSR: Fast 3D Object Reconstruction from a Single Image

    Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image.arXiv preprint arXiv:2403.02151,

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Improving and generalizing flow-based generative models with minibatch optimal transport

    Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.00482,

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    Face: A face-based autoregressive representation for high-fidelity and efficient mesh generation

    Hanxiao Wang, Yuan-Chen Guo, Ying-Tian Liu, Zi-Xin Zou, Biao Zhang, Weize Quan, Ding Liang, Yan-Pei Cao, and Dong-Ming Yan. Face: A face-based autoregressive representation for high-fidelity and efficient mesh generation. arXiv preprint arXiv:2603.01515,

    work page arXiv

  23. [23]

    Native and Compact Structured Latents for 3D Generation

    Jianfeng Xiang, Xiaoxue Chen, Sicheng Xu, Ruicheng Wang, Zelong Lv, Yu Deng, Hongyuan Zhu, Yue Dong, Hao Zhao, Nicholas Jing Yuan, and Jiaolong Yang. Native and compact structured latents for 3d generation.arXiv preprint arXiv: 2512.14692, 2025a. Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong ...

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models

    Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models.arXiv preprint arXiv:2404.07191,

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    Strips as Tokens: Artist Mesh Generation with Native UV Segmentation

    Rui Xu, Dafei Qin, Kaichun Qiao, Qiujie Dong, Huaijin Pi, Qixuan Zhang, Longwen Zhang, Lan Xu, Jingyi Yu, Wenping Wang, et al. Strips as tokens: Artist mesh generation with native uv segmentation.arXiv preprint arXiv:2604.09132,

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    Fast3dcache: Training-free 3d geometry synthesis acceleration.arXiv preprint arXiv:2511.22533,

    Mengyu Yang, Yanming Yang, Chenyi Xu, Chenxi Song, Yufan Zuo, Tong Zhao, Ruibo Li, and Chi Zhang. Fast3dcache: Training-free 3d geometry synthesis acceleration.arXiv preprint arXiv:2511.22533,

    work page arXiv

  27. [27]

    IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721,

    work page internal anchor Pith review Pith/arXiv arXiv

  28. [28]

    Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation

    29 Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501.12202, 2025b. Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen, and Siwei Lyu. Dice: Distilling classi...

    work page internal anchor Pith review Pith/arXiv arXiv

  29. [29]

    DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity

    Haowei Zhu, Ji Liu, Ziqiong Liu, Dong Li, Junhai Yong, Bin Wang, and Emad Barsoum. Diffsparse: Accelerating diffusion transformers with learned token sparsity.arXiv preprint arXiv:2604.03674,

    work page internal anchor Pith review Pith/arXiv arXiv