Pith. sign in

REVIEW 4 major objections 7 minor 30 references

Routing each 3D voxel to the single image that best explains it is enough to make frozen single-image 3D models handle messy multi-image inputs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 18:51 UTC pith:7ZSP2HVK

load-bearing objection Clean training-free routing trick that mostly works; the VRS proxy is clever but under-checked against real correspondence. the 4 major comments →

arxiv 2607.24298 v1 pith:7ZSP2HVK submitted 2026-07-27 cs.CV

UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routing

Zefan Qu , Zhenwei Wang , Gerhard Petrus Hancke , Rynson W.H. Lau This is my paper
classification cs.CV
keywords 3D generationmulti-image conditioningcross-attention routingvoxel reference scoretraining-freeunconstrained multi-viewfoundation models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Single-image 3D foundation models make strong assets from one photo, but fall apart when given several unconstrained views of the same object—views that differ in pose, style, lighting, occlusion, or local detail. The paper argues the failure is not a capacity problem: it is a mismatch in how cross-attention is wired. Those models have no rule for which image each 3D voxel should trust at each denoising step, so cues interfere and geometry and texture collapse. UMI3D is a training-free plug-in that fixes the wiring. At every step it keeps all images active, but routes each voxel to one image—the one that best supports it—using a score read out from the model’s own attention. If that claim holds, existing single-image backbones become usable for real multi-image workflows without retraining or external matchers.

Core claim

The degradation of single-image 3D foundation models on unconstrained multi-image inputs comes from single-image cross-attention, not from weak capacity. Explicitly routing each voxel to its most informative conditioning image—via a model-intrinsic Voxel Reference Score and Simultaneous Focus Cross-Attention—is sufficient to unlock coherent multi-image 3D generation on frozen backbones, with no retraining and no external correspondence, matching, or segmentation models.

What carries the argument

Voxel Reference Score (VRS) plus Simultaneous Focus Cross-Attention (SFC-Attn). VRS is the share of a voxel’s cross-attention mass on local (patch) tokens rather than global (CLS/register) tokens for a given image; higher VRS means stronger local support. SFC-Attn groups voxels by highest VRS (refined by 3D KNN), then runs per-group cross-attention so every image is available in the same denoising step while each voxel focuses on one image.

Load-bearing premise

That how much attention a voxel puts on local image patches versus global image tokens is a trustworthy, comparable signal of which photo truly supports that voxel—even when the photos differ a lot in style or appearance.

What would settle it

On the paper’s unconstrained multi-image benchmark, force opposite-VRS routing (each voxel attends the image with lowest VRS) or random routing and check whether geometry/texture invert or collapse relative to correct VRS routing; if opposite routing does not clearly hurt and correct VRS does not clearly win on CLIP/DINO/LPIPS and user preference, the affinity proxy fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Frozen single-image voxel 3D generators can be used on inconsistent multi-view concept art and photo sets without multi-view retraining.
  • The same routing supports multi-view enhancement, single-view local detail refinement from crops, and controlled 3D morphing between source and target images.
  • Naive step-wise switching of the conditioning image can be replaced by simultaneous per-voxel focus, reducing oscillation and over-smoothing.
  • No external 2D–3D matcher or segmenter is required at inference if the backbone’s cross-attention already separates global and local tokens.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any DiT-style 3D generator whose conditioner exposes CLS/register vs patch tokens could adopt the same readout as a free multi-image adapter.
  • Failure on highly symmetric objects suggests VRS needs a tie-break when several views look equally ‘local’ to a voxel.
  • If VRS is stable across denoising steps, one could cache routing early and cut the cost of parallel single-image VRS branches.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper addresses 3D generation from "unconstrained" multi-image inputs — sets of images of the same object that disagree in pose, style, occlusion, illumination, and detail. The authors argue that single-image 3D foundation models (TRELLIS.2, Direct3D-S2) fail in this regime not from capacity limits but because cross-attention has no mechanism for deciding which image each voxel should trust. They propose UMI3D, a training-free inference-time modification: (i) a Voxel Reference Score (VRS, Eq. 2) defined as the cross-attention mass a voxel assigns to an image's local (patch) tokens rather than its global (CLS/register) tokens, read out from parallel single-image branches; (ii) Simultaneous Focus Cross-Attention (SFC-Attn, Eqs. 3–7), which routes each voxel to its argmax-VRS image (with 3D-KNN mode smoothing) and runs per-group cross-attention within each DiT block. Experiments on a new 40-case benchmark of edited Sketchfab renders show improvements over six baselines on CLIP/DINO/LPIPS and a 29-person user study (60.5% preference), plus ablations and destructive routing controls (random/opposite VRS), and extensions to morphing and view enhancement.

Significance. If the central claim holds, this is a useful and timely contribution: unconstrained multi-image conditioning is a realistic and under-served setting, and a training-free, plug-and-play routing mechanism that works across backbones (TRELLIS.2 main, TRELLIS in A.5.1) with no external correspondence model would be broadly applicable. Specific strengths worth naming: the mechanism is fully specified (Eqs. 2–7) and cheap to reproduce; the ablations isolate geometry routing, texture routing, and 3D-KNN (Fig. 6); and the random/opposite routing controls (Fig. 7, Tab. 3) are genuine destructive tests — opposite routing measurably inverts the evidence (texture attributes swapped), which is non-trivial evidence that VRS carries voxel-level signal rather than noise. The user study and multi-metric evaluation, plus the honest failure-case discussion (symmetric objects, Fig. 14), are welcome. The main reservation, developed below, is that the load-bearing assumption — that VRS is comparable across images and tracks per-voxel visibility rather than image-level properties — is asserted with a controls-the-voxel-side argument (Sec. 3.3) and validated only indirectly; the benchmark was constructed,

major comments (4)
  1. [Sec. 3.3, Eq. (3) — VRS cross-image comparability] The central claim rests on argmax_k VRS_{i,k} (Eq. 3) identifying the image that 'best explains' voxel v_i. The comparability argument in Sec. 3.3 controls the voxel side (same step, block, positions, parameters) but not the image side: an image's global/local attention split is plausibly a function of image-level content — sharpness, texture density, style — independent of which voxels it geometrically supports. Under the paper's own unconstrained regime (style/lighting/occlusion edits, Sec. A.3.2), a stylized or low-detail image may attract less local-token mass for ALL voxels, making the argmax encode an image-level bias rather than voxel-level correspondence. The offered validations do not exclude this confound: the global-token swap (Fig. 4) shows invisible regions depend more on global tokens, not that local mass ranks images correctly per voxel; and opposite routing (Tab. 3) would
  2. [Sec. 3.2 / Tab. 2 — Insight 1 generality] Insight 1 is stated as a general property ('Global tokens dominate the attention budget') motivating the VRS definition, but Tab. 2 contradicts it for one of the three backbones examined: Direct3D-S2 allocates 84.4% of attention to local tokens (vs. 15.4% on registers), and TRELLIS.2 sits near 50/50 in both stages. Only TRELLIS shows the claimed dominance (81% on registers). This matters because VRS = 1 − global mass is interpretable as 'support' only when the fallback-to-global behavior of Insight 2 holds; in a backbone where local tokens dominate unconditionally, VRS may carry little signal. Since the method is advertised as plug-and-play, the text should (a) qualify Insight 1 as backbone-dependent, and (b) clarify what VRS means, and whether routing still works, when applied to a local-dominated backbone — the generalization experiment (A.5.1) uses only TRELLIS, the backbone that alre
  3. [Sec. 4.1.3 / A.4 — quantitative evaluation] The image-space metrics (CLIP/DINO/LPIPS) are computed between input images and manually selected rendered views 'that best match the viewpoints of the input images' (Sec. A.4). This measures consistency at observed viewpoints only — by the authors' own admission — and the manual view selection introduces an unquantified evaluator degree of freedom (was selection done blind to method identity?). The main quantitative evidence for complete-asset quality thus rests on the user study (60.52% preference), which is reported as a single aggregate number with no per-case breakdown, inter-rater agreement, or significance testing over 29 participants × 20 cases. Given that the DINO improvement (1.63 points) and LPIPS improvement (0.0057) are modest, the load-bearing quantitative claim would be strengthened by (i) a geometry-side metric (e.g., Chamfer/normal consistency against the source Sketchfa
  4. [Sec. 4.2 / A.3.1 — ablation completeness and hyperparameters] Several free parameters are introduced with little or no sensitivity analysis: the SFC-Attn activation schedule in the sparse-structure stage (last 25% of steps, Sec. A.3.1), the 3D-KNN neighborhood size (K_nn = 10), the per-stage CFG scales (7.5/7.5/4.5), and the random anchor-image selection for sparse structure. Since routing in the sparse stage shapes all downstream geometry, the 75/25 split and the anchor choice are potentially load-bearing; a small sweep (or at least a statement of robustness across 2–3 settings and anchor choices) would de-risk the reproducibility of the main results. Relatedly, the inference cost of K parallel VRS-estimation branches per denoising step is never reported — for a 'plug-and-play' claim, runtime/memory overhead vs. the baseline on the stated RTX 4090 should be quantified.
minor comments (7)
  1. [Sec. 3.2] Eq. (2): defining VRS as a sum over local tokens and then 'computed by subtracting the attention mass allocated to the global tokens' is slightly redundant; also state explicitly that AttnMap rows sum to 1 so the two forms are equivalent.
  2. [Tab. 2] Tab. 2 header 'Ratio (Per-Token Ratio)' is confusing; the per-token column appears to be a further division by token count, but this is not explained in the caption. Also clarify over which denoising steps/blocks the accumulation is performed.
  3. [Sec. 3.2, Fig. 4] The Insight-2 swap experiment (Fig. 4) is described only qualitatively ('only mildly affected... much more significantly'). A quantitative version (e.g., feature drift in visible vs. invisible regions) would make this control more convincing.
  4. [Sec. A.4] User study protocol details are thin: how were the 20 cases sampled from 40, were renderings anonymized/order-randomized across the six methods, and were participants shown identical viewpoint sets? Please add to A.4.
  5. [Sec. 4 / A.3.2] Sec. 4 begins with a capitalized 'We' mid-sentence ('In this section, We first conduct...'); Sec. A.3.2 'The prompt to the Nano-Banana-Pro is illustrated in the Fig. 11' — minor copyediting pass needed. The footnote marking on the user-study sentence in Sec. 4.1.3 appears dropped in the preprint.
  6. [Sec. A.3.2] The benchmark's edited images are produced by a proprietary image model (Nano-Banana-Pro) and manually filtered; please state the rejection rate during filtering and whether the 40 cases, prompts, and anchor choices will be released, since the benchmark is a stated contribution and is needed to reproduce Tab. 1.
  7. [Fig. 2] Fig. 2(b) would benefit from showing actual VRS partition maps (G_k) on a real example rather than a schematic; the claimed spatial coherence of VRS groups before KNN smoothing is currently asserted, not shown.

Circularity Check

0 steps flagged

No circular derivation: VRS is an attention readout validated by destructive controls and external metrics, not a quantity forced by its own definition or self-citation.

full rationale

UMI3D’s load-bearing chain is observational and engineering, not a closed prediction loop. Insights 1–2 report empirical attention mass on global vs local DINO tokens in frozen TRELLIS/Direct3D/TRELLIS.2 backbones (Tab. 2, Figs. 3–4). VRS is then defined as local-token attention mass (Eq. 2) and used for hard voxel–image assignment (Eqs. 3–7). That definition does not encode the evaluation targets: CLIP/DINO/LPIPS and the user study compare rendered views to inputs and are independent of how VRS is computed. The paper’s own controls (random routing and opposite-VRS routing, Tab. 3 and Fig. 7) degrade metrics relative to correct VRS routing, which would be unnecessary if success were definitional. There is no parameter fit to a data subset that is then re-reported as a prediction; the method is training-free. Self-citations (e.g., Qu et al. on related 3D tasks) appear only as related work and do not justify uniqueness or force the routing rule. Author-built edited Sketchfab benchmarks and the open question whether local-token mass tracks true per-voxel visibility rather than image-level bias are validity/generalization concerns, not circularity under the stated criteria. The derivation is therefore self-contained against external benchmarks with no step that reduces by construction to its inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

The claim rests on pretrained single-image 3D DiT backbones and DINO features as fixed oracles, plus the modeling choice that local-vs-global attention mass is a sufficient affinity signal for hard routing. A few inference hyperparameters are set by hand. No new physical entities; invented constructs are algorithmic (VRS, SFC-Attn, the synthetic benchmark protocol).

free parameters (4)
  • 3D-KNN neighborhood size K_nn = 10
    Set to 10 for mode-based spatial smoothing of voxel–image assignments; chosen by authors, not derived.
  • SFC-Attn activation schedule in sparse structure stage = first 75% anchor-only; last 25% SFC-Attn
    Only the last 25% of denoising steps use SFC-Attn (first 75% single anchor image) to keep coarse shape stable; hand-chosen schedule.
  • Classifier-free guidance scales per stage = 7.5 / 7.5 / 4.5; 12 steps
    CFG set to 7.5, 7.5, and 4.5 across three stages with 12 steps each; standard tuning knobs affecting output fidelity.
  • Anchor image selection for sparse structure = random among inputs
    One input is randomly designated as the sparse-structure reference; choice can affect geometry when views disagree strongly.
axioms (5)
  • domain assumption Frozen TRELLIS/TRELLIS.2-style voxel DiT cross-attention to DINO tokens is a valid substrate for multi-image conditioning without weight updates.
    Entire method is training-free and only rewires inference-time attention (Sec. 3.1, A.3.1).
  • ad hoc to paper Global tokens (CLS + registers) dominate structural reasoning; local tokens carry spatially grounded evidence, so 1 − attention_on_globals is a meaningful support score (VRS).
    Stated as Insights 1–2 from attention statistics (Fig. 3–4, Tab. 2) and used as the definition of VRS (Eq. 2).
  • ad hoc to paper VRS values from parallel single-image branches at the same step/block/voxel positions are comparable enough for argmax routing across images.
    Explicitly claimed in Sec. 3.3 despite branch-specific queries; load-bearing for κ(i) = arg max_k VRS_i,k.
  • domain assumption Hard assignment of each voxel to exactly one image (plus KNN mode cleanup) is preferable to soft multi-image attention mixing for inconsistent inputs.
    Design choice underlying SFC-Attn grouping (Eqs. 3–7); alternatives not fully explored beyond random/opposite ablations.
  • domain assumption Gemini-edited multi-view renders of Sketchfab assets are a valid proxy for real unconstrained multi-image inputs.
    Benchmark construction in A.3.2; all quantitative claims in Tab. 1 rest on these 40 cases.
invented entities (3)
  • Voxel Reference Score (VRS) no independent evidence
    purpose: Scalar voxel–image affinity from attention mass on local vs global condition tokens, used to choose which image each voxel trusts.
    Defined in Eq. 2 from model-internal maps; no external labels. Independent handle is limited to ablations (opposite routing) inside the same models.
  • Simultaneous Focus Cross-Attention (SFC-Attn) no independent evidence
    purpose: Inference module that activates all images each step while restricting each voxel group to one chosen image’s keys/values.
    Algorithmic construct (Sec. 3.3); evaluated only within this paper’s experiments.
  • Unconstrained multi-image 3D generation benchmark (40 cases) no independent evidence
    purpose: Author-built testbed of 2–4 inconsistent views per object via Gemini attribute edits.
    Not a community dataset; central quantitative comparisons depend on it (A.3.2, Tab. 1).

pith-pipeline@v1.2.0-grok45-kimik3 · 22778 in / 4138 out tokens · 71023 ms · 2026-07-31T18:51:28.550885+00:00 · methodology

0 comments
read the original abstract

Recent 3D foundation models can generate high-quality assets from a single image, but degrade markedly on unconstrained multi-image inputs, often producing distorted geometry, over-smoothed textures, and chaotic colors. We argue that this failure stems not from limited model capacity, but from a mismatch between single-image cross-attention and the multi-image setting: existing models lack a principled way to decide which image each 3D voxel should trust at each denoising step. Revisiting recent single-image 3D foundation models, we show that explicitly routing each voxel to its most informative image is sufficient to unlock strong performance on inconsistent multi-image inputs. Based on this observation, we propose UMI3D, a training-free and plug-and-play framework that restructures cross-attention for unconstrained multi-image 3D generation. Its core, Simultaneous Focus Cross-Attention (SFC-Attn), activates all conditioning images at each denoising step while allowing each voxel to focus on the single image that best explains it. To enable this routing, we derive the Voxel Reference Score (VRS), a model-intrinsic metric for voxel--image affinity that requires no external matching, segmentation, or correspondence models. Extensive experiments show that UMI3D unlocks the multi-image potential of single-image 3D generation frameworks across diverse tasks. Project Page: UMI3D-Project.github.io.

Figures

Figures reproduced from arXiv: 2607.24298 by Gerhard Petrus Hancke, Rynson W.H. Lau, Zefan Qu, Zhenwei Wang.

Figure 1
Figure 1. Figure 1: UMI3D can generate a coherent 3D asset with unconstrained multi-image input and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of UMI3D. (a) Overall pipeline. (b) Voxel Reference Score (VRS). (c) Simulta [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Insight 1 of the VRS metric. Global Tokens Local Tokens + Reference Region Non-Reference Region TRELLIS.2 Cross-Attn Map Direct3D-S2 Cross-Attn Map TRELLIS.2 Top-10 tokens Attention Score Direct3D-S2 Top-10 tokens Attention Score [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison to multi-view 3D generation model, Tripo-3D (Li et al., 2025b), [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overall ablation study on UMI3D. (a) The Baseline Model (TRELLIS.2). (b) w/o Geom [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Insight validation of the SFC-Attn module. (a) Baseline. (b) Random Routing: Voxels [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The multi-view 3D generation enhancement results of UMI3D. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The 3D morphing results of UMI3D. α controls the 3D morphing progress. images to effectively integrate complementary visual information during generation. To address the ambiguity of voxel-image association in existing 3D generation models, UMI3D leverages the in￾trinsic behavior of cross-attention layers to derive the Voxel Reference Score (VRS), allowing the model to automatically establish reliable voxe… view at source ↗
Figure 10
Figure 10. Figure 10: Limitations of existing multi-image 3D generation approaches under inconsistent inputs. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Example of the prompt to the image generation model. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: The generalization experiment on TRELLIS backbone. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: The single-view 3D generation refinement results of UMI3D. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Failure Cases. • ”Luffy” by MONKEY. D. LUFFY. • ”tangerine peel man” by Sksskd. • ”Swan” by Diana Liu. • ”Colorful Frog Free” by moxstudios. • ”owlbear-3d-model” by AidanYT55Twt. • ”Wrecked” by Jacek Jaskolski. ´ • ”Mega Construx Pokemon (10)” by Emm. ´ • ”Teenage Mutant Ninja Turtles - Raphael ” by Hellbrush. • ”Apple Watch Ultra 2” by polyman Studio. • ”Knight - includes file for 3d printing” by Andy Wo… view at source ↗
Figure 15
Figure 15. Figure 15: More qualitative comparison with other 3D generation methods. [PITH_FULL_IMAGE:figures/full_fig_p021_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 19 linked inside Pith

  1. [3]

    Neural wavelet-domain diffusion for 3d shape generation

    Ka-Hei Hui, Ruihui Li, Jingyu Hu, and Chi-Wing Fu. Neural wavelet-domain diffusion for 3d shape generation. InSIGGRAPH Asia 2022 Conference Papers, pp. 1–9,

    2022

  2. [5]

    Wildgaus- sians: 3d gaussian splatting in the wild.arXiv preprint arXiv:2407.08447,

    Jonas Kulhanek, Songyou Peng, Zuzana Kukelova, Marc Pollefeys, and Torsten Sattler. Wildgaus- sians: 3d gaussian splatting in the wild.arXiv preprint arXiv:2407.08447,

    Pith/arXiv arXiv

  3. [6]

    Step1x-3d: Towards high-fidelity and controllable generation of textured 3d assets.arXiv:2505.07747, 2025a

    Weiyu Li, Xuanyang Zhang, Zheng Sun, Di Qi, Hao Li, Wei Cheng, Weiwei Cai, Shihao Wu, Jiarui Liu, Zihao Wang, et al. Step1x-3d: Towards high-fidelity and controllable generation of textured 3d assets.arXiv:2505.07747, 2025a. Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et ...

    Pith/arXiv arXiv

  4. [7]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003,

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003,

    Pith/arXiv arXiv

  5. [8]

    Point-e: A system for generating 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751,

    Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts.arXiv preprint arXiv:2212.08751,

    Pith/arXiv arXiv

  6. [9]

    Scalable diffusion models with transformers.arXiv preprint arXiv:2212.09748,

    William Peebles and Saining Xie. Scalable diffusion models with transformers.arXiv preprint arXiv:2212.09748,

    Pith/arXiv arXiv

  7. [10]

    (b) Naive extensions of single-image 3D pipelines switch conditioning images across denoising steps, resulting in conditioning oscillation and detail loss

    (a) Pre-trained multi-view 3D generation backbones assume relatively con- sistent inputs and fail under style or viewpoint mismatch. (b) Naive extensions of single-image 3D pipelines switch conditioning images across denoising steps, resulting in conditioning oscillation and detail loss. (c) UMI3D resolves the above issues via VRS-guided voxel routing. (b...

    2024

  8. [11]

    Tailor3d: Customized 3d assets editing and gen- eration with dual-side images.arXiv preprint arXiv:2407.06191,

    Zhangyang Qi, Yunhan Yang, Mengchen Zhang, Long Xing, Xiaoyang Wu, Tong Wu, Dahua Lin, Xihui Liu, Jiaqi Wang, and Hengshuang Zhao. Tailor3d: Customized 3d assets editing and gen- eration with dual-side images.arXiv preprint arXiv:2407.06191,

    Pith/arXiv arXiv

  9. [12]

    Lush-nerf: Lighting up and sharpening nerfs for low-light scenes.arXiv preprint arXiv:2411.06757,

    Zefan Qu, Ke Xu, Gerhard Petrus Hancke, and Rynson WH Lau. Lush-nerf: Lighting up and sharpening nerfs for low-light scenes.arXiv preprint arXiv:2411.06757,

    Pith/arXiv arXiv

  10. [13]

    Stylesculptor: Zero-shot style-controllable 3d asset generation with texture-geometry dual guid- ance

    Zefan Qu, Zhenwei Wang, Haoyuan Wang, Ke Xu, Gerhard Petrus Hancke, and Rynson WH Lau. Stylesculptor: Zero-shot style-controllable 3d asset generation with texture-geometry dual guid- ance. InProceedings of the SIGGRAPH Asia 2025 Conference Papers, pp. 1–12,

    2025

  11. [14]

    Dinov3.arXiv preprint arXiv:2508.10104,

    Oriane Sim´eoni, Huy V V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha ¨el Ramamonjisoa, et al. Dinov3.arXiv preprint arXiv:2508.10104,

    Pith/arXiv arXiv

  12. [16]

    Themestation: generating theme-aware 3d assets from few exemplars

    Zhenwei Wang, Tengfei Wang, Gerhard Hancke, Ziwei Liu, and Rynson WH Lau. Themestation: generating theme-aware 3d assets from few exemplars. InACM SIGGRAPH 2024 Conference Papers, pp. 1–12,

    2024

  13. [17]

    Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer.arXiv preprint arXiv:2405.14832,

    Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer.arXiv preprint arXiv:2405.14832,

    Pith/arXiv arXiv

  14. [18]

    Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention.arXiv preprint arXiv:2505.17412,

    Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Xun Cao, Philip Torr, et al. Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention.arXiv preprint arXiv:2505.17412,

    Pith/arXiv arXiv

  15. [19]

    Structured 3d latents for scalable and versatile 3d generation.arXiv preprint arXiv:2412.01506,

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation.arXiv preprint arXiv:2412.01506,

    Pith/arXiv arXiv

  16. [20]

    Native and compact structured latents for 3d generation.arXiv preprint arXiv:2512.14692,

    Jianfeng Xiang, Xiaoxue Chen, Sicheng Xu, Ruicheng Wang, Zelong Lv, Yu Deng, Hongyuan Zhu, Yue Dong, Hao Zhao, Nicholas Jing Yuan, et al. Native and compact structured latents for 3d generation.arXiv preprint arXiv:2512.14692,

    Pith/arXiv arXiv

  17. [21]

    Textured 3d regenerative morphing with 3d diffusion prior.arXiv preprint arXiv:2502.14316,

    Songlin Yang, Yushi Lan, Honghua Chen, and Xingang Pan. Textured 3d regenerative morphing with 3d diffusion prior.arXiv preprint arXiv:2502.14316,

    Pith/arXiv arXiv

  18. [22]

    Wukong’s 72 transformations: High-fidelity textured 3d morphing via flow models.arXiv preprint arXiv:2511.22425,

    Minghao Yin, Yukang Cao, and Kai Han. Wukong’s 72 transformations: High-fidelity textured 3d morphing via flow models.arXiv preprint arXiv:2511.22425,

    arXiv

  19. [24]

    Splatpainter: Interactive authoring of 3d gaussians from 2d edits via test-time training

    Yang Zheng, Hao Tan, Kai Zhang, Peng Wang, Leonidas Guibas, Gordon Wetzstein, and Wang Yifan. Splatpainter: Interactive authoring of 3d gaussians from 2d edits via test-time training. arXiv preprint arXiv:2512.05354,

    Pith/arXiv arXiv

  20. [26]

    Such pipelines typically consist of two or three stages, each responsible for generating a coarse 3D structure, detailed geometry, or texture, respectively

    serve as backbones for synthesizing 3D assets from user-provided images, and can produce multiple forms of 3D representations (e.g., NeRF, 3DGS, and meshes). Such pipelines typically consist of two or three stages, each responsible for generating a coarse 3D structure, detailed geometry, or texture, respectively. All stages are commonly implemented using ...

    2022

  21. [27]

    Cross-attention conditioning.During 3D generation, the 2D informationcis injected into the generative network at each layer via cross-attention (cross-attn)

    composed of multiple stacked transformer layers. Cross-attention conditioning.During 3D generation, the 2D informationcis injected into the generative network at each layer via cross-attention (cross-attn). Given voxel tokensZ∈R Nv×d and image featuresc∈R Nc×dc, the queries, keys, and values are computed as: Q=ZW Q,K=cW K,V=cW V ,(10) whereW Q,W K, andW V...

    2025

  22. [28]

    from human to cat)

    We manually filter out generated images that deviate significantly from the semantic information of the original objects (e.g. from human to cat). The dataset includes a wide range of inconsistencies, such as variations in lighting, motion, occlusion, color, style, and local details, ensuring the dataset’s diversity and the benchmark’s robustness. Generat...

    2021

  23. [29]

    (b) w/o Geometry Routing: Replacing SFC-Attn in geometry generation with original cross-attn

    The variants of our approach are selected as follows: (a) The Baseline Model (TRELLIS.2): Original cross-attn in both stages. (b) w/o Geometry Routing: Replacing SFC-Attn in geometry generation with original cross-attn. (c) w/o Texture Routing: Replacing SFC-Attn in texture generation with original cross-attn. (d) w/o 3D-KNN: Removing the 3D-KNN refinemen...

    2024

  24. [30]

    Portrait with Hat Van Gogh

    Specifically, instead of using theargmaxoperation in the SFC-Attn module to compare VRS scores, we directly inject conditions according to the VRS scores from the target-image branch, where the degree of injection is controlled by the desired morphing extent. The morphing process is achieved by progressively increasing the number of voxels conditioned on ...

    2018

  25. [2018]

    Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation.arXiv preprint arXiv:2501.12202,

    Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation.arXiv preprint arXiv:2501.12202,

    Pith/arXiv arXiv

  26. [2021]

    Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805,

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805,

    Pith/arXiv arXiv

  27. [2022]

    Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988,

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988,

    Pith/arXiv arXiv

  28. [2023]

    Sam 3d: 3dfy anything in images.arXiv preprint arXiv:2511.16624,

    Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, et al. Sam 3d: 3dfy anything in images.arXiv preprint arXiv:2511.16624,

    Pith/arXiv arXiv

  29. [2024]

    Fuse3d: Generating 3d assets controlled by multi-image fusion

    Xuancheng Jin, Rengan Xie, Wenting Zheng, Rui Wang, Hujun Bao, and Yuchi Huo. Fuse3d: Generating 3d assets controlled by multi-image fusion. InProceedings of the SIGGRAPH Asia 2025 Conference Papers, pp. 1–12,

    2025

  30. [2025]

    Charactershot: Controllable and consistent 4d character animation.arXiv preprint arXiv:2508.07409,

    Junyao Gao, Jiaxing Li, Wenran Liu, Yanhong Zeng, Fei Shen, Kai Chen, Yanan Sun, and Cairong Zhao. Charactershot: Controllable and consistent 4d character animation.arXiv preprint arXiv:2508.07409,

    Pith/arXiv arXiv