{"id":"12411670-ea9b-4dc1-8884-5d3973405cc4","arxiv_id":"2412.16919","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"TAR3D uses a triplane VQ-VAE to turn 3D shapes into discrete codebook tokens and a GPT-style transformer to generate those tokens autoregressively from text or image prompts.","lead":"TAR3D generates 3D objects by first compressing shapes into discrete triplane tokens and then predicting those tokens one by one with a transformer, like an image or text generator. The paper reports better image-to-3D and text-to-3D quality than several recent systems on ShapeNet, Objaverse, and Google Scanned Objects.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Abstract's text-to-3D superiority claim has no quantitative support: Section 4.3 benchmarks only image-to-3D, so the second half of the central claim is unverified; a text-conditioned metric comparison is needed before acceptance.","rationale":"The reader's weakest assumption points at tokenizer resolution; I partially agree but think the decisive issue is the absence of text-to-3D evidence. The tokenizer may be a ceiling, but the paper's own ablations (Table 4) show that going from 3x32x32 to 3x48x48 improves Chamfer Distance only from 0.066 to 0.062, so the current generation deficit is not primarily tokenizer resolution. The central claim explicitly bundles text-to-3D with image-to-3D, and the manuscript itself concedes at Section 4.3 that the only tabulated comparison is image-to-3D. That is an internal mismatch between the claim and the evidence, not a matter of scientific consensus. A quantitative text-to-3D comparison, using the same metrics and evaluation set as Table 1, would settle whether the paper can claim both modalities. Since this is an evidence gap that additional experiments can fill, the correct verdict remains CONDITIONAL (the same as the reader), not REJECT; the concern does not refute the image-to-3D results but does narrow the scope of what has been proven. I therefore keep the reader's verdict unchanged while adding a more pointed condition: the text-to-3D claim must be backed by numbers or removed.","tokens_in":15911,"tokens_out":6244,"duration_ms":60963,"concrete_test":"Run the paper's own Section 4.3 evaluation for text-to-3D on a fixed sample of prompts from the ShapeNet/Objaverse test split: generate one mesh per prompt with TAR3D and with the strongest baselines (e.g., Shap-E, Michelangelo, SDFusion) using their official released checkpoints; compute Chamfer Distance, F-Score@0.02, and a semantic alignment metric (CLIP-Score or T3Bench) against ground-truth meshes or human preference ratings, reporting paired bootstrap confidence intervals. If TAR3D is not significantly better than the best baseline on geometry and semantic alignment, the 'superior text-to-3D' claim in the Abstract is not supported and should be weakened to image-to-3D only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the Abstract, 'superior generation quality over existing methods in text-to-3D and image-to-3D tasks,' has two legs, and only the image-to-3D leg is quantitatively tested. In Section 4.3 the authors state that they 'conduct experiments on image-to-3D tasks to ensure an accurate comparison,' and Table 1 is the only quantitative comparison table in the paper. The text-to-3D results in Fig. 5 are a small set of selected prompts with no metric, no error bar, and no user study; that is not enough to establish superiority over Diffusion-SDF, SDFusion, Shap-E, Fantasia3D, or Michelangelo. This matters because the paper's stated contribution is a unified next-token-prediction framework across both conditional modalities: if text conditioning only looks good on hand-picked examples, the central claim overstates what is demonstrated. A second supporting weakness is that the image-to-3D evidence itself is fragile: the generation F-Score of 0.303 (Table 1) is far below the tokenizer's own reconstruction F-Score of 0.822 (Table 2), and Table 4 shows that increasing triplane size from 32 to 48 changes Chamfer Distance only from 0.066 to 0.062, so the autoregressive stage, not merely tokenizer capacity, is the current bottleneck. The comparison also lacks error bars and any discussion of baseline scale or normalization, so the large Table 1 margins could be partly an artifact of evaluation protocol. The load-bearing requirement is therefore a quantitative, controlled text-to-3D comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TAR3D, a two-stage framework for conditional 3D object generation. A 3D VQ-VAE encodes point clouds into a fixed-size triplane latent, quantizes it with a trainable codebook, and reconstructs neural occupancy fields through a decoder with a plane-interaction attention (PII) design. A decoder-only transformer, augmented with a custom triplane positional encoding (TriPE), autoregressively predicts the codebook index sequence conditioned on image or text prompts. Image-to-3D experiments on ShapeNet, Objaverse, and Google Scanned Objects report large margins over multiview-based and 3D-diffusion baselines; text-to-3D results are presented through qualitative comparisons only.","tokens_in":16281,"tokens_out":4826,"duration_ms":41382,"significance":"If the quantitative claims survive a more rigorous evaluation, TAR3D is a solid autoregressive alternative to diffusion-based 3D generation. The fixed-length discrete tokenization of arbitrary meshes is practically valuable, and the TriPE and PII design choices are clearly motivated and supported by ablations (Table 3, Fig. 6). The paper ships detailed implementation settings and a clear architectural description, which strengthens reproducibility. The principal weakness is that the abstract's equally prominent text-to-3D superiority claim is not quantitatively supported, and the image-to-3D comparison lacks statistical grounding.","major_comments":[{"comment":"The abstract claims superiority in both text-to-3D and image-to-3D, but §4.3 only quantifies image-to-3D (Table 1). The text-to-3D evidence is a small set of selected qualitative examples (Fig. 5) with no metric, user study, or error bars. Because the paper's stated contribution is a unified next-token-prediction framework across both modalities, the missing quantitative text-to-3D evaluation is load-bearing; please add a quantitative comparison (e.g., CLIP-Score, user study, or a standard text-to-3D benchmark) or revise the central claim to image-to-3D only.","section":"Abstract; §4.3"},{"comment":"The quantitative comparison lacks statistical and protocol detail. Only single-run numbers are reported, with no standard deviation, significance test, or description of how baselines were configured (checkpoints, mesh extraction threshold, post-processing, normalization). Large margins can arise from evaluation-protocol differences, particularly when comparing normal maps rendered from meshes produced by different pipelines. Please report multiple runs or provide the evaluation code and baseline settings to rule out protocol artifacts.","section":"§4.3, Table 1"},{"comment":"The paper does not analyze the relationship between tokenizer capacity and generation quality. Reconstruction F-Score is 0.822, while generation F-Score is only 0.303, and increasing triplane size from 3×32×32 to 3×48×48 changes Chamfer Distance only from 0.066 to 0.062. This suggests the autoregressive stage, not merely tokenizer capacity, is the current bottleneck; the paper should discuss this gap to calibrate the claim that the fixed triplane codebook preserves sufficient geometric detail for high-quality generation.","section":"§3.1, Tables 2 and 4"}],"minor_comments":[{"comment":"The second term of the codebook loss should read ||\\tilde z − sg[z_q]||_2^2; as printed it lacks the square and is dimensionally inconsistent.","section":"§3.3, Eq. (4)"},{"comment":"The prompt construction is described only loosely ('we adopt the rendered images and textual descriptions from ULIP'); please clarify how the 4 images and top-1 captions are selected and whether the same captions are used for all views.","section":"§4.1"},{"comment":"The header '3D V AE' should read '3D VAE'.","section":"Table 2"},{"comment":"The text says 'three multiview-based methods' but lists four (SyncDreamer, InstantMesh, OpenLRM, LGM); additionally, OpenLRM is discussed but does not appear in Table 1.","section":"§4.2"},{"comment":"The baseline is labeled 'DiffSDF' while the reference list names the method 'Diffusion-sdf'; please unify the naming.","section":"Fig. 5"},{"comment":"The heading uses inconsistent spacing ('3 ×16×16' vs '3×32×32'); please standardize and explicitly state that 3×32×32 is the default setting.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a computer vision journal. The single-run quantitative results and missing text-to-3D metrics are the main risks; I would support acceptance after a major revision that adds those elements. There is no circularity concern: the model is trained and evaluated on held-out splits, and the reported generation numbers are measured outputs. The main uncertainty is whether the large Table 1 margins survive a controlled protocol comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"TAR3D is a real system, and the core idea is new: instead of tokenizing mesh faces or voxels, it quantizes triplane latents into a fixed-length sequence and trains a GPT to predict codebook indices, with TriPE and plane-interaction attention as the main contributions. The architecture is described well enough to reimplement, and the ablations show that both the PII decoder and TriPE help. Reconstruction F-Score of 0.822 is respectable.\n\nThe weaknesses are in the evaluation, not the architecture. I agree with the stress-test: the abstract claims superiority in both text-to-3D and image-to-3D, but only the image-to-3D leg has quantitative support. Table 1 shows large margins, but the text-to-3D section is entirely qualitative—a few selected renders with no metrics or user study. That is a real gap for a paper whose framing is a unified conditional generation framework. Also, Table 1 reports single runs with no error bars, and there is no detail on baseline checkpoints or training data, so some of that margin could be a protocol artifact. The gap between generation F-Score (0.303) and reconstruction F-Score (0.822) shows the autoregressive stage is the current bottleneck, which the paper does not analyze.\n\nNone of this is fatal. The central argument holds, and the paper deserves a serious referee. I would encourage revision: add a quantitative text-to-3D comparison (or at least a user study), report multiple seeds or error bars, clarify the baseline setup, and discuss the reconstruction-generation gap. If the authors deliver that, this is a solid contribution to autoregressive 3D generation.","headline":"TAR3D is a credible new triplane-token autoregressive 3D generation system, but the text-to-3D superiority claim is not quantitatively supported and the image-to-3D evidence needs hardening.","tokens_in":16818,"tokens_out":3445,"would_cite":true,"duration_ms":28406,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Autoregressive 'next-part' prediction generates 3D assets from text and images.","keywords":["3D generation","autoregressive transformer","VQ-VAE","triplane representation","next-token prediction","text-to-3D","image-to-3D","positional encoding"],"falsifier":"Render a set of thin or highly structured objects, such as chair backs, bicycle spokes, or ropes, through the 3D VQ-VAE and compare occupancy F-Score against the continuous VAE counterpart; a large drop on those shapes would show the tokenizer is the bottleneck. An independent benchmark in which multiview or diffusion methods beat TAR3D on geometric fidelity for the same prompts would also contradict the claimed margin.","tokens_in":15750,"feed_emoji":"🧊","tokens_out":6485,"duration_ms":53165,"temperature":0.7,"pith_summary":"TAR3D proposes that 3D object generation can be treated as next-token prediction over a discrete codebook of geometric parts. The paper builds a triplane-based VQ-VAE that converts any mesh into a fixed-length sequence of codebook indices, then trains a decoder-only transformer to predict that sequence left-to-right, conditioned on an image or text prompt. It claims this autoregressive 'next-part' approach produces better geometry than multiview-reconstruction and 3D-diffusion baselines, reporting the largest margins on Chamfer Distance and F-Score. The practical payoff is that sequence length no longer grows with polygon count, making autoregressive modeling feasible for high-complexity assets.","feed_headline":"Next-part prediction beats diffusion at 3D asset generation","feed_subtitle":"A triplane codebook turns any mesh into a token sequence, letting GPT compose objects part by part.","key_machinery":"The load-bearing object is the triplane VQ-VAE: three axis-aligned 2D feature maps (XY, YZ, XZ) that compress a mesh into fixed-length discrete codes. The quantizer maps each of the $3\\times32\\times32$ latent vectors to one of $16{,}384$ entries, and the GPT models the index sequence in a raster-scan-within-plane, plane-adjacent order. TriPE fuses 2D and 1D rotary position embeddings so the transformer knows both where a token sits in its plane and which plane it belongs to; the decoder then upsamples to $256\\times256$ and predicts occupancy, with cross-plane attention carrying the information exchange between planes.","core_discovery":"The central claim is that a 3D shape can be encoded as a $3\\times32\\times32$ grid of triplane features, quantized with a $16{,}384$-entry codebook, and then generated part by part by a GPT predicting the resulting $3{,}072$-token index sequence. To keep spatial structure during autoregression, the paper introduces TriPE, which adds 2D rotary positions within each plane and 1D rotary positions across the three planes. The decoder adds feature deformation and plane-interaction attention to reconstruct fine geometry, upsampling to $256\\times256$ and predicting occupancy at query points. On ShapeNet and Objaverse, the method reports Chamfer Distance $0.066$ versus $0.117$ for the best baseline and F-Score $0.303$ versus $0.226$, with visual results showing more coherent geometry than multiview and diffusion competitors.","pith_inferences":["Quality gains from larger triplane grids are small ($0.066$ to $0.062$ Chamfer Distance from $3\\times32\\times32$ to $3\\times48\\times48$) while inference time doubles, suggesting the tokenizer and decoder, not sequence length, are the current quality ceiling.","The sequence ordering is a design choice; alternative orderings that better respect 3D locality could yield further gains, since autoregressive models are sensitive to token order.","Sharing a codebook with 2D image tokenizers and predicting image and 3D tokens in one sequence is a natural extension that could unify 2D and 3D generation in a single next-token model."],"forward_implications":["Sequence length for a 3D asset becomes a fixed $3{,}072$ tokens regardless of mesh complexity, so autoregressive generation can scale to industrial assets without the polygon-count blowup of mesh-face methods.","The same GPT with prefilling tokens handles both image and text conditioning, pointing toward a single autoregressive model for multimodal 3D generation.","TriPE's fused 2D and 1D positions preserves geometry that plain 1D rotary encoding loses, making position encoding a first-class factor in autoregressive 3D quality.","Plane-interaction attention raises reconstruction F-Score from $0.661$ to $0.822$, showing cross-plane information exchange is needed for fine-grained occupancy prediction.","Classifier-free guidance at scale 7.5 improves prompt alignment, so inference-time guidance transfers from image and text autoregression to 3D generation."],"supporting_citations":[{"why":"supplies the vector-quantized codebook mechanism the 3D tokenizer builds on","marker":"[64]"},{"why":"supplies codebook training with adversarial losses that improve reconstruction fidelity","marker":"[22]"},{"why":"prior autoregressive mesh generation whose long face sequences motivate fixed-length triplane encoding","marker":"[56]"},{"why":"supplies the decoder-only transformer backbone and autoregressive training recipe used for the 3D GPT","marker":"[59]"},{"why":"supplies the triplane latent representation and the upsampling decoder design","marker":"[72]"},{"why":"supplies rotary position embedding, which TriPE extends into fused 2D and 1D positions","marker":"[57]"},{"why":"the strongest image-to-3D baseline that TAR3D compares against in Table 1","marker":"[73]"},{"why":"multiview generation baseline used in the quantitative and qualitative comparisons","marker":"[41]"}],"fun_headline_variants":["GPT models 3D shapes part by part, beating diffusion","Triplane tokenization: a new SOTA for 3D asset generation","Autoregressive next-part prediction outperforms diffusion in 3D","TAR3D turns meshes into tokens for superior 3D generation","Part-by-part GPT reconstruction edges out diffusion in 3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach depends on the fixed $3\\times32\\times32$ triplane codebook preserving enough geometric detail; if thin structures or fine topology are lost in quantization, no amount of sequence modeling can recover them.","fun_headline_variants_meta":{"raw":{"variants":["GPT models 3D shapes part by part, beating diffusion","Triplane tokenization: a new SOTA for 3D asset generation","Autoregressive next-part prediction outperforms diffusion in 3D","TAR3D turns meshes into tokens for superior 3D generation","Part-by-part GPT reconstruction edges out diffusion in 3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001018,"raw_usage":{"total_tokens":4287,"prompt_tokens":927,"completion_tokens":3360,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":3267}},"tokens_in":543,"tokens_out":3360,"duration_ms":20762,"temperature":1.0,"reasoning_tokens":3267,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:58:20.787548+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a set of thin or highly structured objects, such as chair backs, bicycle spokes, or ropes, through the 3D VQ-VAE and compare occupancy F-Score against the continuous VAE counterpart; a large drop on those shapes would show the tokenizer is the bottleneck. An independent benchmark in which multiview or diffusion methods beat TAR3D on geometric fidelity for the same prompts would also contradict the claimed margin.","supporting_citations":[{"cited_title":"Meshgpt: Generating triangle meshes with decoder-only transformers","cited_arxiv_id":null,"evidence_quote":"prior autoregressive mesh generation whose long face sequences motivate fixed-length triplane encoding"}],"review_version":1}