Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A shape-conditioned transformer trained on 120K human-annotated primitive assemblies can regenerate a broad range of 3D objects as sequences of cuboids, elliptical cylinders, and ellipsoids, with geometric fidelity close to…

desk verdict A solid auto-regressive framework and a large human-annotated primitive dataset, but the human-perception claim rests on thinner evidence than the paper suggests. read the letter →

arxiv 2505.04622 v1 pith:YTHKSV2D submitted 2025-05-07 cs.GR cs.CV

Jingwen Ye , Yuze He , Yanning Zhou , Yiqin Zhu , Kaiwen Xiao , Yong-Jin Liu , Wei Yang , Xiao Han This is my paper · ORCID
classification cs.GRcs.CV
keywords 3Dshapeabstractionprimitiveassemblygenerationauto-regressivetransformerhuman-annotateddecompositionsdecompositionpoint-cloudconditioningparameterizationcontent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that shape primitive abstraction—breaking a 3D mesh into simple geometric elements—can be recast as an autoregressive sequence-generation task learned from human-annotated examples. The payoff would be a single primitive-based representation that transfers across many object categories, is editable by non-experts, and is compact enough for game-ready user-generated content. The proposed model, PrimitiveAnything, directly learns the order and parameters of primitive assembly from 120K human-crafted decompositions, rather than optimizing geometric fit alone, and claims to produce assemblies that both match geometry and align with how people perceive and build shapes.

What carries the argument

The central mechanism is the ambiguity-free parameterization scheme: each primitive is represented by a type label plus translation, rotation in Euler angles, and scale, and symmetry-equivalent parameter tuples are canonicalized by composing the primitive's rotational symmetry set with the current transform and selecting the tuple with minimal L1 rotation norm. This removes the one-to-many ambiguity that symmetry creates—for example, a cuboid's 90-degree rotation combined with an axis swap of scale factors produces an identical shape—so the autoregressive transformer is trained on a stable sequence-to-primitive mapping. The architecture couples a primitive encoder, a decoder-only transformer, a cascaded primitive decoder that predicts class, translation, rotation, and scale in that order, and an EOS decoder, with Michelangelo point-cloud tokens serving as the conditioning signal.

What would settle it

Run a forced-choice user study on fresh shapes from categories absent from HumanPrim, where participants compare PrimitiveAnything's output against human-authored primitive assemblies, and also measure inter-annotator agreement on those same shapes; if participants do not significantly prefer the human assemblies over the model's, or if pairwise human agreement is low, then the central claim that the model reproduces human decomposition patterns is falsified.

Watch

Extended reading notes

Core claim

On its own terms, PrimitiveAnything claims that a shape-conditioned decoder-only transformer, trained on the HumanPrim dataset of 120K manually assembled primitive decompositions, can generate variable-length sequences of primitives from three types—cuboids, elliptical cylinders, and ellipsoids—whose union reconstructs the input shape. On the HumanPrim test set, the method reports lower Chamfer distance and Earth Mover's distance than optimization-based baselines, higher Voxel-IoU, and better scores on the segmentation metrics RI, VOI, and SC, which the paper interprets as closer alignment with human decomposition patterns. The model also produces plausible primitive assemblies for shapes from Objaverse and for ShapeNet chair categories it was not trained on, suggesting that the learned assembly logic generalizes beyond the training distribution.

Load-bearing premise

The load-bearing premise is that the HumanPrim annotations are a consistent, learnable ground truth for 'human perception' of shape decomposition; the paper itself acknowledges in the limitations that annotation styles vary and some assemblies over-segment, and no inter-annotator agreement statistic is reported, so if the annotations are idiosyncratic or do not transfer to unseen categories, the perceptual-alignment claim loses its foundation.

Editorial extensions

If this is right

  • Primitive abstraction becomes a scalable generation problem: a single model can output variable-length assemblies of multiple primitive types from an arbitrary point cloud, without per-category training.
  • The resulting representations are compact and editable—the paper reports over 95% storage reduction relative to meshes—which could enable primitive-based user-generated content in games and simple manipulation in standard graphics interfaces.
  • Because primitive type is handled as a learnable token, the framework can be extended to new primitive types without architectural changes, making the approach adaptable to richer primitive vocabularies.
  • When chained with text-to-3D or image-to-3D generators, the method can produce primitive-based 3D content directly from natural user conditions, as demonstrated with TRELLIS and SDXL in the paper.
  • The improved segmentation metrics indicate that the assemblies carry part-level structure useful for downstream recognition or editing tasks, not just surface fidelity.
  • The method's generalization across Objaverse suggests that human assembly logic, as captured by the annotations, transfers to shape categories beyond the training set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same canonicalization recipe could be applied to richer primitive vocabularies—superquadrics, convex polytopes, or CSG operators—because the framework treats primitive type as a token and keeps the sequence mechanics unchanged.
  • The z-y-x centroid sorting of primitives is a design choice; testing alternative orderings (by size, by semantic part, or by connected component) could reveal how much of the learned 'human logic' actually resides in the ordering versus the transformer's predictive capacity.
  • A direct test of the human-alignment claim would be an inter-annotator agreement study on fresh meshes: without a high baseline agreement among human annotators, the ground-truth 'human perception' is itself an unstable target, regardless of model scores.
  • The paper's storage-efficiency and editability claims could be quantified further by measuring how much a user must manipulate the generated primitives to achieve a target edit, which the paper does not currently report.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes PrimitiveAnything, an autoregressive transformer that reformulates 3D shape primitive abstraction as a sequence-generation task. It introduces HumanPrim, a dataset of 120K human-crafted primitive assemblies; an ambiguity-free parameterization that unifies cuboids, elliptical cylinders, and ellipsoids; and a cascaded decoder that predicts primitive class, translation, rotation, and scale autoregressively. The model is conditioned on point clouds and trained with cross-entropy, EOS, and auxiliary Chamfer losses. Experiments compare against optimization-based methods (EMS, Marching-Primitives) and learning-based methods (Tulsiani et al. 2017, Paschalidou et al. 2019) on geometric and segmentation metrics, with ablations, a small user study, and qualitative generalization results. The paper claims that the method generates high-quality primitive assemblies that better align with human perception while maintaining geometric fidelity across diverse shape categories.

Significance. If the results hold, the work is significant: it provides a large-scale human-annotated dataset for primitive abstraction, a unified tokenization for multiple primitive types, and a generative formulation that can produce variable-length assemblies across categories. The reported geometric improvements over the considered baselines and the component ablations support the usefulness of the framework for reconstruction and editing. The main weakness is that the perceptual-alignment claim is not matched by the evidence: the annotation ground truth is not validated for inter-annotator consistency, and the primary quantitative evidence for 'human alignment' is measured against the same annotation pool used for training. The strengths are the dataset construction effort, the explicit ambiguity-free parameterization design with an ablation, and the inclusion of both geometric and segmentation metrics plus a user study.

major comments (5)
  1. [Section 4.1, Appendix C, Tables 2 and 4] The central claim of better alignment with human perception rests on the assumption that HumanPrim is a consistent, learnable signal of human decomposition, but the paper reports no inter-annotator agreement and Appendix C explicitly concedes that 'some annotators tend to use more primitives than necessary ... resulting in over-segmentation in certain cases.' Because the RI/VOI/SC metrics in Tables 2 and 4 are computed by transferring labels from the same HumanPrim annotations used for training, these numbers chiefly measure agreement with one annotation pool's conventions rather than with human perception generally. Please report pairwise annotator agreement on a double-annotated subset and validate the perceptual claim on independent annotations or a substantially larger user study.
  2. [Tables 1-5 and 7] No quantitative table reports error bars, confidence intervals, or the number of independent runs or seeds over which the means are computed. Without this information, the reported margins (e.g., CD 0.0404 versus 0.0546 in Table 1) cannot be assessed for statistical significance. Please provide mean plus/minus standard deviation (or confidence intervals) and a repeated-seed or resampling protocol.
  3. [Appendix A, Table 6] The user study uses 30 participants, 20 shapes, and only two optimization-based baselines, and it reports average scores without any dispersion or significance test. This is too narrow to independently support the perceptual-alignment claim made in the abstract. Please expand the study to more shapes and include at least one learning-based baseline, with per-participant variance and a significance test, or soften the perceptual claim accordingly.
  4. [Section 4.2 and Fig. 9] The generalization claim across diverse shape categories is supported quantitatively only on the ShapeNet chair category, which is a single-category cross-dataset test rather than a test of cross-category generalization, while the Objaverse evaluation is qualitative. Please provide quantitative held-out-category evaluation (e.g., a test split excluding categories seen in training) or an explicit analysis of train/test nearest-neighbor distances with numbers, to substantiate the 'across diverse shape categories' claim.
  5. [Equations (3)-(6)] The 'ambiguity-free' property is not proven. Equation (4) selects the symmetry rotation with minimal L1 norm, but the paper does not show that this arg min is unique across the discretized parameter space, and ties would reintroduce the mode confusion the scheme is designed to remove. Please prove uniqueness under the discretization or explicitly state and analyze the tie-breaking rule.
minor comments (6)
  1. [Equation (5)] Equation (5) uses \hat s_k, \hat r_k, \hat t_k on the left-hand side but s_k, r_k on the right-hand side; the composition order and notation should be clarified.
  2. [Figure 3] Figure 3 contains confusing labels ('rb rbra', 'ra''); please redraw or clarify the caption and the text that refers to it.
  3. [Section 4.1] The dataset name is misspelled as 'Objarverse' in Section 4.1 and the misspelling is repeated in Appendix A; please correct it to 'Objaverse'.
  4. [Section 4.4] The 'over 95% storage reduction' claim lacks a definition of the compared mesh representation and measurement protocol; please specify how the comparison was performed.
  5. [Section 4.1] The label-transfer procedure for RI/VOI/SC is described in one sentence; please provide explicit steps, including how empty or duplicate predicted primitives are handled.
  6. [Appendix A, Generalization Analysis] 'We employs the pointbert-vitg14' contains a grammatical error; the retrieval experiment would also be stronger if reported with quantitative similarity scores rather than qualitative examples only.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: predictions are trained and held-out tested on HumanPrim annotations with external geometric and user-study benchmarks; the only self-citation is non-load-bearing.

full rationale

The paper's derivation chain is self-contained. PrimitiveAnything is trained with a next-primitive cross-entropy objective plus an auxiliary Chamfer loss against ground-truth HumanPrim primitives (Sec. 3.3, Eqs. 14-15), and the central 'human alignment' evidence is a held-out test set (314 samples, Sec. 4.1) measured by RI/VOI/SC against the same annotation protocol, plus an external 30-participant user study on Objaverse (Appendix A) and geometric metrics against original meshes (Sec. 4.1). Held-out agreement with the training-label distribution is standard supervised generalization, not a construction that forces the result; no parameter is fitted to the test labels or to the user-study ratings. The ambiguity-free parameterization (Sec. 3.1) is a canonicalization of symmetric primitives, not a result assumed from its conclusion. The only self-citation, [He et al. 2024] for the label-transfer protocol, is not load-bearing, since the protocol is fully described in the paper. Appendix C's conceded annotation-style diversity and over-segmentation, and the absence of inter-annotator agreement, undermine the strength of the 'human perception' claim as a validity/correctness issue, but they do not make any equation or prediction reduce to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the dataset, the primitive vocabulary, the canonicalization rule, and the conditioning signal. No physical or mathematical constants are derived; the assumptions are about data quality, representation adequacy, and the validity of the ambiguity-free scheme.

free parameters (3)
  • Primitive type set = cuboid, elliptical cylinder, ellipsoid
    Chosen by the authors; limits expressiveness, and the limitations section admits that topological structures like ring shapes fail.
  • Discretization levels = rotation 180, scale 128, translation 128 per dimension
    Hand-set design choices that control prediction granularity and training difficulty; no sensitivity analysis is reported.
  • Model hyperparameters = 12 layers, hidden 768, LR 1e-3, batch 128
    Standard architecture choices; not justified by controlled experiments in the paper.
assumptions (4)
  • domain assumption HumanPrim annotations reflect human perception of shape decomposition.
    The central perceptual-alignment claim rests on treating the annotations as ground-truth human behavior; no independent cognitive benchmark or inter-annotator agreement is provided.
  • domain assumption Three primitive types (cuboids, elliptical cylinders, ellipsoids) suffice to represent diverse shape categories.
    The method's expressiveness is bounded by this vocabulary; the limitations section admits failure on ring-like shapes with holes.
  • ad hoc to paper The canonicalization rule in Eq. 4 yields a unique, ambiguity-free parameterization.
    The min-L1-norm selection over the symmetry group is a heuristic; no proof of uniqueness across all symmetry combinations is given.
  • domain assumption Point clouds are a sufficient conditioning signal for primitive assembly generation.
    The model uses only sampled point clouds as input; no analysis of sensitivity to point density or noise is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer." pith.science (2026). https://pith.science/paper/YTHKSV2D

@misc{pith2026250504622,
  author       = {Pith},
  title        = {Pith review of: PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YTHKSV2D}},
  note         = {Machine review of arXiv:2505.04622}
}
read the original abstract

Shape primitive abstraction, which decomposes complex 3D shapes into simple geometric elements, plays a crucial role in human visual cognition and has broad applications in computer vision and graphics. While recent advances in 3D content generation have shown remarkable progress, existing primitive abstraction methods either rely on geometric optimization with limited semantic understanding or learn from small-scale, category-specific datasets, struggling to generalize across diverse shape categories. We present PrimitiveAnything, a novel framework that reformulates shape primitive abstraction as a primitive assembly generation task. PrimitiveAnything includes a shape-conditioned primitive transformer for auto-regressive generation and an ambiguity-free parameterization scheme to represent multiple types of primitives in a unified manner. The proposed framework directly learns the process of primitive assembly from large-scale human-crafted abstractions, enabling it to capture how humans decompose complex shapes into primitive elements. Through extensive experiments, we demonstrate that PrimitiveAnything can generate high-quality primitive assemblies that better align with human perception while maintaining geometric fidelity across diverse shape categories. It benefits various 3D applications and shows potential for enabling primitive-based user-generated content (UGC) in games. Project page: https://primitiveanything.github.io

Figures

Figures reproduced from arXiv: 2505.04622 by the authors.

Figure 1
Figure 1. 3D primitive assemblies created by PrimitiveAnything span diverse shape categories, enabling versatile primitive-based 3D content creation. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview. We propose PrimitiveAnything to decompose complex shapes into 3D primitive assembly via the auto-regressive transformer. Given [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Demonstration of primitive attribute ambiguity. A primitive with [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons on the HumanPrim test set: In our method, colors indicate different primitive types, while in Marching Primitives and EMS, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: PrimitiveAnything interfaces with state-of-the-art 3D shape gen [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: More qualitative comparisons with optimization-based methods on the HumanPrim dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Comparisons on the HumanPrim test set (Chair subset). [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Comparisons on the Objaverse dataset [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: More visualizations of primitive-based 3D content generation on text and image conditions. [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparisons of ablation study. [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Generalization capability of our method. Top row: unseen test shapes; Middle row: our primitive decomposition results; Bottom row: geometrically [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Statistical characteristics of our HumanPrim dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 15
Figure 15. Figure 15: This challenge stems partly from our current primitive types [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]
Figure 14
Figure 14. Figure 14: Symmetry order calculation for the x-axis of a cuboid. Axis permutation (swapping y and z axes) after 90 [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Failure cases with out-of-distribution inputs. [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers

    cs.CV 2025-06 conditional novelty 6.0 of 10

    PartCrafter generates several separable 3D part meshes at once from a single image by fine-tuning a pretrained 3D diffusion transformer with part identity tokens and local-global attention.

Reference graph

Works this paper leans on

8 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [4]

    In ACM SIGGRAPH 2024 Conference Papers

    Diffusion texture painting. In ACM SIGGRAPH 2024 Conference Papers . 1–12. Xiaoyang Huang, Yi Zhang, Kai Chen, Teng Li, Wenjun Zhang, and Bingbing Ni. 2023. Learning shape primitives via implicit convexity regularization. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 3642–3651. Ka-Hei Hui, Aditya Sanghi, Arianna Rampini, Kam...

    arXiv 2023

  2. [6]

    In ACM SIGGRAPH 2024 Conference Papers

    Gem3d: Generative medial abstractions for 3d shape synthesis. In ACM SIGGRAPH 2024 Conference Papers . 1–11. Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023). Ben Poo...

    arXiv 2023

  3. [2017]

    ACM Transac- tions on Graphics (TOG) 36, 4 (2017), 1–14

    Grass: Generative recursive autoencoders for shape structures. ACM Transac- tions on Graphics (TOG) 36, 4 (2017), 1–14. Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas J Guibas. 2019. Supervised fitting of geometric primitives to 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 265...

    arXiv 2017

  4. [2019]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Bae-net: Branched autoencoder for shape co-segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 8490–8499. Laurent Chevalier, Fabrice Jaillet, and Atilla Baskurt. 2003. Segmentation and su- perquadric modeling of 3D objects. (2003). Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt,...

    arXiv 2003

  5. [2020]

    Advances in neural information processing systems 33 (2020), 1877–1901

    Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901. Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository . Technica...

    arXiv 2020

  6. [2022]

    Advances in Neural Information Processing Systems 35 (2022), 10021–10039

    Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems 35 (2022), 10021–10039. Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. 2023a. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vi...

    arXiv 2022

  7. [2023]

    arXiv preprint arXiv:2302.13971 (2023)

    Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023). Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. 2017. Learning shape abstractions by assembling volumetric primitives. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2635–2643. Arash Vahdat, Fra...

    arXiv 2023

  8. [2024]

    AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videos

    AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videos. arXiv preprint arXiv:2411.19950 (2024). Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. 2024. LRM: Large Reconstruction Model for Sin- gle Image to 3D. In The Twelfth International Confere...

    work page Pith review arXiv 2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.