{"id":"6015aaa1-7a69-4993-b4ab-b5f53f2ca0df","arxiv_id":"2412.18608","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A multi-view diffusion pipeline that segments 3D objects into parts, completes occluded or invisible parts, and reconstructs them into a compositional 3D asset.","lead":"PartGen is a pipeline that takes a text prompt, an image, or an unstructured 3D model and produces a 3D object broken into meaningful, editable parts. It uses two diffusion models: one to label parts across multiple views, and one to fill in parts that are hidden or missing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The artist-GLTF part assumption is unvalidated: if node decompositions reflect modeling convenience rather than semantics, the 'meaningful parts' claim and all downstream metrics are dataset-specific.","rationale":"The reader's weakest assumption is the artist-GLTF part semantics, and I agree that is the right focus. The strongest claim in the abstract is not just that the method can produce 3D assets with parts, but that those parts are meaningful and that PartGen beats baselines 'by a large margin.' Meaningfulness is a semantic property that the paper never directly validates: the only ground truth is the internal licensed dataset's node decomposition, filtered by volume and count, and the paper itself flags the dependence on dataset quality and diversity in the Limitations. Without evidence that the GLTF node hierarchies correspond to human-recognizable parts (or a transfer evaluation to an external part benchmark), the quantitative superiority in Table 1 may only show that a generative model trained on artist decompositions matches those decompositions better than SAM2, which is not the same as the general-purpose part understanding claimed. I do not see an internal inconsistency in the method: the architecture, training losses, and evaluation protocol are described coherently, and the completion ablations (Table 2) support the importance of context and multi-view joint reasoning. The semantic foundation, however, is external and unmeasured. The proposed concrete test is a standard cross-dataset transfer: apply the same segmentation model to ShapeNetPart or PartNet-Mobility views and compare with the same baselines. That would test whether the learned part concept transfers beyond the licensed data distribution. Because the concern is about generalization rather than a demonstrable internal flaw, and because the paper already acknowledges dataset dependence, the conditional verdict remains appropriate: with code/data and a transfer evaluation, the claim would be much stronger; without them, the 'large margin' claim should be read as dataset-specific.","tokens_in":21949,"tokens_out":6256,"duration_ms":62842,"concrete_test":"Render a held-out set of objects from an external dataset with human semantic part annotations (e.g., ShapeNetPart or PartNet-Mobility) using the same 2x2 multi-view protocol, run the published PartGen segmentation model (or, failing code release, a reimplementation by the authors) on those views, and compute mAP50/recall@K with Hungarian matching after aligning predicted parts to annotated parts by majority overlap. Compare the resulting numbers to the same SAM2 baselines from Table 1. If the PartGen-vs-SAM2 margin largely disappears or the external mAP50 falls below roughly 40, the learned part concept is dataset-specific; if the margin persists, the data assumption is substantially supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that PartGen outputs objects composed of meaningful, human-interpretable parts and that it outperforms segmentation/part-extraction baselines by a large margin. The entire notion of 'meaningful' is inherited from the Sec. 3.5 training data assumption: the 140k licensed GLTF assets' native mesh decompositions 'often align with semantic parts due to being created by a human who likely aimed to create an editable asset.' The paper filters small parts and objects with more than 10 parts, but provides no evidence about how often the GLTF node hierarchy actually matches semantic parts, no inter-annotator agreement statistics, and no cross-dataset evaluation of the segmentation concept. The self-stated limitation in Sec. E ('heavily dependent on the quality and diversity of the dataset') explicitly concedes this point. If many node decompositions are driven by modeling convenience (separate materials, export groups, geometry islands) rather than semantics, the diffusion segmentation model learns a dataset-specific partition, and Table 1's mAP numbers, all measured against the same artist-decomposition distribution, overstate generalization to arbitrary assets. Since completion, reconstruction, editing, and reassembly all operate on these parts, every downstream claim inherits this potential bias.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"PartGen is a two-stage pipeline for generating or decomposing 3D assets into parts. Given text, an image, or an unstructured 3D object, the method first renders a four-view grid and uses a fine-tuned multi-view diffusion model to sample color-coded part segmentation maps, merging multiple stochastic samples into a ranked list of part proposals. It then uses a second multi-view diffusion model to complete each part's masked views, conditioned on the full-object context, and reconstructs the completed part with LightplaneLRM. Parts are finally assembled into a compositional 3D object. The paper reports that the segmentation stage substantially outperforms SAM2 and Part123 on mAP (Table 1), that contextual completion improves part-view PSNR/LPIPS/CLIP over no-completion and context-free baselines (Table 2), and that part-based reassembly matches monolithic reconstruction (Table 3). Applications demonstrated include part-aware text/image-to-3D, decomposition of Google Scanned Objects, iterative part addition, and text-guided part editing.","tokens_in":22183,"tokens_out":5071,"duration_ms":54005,"significance":"If the underlying artist-decomposition assumption holds, PartGen is a strong and timely contribution. The core ideas—treating part segmentation as stochastic multi-view coloring and treating part completion as generative, context-conditioned inpainting—are natural fits for the inherent ambiguity of part boundaries and amodal completion, and the reported quantitative gains are large: Table 1 shows 59.3 mAP50 for PartGen versus 35.3 for SAM2, and Table 2 shows PSNR increasing from 13.24 without completion to 21.38 with contextual completion. The ablations in Table 2 isolate the contributions of multi-view reasoning and object context, which is good experimental practice. The supplementary material is detailed and the method is built from reasonably described components. The main caveat is that the definition of 'meaningful part' is inherited from the raw node hierarchies of licensed artist-created GLTF assets, and all quantitative evaluation is performed on a held-out split of the same collection. The paper's self-stated limitation in Sec. E that the approach is 'heavily dependent on the quality and diversity of the dataset' is directly relevant to this concern.","major_comments":[{"comment":"The central assumption that GLTF node decompositions 'often align with semantic parts due to being created by a human who likely aimed to create an editable asset' is not validated. The paper culls parts smaller than 5% of object volume and removes assets with more than 10 parts, but it provides no evidence about how often the node hierarchies actually match human semantic judgments, no inter-annotator agreement statistics, and no evaluation on any external dataset. Since both training and the held-out test set come from the same licensed collection, the mAP numbers in Table 1 measure how well the model reproduces that collection's particular decomposition convention, not whether the resulting parts are semantically meaningful for arbitrary assets. This is load-bearing for the abstract's claim of 'meaningful parts' and for all downstream applications. Please add a human study on part validity, or an external benchmark with independent part annotations (e.g., PartNet, PartImageNet, or manually annotated GSO objects), and report results per category and across datasets; alternatively, the generalization claims should be explicitly narrowed.","section":"Sec. 3.5 and Table 1"},{"comment":"The completion and reconstruction evaluation uses ground-truth masks: the test tuples are (S, I, M^k, J^k), and one random part per object is selected. This isolates completion quality from segmentation errors, which is a reasonable design, but it means the main quantitative evidence for the full pipeline is Table 3 combined with qualitative figures. Table 3 compares the reassembled part-based reconstruction with an unstructured reconstruction by the same reconstruction model; this shows that part-based assembly does not lose much reconstruction fidelity, but it does not establish that the parts are semantically correct or that they match artist intent. Please either report an end-to-end metric that feeds predicted masks into the completion and reconstruction stages and evaluates part-level accuracy, or explicitly state in the main text that Table 2 is an oracle-mask evaluation and that end-to-end part quality is demonstrated only qualitatively.","section":"Sec. 4.2 and Table 2"},{"comment":"The automatic-segmentation comparison may not be proposal-matched. PartGen is evaluated with 1, 5, and 10 stochastic samples and then uses an overlap-based reliability score with non-maximum suppression to build a ranked list, whereas the SAM2 baselines are seeded with a set of query points and return up to three regions per seed. The paper does not control for the number of proposals per object, and a retrieval-style mAP with ten merged samples can be inflated simply by the increased diversity of the candidate pool. The recall-at-K curves in Fig. 9 are helpful, but please also report precision at fixed recall, recall at equal proposal counts, or precision-recall curves for each method, and state explicitly how many proposals each baseline produces on average. Without this, the 'large margin' over SAM2 in the automatic setting is not fully quantified.","section":"Sec. 4.1 and Sec. B"},{"comment":"Several hand-set thresholds control the data and the method: the 5% volume culling threshold, the maximum of 10 parts per object, the four orthogonal views, and the number of quantized colors Q. No sensitivity analysis is reported for any of these choices. Because the segmentation network is trained only after these filters are applied, the reported mAP and completion metrics could depend substantially on them. Please report the effect of varying the volume threshold (e.g., 2%/5%/10%), the maximum part count (e.g., 6/10/15), and the number of views, or justify these choices with evidence that the results are stable.","section":"Sec. 3.5 and Sec. 4"}],"minor_comments":[{"comment":"The abstract and introduction state that the segmentation maps are 'view-consistent,' but no quantitative view-consistency metric is reported; consider adding a measure such as cross-view mask IoU or projecting masks into 3D and measuring 3D consistency.","section":"Abstract and Sec. 1"},{"comment":"The table uses SAM2† and SAM2∗ with a caption explanation, but the symbols are easy to confuse; consider renaming them 'SAM2-MV' and 'SAM2-FT' or adding an explicit legend in the table itself.","section":"Table 1"},{"comment":"The text says 'We choose one random part index k per model' for evaluation; since the choice is random, the reported numbers may depend on the random seed. Please report mean and standard deviation over multiple random choices, or evaluate all parts.","section":"Sec. 4.2"},{"comment":"The sentence 'This results in the dataset of 45k objects contain the total of 210k parts' contains a grammatical error ('contain'); please revise to 'containing a total of 210k parts.'","section":"Sec. 3.5"},{"comment":"The failure-case section notes that the implementation is limited to objects with no more than 10 parts, but this limitation is not mentioned in the main-text Limitations section (Sec. E); please state it prominently in Sec. E as well.","section":"Sec. D"},{"comment":"The conditioning of the completion network includes the mask M, but the mask is provided as a single unencoded channel while the masked image and context are encoded; a sentence explaining why the mask is not VAE-encoded would help reproducibility.","section":"Sec. 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's quantitative evaluation is solid internally, but the proprietary nature of the training data and the lack of any external validation of the part semantics are the main risks. A human study or a cross-dataset evaluation would substantially increase confidence. Given the paper's strong ablations and clear writing, I believe the concerns are addressable within the scope of a revision rather than requiring rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: PartGen is a serious, well-built pipeline paper. It does what it says on its own test set: it segments 3D objects into clean, view-consistent parts and—more interestingly—completes heavily occluded or fully invisible parts using a second multi-view diffusion model conditioned on the whole object. The numbers back the core claim. Segmentation mAP50 is 59.3 with 10 samples versus 35.3 for SAM2 and 11.5 for Part123. Completion improves part-view PSNR from 12.32 (no completion) to 21.38, and the ablation shows both context and multi-view reasoning matter. The oracle reconstruction bound is 18.91, so the system is near the ceiling of what the reconstruction model allows.\n\nThe genuinely new piece is the integration: stochastic multi-view diffusion for part masks, then a context-conditioned completion network that amodally reconstructs each part, then standard LRM reconstruction. Prior part-level works skip the completion or do it without the global context. The method is well described and the ablations are informative.\n\nThe soft spot is exactly what you'd worry about. The training data is 140k licensed GLTF assets, and the 'ground-truth' parts are the artist node decompositions. The paper assumes these are semantic because artists made them, then filters parts below 5% volume and objects with more than 10 parts. There is no validation of that assumption: no human agreement study, no cross-dataset evaluation. All the mAP numbers are measured on a held-out set from the same distribution, so they measure how well the model mimics the average artist decomposition in that particular store, not how well it matches general semantic parts. The paper's own limitation section concedes dataset dependence. This is a genuine limitation, but not a fatal flaw. The system is still useful: it clearly produces editable, plausible parts for a large class of objects, and the completion mechanism is valuable regardless of the exact taxonomy.\n\nOther quibbles: no error bars on the main tables, no code or data release, and the 'hallucinate invisible parts' claim is only shown qualitatively. These are standard for an applied paper of this scale but should be flagged.\n\nWho should read this: anyone working on 3D asset generation, editing, or decomposition. It deserves a serious referee, and I'd accept it for peer review with the expectation that the part-semantics question be addressed with an external dataset or human study.","headline":"A well-engineered pipeline that convincingly demonstrates part segmentation and amodal completion on its own dataset, but the part-semantics assumption needs external validation.","tokens_in":22708,"tokens_out":2801,"would_cite":true,"duration_ms":26445,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PartGen uses two multi-view diffusion models to split 3D objects into meaningful, complete parts that can be edited and reassembled, and the paper reports it outperforms segmentation baselines by a large margin.","keywords":["part-level 3D generation","multi-view diffusion","3D part segmentation","part completion","amodal reconstruction","3D part editing","text-to-3D","image-to-3D"],"falsifier":"Give PartGen object classes absent from its training corpus, such as mechanical devices or biological specimens, and compare its segmentations against parts annotated by independent human judges using a functional definition of 'part' (a component that can be removed, moved, or replaced without destroying the object); if mean average precision against the functional ground truth is no better than against random groupings, the artist-GLTF grouping assumption fails.","tokens_in":21768,"feed_emoji":"🧩","tokens_out":7025,"duration_ms":57626,"temperature":0.7,"pith_summary":"PartGen aims to show that unstructured 3D content, whether generated from text or an image or captured by a scanner, can be converted automatically into a composition of meaningful, standalone parts, the way a human artist would build the asset. It claims that two fine-tuned multi-view diffusion models are enough: one paints view-consistent colour-coded part masks across a four-view grid, and a second completes the masked views of each part using the whole object as context, so that hidden or entirely invisible geometry is plausibly filled in. The completed multi-view images are then lifted to 3D by a feed-forward reconstruction network. The paper reports that this beats segmentation baselines by a large margin and that the reassembled parts match the quality of unsegmented reconstruction, which would mean structure can be added to modern 3D generation and scanning pipelines without sacrificing fidelity.","feed_headline":"Two diffusion passes turn 3D assets into editable parts","feed_subtitle":"Generative completion even hallucinates hidden parts, so scanned or generated assets become editable compositions.","key_machinery":"The load-bearing machinery is a pair of fine-tuned multi-view diffusion generators built from a pretrained text-to-image diffusion model in an 8-channel latent space. The segmentation network takes a four-view grid image as conditioning and generates a colour-coded segmentation grid, with the mapping from parts to colours randomly permuted per training sample so the model learns part boundaries rather than part identities. The completion network takes a 25-channel input consisting of the noised latent, the encoded masked part views, the encoded full object views for context, and the unencoded part mask, and generates completed multi-view images of the part; conditioning on the intact object is what lets the model hallucinate invisible parts that still fit the whole. The reconstruction step uses a feed-forward reconstruction model that turns the completed multi-view images into a 3D field per part, and parts are assembled with an emission-absorption renderer that sums per-part opacities and features along each ray.","core_discovery":"PartGen's central claim is that part-level 3D structure can be produced by reusing the two-stage paradigm of modern 3D generation, multi-view diffusion followed by a reconstruction network, rather than by inventing a dedicated 3D segmentation architecture. The first contribution is to frame segmentation as a stochastic multi-view colouring problem: a diffusion model, conditioned on a four-view grid image of the object, generates colour-coded masks whose colour-to-part assignment is randomized at training time, which lets the model capture the ambiguity in how artists split an object into parts and sidesteps instance-naming issues. The second contribution is contextual completion: a diffusion model takes the masked views of one part together with the full object views and generates complete, view-consistent images of that part, inferring occluded or completely invisible geometry from the object's overall context. Completed part views are fed to a reconstruction model to obtain full 3D parts, and the parts are composited by a modified emission-absorption renderer. The paper demonstrates the pipeline on text-to-3D, image-to-3D, decomposition of real scans, and text-guided part editing.","pith_inferences":["The random colour-permutation trick is a transferable idea: any multi-view instance-segmentation problem with unknown identity counts could use a generative model conditioned on a permuted-colour target to avoid matching and naming, not just 3D part decomposition.","Conditioning part completion on the whole object suggests a scalable route to amodal scene decomposition: if training data with scene-level part or object decompositions existed, the same context-conditioned completion could plausibly recover objects hidden behind other objects, not just parts hidden inside one asset.","A direct test of the artist-intent assumption would be to compare PartGen's parts against functional part annotations from naive users on object categories outside the training distribution; the paper's reliance on artist GLTF groupings makes this the main open empirical question.","The paper stops short of evaluating whether completed parts are physically usable for animation rigging or physics simulation; a follow-up measuring part-boundary alignment, watertightness of completed parts, and joint placement would tell whether the parts are not just visually plausible but mechanically meaningful."],"forward_implications":["Text- and image-to-3D generators that adopt this pipeline will output assets already split into meaningful parts, removing a manual step in creative workflows.","Decomposing a scanned or generated object no longer requires the part to be visible: the completion model can infer occluded internals or hallucinate entirely hidden parts while keeping them coherent with the whole.","The modular design means reconstruction models can be swapped without retraining the segmentation or completion networks, so the method tracks improvements in feed-forward 3D reconstruction.","Reassembled part-based reconstructions match the fidelity of unsegmented reconstructions, so structure is obtained without a fidelity penalty.","Because parts are independently reconstructed, applications like text-guided part editing, part replacement, and iterative part addition follow directly from the same machinery."],"supporting_citations":[{"why":"Supplies the multi-view generator recipe, including the Emu-style architecture, v-prediction, and SNR rescaling, and the captioning protocol for text-to-3D training data.","marker":"[76]"},{"why":"Establishes the two-stage sparse-view generation plus large reconstruction model paradigm and the four-view rendering setup used throughout PartGen.","marker":"[39]"},{"why":"Provides the LightplaneLRM reconstruction network that lifts completed multi-view part images to 3D.","marker":"[5]"},{"why":"The SAM2 baseline whose original and fine-tuned variants the segmentation method is compared against.","marker":"[70]"},{"why":"The Part123 part-aware 3D reconstruction baseline used for segmentation comparison.","marker":"[44]"},{"why":"The pre-trained text-to-image diffusion architecture that the multi-view generators are fine-tuned from.","marker":"[13]"},{"why":"The CAP3D pipeline used to generate the text captions that train the text-to-multi-view generator.","marker":"[52]"},{"why":"The Llama 3 model used inside the captioning pipeline to produce the training captions.","marker":"[16]"},{"why":"The latent diffusion inpainting conditioning setup that motivates the design of the completion network's context conditioning.","marker":"[71]"}],"fun_headline_variants":["Diffusion splits 3D into editable parts","PartGen: multi-view diffusion for part-level editing","3D parts from diffusion, even hidden ones","Turn any 3D object into editable parts","Diffusion recovers parts, even invisible ones"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method learns what counts as a 'part' from the native mesh groupings in a licensed collection of 140k artist-made GLTF assets, and it assumes those groupings reflect the semantic parts users actually want to manipulate.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion splits 3D into editable parts","PartGen: multi-view diffusion for part-level editing","3D parts from diffusion, even hidden ones","Turn any 3D object into editable parts","Diffusion recovers parts, even invisible ones"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1373,"prompt_tokens":1040,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":261}},"tokens_in":656,"tokens_out":333,"duration_ms":3855,"temperature":1.0,"reasoning_tokens":261,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:35:59.581834+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give PartGen object classes absent from its training corpus, such as mechanical devices or biological specimens, and compare its segmentations against parts annotated by independent human judges using a functional definition of 'part' (a component that can be removed, moved, or replaced without destroying the object); if mean average precision against the functional ground truth is no better than against random groupings, the artist-GLTF grouping assumption fails.","supporting_citations":[{"cited_title":"Meta 3D Asset Gen: Text-to-mesh gener- ation with high-quality geometry, texture, and PBR mate- rials","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-view generator recipe, including the Emu-style architecture, v-prediction, and SNR rescaling, and the captioning protocol for text-to-3D training data."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"The latent diffusion inpainting conditioning setup that motivates the design of the completion network's context conditioning."}],"review_version":1}