{"id":"16e1a11f-7ed4-48c7-8294-15a0254e0f02","arxiv_id":"2507.15454","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ObjectGS unifies 3D Gaussian scene reconstruction with object-level segmentation by binding each object to local anchors with fixed one-hot ID encodings, improving open-vocabulary and panoptic segmentation.","lead":"ObjectGS is a new method for 3D scene reconstruction that separates and understands individual objects while building the scene, using object-specific anchors that generate Gaussian splats. It outperforms prior methods on open-vocabulary and panoptic segmentation benchmarks and supports object-level editing and mesh extraction.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method drops background (ID 0) from the one-hot encoding and semantic loss, yet inference by Eq. 6 forces every background pixel into an object, so the reported panoptic segmentation numbers are either foreground-only or rely on an undocumented mechanism.","rationale":"The reader's weakest-assumption flags reliance on DEVA mask quality; that is a real external-validity concern, but it does not threaten the internal validity of the reported numbers when DEVA is accurate. The background-encoding gap is more load-bearing because it concerns the method's own definition: the one-hot ID encoding and semantic loss explicitly exclude ID 0, yet inference has no fallback label. Unless the implementation has a hidden background class (not described in Sec. 3.3 or the supplementary), the model cannot produce background predictions, making the 'panoptic' claim internally inconsistent with the method. The high accuracy in Table 2 suggests either the evaluation omits background pixels or an undocumented threshold is applied; either way, the central claim is not supported as written. This is addressable by adding a background dimension to the one-hot encoding or by clearly limiting claims to instance segmentation. I therefore recommend maintaining the CONDITIONAL verdict, with the added condition that the authors specify or revise the background handling and re-report the panoptic metrics accordingly.","tokens_in":15380,"tokens_out":13167,"duration_ms":145335,"concrete_test":"Run the released code (or a re-implementation) on the Replica scene from Table 2 and compute per-class IoU including the background class (ID 0) using the standard panoptic protocol. Additionally, render a held-out view and compare the predicted ID map to the DEVA ground truth: check whether any ground-truth background pixel receives a positive object ID. If the metric computation excludes ID=0 pixels, recompute Table 2 over the full label set; the reported panoptic numbers would likely drop.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 3.3(c) defines one-hot encodings only for object IDs 1..n (Eq. 4), and Section 3.4's cross-entropy loss (Eq. 7) sums over i=1..n, so pixels whose DEVA label ID'=0 (background/unclassified, as defined in Sec. 3.1) contribute no gradient. At inference, Eq. 6 takes the argmax over the n-dim probability vector P(x); a background pixel therefore receives the ID of whichever object has the largest (possibly spurious) alpha-blended probability. No background class exists in the representation. The paper's evaluation of 'panoptic segmentation' (Table 2) reports high Acc/IoU/Dice, but if those metrics include background pixels, the model as described would misclassify all of them; if they exclude ID=0 pixels, the evaluation is instance-only, not panoptic. The paper never specifies how background is predicted, whether a confidence threshold is used, or whether ID=0 anchors are simply discarded. This is a gap between the method as specified and the central claim of panoptic segmentation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"ObjectGS proposes a 3D Gaussian Splatting framework that couples object-level reconstruction with instance segmentation. Starting from DEVA/Grounded-SAM 2D masks, it lifts object IDs to the COLMAP point cloud by majority, probability-based, or correspondence-based voting, initializes Scaffold-GS anchors with object IDs, and grows or prunes anchors while inheriting IDs. Each generated Gaussian carries a fixed one-hot ID encoding; alpha-blended rendering of these encodings gives per-pixel classification probabilities, which are supervised by a cross-entropy loss. The paper reports open-vocabulary segmentation on LERF-Mask and 3DOVS, panoptic segmentation on Replica and ScanNet++, and applications in mesh extraction and scene editing.","tokens_in":15629,"tokens_out":5975,"duration_ms":68185,"significance":"The central design hypothesis—that discrete one-hot ID semantics avoid the alpha-blending ambiguity of learnable continuous semantic features—is well motivated and supported by the ablation in Table 5, where one-hot encoding (88.19 mIoU) clearly outperforms learnable semantics (69.57 mIoU) and object-independent constraints (37.48 mIoU) on the Figurines scene. The object-anchor coupling also yields clean 3D instance labels without per-Gaussian feature distillation, and the variable-length feature rasterizer is a practical engineering contribution. If the open questions about background handling and evaluation protocol are resolved, this is a useful step for object-aware 3DGS.","major_comments":[{"comment":"The background class is not representable in the model as specified. Section 3.1 assigns unclassified pixels ID 0, but Eq. (4) defines one-hot vectors only for IDs 1..n, Eq. (7) has no loss term for ID'(x)=0, and Eq. (6) takes argmax over the n object-only probabilities. A background pixel therefore receives the ID of the most probable object (or an undefined argmax where no Gaussian covers the pixel). Since Table 2 reports panoptic IoU/Dice/Acc on Replica and ScanNet++, which contain large background regions, the high scores imply either that ID 0 is represented somewhere outside the described equations or that background pixels are excluded from the metrics. Please specify how background is predicted (a background anchor, a confidence threshold, or exclusion), and state clearly whether the reported metrics include ID 0. If ID 0 pixels are excluded, the task should be described as instance-level rather than panoptic segmentation.","section":"§3.3(c), Eq. (4)–(7); Table 2"},{"comment":"The claim of 'significantly outperforming' state-of-the-art on LERF-Mask is not uniform: on the Figurines scene, Gaga achieves mIoU 90.7/mBIoU 89.0 while ObjectGS achieves 88.2/85.2. The paper reports no average or variance across scenes, so the statement in Section 4.2 overstates the evidence. Please report per-scene means with standard deviations, or rephrase the claim to identify the scenes where the improvement holds.","section":"§4.2, Table 1"},{"comment":"The robustness argument that the grow-and-prune mechanism 'can naturally correct some mislabeled points' is supported only qualitatively. Table 6 compares three voting strategies on the same clean DEVA masks, but it does not inject label noise into the initialization to measure correction. Given that the whole pipeline inherits object IDs from the initial point-cloud voting, a quantitative robustness test (e.g., randomly flipping a controlled fraction of initial point IDs and measuring final mIoU) would substantiate the claim; otherwise the limitation statement in Section 5 should be strengthened.","section":"§4.3, Table 6; §5 Limitation"},{"comment":"The ScanNet++ evaluation protocol is underspecified: the paper says 'randomly select 7 scenes' without naming them or reporting a seed, and Table 3 does not define how precision, recall, and F1 are aggregated over instances or whether the Chamfer distance is computed per predicted instance or per ground-truth instance. This information is needed to reproduce the reported numbers and to interpret the large recall gap (66.5% vs. 86.1%) in Table 3.","section":"§4.1–4.2, Tables 2 and 3"}],"minor_comments":[{"comment":"The caption mentions 'Grounded SAM 2' but the text and references use Grounded-SAM [37]; please align the terminology.","section":"Fig. 4 caption"},{"comment":"The headings 'Object ID V oting' and 'V oting Algorithm' contain a typo ('V oting' should be 'Voting').","section":"§3.1 and Supplementary §8"},{"comment":"Algorithm 3 is structurally identical to Algorithm 1 except for renaming 'Camera poses' to 'Correspondences'; the pseudocode should show how correspondences actually replace projection.","section":"Supplementary, Algorithm 3"},{"comment":"The scene names '1ada', '3e8b', and '0d2e' in Table 8 are not defined, and the table heading alternates between 'Scannet++' and 'ScanNet++'; please provide the full list of the seven ScanNet++ scenes used and unify the naming.","section":"§4.2, Table 2 and Table 8"},{"comment":"The text says Gaussian color attributes are extended from 3 to N+3, but Eq. (5) suggests the ID encoding is rendered as a separate channel; please clarify how the N-dimensional semantic vector is split from the color attributes during rasterization.","section":"§4.1 Implementation Details"},{"comment":"The indicator notation 1[ID'(x)=i] is used without definition; please define the indicator function explicitly.","section":"Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sound and the paper is a reasonable fit for the journal. The main technical risk is the background-handling gap in the panoptic evaluation; if the authors clarify that ID 0 is excluded from the metrics or add a background class to the model, the central claim can be fixed. I see no novelty or citation concerns beyond the need to position against Gaussian Grouping and Scaffold-GS."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take after reading ObjectGS. The core idea is worth taking seriously: bind a fixed one-hot object ID to each Scaffold-GS anchor, let anchors grow/prune while inheriting IDs, and supervise rendered ID maps with cross-entropy. That is a real departure from the learnable-semantics crowd (LangSplat, Gaussian Grouping, Gaga) and the ablations in Tables 5–7 support the claim that one-hot beats learnable features. The voting strategies are clearly described, the implementation details are there, and the mesh/edit applications make the contribution tangible. Credit where due: this is not a recycled baseline; the object-aware anchor with deterministic ID semantics is new in the Gaussian-splatting literature.\n\nThe soft spot is the background/ID-0 problem. Section 3.1 defines unclassified pixels as ID 0, but Eq. 4 defines one-hot vectors only for IDs 1..n and Eq. 7 sums only i=1..n. So background pixels are unsupervised, and at inference argmax over the n-dimensional vector forces every background pixel into an object. The paper never says whether the panoptic metrics in Table 2 include background, whether a confidence threshold is used, or whether ID-0 anchors are dropped. As written, the method either misclassifies all background or the evaluation excludes it, which would make it instance-only, not panoptic. This is a load-bearing gap in the central claim of \"panoptic segmentation,\" not a cosmetic issue. It's fixable—add a background dimension, a zero-vector for ID 0 with a threshold, or explicitly report foreground-only metrics—but it has to be fixed before the numbers are interpretable.\n\nOther concerns are minor by comparison. The OVS comparison is uneven: Gaga beats ObjectGS on figurines (90.7 vs 88.2 mIoU), so the \"outperforms SOTA\" claim is too broad. There is a single baseline for panoptic, no error bars, and only seven ScanNet++ scenes. The robustness claim that grow/prune corrects mislabeled points is qualitative. Still, none of these are fatal; they are standard referee requests.\n\nWho is this for? Someone working on 3D instance segmentation or editable Gaussian representations will find the anchor-ID idea useful and the writing mostly clear. It deserves a serious referee—the core mechanism is interesting and the empirical evidence is substantial, even if the background question changes how the evidence is read. I'd send it to review, but I'd want the authors to clarify or fix the background handling before acceptance.\n\nFor your reading group, it's a maybe: the paper will spark a good argument about what \"panoptic\" means in this setting.","headline":"A clever, well-engineered pipeline with a genuine spec gap: the paper never says how background (ID 0) is handled, so the panoptic numbers are not yet trustworthy.","tokens_in":16184,"tokens_out":2049,"would_cite":true,"duration_ms":24049,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Objects modeled as anchors with fixed one-hot IDs improve 3D Gaussian segmentation.","keywords":["3D Gaussian splatting","object-aware reconstruction","3D instance segmentation","open-vocabulary segmentation","panoptic segmentation","one-hot ID encoding","scene editing","mesh extraction"],"falsifier":"Take a scene with known object IDs, progressively corrupt a growing fraction of views' 2D masks by swapping two adjacent objects' IDs, and measure the final 3D segmentation IoU. If the cross-entropy loss simply locks in the swapped supervision, the IoU curve will fall with corruption; if the claimed geometry-guided correction works, the curve will stay high until some corruption threshold.","tokens_in":15193,"feed_emoji":"🎯","tokens_out":6221,"duration_ms":66089,"temperature":0.7,"pith_summary":"This paper proposes a way to make 3D Gaussian Splatting understand objects while it reconstructs a scene, rather than treating geometry and semantics as separate steps. The method assigns each object a fixed one-hot ID, binds that ID to anchor points that spawn neural Gaussians, and supervises rendering with a classification loss. The authors claim this avoids the semantic blur that continuous learnable features suffer during alpha blending, and that it beats current open-vocabulary and panoptic segmentation methods on standard benchmarks. A reader should care because the same object-aware representation directly supports downstream uses such as pulling out an object's mesh and editing a scene by removing or recoloring a selected object.","feed_headline":"One object ID per Gaussian sharpens 3D segmentation","feed_subtitle":"Object-aware anchors plus a one-hot classification loss give cleaner object boundaries and support mesh extraction and editing.","key_machinery":"The load-bearing mechanism is the pairing of object-aware anchors with a fixed one-hot ID encoding. Anchors are initialized from the object-voted point cloud, each carrying an object ID; they grow and prune in the voxel grid while replicating or removing their ID, and each anchor spawns several neural Gaussians that inherit the ID. The Gaussian's semantic attribute is a one-hot vector whose length equals the number of objects, so alpha blending of these vectors yields a classification probability vector at each pixel; taking the argmax predicts the pixel's object ID, and a cross-entropy loss enforces the label. A variable-length feature rasterizer extends the Gaussian color attribute to carry the ID channels, making the discrete semantic rendering efficient and scene-level, so all objects are rendered in one pass and occluded objects are handled.","core_discovery":"The paper's central claim is that object-level 3D understanding can be built directly into Gaussian Splatting reconstruction by making each object a set of anchors, each anchor carrying a fixed object ID, and by representing Gaussian semantics as one-hot ID vectors rather than learnable continuous features. During alpha blending these vectors accumulate independently, so the rendered pixel vector is a true classification distribution over object IDs; a cross-entropy loss against ID-consistent 2D masks then supervises both geometry and semantics at once. ObjectGS claims this design resolves the semantic ambiguity that arises when continuous learned features are blended, and the experiments report gains over prior methods on open-vocabulary segmentation and panoptic segmentation, plus cleaner 3D instance point clouds and direct mesh extraction and scene editing.","pith_inferences":["The discrete-semantics principle likely transfers to other neural rendering backbones: any continuous feature field used for classification could be replaced by a one-hot head with cross-entropy, at the cost of needing ID-consistent training masks.","The method's ceiling is set by the 2D mask source; a natural extension is to measure robustness under controlled mask corruption, and to let the geometry branch actively re-label uncertain points instead of merely inheriting votes.","Because the ID encoding length equals the object count, scenes with long-tailed or hierarchical categories could use coarse-to-fine one-hot groups or per-category subset encoding to keep memory flat, something the paper hints at in its overhead discussion.","The current open-vocabulary ability is realized at label-extraction time; a further step would attach text embeddings to each ID so that arbitrary queries can be resolved at render time without running the segmentation model again."],"forward_implications":["Objects can be selected by their ID directly after training: removing an object is deleting its anchors, and recoloring is editing the associated Gaussian color attributes.","Because semantic rendering runs once at scene level, occluded and partially invisible objects remain queryable, which per-object rendering cannot do.","The 2DGS variant of the framework exports individual object meshes via TSDF fusion using the anchors of the selected ID.","Segmentation and reconstruction improve together: adding the semantic loss also improves PSNR, SSIM, and LPIPS in the reported ablations.","The one-hot formulation needs no post-processing or classifier call at query time; text or click prompts are resolved at mask-extraction time and map directly to IDs."],"supporting_citations":[{"why":"Supplies the anchor-based hierarchical Gaussian generation that ObjectGS extends with object IDs.","marker":"[27]"},{"why":"Provides ID-consistent 2D object masks used for labeling and for supervising the semantic loss.","marker":"[7]"},{"why":"Gives the learnable-semantics baseline and the editing pipeline that the paper compares against and extends.","marker":"[47]"},{"why":"Establishes the base 3D Gaussian Splatting representation and rendering.","marker":"[16]"},{"why":"Provides the 2D Gaussian primitive variant used for object surface reconstruction and mesh extraction.","marker":"[12]"},{"why":"Enables text- and click-prompted open-vocabulary masks for object selection.","marker":"[37]"},{"why":"Supplies the rasterizer that the variable-length feature alpha blending extends.","marker":"[48]"},{"why":"Represents the continuous language-feature approach that the paper contrasts with discrete one-hot semantics.","marker":"[33]"}],"fun_headline_variants":["Object IDs in Gaussian splatting sharpen 3D segmentation","Object-aware Gaussians: cleaner 3D segmentation and editing","One-hot object IDs boost Gaussian segmentation accuracy","Anchored objects unify 3D reconstruction and understanding","Gaussian splatting with object anchors improves segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes the 2D segmentation model produces accurate, ID-consistent masks across views; if the masks are wrong or inconsistent, the voted object IDs on the point cloud are wrong, and the cross-entropy loss will strengthen those wrong IDs rather than fix them.","fun_headline_variants_meta":{"raw":{"variants":["Object IDs in Gaussian splatting sharpen 3D segmentation","Object-aware Gaussians: cleaner 3D segmentation and editing","One-hot object IDs boost Gaussian segmentation accuracy","Anchored objects unify 3D reconstruction and understanding","Gaussian splatting with object anchors improves segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000639,"raw_usage":{"total_tokens":2893,"prompt_tokens":845,"completion_tokens":2048,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":1968}},"tokens_in":461,"tokens_out":2048,"duration_ms":14865,"temperature":1.0,"reasoning_tokens":1968,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:31:37.347766+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a scene with known object IDs, progressively corrupt a growing fraction of views' 2D masks by swapping two adjacent objects' IDs, and measure the final 3D segmentation IoU. If the cross-entropy loss simply locks in the swapped supervision, the IoU curve will fall with corruption; if the claimed geometry-guided correction works, the curve will stay high until some corruption threshold.","supporting_citations":[{"cited_title":"Scaffold-gs: Structured 3d gaussians for view-adaptive rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the anchor-based hierarchical Gaussian generation that ObjectGS extends with object IDs."},{"cited_title":"Tracking anything with decoupled video segmentation","cited_arxiv_id":null,"evidence_quote":"Provides ID-consistent 2D object masks used for labeling and for supervising the semantic loss."},{"cited_title":"Gaussian grouping: Segment and edit anything in 3d scenes","cited_arxiv_id":null,"evidence_quote":"Gives the learnable-semantics baseline and the editing pipeline that the paper compares against and extends."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Establishes the base 3D Gaussian Splatting representation and rendering."},{"cited_title":"2d gaussian splatting for geometrically accu- rate radiance fields","cited_arxiv_id":null,"evidence_quote":"Provides the 2D Gaussian primitive variant used for object surface reconstruction and mesh extraction."},{"cited_title":"gsplat: An open-source library for gaussian splatting","cited_arxiv_id":null,"evidence_quote":"Supplies the rasterizer that the variable-length feature alpha blending extends."},{"cited_title":"Langsplat: 3d language gaussian splatting","cited_arxiv_id":null,"evidence_quote":"Represents the continuous language-feature approach that the paper contrasts with discrete one-hot semantics."}],"review_version":1}