{"id":"af87f781-a71f-4043-a972-1e6976c10665","arxiv_id":"2412.02140","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A language-guided robotic grasping system that reconstructs a 3D semantic scene from three RGB views and updates moved objects in about 200 ms, reporting higher grasp success than F3RM and LERF-TOGO.","lead":"SparseGrasp lets a robot build a 3D scene model from just three camera views and then grasp objects named in plain language, updating the model in about 200 milliseconds when objects move. It combines existing computer vision tools, including DUSt3R, 3D Gaussian Splatting, and CLIP, to make language-guided grasping work with fewer cameras and faster updates than prior systems.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 55/70 grasp result rests on a retrained GraspNet (Sec. III-D) whose sim-to-real transfer is unvalidated; no held-out objects, no baseline versus original GraspNet, and no error bars are reported, so the central claim is not yet supported.","rationale":"The reader's CONDITIONAL verdict and weakest-assumption identification are correct: the retrained GraspNet is the most load-bearing component because Table I's grasp success rate is the central quantitative claim. The real-robot trials are genuine evidence, and the reconstruction and update times are plausible, but the comparative claim rests on a single set of seven objects with no held-out evaluation. Since F3RM and LERF-TOGO use different grasp pipelines, the 55 vs 38/33 gap at 17 views could be due to the grasp-pose generation module rather than the sparse-view semantic reconstruction, which is the paper's stated contribution. The proposed held-out real-robot test directly checks whether the retrained GraspNet transfers to unseen objects; conditional on the outcome, the verdict stays CONDITIONAL or moves to REJECT. Because the concern stems from missing evidence rather than a demonstrated internal inconsistency, the reader's CONDITIONAL verdict is unchanged.","tokens_in":10385,"tokens_out":9085,"duration_ms":101268,"concrete_test":"Retrain the GraspNet module per Section III-D using only the 100 specified scenes, then execute 10 grasps per object on five novel objects that are absent from both the 100 training scenes and the original seven test objects, under the identical 3-view KINOVA setup. Compare the held-out success rate with the original 55/70 using a one-sided two-proportion test at the 5% level; if the held-out rate is significantly lower, the sim-to-real transfer is fragile, and if it is not, the concern is resolved. In the same run, also score original GraspNet on point clouds extracted from the same 3DGS scenes to isolate the retraining contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline number is Table I's 55/70 grasp successes. All upstream contributions (DUSt3R initialization, MaskCLIP/FastSAM distillation, PCA compression) are evaluated only indirectly; the actual grasp poses come from a retrained GraspNet described in Section III-D in five sentences. The training on 100 modified GraspNet-1B scenes separately reconstructs objects and backgrounds with 3DGS, labels object Gaussians as 'objectness', adds random Gaussian noise, and varies densification thresholds, but the paper gives no architecture details for ingesting pi, si, qi, no comparison against original GraspNet on the same 3DGS output, no evaluation on held-out synthetic scenes, and no confidence intervals for the seven real objects. Since the 55/70 result is the only quantitative evidence that the full pipeline improves over F3RM and LERF-TOGO at 17 views (38/33), fragile sim-to-real transfer of the retrained GraspNet would invalidate the central claim even if reconstruction and semantic fields are correct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SparseGrasp, a language-guided robotic grasping system that reconstructs a 3D semantic Gaussian Splatting scene from as few as three RGB images. It combines DUSt3R-based dense point initialization, MaskCLIP and FastSAM semantic feature extraction with PCA compression, and a render-and-compare module for updating moved objects. The authors report a 55/70 grasp success rate on a KINOVA Gen2 robot using three views, outperforming F3RM (38/70) and LERF-TOGO (33/70) at seventeen views, and scene updates in about 200 ms. The central claim is that sparse views and fast updates are sufficient for practical multi-turn open-vocabulary grasping.","tokens_in":10632,"tokens_out":4233,"duration_ms":42325,"significance":"If the reported results hold, SparseGrasp would be a meaningful advance in making language-guided manipulation practical in changing environments, reducing reconstruction from minutes with dense views to about four minutes with three views and enabling near-instant scene updates. The use of DUSt3R for dense initialization, PCA for feature compression, and a retrained GraspNet operating directly on Gaussian parameters are sensible engineering contributions. The real-robot evaluation on seven objects is a strength. However, the significance is currently limited by the absence of statistical rigor, the brevity of the grasp-network description, and the lack of validation for the sim-to-real transfer of the retrained GraspNet, which is the sole source of grasp poses.","major_comments":[{"comment":"The retrained GraspNet is the only component that generates grasp poses, yet its description is only five sentences: no architecture, loss function, training schedule, or inference procedure is given for ingesting pi, si, qi. There is no held-out synthetic evaluation and no comparison against the original GraspNet on the same 3DGS output. The headline 55/70 success rate therefore depends on unvalidated sim-to-real transfer of this network. Please provide a standalone evaluation of the grasp model (e.g., held-out objects or scenes, an original-GraspNet baseline, or simulation results) and full implementation details.","section":"III-D and Table I"},{"comment":"All success rates are reported as raw counts with no confidence intervals or statistical tests. With 10 trials per object, the 95% binomial confidence interval for 8/10 spans roughly 44% to 97%, so differences such as 8/10 vs 7/10 are not meaningful. Table II uses only five scenes per condition, and the improvement from 3/5 to 5/5 at different update times is not statistically reliable. The claim that SparseGrasp 'significantly outperforms' the baselines needs error bars, confidence intervals, or a paired statistical test across objects and scenes.","section":"Tables I and II"},{"comment":"The render-and-compare loss in Eq. (6) uses d_gt, the MOG2-detected center of moved pixels, as supervision for the predicted object position. As presented, this is an alignment procedure that assumes a known foreground mask or center, not a predictor that autonomously detects arbitrary scene changes. The paper should explicitly state the assumptions on d_gt and evaluate robustness when MOG2 is imperfect, for example with overlapping objects, lighting changes, or partial occlusion, before Table II can support the claim of rapid updating in general changeable environments.","section":"III-C, Eq. (6)"},{"comment":"The 2D IoU evaluation is the only quantitative semantic distillation result, but the paper does not specify how the continuous feature heatmaps are thresholded to binary masks, how novel-view ground-truth regions are obtained, or whether the IoU is averaged over all queries. Without this protocol detail, the reported values (e.g., Ours 0.71 vs F3RM 0.75 at 17 views) cannot be interpreted or reproduced. Please describe the evaluation protocol and report per-query or per-object variability.","section":"IV-B, Table III"}],"minor_comments":[{"comment":"The phrase 'handles scene updates fastly' is ungrammatical; use 'quickly' or 'fast'.","section":"Abstract and Introduction"},{"comment":"The column labels in Fig. 5 appear fused as 'F3RMLERFOurs', making the comparison difficult to read; please separate the labels.","section":"Fig. 5"},{"comment":"The caption references the query 'metal hug' while the text uses 'metal mug'; this typo should be corrected.","section":"Fig. 8"},{"comment":"The definition of the 'objectness' label for Gaussians is ambiguous for objects that touch or occlude each other; clarify how boundary Gaussians are assigned.","section":"III-D"},{"comment":"The description 'equipped with three common cameras' is vague; please specify the camera models, resolution, and the extrinsic calibration procedure used to provide poses to DUSt3R.","section":"IV, Environment and Setup"},{"comment":"References [4] and [43] are the same work (Object-aware Gaussian Splatting for Robotic Manipulation) and should be consolidated.","section":"References"},{"comment":"The phrase 'significantly outperforms' in the Introduction and Conclusion should be qualified with the statistical evidence, or deferred to the results section where the uncertainty can be reported.","section":"I, Contributions"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant problem, and the core idea of combining DUSt3R initialization, PCA-compressed semantic features, and render-and-compare updates is promising. The main weakness is that the quantitative evidence rests on the retrained GraspNet, which is not adequately described or validated, and the success rates lack statistical support. The revisions required are substantial but feasible within the manuscript's scope: add a dedicated grasp-model evaluation, report confidence intervals or error bars, and clarify the assumptions of the render-and-compare module. I recommend major revision rather than rejection, as the central claim is defensible but not yet supported with the current evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, SparseGrasp is a real systems contribution: it shows that three RGB views can drive a language-guided grasp pipeline that beats F3RM and LERF-TOGO at seventeen views, and it does so with a clean integration rather than a new foundation model. The render-and-compare update, the PCA feature compression, and the GraspNet retrained directly on Gaussian parameters are genuinely new in this combination, and the 4-minute reconstruction plus 200 ms update times are the kind of numbers that make multi-turn manipulation practical.\n\nThe experiments are honest but modest. Seven objects, one robot, ten trials each, no error bars or statistical tests. The gap over F3RM at 17 views (55/70 vs 38/70) is large enough that I do not think the basic claim is a fluke, but a skeptic would want a second object set and a different gripper. I would also want the GraspNet retraining described in more detail: Section III-D's five sentences leave out the network architecture, the input encoding of pi, si, qi, and any comparison against original GraspNet. The sim-to-real transfer is the load-bearing assumption for the grasp results.\n\nThe stress-test note focuses on that same section and calls the central claim unsupported. I think that is a bit too harsh: the 55/70 is real-robot evidence that the full pipeline, including the retrained GraspNet, works on the tested objects. The weakness is generalization, not internal validity. Also, the render-and-compare update is a fitting procedure, not a prediction, so there is no circularity problem.\n\nWhat would strengthen this: an ablation running original GraspNet on the same 3DGS output, a few held-out synthetic scenes, and confidence intervals. Those are addressable. The discussion already flags the refresh limitation for added or removed objects, which is fair.\n\nBottom line: the paper deserves a serious referee. I would send it to review for a robotics venue, and ask for the above ablations plus code/data release. It is not a home run, but it is a solid step.","headline":"A solid sparse-view language-guided grasping system; the real-robot results are plausible but the retrained GraspNet is under-specified and generalization beyond seven objects remains open.","tokens_in":11171,"tokens_out":1984,"would_cite":false,"duration_ms":20342,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SparseGrasp reconstructs a language-queryable 3D scene from three RGB images in about four minutes and uses it to grasp objects on a real robot, reporting 55/70 successes.","keywords":["3D Gaussian Splatting","language-guided robotic grasping","sparse-view RGB reconstruction","semantic distillation","open-vocabulary grasping","render-and-compare scene updating","PCA feature compression","GraspNet"],"falsifier":"Replace the retrained GraspNet with the original GraspNet on the same three-view 3DGS scenes and measure grasp success on the same seven objects; a success rate that stays near 55/70 would show the retraining is not load-bearing, while a sharp drop would confirm that the system's performance rests on the simulated training transfer.","tokens_in":10228,"feed_emoji":"🤖","tokens_out":5479,"duration_ms":48494,"temperature":0.7,"pith_summary":"Language-guided grasping usually needs dense camera coverage and full re-reconstruction when a scene changes. SparseGrasp claims that a 3D semantic Gaussian Splatting scene, initialized from DUSt3R's dense point cloud and distilled with compressed CLIP features, can be built from just three RGB images in about four minutes. The paper reports 55 successful grasps out of 70 on a real KINOVA robot, against 38/70 for F3RM and 33/70 for LERF-TOGO even with 17 views. It also claims a render-and-compare step that relocates moved objects in about 200 ms, making multi-turn, open-vocabulary grasping in changing scenes practical. If correct, dense multi-view capture and slow full-scene retraining are no longer prerequisites for language-guided manipulation.","feed_headline":"Three camera views: 55/70 robotic grasps","feed_subtitle":"Open-vocabulary grasping that beats a 17-view baseline and updates moved objects in 200 ms.","key_machinery":"The load-bearing mechanism is a 3D semantic Gaussian field: each Gaussian carries its usual position, scale, rotation, opacity, and color, plus a 16-dimensional semantic feature obtained by PCA-compressing MaskCLIP/FastSAM features. DUSt3R provides a dense point cloud as the initial geometry, which prevents sparse-view overfitting; during training the RGB loss and the semantic-feature L1 loss are optimized jointly with the same tile-based rasterizer. For scene updates, a render-and-compare loop uses MOG2 to detect moved pixels, matches them to Gaussians by mean semantic-feature cosine similarity, and optimizes only the moved object's translation and rotation against a combined pixel and 2D-position loss. Grasp poses come from a retrained GraspNet that consumes Gaussian parameters directly, so no voxelization is needed.","core_discovery":"On its own terms, the paper's central discovery is that 3D Gaussian Splatting can carry both geometry and open-vocabulary semantics well enough for robotic grasping when it is initialized with dense points from DUSt3R rather than COLMAP's sparse points, and when patch-level MaskCLIP features are averaged inside FastSAM masks, compressed by PCA from 768 to 16 dimensions, and distilled into Gaussians with a joint RGB-plus-semantics loss. From that representation, the system generates six-degree-of-freedom grasps directly from the Gaussian parameters via a retrained GraspNet, bypassing the voxelization and depth back-projection that the NeRF-based baselines need. For changed scenes, a render-and-compare optimization that adjusts only the translation and rotation of the moved object's Gaussians restores graspability in roughly 200 ms. The paper supports these claims with real-robot trials across seven objects, reporting 55/70 successes from three views, and with a moved-object study where the 200 ms update reaches 22/25 successes across five scenes.","pith_inferences":["The same render-and-compare trick might extend beyond rigid moves to object removal or addition if combined with 3D inpainting, a direction the paper itself flags as future work.","If 16 PCA components suffice for grounding, then semantic fields for grasping may be intrinsically low-rank, which suggests that even cheaper feature distillation is possible.","The retrained GraspNet's inputs are Gaussian parameters; this design implies any representation expressible as anisotropic Gaussians could reuse the grasp head, but the sim-to-real gap remains the main uncertainty."],"forward_implications":["A robot can complete a multi-turn pick-and-place sequence from a single three-image reconstruction, without rescanning between commands.","Scene updates for moved objects drop from minutes of full reconstruction to about 200 ms, so the system can react to a rearrangement while the operator waits.","The PCA compression shows that 16 dimensions suffice for open-vocabulary grounding in these scenes, which keeps the 3DGS memory and rendering cost near plain color rendering.","Because grasp generation runs directly on Gaussian parameters, the method avoids the precision loss that voxelization introduces in NeRF-based pipelines."],"supporting_citations":[{"why":"DUSt3R supplies the dense point cloud that initializes 3DGS, preventing sparse-view overfitting.","marker":"[3]"},{"why":"3D Gaussian Splatting is the base representation whose Gaussians carry both color and semantic features.","marker":"[7]"},{"why":"F3RM is the primary NeRF-based baseline the method compares against for reconstruction, grounding, and grasping.","marker":"[1]"},{"why":"LERF-TOGO is the task-oriented grasping baseline that needs dense views and full reconstruction for scene changes.","marker":"[2]"},{"why":"MaskCLIP provides patch-level language features, which the method averages within masks.","marker":"[8]"},{"why":"FastSAM generates the object masks used to make the semantic features boundary-aware.","marker":"[37]"},{"why":"GraspNet is the base grasp generator, retrained to consume Gaussian parameters instead of depth voxels.","marker":"[10]"},{"why":"MOG2 background subtraction detects moved pixels that seed the render-and-compare update.","marker":"[40]"}],"fun_headline_variants":["SparseGrasp: 55/70 grasps from just 3 RGB views","Open-vocab grasping from 3 views beats 17-view baseline","SparseGrasp: update moved objects in 200 ms","3-view open-vocab grasping with 200ms scene updates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported grasp rates depend on the retrained GraspNet, trained on a modified GraspNet-1B dataset, transferring to the real KINOVA robot; if that sim-to-real transfer fails, the success numbers would not hold even if the reconstruction and semantics are accurate.","fun_headline_variants_meta":{"raw":{"variants":["SparseGrasp: 55/70 grasps from just 3 RGB views","Open-vocab grasping from 3 views beats 17-view baseline","SparseGrasp: update moved objects in 200 ms","3-view open-vocab grasping with 200ms scene updates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000689,"raw_usage":{"total_tokens":3142,"prompt_tokens":989,"completion_tokens":2153,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":2072}},"tokens_in":605,"tokens_out":2153,"duration_ms":14127,"temperature":1.0,"reasoning_tokens":2072,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:46:41.177606+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the retrained GraspNet with the original GraspNet on the same three-view 3DGS scenes and measure grasp success on the same seven objects; a success rate that stays near 55/70 would show the retraining is not load-bearing, while a sharp drop would confirm that the system's performance rests on the simulated training transfer.","supporting_citations":[{"cited_title":"Language embedded radiance fields for zero- shot task-oriented grasping,","cited_arxiv_id":null,"evidence_quote":"LERF-TOGO is the task-oriented grasping baseline that needs dense views and full reconstruction for scene changes."},{"cited_title":"Graspnet-1billion: A large- scale benchmark for general object grasping,","cited_arxiv_id":null,"evidence_quote":"GraspNet is the base grasp generator, retrained to consume Gaussian parameters instead of depth voxels."},{"cited_title":"Improved adaptive gaussian mixture model for back- ground subtraction,","cited_arxiv_id":null,"evidence_quote":"MOG2 background subtraction detects moved pixels that seed the render-and-compare update."}],"review_version":1}