{"id":"bc09f80c-0db3-4971-8d13-10419f99db5c","arxiv_id":"2505.13633","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"IPENS lifts SAM2's 2D segmentations into 3D via NeRF mask inverse rendering, enabling interactive multi-target point cloud extraction and trait estimation for rice and wheat.","lead":"A framework called IPENS combines the SAM2 image segmenter with a neural 3D scene model (NeRF) to extract rice and wheat organs as point clouds from 360-degree videos without manual annotation. It reports grain-level segmentation (mIoU 63.72% for rice, 89.68% for wheat) and trait estimates such as grain volume and leaf dimensions with R² values up to 0.97 and near 1.0, but code, data, and key calibration details are not yet public.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Volume trait is computed from surface-voxel counts, so the reported grain/panicle volume accuracies do not validate physical volume estimation.","rationale":"The paper's headline quantitative achievements are the segmentation mIoU values and the phenotypic R²/RMSE values. The segmentation pipeline is plausible: SAM2 provides 2D masks, inverse rendering lifts them to 3D, and the reported mIoU (63.72% rice, 89.68% wheat) is internally consistent with per-organ IoU tables, though the instance-matching protocol is not described. The more serious internal inconsistency is in the volume trait. Section 2.10 explicitly defines volume from point count after voxel downsampling at 0.01, but a NeRF-derived point cloud is a surface reconstruction. The occupied voxels form a shell, not a volume, so V = num_points * (0.01)^3 conflates surface voxel count with volume. This is not a consensus disagreement; it is an internal dimensional error. The reader identified the unstated metric scale as the weakest assumption; I agree that scale is needed, but even with scale, Eq. (4) is wrong for surface point clouds. This concern directly affects the abstract's volume claims, which are part of the central claim. I recommend keeping the CONDITIONAL verdict: the segmentation results may be sound, but the volume trait numbers must be recomputed with a proper mesh-based or interior-filled volume method, and the evaluation rerun, before the phenotypic claims can be accepted. This does not require rejecting the paper, because the error is localized and fixable; it does require changing the volume-extraction methodology and re-reporting the trait accuracies.","tokens_in":19081,"tokens_out":5796,"duration_ms":58734,"concrete_test":"Generate a synthetic sphere of radius 1 cm sampled only on its surface with ~100k points, run the Section 2.10 preprocessing exactly: voxel downsampling with voxel size 0.01, then compute V = num_points * 0.01^3. Compare the result to the true sphere volume 4.19 cm³ and to the volume of the Alpha Shapes mesh constructed from the same downsampled point cloud. If the Eq. (4) result differs from the true volume by more than 20%, or if repeating with a 0.5 cm sphere gives a volume ratio far from 8:1 (true volume ratio), Eq. (4) does not measure object volume.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim includes phenotypic trait estimation, and the abstract highlights grain/panicle volume R². Section 2.10 states: after setting voxel size to 0.01, performing voxel downsampling, then constructing a mesh via Alpha Shapes, the volume is computed as V = num_points * 0.01^3 (Eq. 4). This formula is only valid if num_points counts voxels that tile the object's interior. But the point cloud comes from a NeRF density field, whose high-density points lie on object surfaces, and voxel downsampling retains one point per occupied voxel. The occupied voxels therefore form a surface shell, not a filled volume. Multiplying the number of surface voxels by voxel volume gives a shell measure that systematically underestimates the true volume, with a bias that depends on object size and shape. For a sphere of radius R, the estimated 'volume' scales roughly as R² instead of R³, so the ratio between predicted volumes of different-sized grains will be wrong. This means the reported R²=0.7697 for rice grain volume, R²=0.9956 for wheat panicle volume, and the associated RMSE values are not measuring physical volume, independent of the metric-scale concern raised by the Reader. Even if the reconstruction scale were calibrated to centimeters, Eq. (4) would still be dimensionally incorrect for a surface point cloud. The reader's weakest assumption is valid but secondary: the formula is wrong even at known scale.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IPENS, an interactive unsupervised pipeline for extracting 3D organ point clouds from rice and wheat image sequences. The method combines SAM2 2D mask propagation with a NeRF mask inverse-rendering optimization to lift multi-target masks into 3D, adds morphological post-processing and an SSIM-based re-prompting heuristic, and derives phenotypic traits (grain/panicle volume, leaf area, leaf length/width) from the extracted point clouds. On the authors' MMR and MMW datasets, IPENS reports mIoU of 63.72% (rice) and 89.68% (wheat), with trait R² values up to 0.9956 for wheat panicle volume and 1.00 for wheat leaf area. The paper also compares against CrossPoint, Agile3D, and oneformer3D and reports inference times of about 3 minutes after reconstruction.","tokens_in":19446,"tokens_out":6202,"duration_ms":54147,"significance":"If the quantitative claims held, IPENS would be a practically useful zero-training interactive phenotyping tool: it avoids 3D annotation, handles multiple targets in one interaction, and reports competitive organ segmentation relative to the included baselines on two crop species. The strengths of the manuscript are the concrete system design, the use of standard components (nerfacto, SAM2), and the explicit multi-target collaborative optimization and post-processing heuristics. However, the trait-validation pipeline and evaluation protocol have load-bearing gaps, notably the volume formula, the missing metric-scale calibration, and the undefined instance-matching rule, so the reported numerical claims cannot currently be taken at face value. The paper does not provide code, data, or an evaluation script, which limits reproducibility.","major_comments":[{"comment":"The formula V = num_points × 0.01³ does not compute a physical volume for the point clouds produced by the pipeline. The extracted point cloud comes from a NeRF density field, so the high-density points are concentrated on object surfaces; after voxel downsampling with voxel size 0.01, each occupied voxel contains one representative point, and the occupied voxels form a surface shell rather than a filled interior. Multiplying the number of surface voxels by the voxel volume therefore measures a shell-like quantity that scales roughly as R² for a sphere while the true volume scales as R³, and the bias depends on object size and shape. Consequently, the reported grain-volume R² = 0.7697 (Figure 9), panicle-volume R² = 0.9956 (Figure 10), and their RMSEs do not validate physical volume estimation. The authors need to fill the interior voxels (e.g., flood-fill or occupancy carving) or compute volume from a watertight mesh, and then re-report all volumetric trait metrics.","section":"§2.10.1, Eq. (4)"},{"comment":"The manuscript never explains how the NeRF/COLMAP reconstruction is scaled to metric units. COLMAP sparse reconstruction and nerfacto determine geometry up to an unknown global scale unless a reference length is provided, yet Section 2.10 hard-codes a voxel size of 0.01 and reports volumes in cm³, areas in cm², and lengths/widths in cm (Tables 9–12). Without a documented calibration procedure (calibration object, known camera baseline, or manual reference measurement), the dimensional RMSE values are unanchored, and the '0.01' voxel size is an arbitrary unitless constant. The authors should state how scale was recovered and verify it on objects not used for calibration.","section":"§2.2 and §2.10"},{"comment":"The mIoU evaluation protocol is incomplete because the predicted-to-ground-truth instance matching rule is never specified. In scenes with thousands of grain instances and many leaves/stems (Table 1), different matching conventions, such as Hungarian assignment, greedy best-overlap, a minimum IoU threshold, or allowing many-to-one matches, can materially change the reported mIoU. The paper also defines N in Eq. (19) as the number of categories, which is inconsistent with an instance-level IoU computation; clarify whether mIoU averages per-category or per-instance values. No confidence intervals or significance tests are provided for the IoU or trait comparisons, so differences such as the 8.12% gap against Agile3D@1 in Table 6 cannot be assessed statistically.","section":"§3.1, Eqs. (18)–(19)"},{"comment":"The baseline comparison protocol is under-specified. The text states that 30% of the data was held out as a validation set and the remainder used for training comparison algorithms, but it does not say whether IPENS reports results on the same 30% split, how the same prompts were administered to Agile3D and oneformer3D, or how the 'first interaction' for Agile3D was standardized. Since the comparison is a central claim of the paper, the shared evaluation split, prompt protocol, and matching code must be specified or released. Relatedly, the paper gives no ablation of the two proposed post-processing steps (morphological inpainting and SSIM-based re-prompting), so their individual contributions to the reported IoU gains are unknown.","section":"§3.2.1"}],"minor_comments":[{"comment":"The header '1≪11' appears to be a typographical corruption of '<< 11'; also clarify that 'Batch Rays' is rays per chunk to match the surrounding text.","section":"Table 3"},{"comment":"The prompting strategy description is ambiguous: it first says 'five adjacent center points' are selected as negative prompts, then describes fixed grid sampling with grid size 3 and radius 1; please reconcile these statements and define the sampling parameters in one place.","section":"§2.7"},{"comment":"Report unrounded R² values and sample sizes; an entry of 1.00 with RMSE 0.67 on 35 plants is likely a rounded value and should be given to more decimal places.","section":"Tables 11 and 12"},{"comment":"In the bullet list, 'Medium-Speed Models ( 1 hour)' has a missing symbol; use '~1 hour' or '≈1 hour'. Also clarify whether the FPS column refers to rendering speed during reconstruction or to some other quantity.","section":"Appendix B"},{"comment":"The paper would benefit from a data availability statement: no code, datasets, or evaluation scripts are provided, which hinders reproduction of the numerical claims.","section":"General"},{"comment":"The term 'unsupervised' is used loosely: the system relies on pre-trained SAM2 and YOLOv11 models and on user prompts; consider 'training-free' or 'zero-shot interactive' to avoid confusion with unsupervised learning.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty and results are potentially interesting for the phenotyping community, but the missing code and the unresolved volume, scale, and instance-matching issues make the reported numbers impossible to verify as currently written. I would consider a revision only if the authors provide corrected trait evaluations, a documented scale-calibration procedure, and a precise evaluation protocol; otherwise the empirical claims would remain unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline first: the engineering here is real and the segmentation results are worth a look, but the volume traits are not. Eq. (4) computes volume as the number of points (after voxel downsampling) times voxel volume. The point cloud from a NeRF density field is a surface shell, not a filled solid, so that formula measures something proportional to surface area, not volume. For grains of different sizes the bias is systematic: estimated \"volume\" scales roughly as R^2 instead of R^3. The reported R² of 0.7697 for rice grain volume and 0.9956 for wheat panicle volume do not validate physical volume estimation, even if the reconstruction scale were perfectly calibrated. The reader's concern about missing metric-scale calibration is valid but secondary; the formula is wrong at any scale.\n\nWhat is actually new: the SAM2 + NeRF mask-lifting combination exists already (SA3D, PanicleNeRF), but the multi-target collaborative optimization, YOLOv11-assisted prompting, morphological inpainting, and SSIM-based re-prompting are practical additions. The system runs end-to-end in roughly three minutes and produces 3D segmentations of rice and wheat organs from a single round of interaction. The wheat numbers (mIoU 89.68) and the comparison against oneformer3D, Agile3D, and CrossPoint are informative. This is a useful engineering contribution for breeding workflows.\n\nSoft spots, in proportion: the volume formula is a load-bearing flaw and should be fixed by computing volume from the closed mesh (e.g., via the divergence theorem on the Alpha Shapes mesh). No code or data is provided, so the numbers are not independently checkable. There is no instance-matching protocol for the IoU metric, which matters when objects are occluded or touching. There are no error bars or significance tests; wheat leaf area with R² = 1.00 on a small sample needs a scatter plot to be credible. And the scale from COLMAP/nerfacto to metric units is never stated, so leaf length and width in cm are unanchored.\n\nBottom line: this paper deserves a serious referee. I would send it to review, but with a clear expectation of major revision: fix the volume estimation, release code and data, add metric calibration and error bars, and specify the instance-matching protocol. As it stands, the volume claims should not be cited.","headline":"The system is a plausible engineering contribution and the segmentation numbers are worth discussing, but the grain/panicle volume claims are undermined by Eq. (4), which counts surface voxels as if they filled the interior.","tokens_in":19937,"tokens_out":2196,"would_cite":false,"duration_ms":21337,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"IPENS claims that SAM2-propagated 2D masks, lifted into 3D by NeRF mask inverse rendering, turn single-round click prompts into grain-level 3D point clouds of rice and wheat organs, with mIoU 63.72% (rice) and 89.68% (wheat) and trait R²…","keywords":["rice phenotyping","wheat phenotyping","NeRF","SAM2","3D instance segmentation","unsupervised interactive segmentation","mask inverse rendering","point cloud traits"],"falsifier":"Capture a calibration object with known physical dimensions (a ruler, a cube, or a sphere) with the identical robotic-arm capture and nerfacto reconstruction pipeline, run IPENS on the sequence, and compare the predicted volume and leaf length/width against the object's true measurements; any systematic scale error directly invalidates the reported RMSE values and the physical interpretation of the trait estimates.","tokens_in":18924,"feed_emoji":"🌾","tokens_out":7120,"duration_ms":56860,"temperature":0.7,"pith_summary":"IPENS claims that a user can extract grain-level 3D point clouds of rice and wheat organs from a single round of click prompts on a few images, with no annotated training data, by combining SAM2's video mask propagation with a NeRF radiance field. The key move is mask inverse rendering: SAM2's 2D masks are projected back into the 3D voxel grid along NeRF rays, and a multi-target loss optimizes several organ masks simultaneously. On custom rice and wheat datasets the framework reports segmentation mIoU of 63.72% and 89.68%, and trait estimates for grain/panicle volume, leaf area, length and width with R² up to 0.97 for rice leaf length and 0.9956 for wheat panicle volume. A sympathetic reading is that this makes per-organ 3D phenotyping feasible in minutes, replacing labor-intensive 3D annotation with cheap single-round interaction.","feed_headline":"Single-round prompts extract rice and wheat 3D traits","feed_subtitle":"IPENS lifts SAM2 masks into NeRF point clouds, measuring grain volume, leaf area, length and width without training labels.","key_machinery":"The central object is mask inverse rendering: a 3D voxel mask grid $V\\in\\mathbb{R}^{L\\times W\\times H}$ whose per-object mask scores $G(r(t))$ are rendered along NeRF rays by Eq. (1), then optimized with the projection loss $L_{\\text{proj}}$ in Eq. (3) so that the grid's 2D projections match SAM2's propagated masks across all views. Each object gets its own mask-score channel, and all channels are optimized together, which is what turns one round of 2D prompts into separated 3D instances. The pipeline also relies on SAM2's video-propagation memory for mask continuity, morphological closing/opening for mask repair, and an SSIM-based rear-frame detector (Algorithm 2) that locates where an occluded target reappears so the user can re-prompt; together these components let the method handle small self-occluded grains that defeat earlier interactive approaches such as SA3D.","core_discovery":"IPENS claims that SAM2's streaming-memory video segmentation, followed by mask inverse rendering through a pre-trained nerfacto NeRF field, can segment small, self-occluded organs at the grain level in 3D from single-round prompts. The multi-target collaborative optimization in Eq. (3) lets one interaction simultaneously produce separated 3D masks for grains, leaves, stems and panicles. Two post-processing stages—morphological closing/opening to repair 2D masks and an SSIM-based algorithm that flags rear-view frames where the target reappears—recover masks lost to self-occlusion. On the MMR and MMW datasets the method reports grain IoU 61.48% (rice) and 92.82% panicle IoU (wheat), exceeding the unsupervised baseline CrossPoint (23.41% and 16.50% mIoU) and approaching the first-interaction results of the supervised interactive method Agile3D, with the fully supervised oneformer3D serving as the labeled-data upper bound.","pith_inferences":["The reported physical traits (cm³, cm², cm) are only meaningful if the reconstruction is metric or at fixed scale; the paper never states how scale is recovered, so a calibrated-capture extension (embedding a known-size reference in the acquisition cube) is the natural next step and would convert relative rankings into absolute trait values.","The mask-inverse-rendering mechanism is not specific to NeRF; the same 2D-to-3D mask lifting could operate on 3D Gaussian Splatting fields, trading some of the robustness the paper credits to NeRF for faster reconstruction.","The framework already contains the pieces of full automation: YOLOv11 generates first-frame prompts and the SSIM detector locates re-prompt frames, so removing the human click entirely is a testable extension rather than a new method.","Because segmentation quality appears to follow 2D mask quality, replacing the heuristic SSIM rear-frame test with a learned re-identification of the reappeared target could improve accuracy on crops whose rear views are not approximate mirror images."],"forward_implications":["Grain-level 3D instance extraction is possible without annotated training data: single-round prompts on a few key frames suffice, because SAM2 propagates masks through the video and mask inverse rendering lifts them into 3D.","Multiple organ types (grains, leaves, stems, panicles) can be segmented in parallel from one interaction; inference time grows nearly linearly with target count, and IPENS runs roughly 3.3 times faster than SA3D on the reported data.","Morphological inpainting of 2D masks and SSIM-guided re-prompting of rear frames raise 3D mask quality, which is the stated reason the grain-level results are achievable despite self-occlusion.","Phenotypic traits computed from the extracted point clouds track manual labels well: rice grain volume R²=0.7697, rice leaf area R²=0.84, rice leaf length/width R²=0.97/0.87, wheat panicle volume R²=0.9956, and wheat leaf length/width R²=0.99/0.92.","The same interactive pipeline extends to other crops without retraining; the appendix shows segmentations for apple, cherry tomato, maize, and multiple wheat plants."],"supporting_citations":[{"why":"Supplies SAM2, the video mask propagation engine that produces 2D masks across the image sequence from single prompts.","marker":"[33]"},{"why":"Supplies NeRF, the radiance field representation that the mask inverse rendering projects 2D masks through.","marker":"[11]"},{"why":"Supplies nerfacto, the specific NeRF variant used for reconstruction and point cloud export.","marker":"[39]"},{"why":"Estimates camera poses from the captured image sequence, a precondition for NeRF training and mask projection.","marker":"[37]"},{"why":"SA3D, the interactive NeRF-SAM segmentation baseline that IPENS is compared against and that fails on small grains.","marker":"[30]"},{"why":"PanicleNeRF, the prior NeRF-based rice panicle phenotyping work that motivates the volume-to-trait correlation approach.","marker":"[14]"},{"why":"CrossPoint, the unsupervised 3D baseline that IPENS must beat without labels.","marker":"[40]"},{"why":"Agile3D, the interactive supervised baseline whose first-click results IPENS is compared with.","marker":"[41]"},{"why":"oneformer3D, the fully supervised method representing the labeled-data upper bound.","marker":"[42]"},{"why":"SSIM, the similarity measure used in the rear-frame detection algorithm to re-prompt occluded targets.","marker":"[36]"}],"fun_headline_variants":["NeRF-SAM2 fusion lifts 2D masks to 3D grain traits","No labels, one click: rice and wheat 3D phenotyping","Interactive unsupervised 3D phenotyping hits 89% IoU","Grain-level 3D traits from single prompts, no training data","IPENS: SAM2 masks lifted into NeRF point clouds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reconstructed point cloud is assumed to be in metric units (or at least at the same fixed scale as the manual annotations) so that the volume formula $V = \\text{num\\_points} \\times (0.01)^3$ and the leaf length/width values in cm are physically meaningful; the paper never states how this scale is recovered from the COLMAP/nerfacto reconstruction.","fun_headline_variants_meta":{"raw":{"variants":["NeRF-SAM2 fusion lifts 2D masks to 3D grain traits","No labels, one click: rice and wheat 3D phenotyping","Interactive unsupervised 3D phenotyping hits 89% IoU","Grain-level 3D traits from single prompts, no training data","IPENS: SAM2 masks lifted into NeRF point clouds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1453,"prompt_tokens":1134,"completion_tokens":319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":750,"completion_tokens_details":{"reasoning_tokens":223}},"tokens_in":750,"tokens_out":319,"duration_ms":3054,"temperature":1.0,"reasoning_tokens":223,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:12:14.305294+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Capture a calibration object with known physical dimensions (a ruler, a cube, or a sphere) with the identical robotic-arm capture and nerfacto reconstruction pipeline, run IPENS on the sequence, and compare the predicted volume and leaf length/width against the object's true measurements; any systematic scale error directly invalidates the reported RMSE values and the physical interpretation of the trait estimates.","supporting_citations":[{"cited_title":"Mildenhall, P","cited_arxiv_id":null,"evidence_quote":"Supplies NeRF, the radiance field representation that the mask inverse rendering projects 2D masks through."},{"cited_title":"Tancik, E","cited_arxiv_id":null,"evidence_quote":"Supplies nerfacto, the specific NeRF variant used for reconstruction and point cloud export."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Estimates camera poses from the captured image sequence, a precondition for NeRF training and mask projection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SA3D, the interactive NeRF-SAM segmentation baseline that IPENS is compared against and that fails on small grains."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PanicleNeRF, the prior NeRF-based rice panicle phenotyping work that motivates the volume-to-trait correlation approach."},{"cited_title":"Afham, I","cited_arxiv_id":null,"evidence_quote":"CrossPoint, the unsupervised 3D baseline that IPENS must beat without labels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Agile3D, the interactive supervised baseline whose first-click results IPENS is compared with."},{"cited_title":"Kolodiazhnyi, A","cited_arxiv_id":null,"evidence_quote":"oneformer3D, the fully supervised method representing the labeled-data upper bound."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SSIM, the similarity measure used in the rear-frame detection algorithm to re-prompt occluded targets."}],"review_version":1}