{"id":"2b163b0f-9f55-474a-b7f5-8be360254e14","arxiv_id":"2411.19950","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A rectangle-with-alpha-channel plane representation with differentiable rasterization gives state-of-the-art 3D planar reconstruction from monocular videos on ScanNet.","lead":"The paper introduces AlphaTablets, a way to represent 3D planes as rectangles with transparency maps, and uses them to reconstruct 3D indoor scenes from ordinary monocular video. The method reports better coverage and segmentation of planar surfaces than prior systems on the ScanNet benchmark, with a demonstration of scene editing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (9)'s depth normalization is inconsistent with standard alpha compositing, and the reported F-score depends on whether this is a typo or a real implementation bug.","rationale":"The reader's weakest_assumption focused on the accuracy and cross-view consistency of pretrained monocular depth and normal cues. That is a legitimate limitation, but it is not the single most load-bearing issue because the paper's own ablations show the method can still achieve strong results when those cues are present (as on ScanNet), and the comparison against Metric3D+Seq-RANSAC partially controls for cue quality. The Eq. (9) depth-normalization inconsistency is more directly load-bearing: the quantitative claim depends on an optimization that is described with an equation that is numerically incorrect under the paper's own alpha-compositing framework. If the implementation matches the write-up, the optimization could not reliably produce the reported numbers; if it does not match, the paper's method description is inaccurate. Either way, the reader cannot verify the central claim from the manuscript alone. This does not change the reader's CONDITIONAL verdict, which already flagged the need for clarification, but it sharpens the condition: the authors must confirm whether Eq. (9) is a typo or the actual implementation. The reader's rationale did mention Eq. (9) as an inconsistency, so there is partial agreement, but the reader's formal weakest_assumption was a different concern.","tokens_in":15251,"tokens_out":7758,"duration_ms":73039,"concrete_test":"Take a single pixel with two layers (α1 = α2 = 0.5, depths d1 = 1 and d2 = 2), compute the depth loss using Eq. (9) as written and using the corrected denominator 1 - Π(1 - α_l), and compare gradients with respect to α and d. Then inspect the authors' released code (or ask them directly) to see which normalization is actually used in the depth loss. If the code uses 1 - Π(1 - α_l), the paper has a typo and the claim is unscathed; if it uses Π(1 - α_l), the optimization is numerically broken and the reported results are suspect.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim rests on the optimization in Sec. 3.3. In Eq. (9), the rendered depth is normalized as d = d_r / Π_l (1 - α_l), where d_r = Σ T_l α_l d_l from Eq. (8). Under the same alpha compositing used in Eq. (6), the sum of blending weights is 1 - Π_l (1 - α_l), so the correct expected-depth denominator is 1 - Π_l (1 - α_l), not the product. As written, the denominator is the transmittance after all layers: for a single opaque layer (α = 1) it is zero, and for two layers with α = 0.5 it gives d = 4.0 instead of 1.33. Because the ablation in Table 3 shows that removing the depth loss drops F-score from 0.456 to 0.271, the depth loss is a critical component. If Eq. (9) is what the implementation actually uses, the loss would be numerically unstable or degenerate, making the reported results hard to reproduce from the paper alone. If it is simply a typographical error and the implementation uses the standard denominator 1 - Π(1 - α_l), the method may be sound, but the paper as written does not let a reader verify this. Since no code is provided, this inconsistency is the most load-bearing unresolved issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces AlphaTablets, a 3D plane representation in which each plane is a rectangle with a learnable alpha channel, texture map, and canonical coordinates. The authors derive differentiable rasterization for AlphaTablets by converting them to pseudo-meshes and using multi-layer alpha compositing, then propose a bottom-up reconstruction pipeline from posed monocular videos: SLIC superpixels are initialized as 3D tablets using pretrained monocular depth and normal models, optimized with photometric, depth, normal, distortion, and alpha-inverse losses, and merged iteratively into larger planes. Experiments on ScanNet report an F-score of 0.456 versus 0.372 for PlanarRecon, along with improved plane segmentation metrics (VOI 3.468 vs. 3.622, SC 0.273 vs. 0.248). The paper also shows qualitative generalization to TUM-RGBD and Replica and demonstrates plane-based scene editing.","tokens_in":15534,"tokens_out":5817,"duration_ms":52791,"significance":"If the reported results hold, AlphaTablets is a valuable representation: it combines the completeness of 3D primitives with the boundary precision of 2D masks, supports per-scene optimization without training on the target dataset, and enables straightforward plane-based editing. The gains over PlanarRecon are plausible, and the ablation study broadly supports the importance of the depth/normal losses, distortion loss, anti-aliasing, and merging. The main unresolved issue is the depth-normalization formula in Eq. (9), which as written is inconsistent with standard alpha compositing and could not produce the reported results without a correction or clarification. Because no code is provided, this inconsistency blocks full confidence in the central quantitative claim.","major_comments":[{"comment":"Equation (9) defines the rendered depth as d = d_r / ∏_{l=1}^L (1 − α_l). Under the alpha compositing in Eq. (6), the sum of blending weights is ∑_l T_l α_l = 1 − ∏_l (1 − α_l), so the correct expected-depth denominator is 1 − ∏_l (1 − α_l), not the product. As written, the denominator is the final transmittance: for a single opaque layer (α = 1) it is 0, and for two layers with α = 0.5 it overestimates the depth by a factor of 3. Moreover, the paper minimizes Lainv = ∏_l (1 − α_l), driving the denominator toward 0 and making the depth loss ill-posed. The ablation in Table 3 shows that the depth loss contributes to the final F-score (0.425 to 0.456 when added after the normal loss), so this issue is load-bearing. The authors must correct Eq. (9) to the standard normalization, state the actual implementation used, and ideally release code so that the reported numbers can be reproduced.","section":"Sec. 3.3, Eq. (9)"},{"comment":"The depth and normal losses in Eqs. (8)–(10) supervise the tablets with the same pretrained models (Metric3Dv2 and Omnidata) that provide the initialization. Consequently, Ldepth and Lnormal are consistency regularizers toward monocular priors rather than independent geometric supervision. The external ScanNet ground-truth evaluation mitigates this concern, but the ablation study in Table 3 does not reveal whether the gains come from fitting the prior or from genuinely improved geometry. I request a concrete test: on a subset of ScanNet, either supervise Ldepth and Lnormal with ground-truth depth and normals, or ablate the two losses entirely without the prior-based supervision, and report how the F-score and other metrics change.","section":"Sec. 3.3, Initialization and Loss Design"},{"comment":"The merging scheme is essential (Table 3 drops from F-score 0.456 to 0.188 without merging), but the merging thresholds are not fully specified. The implementation details give only a 'normal threshold' of 0.93; the angle thresholds θ and θ_s, the distance threshold d, the color threshold c, and the neighborhood size K used in the KD-tree search are not reported in the main text or appendix. Without these values, the merging behavior cannot be reproduced. Please report all merging hyperparameters and, if they are scene-dependent, describe the selection procedure.","section":"Sec. 3.3 and Sec. 4.1, Merging Scheme"}],"minor_comments":[{"comment":"In Eq. (7), T_i is described as the 'blending weight' of the i-th rasterization layer, but in Sec. 3.2 T is defined as the accumulated transmittance, and the actual blending weight in Eq. (6) is T_l α_l. Please clarify which quantity is used in the distortion loss.","section":"Sec. 3.3, Eq. (7)"},{"comment":"The ablation rows are cumulative, so the depth loss alone contributes +0.031 F-score (from 0.425 to 0.456), not the +0.216 that might be inferred by comparing the first row with the last. The text could make the incremental contributions more explicit.","section":"Table 3"},{"comment":"The discussion of the higher Acc (0.161 vs. 0.105 for PlanarRecon) attributes the gap to incomplete ground-truth coverage. This is plausible, but the paper should quantify it, e.g., by evaluating accuracy only on voxels covered by ground-truth planes, to distinguish genuine error from coverage effects.","section":"Appendix A.3, 3D reconstruction accuracy"},{"comment":"The segmentation evaluation transfers reconstructed planes to ground-truth planes using nearest-neighbor assignment; this is reasonable, but a boundary-focused metric (e.g., boundary IoU) would help substantiate the claim of 'precise boundary delineation.'","section":"Sec. 4.1, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a novel representation and a plausible reconstruction pipeline, but the unresolved Eq. (9) issue is a serious correctness concern that cannot be resolved by reading the manuscript. Since no code is provided, the authors should be asked to correct the formula and either release code or provide a detailed verification of the actual normalization used. The circularity of the depth/normal supervision is a lesser concern given the external ScanNet evaluation, but it should be addressed with an additional experiment. The overall fit for NeurIPS is acceptable, though the contribution is primarily empirical."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this paper is worth reading even though a key equation is wrong. The AlphaTablets representation — rectangles in 3D with learnable alpha channels and tiled textures — is new, and the bottom-up superpixel-to-tablet optimization plus union-find merging is a sensible pipeline I haven't seen before. The ScanNet numbers look credible: F-score 0.456 versus PlanarRecon's 0.372, with better recall and segmentation. I would not be surprised if these are real, given the ablations and the careful baseline comparison.\n\nWhat they do well: the idea of using alpha for boundaries in 3D is elegant and solves a real problem. The differentiable rasterization through NVDiffrast is a practical adaptation. The merging scheme is well thought out, especially the use of unit tablets to define neighborhoods, avoiding the average-center ambiguity. The ablations are informative, and the appendix gives useful implementation details, including a time budget and a tablet-count evolution plot.\n\nNow the soft spots. The most serious is Eq. (9): the rendered depth is divided by Π(1−α) rather than 1−Π(1−α). As written, an opaque tablet (α=1) gives division by zero, and two layers with α=0.5 give a depth four times too large. This is almost certainly a typo — the standard expected-depth denominator is 1−Π(1−α) — but the paper as published does not let a reader verify that the implementation matches the text. The depth loss itself contributes a modest improvement (0.425→0.456 F-score in Table 3), so this is not a load-bearing flaw in the results, but it blocks reproduction. No code or error bars are provided, and several merging thresholds are never given. The generalization claim rests on qualitative figures only. There is mild circularity in using the same pretrained depth/normal models for initialization and supervision, but the final evaluation is against ScanNet ground truth, so it is acceptable.\n\nI disagree with the stress-test note's characterization of the depth loss as critical: the bigger jump comes from the normal loss, not the depth loss. But the note correctly identifies the typo.\n\nWho should read this: anyone working on 3D planar reconstruction, plane representations, or differentiable rendering of primitives. It deserves a serious referee; I'd send it for review with a request for a corrected Eq. (9), a code release, and full hyperparameters. For a desk editor: don't reject it on the basis of the typo alone — the idea is sound and the results are likely real.","headline":"AlphaTablets is a genuinely new plane representation with a strong ScanNet result, but the paper as written contains a wrong depth-normalization equation that must be corrected before the numbers are reproducible.","tokens_in":16097,"tokens_out":4266,"would_cite":true,"duration_ms":37842,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces AlphaTablets, a 3D plane representation as rectangles with alpha channels, and a bottom-up optimization-and-merging pipeline that reconstructs complete 3D planes from posed monocular video, reporting state-of-the-art…","keywords":["AlphaTablets","3D plane reconstruction","differentiable rasterization","monocular video","superpixel merging","planar segmentation","scene editing"],"falsifier":"Run the pipeline on a sequence where the monocular depth is deliberately corrupted by a constant per-view scale error but the images and poses are unchanged; if the final planes drift by more than a small tolerance from ground truth, that would confirm the reconstruction leans on the pretrained cues rather than on multi-view photometric consistency. A second concrete check: on a scene with strong non-Lambertian reflections or lighting changes, if plane boundaries fragment or F-score drops below PlanarRecon's, the claim that alpha channels capture boundaries robustly under color-consistency optimization would be contradicted.","tokens_in":15045,"feed_emoji":"📐","tokens_out":6451,"duration_ms":52760,"temperature":0.7,"pith_summary":"This paper introduces AlphaTablets, a way to represent a 3D plane as a textured rectangle with an alpha channel, so the rectangle can carve out arbitrary irregular boundaries while remaining a single continuous surface in 3D. The authors argue that prior representations force a trade-off: 2D masks delineate boundaries precisely but are inconsistent across views, while 3D volumes or point clouds are view-consistent but lose boundary sharpness and surface continuity. They propose a bottom-up pipeline that initializes many small AlphaTablets from superpixels and pretrained monocular depth and normals, then iteratively optimizes them by differentiable rendering and merges compatible neighbors into larger planes. On the ScanNet benchmark the method reports an F-score of 0.456 and segmentation covering of 0.273, both above the previous state of the art (PlanarRecon: 0.372 and 0.248). If this holds, AlphaTablets would be a generic plane primitive that supports complete, accurate, editable plane reconstruction without per-dataset training.","feed_headline":"AlphaTablets lift 3D plane reconstruction F-score to 0.456","feed_subtitle":"Rectangles with alpha channels rebuild complete, editable planes from posed monocular video, beating prior methods on ScanNet.","key_machinery":"The central object is the AlphaTablet: a 3D rectangle defined by center $p$, orthogonal normal, up, and right vectors, a canonical 2D texture map $c$ with $\\alpha$ channel $\\alpha$, pixel range $(r_u,r_v)$, and distance ratios $\\lambda_u,\\lambda_v$ linking texture pixels to 3D size; arbitrary plane shapes are cut out by the $\\alpha$ channel. The argument is carried by a differentiable rasterization that converts each tablet to two mesh triangles, rasterizes multiple depth layers per pixel through depth peeling, anti-aliases colors with $\\alpha$-aware weights while leaving $\\alpha$ unchanged, and composites layers by $\\alpha$ blending. On top of this, the reconstruction machinery is a per-scene optimization loop: initialize from superpixel masks and pretrained monocular depth and normals, optimize texture, $\\alpha$, normal, and distance under the loss $L = w_1L_{\\mathrm{pho}} + w_2L_{\\mathrm{ainv}} + w_3L_{\\mathrm{dist}} + w_4L_{\\mathrm{depth}} + w_5L_{\\mathrm{normal}}$, and iteratively merge tablets whose normals, projected centers, and colors are close, until a few hundred tablets represent the scene's planes.","core_discovery":"On the paper's own terms, the central discovery is that a 'rectangle soup with alpha channels' can serve as a universal 3D plane representation: each tablet carries a center, normal, up vector, a canonical texture map, an alpha transparency map, and per-axis distance ratios that convert texture pixels into 3D size. Because the alpha channel is learnable and the texture is defined in canonical tablet coordinates, the representation delivers solid surfaces with sharp, arbitrary boundaries, and because all parameters live in 3D, the same tablet is consistent across all views. The paper derives a differentiable rasterizer for these tablets (pseudo-mesh construction, depth-peeled multi-layer rasterization, alpha-weighted anti-aliasing, and alpha compositing) and combines it with a bottom-up pipeline: SLIC superpixels plus pretrained monocular depth and normals initialize dense overlapping tablets; a photometric, alpha-inverse, distortion, depth, and normal loss stack optimizes them; a union-find merging scheme with normal, distance, and color thresholds fuses them into complete planes. The reported results on ScanNet show better geometry and segmentation than PlanarRecon and reconstruct-then-fit baselines, and qualitative results on TUM and Replica indicate generalization.","pith_inferences":["If the per-scene optimization is the real source of generality, then replacing the pretrained monocular cues with better cross-view-consistent depth (for example, a video-depth model) should translate almost directly into higher F-score, since the rest of the pipeline is cue-agnostic.","The alpha-aware anti-aliasing rule (blend colors with weights $w$ and $1-w$ but leave alpha un-blended) could be ported to other semi-transparent primitive renderers such as 2D Gaussian splatting to remove boundary strip artifacts.","The paper's stated limitations (non-planar superpixels and view-dependent appearance) suggest the next test: a version that splits non-planar superpixels or adds per-tablet view-dependent shading would address exactly the cases where the current photometric loss is weakest.","The merging thresholds (normal, distance, color) are fixed hyperparameters; an adaptive or learned merging policy might improve results on scenes with repetitive textures or gradual curvature, but this is an extension the paper does not explore."],"forward_implications":["AlphaTablets can be rendered with existing mesh-based differentiable rasterizers by constructing a pseudo-mesh per tablet, so the representation drops into standard graphics pipelines.","Because plane segmentation is treated as bottom-up merging of 3D tablets rather than as learned instance segmentation, the method does not require dataset-specific plane labels or training, and can in principle generalize to any posed video.","The alpha channel gives a natural way to represent irregular plane boundaries and partial occlusion, so reconstructed planes should have cleaner edges than voxel- or surfel-based planes.","The reconstructed tablets store editable canonical texture maps, so plane-based scene editing (recoloring, texture replacement, style transfer) becomes a direct texture-map operation.","The final tablet count for a scene collapses from tens of thousands of initial superpixel tablets to a few hundred, indicating the representation also acts as a compact, progressive planar scene abstraction."],"supporting_citations":[{"why":"It supplies the SLIC superpixel segmentation used to initialize each tablet from 2D color-homogeneous regions.","marker":"[1]"},{"why":"It provides the pretrained monocular depth estimates that back-project superpixels into 3D and supervise the rendered depth during optimization.","marker":"[19]"},{"why":"It provides the surface normal estimates used both for tablet initialization and as supervision for the rendered normals.","marker":"[12]"},{"why":"It is the differentiable mesh rendering framework that the paper adapts for AlphaTablets via pseudo-mesh construction and depth peeling.","marker":"[24]"},{"why":"It is the main learning-based baseline for monocular-video 3D planar reconstruction and supplies the evaluation protocol and comparison numbers.","marker":"[50]"},{"why":"It is the ScanNet dataset on which the method is evaluated and from which the 3D plane ground truth is taken.","marker":"[11]"},{"why":"It provides the validation split and the Murez 3D metrics used to measure plane geometry and segmentation quality.","marker":"[31]"},{"why":"It motivates the distortion loss that penalizes multiple semi-transparent surfaces occupying the same region, encouraging tablets to merge into solid planes.","marker":"[5]"}],"fun_headline_variants":["AlphaTablets: rectangle soup that reconstructs 3D planes from video","AlphaTablets turn monocular video into sharp 3D planes","AlphaTablets: learnable alpha channels yield precise 3D planes","AlphaTablets: bottom-up 3D plane reconstruction from monocular video","AlphaTablets: differentiable rasterization for consistent 3D planes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline presumes that the pretrained monocular depth and normal estimates are accurate and cross-view consistent enough to initialize every superpixel tablet and supervise geometry during optimization; if these cues are biased or inconsistent, the photometric loss alone may not correct the errors and the final planes inherit the bias.","fun_headline_variants_meta":{"raw":{"variants":["AlphaTablets: rectangle soup that reconstructs 3D planes from video","AlphaTablets turn monocular video into sharp 3D planes","AlphaTablets: learnable alpha channels yield precise 3D planes","AlphaTablets: bottom-up 3D plane reconstruction from monocular video","AlphaTablets: differentiable rasterization for consistent 3D planes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000792,"raw_usage":{"total_tokens":3527,"prompt_tokens":1020,"completion_tokens":2507,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":2411}},"tokens_in":636,"tokens_out":2507,"duration_ms":16045,"temperature":1.0,"reasoning_tokens":2411,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:39:15.435776+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on a sequence where the monocular depth is deliberately corrupted by a constant per-view scale error but the images and poses are unchanged; if the final planes drift by more than a small tolerance from ground truth, that would confirm the reconstruction leans on the pretrained cues rather than on multi-view photometric consistency. A second concrete check: on a scene with strong non-Lambertian reflections or lighting changes, if plane boundaries fragment or F-score drops below PlanarRecon's, the claim that alpha channels capture boundaries robustly under color-consistency optimization would be contradicted.","supporting_citations":[{"cited_title":"Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans","cited_arxiv_id":null,"evidence_quote":"It provides the surface normal estimates used both for tablet initialization and as supervision for the rendered normals."},{"cited_title":"Modular primitives for high-performance differentiable rendering","cited_arxiv_id":null,"evidence_quote":"It is the differentiable mesh rendering framework that the paper adapts for AlphaTablets via pseudo-mesh construction and depth peeling."},{"cited_title":"Planarrecon: Real-time 3d plane detection and reconstruction from posed monocular videos","cited_arxiv_id":null,"evidence_quote":"It is the main learning-based baseline for monocular-video 3D planar reconstruction and supplies the evaluation protocol and comparison numbers."},{"cited_title":"Atlas: End-to-end 3d scene reconstruction from posed images","cited_arxiv_id":null,"evidence_quote":"It provides the validation split and the Murez 3D metrics used to measure plane geometry and segmentation quality."}],"review_version":1}