{"id":"9f669763-cca4-44e6-b5fd-83af4daa1a67","arxiv_id":"2411.11374","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A compact MLP learns to classify occupied versus empty 3D points for large-scale NeRF training, replacing the discrete occupancy grid and improving accuracy at matched training time.","lead":"LeC2O-NeRF trains a small neural network to predict which 3D points in a large urban scene are empty, then uses those predictions to skip empty space when training a NeRF. The authors report that this learned filter is more compact and accurate than the standard occupancy grid, improving image quality and training speed on large-scale benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Aligned-time comparison may mask unequal GPU-hours: occupancy pretraining time is reported without GPU count, while target NeRFs use different GPU counts.","rationale":"The reader's weakest assumption (density-loss circularity) is a soundness concern, but the ablation in Table 6 shows that removing Ld costs only ~0.2 dB PSNR on Building, so even a flawed density loss would not invalidate the core method. The headline claim is about speed without accuracy loss; its evidence is the aligned-training-time tables. Those tables report wall-clock time but not the GPU count for the occupancy pretraining phase, and the target methods use different GPU counts. If pretraining uses 8 GPUs while vanilla INGP uses 2, the comparison is not controlled in GPU-hours, which is the standard measure of total compute in this area. I therefore identify this as the single most load-bearing concern: it directly conditions whether the speedup is algorithmic or merely an effect of resource allocation. The requested disclosure and matched-compute rerun would settle it. This does not conflict with the reader's CONDITIONAL verdict; it sharpens one of the conditions under which the central claim should be accepted.","tokens_in":24520,"tokens_out":25176,"duration_ms":260520,"concrete_test":"Request the exact GPU count for the occupancy pretraining phase and the total GPU-hours for each row of Tables 2 and 3, then rerun INGP+Ours with pretraining limited to 2 GPUs and Switch+Ours with 8 GPUs to measure wall-clock performance under matched hardware. If the PSNR advantage persists at equal total GPU-hours, the speedup claim is genuine; if it shrinks or reverses, the claim should be downgraded to 'faster at fixed wall-clock only when extra parallel GPUs are available for pretraining.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 reports occupancy pretraining as 1.6-1.8h without specifying the GPU count, while Section 4.4 states that INGP is trained on 2 RTX 3090 GPUs and Switch-NeRF on 8. Since the occupancy network is trained with 8 scene sub-networks and then reused for both target methods, pretraining plausibly uses 8 GPUs. Under that assumption, INGP+Ours consumes roughly 8×1.7 + 2×(20−1.7) = 50.2 GPU-hours versus 2×20 = 40 GPU-hours for vanilla INGP, so the reported wall-clock tie does not imply a compute tie. The paper's headline claim 'speed up state-of-the-art NeRF methods without sacrificing accuracy' is thus a wall-clock comparison that may be bought with extra parallel compute during pretraining. This is the most load-bearing uncertainty because the central quantitative claim rests on the benchmark tables, and none of them report GPU-hours or the pretraining GPU count.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes LeC2O-NeRF, a method for learning a continuous, compact occupancy field as a small MLP (0.15M parameters) for large-scale NeRF. The occupancy network dispatches each 3D point either to one of n scene sub-networks or to a tiny empty-space network; the model is trained end-to-end with a rendering loss plus an imbalanced occupancy loss (Eq. 1) and a density loss (Eq. 2) that uses detached densities. After pretraining, the frozen occupancy network is used to discard unoccupied points during point sampling in Instant-NGP and Switch-NeRF. Experiments on Mega-NeRF and Block-NeRF report occupancy classification metrics against a depth-derived ground truth and image reconstruction metrics with wall-clock training time aligned between methods.","tokens_in":24697,"tokens_out":6021,"duration_ms":55987,"significance":"If the claims hold, the paper provides a practical alternative to occupancy grids for large-scale NeRF: a compact continuous predictor that retains roughly 13--16% of points, improves recall substantially over the grid, and improves PSNR/SSIM/LPIPS on most scenes under wall-clock alignment. The core derivation of the imbalanced loss is sound, the ablations are informative, and the paper gives useful occupancy diagnostics (point proportions, density/alpha ratios, memory usage). The main limitation is that the speed claim is made on wall-clock time where GPU counts differ between occupancy pretraining and target NeRF training; until total GPU-hours are reported, the 'speed up without sacrificing accuracy' conclusion is not established. The occupancy ground-truth proxy is generated by the same model family used to train the occupancy network, which weakens the absolute accuracy claims.","major_comments":[{"comment":"The aligned-time comparison does not control for total GPU compute. Occupancy pretraining is reported as 1.6--1.8 h in §4.3 without specifying the number of GPUs, while §4.4 states that INGP is trained on 2 RTX 3090 GPUs and Switch-NeRF on 8. Since the occupancy network is fitted with 8 scene sub-networks and then reused for both target methods, it is plausible that pretraining uses 8 GPUs; under that assumption INGP+Ours costs roughly 8×1.7 + 2×(20−1.7) ≈ 50.2 GPU-hours versus 40 GPU-hours for vanilla INGP. The reported wall-clock tie therefore does not imply a compute tie. Please report GPU-hours for every configuration (including occupancy pretraining) or run all comparisons on identical GPU counts, and state the alignment protocol explicitly. Without this, the central claim that the method can speed up state-of-the-art NeRF methods without sacrificing accuracy is not supported.","section":"§4.3, §4.4, Table 3"},{"comment":"The occupancy accuracy ground truth is derived from depth maps of a fully trained Switch-NeRF, the same model family that the occupancy network is trained with. A learned occupancy model trained jointly with a Switch-NeRF-like radiance field may be systematically favored by this reference, so the reported Accuracy/Precision/Recall/F1 improvements over the grid are not an independent measure of geometric quality. Please corroborate the occupancy evaluation with an external depth or LiDAR source, or at least report how the metrics change with a different reference model, and state this limitation explicitly in the paper.","section":"§4.2, Table 1"},{"comment":"The density loss supervises occupancy with the model's own density predictions, detached, and this signal is assumed to be reliable. Early in training, when the radiance field is poor, these densities may be unreliable and the loss could reinforce routing errors rather than correct them. The paper should include an experiment in which Ld is applied only after a warm-up period or in which the detached densities come from a separate frozen reference, and should report the ratio σe/σs at very early steps; the current Figure 8d starts at 1k steps and does not show the noisiest regime.","section":"§3.3, Eq. (2)"},{"comment":"The statement in §4.4 that the method consistently outperforms Switch, NGP, and Switch+Grid is contradicted on the Residence scene, where Switch+Ours has PSNR 22.10 versus 22.18 for Switch+Grid, even though SSIM and LPIPS are better. No error bars or multiple seeds are reported anywhere in the tables. Please report standard deviations over at least three seeds and either soften the consistency claim or identify the metric (e.g., LPIPS or an aggregate) on which consistency is claimed.","section":"Table 3, Residence row"}],"minor_comments":[{"comment":"The sentence 'Our empty space network O should secure more 3D points' should refer to the occupancy network, not the empty space network; the empty space network is the one whose fraction should increase.","section":"§3.3"},{"comment":"The ratio σe/σs can have a near-zero denominator during early training; add a small positive constant to the denominator for numerical stability and state this in the text.","section":"§3.3, Eq. (2)"},{"comment":"The sampling counts are ambiguous: the text says '512 points for each ray' for occupancy training and later 'usually sample 128 samples along a ray' for guided training; clarify which stage each number applies to.","section":"§4.3"},{"comment":"The 'Para. Number' row compares only the occupancy representation parameters; state whether the scene and empty sub-network parameters are excluded, and give the grid resolution (e.g., 128^3 cells) in the caption for context.","section":"Table 1"},{"comment":"There is a typo in 'Instan-NGP' in the paragraph describing Table 3; it should be 'Instant-NGP'.","section":"§4.4"},{"comment":"The caption label 'GridOurs' should be written 'Grid / Ours' to avoid confusion between the two columns.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reports no code release, and the 'first' novelty claim is difficult to verify without a broader literature search. The main risk is the wall-clock versus GPU-hours accounting in the speed claim, which is load-bearing for the paper's headline. If GPU-hours are added and the occupancy metric is supported by an external ground-truth source, the paper would likely be acceptable after the other revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper makes a credible case that a compact learned occupancy MLP can beat grid-based occupancy for large-scale NeRF, and it does that with clean ablations and honest reporting of the main wall-clock training time. The one load-bearing omission is GPU-hour accounting, which the stress-test note got right: occupancy pretraining is reported without a GPU count, and the target methods use different GPU counts. If pretraining uses 8 GPUs, the INGP+Ours comparison is not compute-neutral. That's fixable and doesn't invalidate the accuracy claims, but it should be fixed before publication.\n\nWhat's new: the specific machinery—an imbalanced loss adapted from GShard to push ~85% of points into a tiny empty-space network, a scene/empty expert split, and a density loss computed from detached densities—is a genuine combination not in the cited grid-based works. The occupancy accuracy and compactness results in Table 1 are convincing: 0.15M parameters versus a 2.0M grid, higher recall/F1, and a lower occupancy ratio across five scenes. The ablations in Tables 4 and 6 show the identity empty-space network and the density loss each earn their keep. Figure 8's statistics on density ratios over training steps support the design.\n\nSoft spots, in order. First, GPU-hours, as above. The wall-clock tie between INGP+Ours and INGP is bought with extra parallel compute during pretraining; report the GPU count and give a GPU-hour table. Second, no error bars. The margins in Table 3 are consistent, but a few seeds would strengthen the claims. Third, the occupancy ground truth comes from a fully-trained Switch-NeRF depth map—reasonable proxy, but it means the \"occupancy accuracy\" numbers are not fully independent. The rendering metrics on held-out views are independent, so this is a minor concern, not a circularity problem. Fourth, the density loss is self-supervised through the model's own detached densities; that is a training signal, not an evaluation loop, so it is acceptable as long as the authors note the risk of early reinforcing errors. They effectively address it by showing convergence in 20k–40k steps.\n\nThe citation pattern is fine; the GShard reference is used correctly. Lack of released code is a reproducibility minus, but not a reason to reject.\n\nWho this is for: people working on large-scale NeRF training acceleration and learned sampling. It deserves a serious referee; I would send it to review and ask for a revision that adds GPU-hour reporting and at least one standard error bar.","headline":"Credible engineering result with a real compute-accounting gap: report GPU-hours and add error bars before this is ready.","tokens_in":25247,"tokens_out":3324,"would_cite":true,"duration_ms":30751,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 0.15M-parameter occupancy network can replace the discrete occupancy grid for large-scale NeRF, speeding up training while improving PSNR, SSIM, and LPIPS.","keywords":["occupancy network","large-scale NeRF","empty space skipping","imbalanced loss","density loss","scene decomposition","urban scenes","volume rendering"],"falsifier":"Train the same architecture on one large-scale scene but swap the density ratio $L_d$ for its inverse, so the loss prefers empty-space points to have higher density than scene points; if the occupancy network still converges to a useful surface-near binary occupancy, then the density loss is not the mechanism driving the dispatch, and the paper's explanation would need revision. A second check: evaluate occupancy accuracy against independent geometry from LiDAR or COLMAP instead of the depth maps of a fully-trained Switch-NeRF.","tokens_in":24275,"feed_emoji":"🏙️","tokens_out":7486,"duration_ms":57912,"temperature":0.7,"pith_summary":"Large-scale neural radiance fields are slowed by the discrete occupancy grid used to skip empty space: grids have fixed resolution, need a tight bounding box, and cost memory to update. This paper claims that a small continuous MLP — an occupancy network with only 0.15M parameters — can learn where a scene is occupied through three self-supervised losses, and that this learned occupancy is more compact, accurate, and smooth than the grid. When frozen and used to guide sampling in existing methods such as Instant-NGP and Switch-NeRF, it speeds up training while improving PSNR, SSIM, and LPIPS. If correct, this would give large-scale NeRF a learned occupancy prior that is not limited by grid resolution or bounding-box regularity.","feed_headline":"A tiny MLP learns scene occupancy faster than a 2M-parameter grid","feed_subtitle":"Continuous occupancy network speeds up large-scale NeRF training while improving PSNR, SSIM, and LPIPS.","key_machinery":"LeC$^2$O-NeRF's central object is the occupancy network: a 4-layer MLP that returns normalized occupancy scores over $n$ scene sub-networks and one empty-space network, acting as a continuous, trainable replacement for the discrete occupancy grid. It is driven by three losses: the imbalanced occupancy loss $L_o = (n+v)\\left(\\frac{f_e p_e}{v} + \\sum_{i=1}^n f_i p_i\\right)$, which generalizes the balanced load loss of GShard by treating the empty-space network as $v$ virtual experts so that roughly $v/(n+v)$ of points are dispatched as unoccupied; the density loss $L_d = \\sigma_e/\\sigma_s$, computed with detached densities, which forces the mean density of empty-space points below that of scene points; and the standard volume-rendering loss. Together these let the MLP learn occupancy from the scene's own radiance field without external geometry supervision.","core_discovery":"The central claim is that occupancy for large-scale NeRF can be encoded as a continuous function learned by a compact MLP, rather than maintained as a discrete momentum-density grid. The occupancy network $O(x)$ outputs $n+1$ normalized values that dispatch each 3D point into one of $n$ scene sub-networks (occupied) or a tiny empty-space network (unoccupied). It is trained end-to-end with the radiance field using the rendering loss plus an imbalanced occupancy loss that controls the occupied/unoccupied ratio and a density loss $L_d = \\sigma_e/\\sigma_s$ that pushes unoccupied points to smaller density. The paper reports that this 0.15M-parameter network classifies occupancy more accurately than a 2.0M-parameter $128^3$ grid, retains only about 13–16% of points as occupied, and when frozen and used as a sampler it improves the training of Instant-NGP and Switch-NeRF at matched training time, including the occupancy-training time itself.","pith_inferences":["If the same occupancy network can be pretrained on a city and then transferred to a new scene, the per-scene 1.6–1.8h occupancy training could be skipped entirely, making the acceleration nearly free at inference time.","The trick of treating empty space as $v$ virtual experts in the load-balancing loss could transfer to other sparse 3D representations, such as pruning Gaussians in splatting or sampling in robotic occupancy mapping, wherever most of the volume is empty.","The density ratio $L_d$ provides a parameter-free, supervision-free diagnostic for how well an occupancy predictor separates surface from free space, potentially useful in settings where ground-truth occupancy is unavailable."],"forward_implications":["Existing grid-based accelerators, including the NerfAcc occupancy grid and Instant-NGP's multi-scale grids, can be replaced by a frozen 0.15M-parameter MLP that needs no per-scene grid-resolution tuning.","Occupancy converges quickly (within about 10k–40k training steps), so the network can be frozen early and then used purely as a sampler to accelerate the remaining NeRF training.","At aligned wall-clock training time, LeC2O-guided Instant-NGP and Switch-NeRF attain higher PSNR, SSIM, and lower LPIPS than the same methods with an occupancy grid, meaning the speed-up does not sacrifice accuracy.","Because the occupancy function is continuous, it represents irregular large-scale urban scenes without the bounding-box and resolution constraints of grids."],"supporting_citations":[{"why":"Supplies the volume-rendering loss and NeRF formulation that the occupancy-guided sampling accelerates.","marker":"(Mildenhall et al., 2020)"},{"why":"Provides the balanced load loss that the imbalanced occupancy loss generalizes by treating empty space as virtual experts.","marker":"(Lepikhin et al., 2021)"},{"why":"The NerfAcc occupancy grid is the discrete baseline and plug-and-play module that the learned occupancy replaces and compares against.","marker":"(Li et al., 2022)"},{"why":"Instant-NGP is one of the state-of-the-art NeRF methods whose grid-based occupancy the learned network replaces for acceleration.","marker":"(Müller et al., 2022)"},{"why":"Switch-NeRF is the large-scale backbone used for end-to-end training and for extracting the depth-based occupancy reference in evaluation.","marker":"(MI & Xu, 2023)"},{"why":"Block-NeRF supplies one of the two large-scale benchmarks used for image reconstruction metrics.","marker":"(Tancik et al., 2022)"},{"why":"Mega-NeRF supplies the other large-scale benchmark with five urban scenes used for occupancy and reconstruction metrics.","marker":"(Turki et al., 2022)"},{"why":"The Mip-NeRF embedding is used for occupancy training on the Block-NeRF dataset.","marker":"(Barron et al., 2021)"}],"fun_headline_variants":["Small MLP beats big grid for NeRF occupancy","Continuous occupancy accelerates large-scale NeRF","MLP occupancy outpaces 2M-param grid in NeRF","Tiny occupancy network speeds up urban NeRF training","Self-supervised MLP learns NeRF occupancy without grids"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The occupancy network trusts the density predictions of the very radiance field it is helping to train: if the density field is initially poor, the density loss can reinforce wrong routing decisions instead of correcting them.","fun_headline_variants_meta":{"raw":{"variants":["Small MLP beats big grid for NeRF occupancy","Continuous occupancy accelerates large-scale NeRF","MLP occupancy outpaces 2M-param grid in NeRF","Tiny occupancy network speeds up urban NeRF training","Self-supervised MLP learns NeRF occupancy without grids"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000789,"raw_usage":{"total_tokens":3552,"prompt_tokens":1091,"completion_tokens":2461,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":707,"completion_tokens_details":{"reasoning_tokens":2383}},"tokens_in":707,"tokens_out":2461,"duration_ms":18291,"temperature":1.0,"reasoning_tokens":2383,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:35:33.732255+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture on one large-scale scene but swap the density ratio $L_d$ for its inverse, so the loss prefers empty-space points to have higher density than scene points; if the occupancy network still converges to a useful surface-near binary occupancy, then the density loss is not the mechanism driving the dispatch, and the paper's explanation would need revision. A second check: evaluate occupancy accuracy against independent geometry from LiDAR or COLMAP instead of the depth maps of a fully-trained Switch-NeRF.","supporting_citations":[{"cited_title":"Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs","cited_arxiv_id":null,"evidence_quote":"Mega-NeRF supplies the other large-scale benchmark with five urban scenes used for occupancy and reconstruction metrics."},{"cited_title":"Nerf in the wild: Neural radiance fields for unconstrained photo collections","cited_arxiv_id":null,"evidence_quote":"The Mip-NeRF embedding is used for occupancy training on the Block-NeRF dataset."}],"review_version":1}