{"id":"f169ab71-8513-4dc7-b088-905df98db6ae","arxiv_id":"2504.19557","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CE-NPBG improves novel view synthesis for autonomous driving by retrieving only visible LiDAR points via a connectivity graph, boosting quality, speed, and scalability, including for 3D Gaussian Splatting.","lead":"This paper presents a method for rendering new camera views in large driving scenes from LiDAR point clouds and photos. It retrieves only the points visible to a camera using a connectivity graph, which improves image quality, speed, and scalability while also boosting a popular 3D Gaussian Splatting method.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 3DGS integration does not isolate the connectivity mechanism: the 28.57 dB result uses 720K points versus 19.4M for the baseline, confounding visibility selection with a 27x reduction in parameter count.","rationale":"The reader's verdict is conditional and identifies retrieval quality as the weakest assumption. I agree that the retrieval precision/recall is unvalidated, but the more acute problem is the absence of a control separating visibility selection from point-count reduction. The 3DGS integration is the strongest evidence for the connectivity mechanism, yet it compares 19.4M points against 720K points; the 27x reduction in parameter count alone could explain the quality and memory gains. Since the adversarial term contributes only about 0.2 dB (Table 2), the bulk of the claimed improvement is attributed to the connectivity graph, making it the load-bearing component. A random-subsample and global-depth-buffer control is straightforward and would settle whether the temporal-window heuristic does causal work. This concern does not require rejecting the paper; it means the current evidence does not yet support the causal claim, and the condition for acceptance should include this control and the precision/recall analysis. The reader's recommendation of conditional acceptance remains appropriate, with the missing control stated as an explicit condition.","tokens_in":10839,"tokens_out":9428,"duration_ms":96825,"concrete_test":"Run a three-arm control on KITTI-4-reduced. Arm 1: optimize 3DGS on a uniformly random 720K-point subset of the 19.4M accumulated cloud. Arm 2: optimize 3DGS on points selected by a global depth-buffer: project all 19.4M points into each training view, keep the nearest point per pixel after culling points with non-positive depth or outside the image plane, union these per-view subsets, and truncate randomly to 720K if the union exceeds that count. Arm 3: the paper's reported 3DGS plus connectivity result. If arms 1 or 2 reach within a small margin of arm 3's PSNR/SSIM, the temporal-window connectivity is not the causal factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a connectivity graph (temporal local-isolation window, n=5 behind and 2n=10 ahead, followed by depth-based pruning) identifies the points actually visible from each camera pose, and that this retrieval is what produces the reported rendering gains and enables 3DGS to scale. The evidence in Table 3 does not isolate this mechanism: 3DGS with the connectivity retrieval uses 720K points versus 19.4M for the 3DGS baseline, so the 19.61 to 28.57 dB PSNR improvement on KITTI-4-reduced is confounded with an approximately 27x reduction in the number of optimized Gaussians. A uniformly random 720K-point subset, or a global depth-buffer (nearest point per pixel) over the full accumulated cloud, is not tested. If either control reaches comparable quality, the improvement is due to point-count reduction or standard z-buffering, not to the temporal-window heuristic that is the paper's named contribution. Moreover, no precision/recall of the retrieval against actual visibility is reported, and no sensitivity to n is given; failure modes such as a future-frame point that projects into the current view and is the nearest sample because the occluder is not scanned are not quantified. The claim that the method 'exceeds state-of-the-art' is also not uniformly supported: on KITTI-0, LPIPS is 0.34 versus 0.32 for NPBG++ (Table 2).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CE-NPBG, a neural point-based rendering method for novel view synthesis in large-scale autonomous driving scenes. The method builds a connectivity graph between posed images and an accumulated LiDAR point cloud by selecting, for each image, a temporal window of LiDAR scans (n behind and 2n ahead, with n=5) followed by depth-based pruning; only the retrieved points are rasterized and processed by a U-Net synthesizer trained with a multi-resolution LSGAN discriminator and a perceptual loss. The paper also integrates the connectivity retrieval into 3D Gaussian Splatting and reports improvements in fitting quality and scalability. Experiments on three KITTI360 sub-sequences compare against NPBG, NPBG++, READ, DS-NeRF, and 3DGS, reporting PSNR/SSIM/LPIPS numbers in Table 2 and 3DGS integration results in Table 3.","tokens_in":11183,"tokens_out":6728,"duration_ms":61111,"significance":"The paper identifies a real and practically important problem: accumulated LiDAR maps contain points that are not visible from a given camera pose, and projecting all of them into the image plane degrades renderings. The connectivity-based retrieval is a simple, plausible heuristic for selecting view-relevant points, and the reported integration with 3DGS shows a large improvement in fitting quality and enables optimization on a 36.3M-point sequence that otherwise fails. If the connectivity mechanism is validated with proper controls, the method would be a useful step toward scalable point-based NVS for driving scenes. The strengths include the use of a public dataset (KITTI360), standard metrics, and external baselines. However, the evidence as presented does not isolate the connectivity contribution from a 27x reduction in point count, and the claim of uniform superiority over the baselines is contradicted on the LPIPS metric for one sequence; the paper also does not release code or data, which limits reproducibility of the non-trivial preprocessing steps.","major_comments":[{"comment":"The 3DGS+Connectivity experiment confounds the connectivity mechanism with a 27x reduction in the number of optimized points; on KITTI-4-reduced, 3DGS uses 19,403,162 points while 3DGS+Connectivity uses only 720,000, and the reported PSNR gain from 19.61 to 28.57 dB cannot be attributed to the temporal-window heuristic without control experiments such as a uniformly random 720K-point subset or a global depth-buffer over the full accumulated cloud.","section":"§4.5, Table 3"},{"comment":"The abstract and §4.4 claim that the method 'exceeds the different state-of-the-art rendering methods', but on KITTI-0 the LPIPS of CE-NPBG is 0.34 versus 0.32 for NPBG++, so the method does not uniformly outperform on all reported metrics; the claim should be qualified or backed with additional evidence such as error bars and per-sequence analysis.","section":"§4.4, Table 2"},{"comment":"The connectivity graph is defined by a fixed local-isolation window (n=5 scans behind, 2n=10 ahead) and depth-based pruning, but the paper provides no sensitivity analysis for n and no precision/recall evaluation of the retrieved points against ground-truth visibility, e.g., points hit by the current LiDAR scan; without these, the load-bearing assumption that the window reliably contains all visible geometry and excludes occluded points remains unverified, particularly for novel poses distant from the nearest training pose.","section":"§3, Visibility Estimation from Connectivity Relationship"},{"comment":"The sentence claiming that 3DGS 'still underperforms our neural point-based rendering approach, even when combined with our connectivity relationship' is a comparative result that is not shown anywhere in the paper; either the results should be added to a table or the sentence should be removed as unsupported.","section":"§4.5"},{"comment":"No error bars, repeated runs, or significance tests are reported; given that the PSNR gaps over NPBG++ are around 1 dB on some sequences and that one LPIPS comparison is reversed, the robustness of the reported improvements is unclear.","section":"Tables 2-4"}],"minor_comments":[{"comment":"The acronym 'CE-NBPG' appears in 'we propose CE-NBPG' and should be 'CE-NPBG'.","section":"Abstract and §1"},{"comment":"The union notation 'T[ t=1' is unclear; an explicit big-union symbol with limits under the summation would be easier to read.","section":"§3, Equations (1)-(2)"},{"comment":"'A NerF-based approach' should be 'a NeRF-based approach'.","section":"§4.2"},{"comment":"The statement that DS-NeRF required 'certain adaptations' is vague; listing the concrete changes (coordinate frame, normalization, learning-rate adjustments) would aid reproducibility.","section":"§4.3"},{"comment":"The caption says 'run-time improvements from using our connectivity relationship on 3DGS', but the table reports only render time and FPS; the effect on training or fitting time is not discussed.","section":"§4.5, Table 4"},{"comment":"The manuscript does not mention code or data release; given the complexity of the connectivity-graph construction, releasing the preprocessing code would significantly help reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and the connectivity heuristic is plausible, but the key experiment in Table 3 is confounded by point-count reduction, and the central claim of uniform superiority is not supported by the reported LPIPS numbers. The authors should be encouraged to add the control experiments and sensitivity analysis described in the major comments; without these, the contribution of the connectivity graph remains unclear. The lack of code release is also a barrier to verification, but not by itself a reason to reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: this paper identifies a real problem—projecting a large accumulated LiDAR map into a camera view drags in points that are not actually visible from that pose, and that hurts learning-based point renderers. Its connectivity graph, which groups LiDAR scans near each camera pose and then prunes by depth, is a simple, plausible fix. Whether the graph itself is responsible for the reported gains is not actually shown.\n\nWhat's new: previous point-based NVS methods (NPBG, NPBG++) use a global depth buffer on the whole cloud. CE-NPBG restricts points to a temporal window of LiDAR scans (5 behind, 10 ahead) before depth pruning. That is a clean idea, and it is cheap to build once per sequence. When fed into the NPBG-style U-Net rasterizer, it gives consistent PSNR/SSIM improvements over NPBG and NPBG++ on three KITTI360 sequences, though the gains are modest (~1.5 dB at best) and the paper's own LPIPS numbers on KITTI-0 are worse than NPBG++ (0.34 vs 0.32), so \"exceeds state-of-the-art\" is overreach. The GAN component adds only a fraction of a dB.\n\nThe soft spots are real. The 3DGS integration, which the authors use as evidence of scalability, is confounded: \"3DGS + our connectivity\" uses 720K points while baseline 3DGS uses 19.4M. A random 720K subset, or a global depth buffer on the full cloud, would be the necessary control. Without it, the 9 dB jump cannot be attributed to the connectivity heuristic; it may just be the 27x reduction in Gaussians. There are also no error bars, no sensitivity analysis for the window size n, and no precision/recall for the retrieval. They don't vary n at all. And the baseline 3DGS was run on a reduced sequence because of memory; that's a fair caveat, but it weakens the comparison.\n\nI don't think the main idea is wrong. The temporal window is a reasonable way to handle driving scenes where the camera forward-motion means the scene seen from each pose is mostly what the nearby scans cover. But the evidence as written doesn't isolate the mechanism, and the 3DGS results oversell it. This deserves a serious referee—the idea is worth engaging, and could become solid with proper controls, code, and hyperparameter sensitivity. But I wouldn't cite it in this form.\n\nRecommendation: send it to peer review with a request for controls and code.\n\nCheers.","headline":"Plausible visibility-retrieval heuristic for point-based NVS, but the 3DGS evidence is confounded and the SOTA claim overreaches.","tokens_in":11724,"tokens_out":4112,"would_cite":false,"duration_ms":41453,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that building a connectivity relationship graph between posed images and LiDAR points, and retrieving only visible points for rasterization, resolves the appearance–geometry mismatch that degrades large-scale point-based…","keywords":["novel view synthesis","neural point-based graphics","LiDAR point clouds","visibility estimation","connectivity graph","autonomous driving","3D Gaussian Splatting","adversarial training"],"falsifier":"One would measure retrieval precision and recall of the connectivity graph on held-out novel poses by comparing its selected points against a reference visibility map computed by ray-casting the accumulated LiDAR cloud (or by dense depth from a multi-view reconstruction). If the retrieved set omits a substantial fraction of visible points or includes many occluded ones, and the omission grows with distance from the nearest training pose, the core explanation for the quality gain would be discredited; likewise, varying the window size n and showing the gain vanishes at n = 1 or n = 20 would indicate the result is an artifact of the chosen window.","tokens_in":10662,"feed_emoji":"🚗","tokens_out":6908,"duration_ms":63375,"temperature":0.7,"pith_summary":"This paper argues that the low quality of large-scale point-based novel view synthesis comes from a visibility mismatch: when the full accumulated LiDAR map is projected into a camera, many occluded or out-of-view points land on the image, so the geometry being rasterized disagrees with the image appearance. To fix this, it builds a connectivity relationship graph that links each posed image to the LiDAR scans around it, then prunes by depth, so only points actually visible from the camera are retrieved and rasterized. The paper claims this restores appearance–geometry compatibility, improves rendering quality on KITTI-360 scenes, and makes fitting more scalable, for example letting 3D Gaussian Splatting fit a 36.3-million-point sequence that otherwise fails. A sympathetic reader would care because the same visibility problem affects any forward-rendering method built on large LiDAR maps in autonomous driving, and the connectivity graph is a reusable fix that does not depend on the renderer.","feed_headline":"Visible-point retrieval lifts driving-scene rendering quality","feed_subtitle":"A connectivity graph between images and LiDAR points beats baselines and lets 3DGS fit 36M points.","key_machinery":"The central object is the connectivity relationship graph between appearance (posed images) and geometry (LiDAR point cloud). It is built in two stages: the greedy stage isolates a local window of scans around each image (n scans behind and 2n ahead, with n = 5), and the pruning stage projects the candidate points and keeps only those inside the image bounds with positive depth and shortest depth per pixel. This graph is constructed once and then queried at fitting and rendering time by retrieving the points attached to the closest camera pose. The other machinery is the rasterization pyramid: visible point descriptors are splatted at five resolutions, and a U-Net synthesizer with gated convolutions converts the rasterized descriptors into an RGB image while a multi-resolution LSGAN discriminator, used only during training, pushes the synthesizer toward realistic outputs. The connectivity graph does the load-bearing work of keeping geometry and appearance consistent; the adversarial branch adds a smaller, measurable gain.","core_discovery":"On the paper's own terms, the discovery is that the failure of point-based rendering on large driving scenes is not primarily a capacity or representation problem but a visibility-selection problem: using the entire accumulated point cloud for every view trains and renders with many unseen points, corrupting both descriptors and pixels. The solution is a two-stage connectivity relationship graph built once from the posed images and LiDAR scans: a greedy stage attaches each image to nearby scans (five behind, ten ahead) to ensure coverage, and a pruning stage keeps only points that lie in front of the camera and are closest to it per pixel. Rendering then retrieves only the connected, visible points for the nearest pose. The paper reports that this raises PSNR/SSIM/LPIPS over NPBG, NPBG++, READ, DS-NeRF, and 3DGS on all tested KITTI-360 sub-sequences, and that applying the same connectivity to 3DGS converts a failed 36.3-million-point optimization into one that fits and renders, with per-frame render time dropping from 0.046 s to 0.00089 s.","pith_inferences":["If the connectivity graph is truly reusable across renderers, the same precomputed visibility index could be plugged into other point-based or Gaussian splatting systems, effectively decoupling visibility estimation from scene representation; the paper demonstrates this only for 3DGS but the mechanism is renderer-agnostic.","The fixed window (five scans behind, ten ahead) assumes a forward-moving camera and a roughly constant LiDAR density; adapting the window size to local scan density or pose spacing could be a straightforward improvement, and the paper's lack of a sweep over n leaves this untested.","The authors treat appearance–geometry mismatch as the primary failure cause, but the same connectivity retrieval also reduces the number of points entering the loss, which speeds convergence independently of visibility correctness; separating these two effects would clarify how much of the gain is purely from pruning.","A testable extension is to use the connectivity graph to supervise visibility learning: the retrieved sets could serve as pseudo-labels to train a fast visibility predictor that works at arbitrary novel poses without a graph lookup."],"forward_implications":["Any forward-rendering method that uses large LiDAR maps can adopt the connectivity graph as a visibility prefilter, since it is built once and queried with a nearest-pose lookup.","Fitting and rendering memory requirements drop to only the visible subset of the point cloud, so scenes with tens of millions of points fit on a single 48 GB GPU.","The reported render-time improvement (from 0.046 s to 0.00089 s per frame for 3DGS) suggests connectivity-based retrieval could move large-scene rendering toward interactive rates.","Because the graph is pose-indexed, novel views near a training pose inherit that pose's visibility, so the method should generalize best along continuous driving trajectories rather than across large pose gaps.","The adversarial branch is optional: the ablation shows most of the quality gain comes from connectivity, with a smaller increment from the discriminator."],"supporting_citations":[{"why":"Defines the neural point-based graphics baseline and the depth-buffer visibility that this paper argues fails on large scenes.","marker":"[1]"},{"why":"NPBG++ baseline, the closest point-based method with view-dependent descriptors, outperformed here via connectivity-based visibility.","marker":"[19]"},{"why":"READ, the large-scale point-based baseline that voxelizes the scene, which this method compares against and surpasses.","marker":"[15]"},{"why":"3D Gaussian Splatting baseline, which fails on the full 36.3M-point sequence and is enabled by the proposed connectivity retrieval.","marker":"[12]"},{"why":"DS-NeRF baseline using LiDAR depth supervision, representing the NeRF-family comparison.","marker":"[4]"},{"why":"LSGAN criterion that defines the adversarial losses used in the joint adversarial-rasterization training.","marker":"[16]"},{"why":"Perceptual (VGG) loss used as part of the training objective alongside the adversarial loss.","marker":"[10]"}],"fun_headline_variants":["Connectivity graph lifts rendering quality in driving scenes","Only visible points matter: fixing point-based view synthesis","Visibility-aware point retrieval improves NVS for autonomous driving","Point-based rendering scaled by selecting visible points, not more data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach rests on the fixed local isolation window (five scans behind, ten ahead) plus a depth check being a reliable predictor of which points are actually visible from a camera pose; if that heuristic misses visible geometry or admits occluded points, the renderings would contain holes or the mismatch it was meant to remove would return.","fun_headline_variants_meta":{"raw":{"variants":["Connectivity graph lifts rendering quality in driving scenes","Only visible points matter: fixing point-based view synthesis","Visibility-aware point retrieval improves NVS for autonomous driving","Point-based rendering scaled by selecting visible points, not more data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000976,"raw_usage":{"total_tokens":4197,"prompt_tokens":1049,"completion_tokens":3148,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":665,"completion_tokens_details":{"reasoning_tokens":3084}},"tokens_in":665,"tokens_out":3148,"duration_ms":24261,"temperature":1.0,"reasoning_tokens":3084,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:48:52.450903+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One would measure retrieval precision and recall of the connectivity graph on held-out novel poses by comparing its selected points against a reference visibility map computed by ray-casting the accumulated LiDAR cloud (or by dense depth from a multi-view reconstruction). If the retrieved set omits a substantial fraction of visible points or includes many occluded ones, and the omission grows with distance from the nearest training pose, the core explanation for the quality gain would be discredited; likewise, varying the window size n and showing the gain vanishes at n = 1 or n = 20 would indicate the result is an artifact of the chosen window.","supporting_citations":[{"cited_title":"Neural point-based graph- ics","cited_arxiv_id":null,"evidence_quote":"Defines the neural point-based graphics baseline and the depth-buffer visibility that this paper argues fails on large scenes."},{"cited_title":"Npbg++: Accelerating neural point-based graphics","cited_arxiv_id":null,"evidence_quote":"NPBG++ baseline, the closest point-based method with view-dependent descriptors, outperformed here via connectivity-based visibility."},{"cited_title":"Read: Large-scale neural scene rendering for autonomous driving","cited_arxiv_id":null,"evidence_quote":"READ, the large-scale point-based baseline that voxelizes the scene, which this method compares against and surpasses."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"3D Gaussian Splatting baseline, which fails on the full 36.3M-point sequence and is enabled by the proposed connectivity retrieval."},{"cited_title":"Depth-supervised NeRF: Fewer views and faster training for free","cited_arxiv_id":null,"evidence_quote":"DS-NeRF baseline using LiDAR depth supervision, representing the NeRF-family comparison."},{"cited_title":"Least squares generative adversarial networks","cited_arxiv_id":null,"evidence_quote":"LSGAN criterion that defines the adversarial losses used in the joint adversarial-rasterization training."}],"review_version":1}