{"id":"69d2ed13-25db-4582-a788-6b6bc5a860b8","arxiv_id":"2504.15776","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A NeRF-based pipeline refines the sensor poses and calibration of KITTI-360, NuScenes, PandaSet, and Waymo, with proxy metrics indicating improved accuracy.","lead":"This paper uses neural radiance fields to refine the camera and LiDAR poses and the calibration of four autonomous driving datasets, and it evaluates the refined poses without ground truth. A reader might use it to obtain better benchmark data or to see how NeRF-based pose refinement can be assessed when true poses are unknown.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No external ground truth validates absolute pose accuracy; all reported metrics are internal consistency measures, so 'significant improvements in sensor pose accuracy' is not established.","rationale":"I read the paper as a dataset-quality application: refine existing poses and calibration with neural rendering and validate without ground truth. The pipeline is thoughtfully designed; strengths include the use of Nerfacto and Splatfacto as independent rendering backbones, the KISS-ICP control in Table III, per-sequence sign plots, and an explicit limitations section. The reader's CONDITIONAL verdict is reasonable, and I do not propose moving it. My stress-test concern is that the central claim is stronger than the evidence: every reported metric is a consistency measure, and the paper itself acknowledges the most circular one, NVS. The COLMAP and mesh metrics are less biased but still lack an absolute reference; none can rule out a global distortion that improves internal agreement while degrading true pose accuracy. The abstract's word 'accuracy' therefore goes beyond what is measured. The proposed synthetic or ground-truth trajectory test would settle this directly and is feasible with the same codebase. I agree with the reader's weakest-assumption analysis and would keep the verdict CONDITIONAL, requiring either external validation or a tempered statement of the claims.","tokens_in":20731,"tokens_out":7389,"duration_ms":73047,"concrete_test":"Run the MOISST and SOAC pipelines on a synthetic autonomous-driving dataset with exact ground-truth poses, such as CARLA with known camera and LiDAR extrinsics and trajectory, using the same subsequence creation and dynamic filtering protocol. Compute absolute trajectory error (ATE) and relative pose error (RPE) for the original poses, the optimized poses, and the KISS-ICP-initialized poses. If ATE and RPE do not improve while the paper's proxy metrics do improve, the proxies are not valid evidence of absolute pose accuracy; if ATE and RPE improve, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that optimized poses improve sensor pose accuracy rests entirely on proxy metrics. The COLMAP reprojection and track length, NVS PSNR/SSIM/LPIPS, and Delaunay mesh-to-LiDAR metrics all measure internal consistency: how well poses agree with image features, how well held-out views render, or how well camera-derived geometry aligns with LiDAR. The optimization directly minimizes photometric and geometric consistency in the same NeRF framework; Section III-C.2 explicitly concedes that NVS improvements are expected because 'we optimize our poses through NeRF training.' COLMAP and mesh metrics are less circular, but they still only show that the sensors are more self-consistent, not that the vehicle trajectory is closer to the physical ground-truth trajectory. Jointly optimizing camera and LiDAR poses can improve mutual alignment while introducing global distortion such as scale drift or datum changes; the reported metrics cannot detect such absolute errors. Table III shows that for NuScenes the baseline is altered by KISS-ICP before optimization, further complicating the 'original vs optimized' comparison. Section V-E's sign-agreement 'optimal pose selection' underscores the residual ambiguity: the method does not identify a single more-accurate pose set, but offers a voting procedure over metrics. Without an external reference, the abstract's 'significant improvements in sensor pose accuracy' is an inference, not a demonstrated result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a NeRF-based optimization pipeline that refines vehicle trajectories and extrinsic sensor calibration for four public autonomous driving datasets (KITTI-360, NuScenes, PandaSet, Waymo). The pipeline builds on the authors' MOISST and SOAC models, adds dynamic-object filtering and subsequence creation, and optimizes an extrinsic-correction embedding and a continuous trajectory-correction MLP. The authors evaluate the corrected poses against the original poses with three families of proxy metrics: COLMAP keypoint reprojection/track length, novel-view-synthesis PSNR/SSIM/LPIPS on NeRF-LiDAR, Nerfacto, and Splatfacto, and Delaunay-mesh-to-LiDAR precision/distance. They report that MOISST generally improves these metrics and release the optimized poses.","tokens_in":20918,"tokens_out":7164,"duration_ms":62581,"significance":"The intended contribution—public corrected sensor poses for widely used driving benchmarks—is potentially valuable to the community, and the paper includes several strengths: evaluation across four datasets, a KISS-ICP baseline for NuScenes, and independent Nerfstudio renderers for the NVS metrics. However, the central claim of 'significant improvements in sensor pose accuracy' overstates what the evidence shows. All reported metrics are internal-consistency measures, and the paper itself acknowledges in Section III-C.2 that NVS improvements are natural because the optimization minimizes the same photometric objective. The results also show regressions for SOAC on KITTI-360 and Waymo and only minimal MOISST gains on KITTI-360, so the claim of consistent improvement is not supported without error bars or significance tests. If the authors re-scope the claims and add an external validation (synthetic ground truth or a high-accuracy reference), the contribution could be solid.","major_comments":[{"comment":"The abstract's claim of 'significant improvements in sensor pose accuracy' is not established because every reported metric is a proxy for internal consistency rather than absolute pose accuracy. The COLMAP reprojection and mesh-to-LiDAR metrics are less circular than the NVS metrics, but they still cannot detect global distortions such as scale drift or datum changes that preserve self-consistency; the optimizer could improve alignment between sensors while moving the trajectory away from the physical one. I recommend adding a synthetic experiment with known pose perturbations (e.g., perturbing a rendered scene or using a simulator with ground-truth poses) and/or comparing against a high-accuracy external reference trajectory to demonstrate that the corrected poses are closer to the true trajectory.","section":"Abstract; Section V (Tables IIa–IIe)"},{"comment":"The text claims a consistent improvement over original poses, but the data contradict this for SOAC: on KITTI-360, reprojection error worsens from 0.586 to 0.735 and mesh precision drops from 0.589 to 0.582; on Waymo, reprojection error worsens from 1.137 to 1.344 and precision drops from 0.441 to 0.417. For MOISST on KITTI-360, the gains are tiny (0.586 to 0.577 for reprojection; 6.53 to 6.68 for track length), and Fig. 9 shows reprojection improves in only 24 of 50 subsequences. The manuscript should report per-sequence distributions, confidence intervals, and a significance test, and should replace 'consistent improvement' with a more conditional statement.","section":"Section V-A; Tables IIa and IIe; Fig. 9"},{"comment":"The NVS evaluation is partially circular because the NeRF-LiDAR model used for evaluation is from the same MOISST family that produced the poses; PSNR/SSIM/LPIPS gains on this model can reflect better optimization of the same photometric objective rather than pose accuracy. The paper acknowledges this in Section III-C.2, but the related-work discussion in Section II-C promises an unbiased evaluation, which is stronger than the evidence. The Nerfacto and Splatfacto evaluations mitigate the renderer-specific concern, but all NVS metrics still reward photometric consistency. I suggest adding a structure-based evaluation that does not use the optimization objective, for example running a full COLMAP bundle adjustment initialized from the optimized poses and reporting reconstruction statistics, or evaluating a downstream geometric task such as monocular depth alignment.","section":"Section III-C.2; Table IIb"},{"comment":"The NuScenes comparison is confounded by KISS-ICP preprocessing. The baseline 'Original' lacks Z-axis information, while MOISST and SOAC are applied after KISS-ICP; Table III shows that KISS-ICP alone improves all metrics (e.g., LPIPS 0.465 to 0.418), so part of the reported MOISST/SOAC improvement is attributable to this preprocessing. To isolate the contribution of the NeRF-based optimization, report MOISST/SOAC applied directly to the original SE2 poses, or otherwise quantify the additive effect beyond the KISS-ICP baseline under matched experimental conditions.","section":"Section V; Table III"},{"comment":"The sign-agreement analysis shows that the different metrics agree only moderately (over 0.5 overall, with the lowest agreement on KITTI-360). This means the method does not deliver a single demonstrably accurate pose set but rather candidate poses subject to a user-selected voting rule. The paper should either present a principled selection criterion or weaken the claim that the pipeline outputs improved poses.","section":"Section V-E; Fig. 8"}],"minor_comments":[{"comment":"'Subsequencies' should be spelled 'subsequences'.","section":"Section IV-B"},{"comment":"'Delauney' should be 'Delaunay' (also in the caption of Table IIe).","section":"Section V-C and Figures 13–16"},{"comment":"The caption label 'Imgine' appears to be a typo for 'Image'.","section":"Figures 17–20"},{"comment":"The text contains visible LaTeX artifacts such as '10%⁄tildelow36%'; please ensure the compiled PDF renders these ranges correctly.","section":"Section V-A"},{"comment":"The entries '15 epochsiterations' and '10 epochsNVS iterations' appear to be missing separators; clarify the intended units for epochs and iterations.","section":"Table I"},{"comment":"The phrase 'the average point reprojection error' should specify that the average is computed over the matched keypoints used in triangulation.","section":"Section III-C.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a natural extension of the authors' own MOISST and SOAC work, so the evaluation relies heavily on their own models; an independent pose optimization method beyond Nerfstudio would strengthen the evidence. The release of optimized poses is useful, but the central claim needs re-scoping. I am not recommending rejection because the issue is fixable with additional experiments and a more cautious narrative."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What should know: this is not a new optimization method. The core is MOISST and SOAC from the authors' prior papers, applied here to dataset-wide pose and calibration refinement on KITTI-360, NuScenes, PandaSet, and Waymo. The genuinely new pieces are the multi-metric evaluation protocol (COLMAP reprojection without bundle adjustment, NVS with three models, and mesh-to-LiDAR geometric consistency) and the promised release of refined poses. The paper is fairly honest about its main weakness: it explicitly concedes in Section III-C.2 that NVS improvements are expected because the same photometric objective is being optimized. That is good to see.\n\nWhat it does well: the evaluation is more thorough than most pose-refinement papers. Per-sequence results are shown, not just averages. The KISS-ICP ablation on NuScenes helps separate the preprocessing effect from the actual optimization. Including a Nerfstudio baseline, even if it can't use LiDAR, is the right instinct. The COLMAP and mesh-to-LiDAR metrics are less circular than NVS and provide genuine, if still indirect, evidence that the poses are more self-consistent.\n\nWhere it is soft: the abstract says 'significant improvements in sensor pose accuracy,' but no external ground truth is used anywhere. Every metric measures internal consistency, not absolute accuracy. That is fine if the claim is 'improved consistency for downstream tasks,' but the word 'accuracy' overreaches. The results are also mixed: MOISST gains on KITTI-360 are minimal (1.5% reprojection, and improvement in only 24 of 50 subsequences), and SOAC regresses on KITTI-360 and Waymo. No error bars, standard deviations, or significance tests appear anywhere. The sign-agreement 'optimal pose selection' in Section V-E is a reasonable heuristic, but it underlines that there is no single best answer. Finally, the poses are promised but no download link or code is provided; for a paper whose contribution is a dataset resource, that is a real missing piece.\n\nThe stress-test note is largely on target. All reported metrics are internal consistency. That doesn't sink the paper, but it means the central claim should be reworded and ideally supported by a small validation subset with known poses (e.g., one sequence with RTK GPS or a synthetic scene) to anchor the results.\n\nWho this is for: anyone using these four datasets for NVS, SLAM, or multi-sensor fusion who suspects the original poses are limiting their results. If the refined poses are actually released, this becomes a useful community resource.\n\nRecommendation: yes, send to peer review. It is an application paper with a transparent evaluation and a potentially useful artifact, but it needs revision: temper the claims, add statistical support, and release the poses. A serious referee should engage with it, not desk-reject it.","headline":"A useful and honest application of the authors' own NeRF-based calibration methods to refine poses on four driving datasets, but the claim of 'significant improvements in sensor pose accuracy' overreaches what internal-consistency metrics can show.","tokens_in":21550,"tokens_out":1879,"would_cite":true,"duration_ms":18895,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"NeRF-based pose refinement measurably improves sensor poses in four public driving datasets.","keywords":["Neural Radiance Fields","sensor pose refinement","extrinsic calibration","trajectory optimization","autonomous driving datasets","novel view synthesis","multi-sensor fusion","pose evaluation without ground truth"],"falsifier":"Run the pipeline on a synthetic or motion-capture urban dataset with known ground-truth poses, starting from deliberately perturbed poses; if the optimized poses do not move closer to the ground truth on absolute trajectory error, or if the proxy metrics improve while true pose error grows, the central claim is refuted.","tokens_in":20466,"feed_emoji":"🚗","tokens_out":10302,"duration_ms":82538,"temperature":0.7,"pith_summary":"Public driving datasets carry small but systematic errors in sensor poses and camera-to-LiDAR calibration, and those errors can dominate the gains reported by downstream models. This paper claims that a neural-radiance-field optimization pipeline can refine both the vehicle trajectory and the extrinsic calibration, making the dataset poses more accurate even when no ground-truth poses are available. It applies the pipeline to KITTI-360, NuScenes, PandaSet, and Waymo, and reports consistent improvements across three families of proxy metrics: keypoint reprojection and track length, novel-view-synthesis quality, and mesh-to-LiDAR geometric alignment. The central contribution, read sympathetically, is a practical cleaning of public benchmarks together with an evaluation protocol for judging pose quality without ground truth.","feed_headline":"NeRF pose refinement improves all four major driving datasets","feed_subtitle":"Optimized trajectories cut reprojection error and sharpen LiDAR alignment across KITTI-360, NuScenes, PandaSet, Waymo.","key_machinery":"The load-bearing mechanism is a differentiable neural scene representation trained with photometric and LiDAR-geometric losses, through which pose and calibration corrections are back-propagated. The optimized pose of sensor $i$ at time $t$ is written $wT_i(t) = E_{\\mathrm{cor}}(i)\\,\\, {}^iT_r\\, T_{\\mathrm{cor}}(t)\\, T_r(t)$, where $T_r$ is the interpolated reference trajectory, $E_{\\mathrm{cor}}(i)$ is a shared per-sensor calibration correction, and $T_{\\mathrm{cor}}(t)$ is a continuous trajectory correction learned by a small network over time. The pipeline first removes dynamic objects by masking annotated bounding boxes, splits long sequences into subsequences small enough for a NeRF, optimizes the shared extrinsic correction across many subsequences in parallel, then freezes calibration and optimizes each trajectory. MOISST is the multi-sensor model that fuses cameras and LiDAR under rigid sensor constraints; SOAC is the overlap-aware alternative with separate scene representations per camera; two open-source rendering models, one NeRF-based and one Gaussian-splatting-based, serve as independent baselines and evaluation renderers.","core_discovery":"The paper's central claim is that jointly optimizing sensor poses and extrinsic calibration inside a neural radiance field yields measurably better dataset poses than the ones shipped with four major autonomous driving datasets. Using MOISST as the main model and SOAC as a robustness-focused alternative, with trajectory correction added to both, the authors report that optimized poses reduce reprojection error, lengthen keypoint tracks, improve PSNR, SSIM, and LPIPS across three separate rendering models, and tighten the alignment between image-derived meshes and accumulated LiDAR. The improvements are consistent for MOISST on all four datasets; SOAC improves NuScenes and PandaSet but regresses on KITTI-360 and Waymo. The paper also shows that KITTI-360's original poses are already precise, with relative gains around one to two percent, while NuScenes benefits the most because its published poses are planar and lack a reliable Z-axis component.","pith_inferences":["My inference: if the proxy metrics genuinely track pose error, the released poses could serve as pseudo-ground truth for calibrating and evaluating localization systems on these datasets, but that status would need independent confirmation on a set with known poses first.","My inference: the method will inherit NeRF's failure modes, such as dark tunnels, repetitive walls, and distant structures, so a natural extension is to swap in a more failure-tolerant renderer, for instance Gaussian splatting with depth priors, and test whether the same pipeline still holds.","My inference: because the rendering metrics are produced by the same family of models used for optimization, the reprojection and geometric metrics carry most of the evidential weight; a useful next step would be a sign-agreement analysis that excludes rendering metrics to see whether geometry alone recommends the same poses.","My inference: the shared-calibration step assumes a fixed sensor rig across a dataset, so datasets captured with multiple vehicles, like Waymo, are forced into per-sequence calibration; estimating per-sequence extrinsic priors from odometry would make the pipeline scale to fleet-scale data."],"forward_implications":["If the optimized poses are accepted, the four datasets become higher-fidelity benchmarks: downstream models are trained and evaluated against geometry closer to the true scene, reducing the risk that reported gains come from overfitting to pose noise.","Neuro-renderers trained from scratch on the optimized poses render with higher PSNR and SSIM and lower LPIPS, so any task built on novel view synthesis inherits the improvement.","Mesh-to-LiDAR precision rises under MOISST on all four datasets, indicating better camera-to-LiDAR alignment and therefore better multi-sensor fusion for mapping and localization.","For NuScenes, where the published poses carry no Z information, trajectory optimization reconstructs a usable vertical component and produces the largest metric gains, with reprojection error dropping from 1.339 to 0.852 pixels."],"supporting_citations":[{"why":"Waymo Open Dataset, one of the four target datasets whose poses and calibration are refined and evaluated.","marker":"[1]"},{"why":"KITTI-360, the target dataset whose original poses are already precise and therefore show the smallest gains.","marker":"[2]"},{"why":"PandaSet, the target dataset providing sequences with semantic labels used for dynamic filtering and evaluation.","marker":"[3]"},{"why":"NuScenes, the target dataset whose planar poses lack a Z component and benefit most from trajectory optimization.","marker":"[4]"},{"why":"MOISST, the core multi-sensor NeRF calibration model that the pipeline extends with trajectory optimization.","marker":"[9]"},{"why":"SOAC, the overlap-aware per-camera NeRF calibration model used as the alternative optimizer in the comparison.","marker":"[10]"},{"why":"Supplies the keypoint triangulation, reprojection-error, track-length, and mesh-construction tools used for the external pose-quality evaluation.","marker":"[12]"},{"why":"The ICP-based LiDAR odometry baseline used to add a Z component to the NuScenes poses and to demonstrate that the reported gains come from the NeRF refinement rather than preprocessing.","marker":"[19]"},{"why":"Supplies the point-to-mesh and precision metrics used to measure geometric consistency between image-derived surfaces and LiDAR.","marker":"[74]"},{"why":"Provides the two open-source rendering models used to verify that the improvements are not specific to a single neural renderer.","marker":"[82]"}],"fun_headline_variants":["NeRF pose optimization sharpens driving dataset accuracy","Refining vehicle poses with neural rendering for better benchmarks","Neural radiance fields correct sensor poses in driving sets","Pose refinement via NeRF improves four driving benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the proxy metrics—reprojection error, rendering quality, and mesh-to-LiDAR distance—actually track true pose accuracy, even though the rendering metrics are partly circular because the poses are optimized by training the same kind of model that then scores them.","fun_headline_variants_meta":{"raw":{"variants":["NeRF pose optimization sharpens driving dataset accuracy","Refining vehicle poses with neural rendering for better benchmarks","Neural radiance fields correct sensor poses in driving sets","Pose refinement via NeRF improves four driving benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000504,"raw_usage":{"total_tokens":2453,"prompt_tokens":932,"completion_tokens":1521,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":1458}},"tokens_in":548,"tokens_out":1521,"duration_ms":11304,"temperature":1.0,"reasoning_tokens":1458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:17:04.434361+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on a synthetic or motion-capture urban dataset with known ground-truth poses, starting from deliberately perturbed poses; if the optimized poses do not move closer to the ground truth on absolute trajectory error, or if the proxy metrics improve while true pose error grows, the central claim is refuted.","supporting_citations":[{"cited_title":"MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration,","cited_arxiv_id":null,"evidence_quote":"MOISST, the core multi-sensor NeRF calibration model that the pipeline extends with trajectory optimization."},{"cited_title":"Soac: Spatio-temporal overlap-aware multi-sensor calibration using neural radiance fields,","cited_arxiv_id":null,"evidence_quote":"SOAC, the overlap-aware per-camera NeRF calibration model used as the alternative optimizer in the comparison."},{"cited_title":"Structure-from-Motion Revisited,","cited_arxiv_id":null,"evidence_quote":"Supplies the keypoint triangulation, reprojection-error, track-length, and mesh-construction tools used for the external pose-quality evaluation."},{"cited_title":"Kiss-icp: In defense of point-to-point icp–simple, accu- rate, and robust registration if done the right way,","cited_arxiv_id":null,"evidence_quote":"The ICP-based LiDAR odometry baseline used to add a Z component to the NuScenes poses and to demonstrate that the reported gains come from the NeRF refinement rather than preprocessing."},{"cited_title":"ViiNeuS: Volumetric Initialization for Implicit Neural Surface reconstruction of urban scenes with limited image overlap","cited_arxiv_id":"2403.10344","evidence_quote":"Supplies the point-to-mesh and precision metrics used to measure geometric consistency between image-derived surfaces and LiDAR."},{"cited_title":"Nerfstudio: A modular framework for neural radiance field development,","cited_arxiv_id":null,"evidence_quote":"Provides the two open-source rendering models used to verify that the improvements are not specific to a single neural renderer."}],"review_version":1}