REVIEW 4 major objections 6 minor 47 references
MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis
T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read MV2 is a multi-vehicle driving dataset that shows novel-view synthesis quality collapses when train and test views come from different vehicles (car, scooter, drone).
desk verdict Useful new cross-vehicle NVS benchmark, but the headline numbers rest on pose self-consistency rather than absolute pose ground truth; worth a serious referee if the authors commit to validating or softening that claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the multi-vehicle capture setup: synchronized car, scooter, and drone cameras looking at the same scene, registered into one coordinate frame by Structure-from-Motion. Poses are certified not by ground truth but by epipolar consistency: human-annotated region correspondences, refined with a dense neural matcher, must keep the mean and maximum epipolar error of every relative pose at or below 30 pixels. That certification step decides which of 200 recorded sequences survive as the 50-scene benchmark, so it is the gatekeeper for every numerical comparison in the paper.
What would settle it
Register a handful of MV2 scenes with surveyed markers or RTK-GPS ground-truth poses, then compare those true poses against the Structure-from-Motion poses that passed the epipolar threshold; if large global pose errors coexist with sub-30-pixel epipolar errors, the certification criterion is insufficient.
Extended reading notes
Core claim
The central claim is that current novel-view-synthesis models, including dynamic Gaussian-splatting methods that lead on standard splits, degrade sharply when asked to render a view from a different vehicle's trajectory, and that existing single-trajectory benchmarks cannot expose this failure because they only interpolate between nearby frames. On MV2, the best dynamic method drops from 27.01 to 20.23 PSNR when moving from the same-trajectory test split to the car-to-scooter split, and aerial-to-ground splits are harder still. The paper also finds that feed-forward pose estimators produce much larger epipolar errors than optimization-based localization under these wide baselines, so the dataset doubles as a pose-estimation stress test.
Load-bearing premise
The benchmark assumes that a maximum epipolar error of 30 pixels on a few dozen manually annotated correspondences is enough to certify that every camera pose is accurate, because no independent ground-truth pose source exists.
Editorial extensions
If this is right
- Benchmarks that sample train and test frames from one vehicle trajectory overestimate generalization; MV2-style cross-vehicle splits should be used to measure extrapolation.
- Aerial-to-ground synthesis, where current methods score below 13 PSNR, is an open problem rather than a solved one.
- Depth supervision from monocular estimators, when filtered to the 10–30 m range, improves dynamic-scene methods but does not close the wide-baseline gap.
- Feed-forward pose estimators need to handle large viewpoint gaps before they can replace optimization-based localization in driving scenes.
- NVS quality on cross-vehicle splits is a direct proxy for whether a simulator can render a plausible lane-change maneuver.
Reading between the lines
- The epipolar-error threshold of 30 pixels could be tested directly: if a subset of scenes were re-certified with surveyed ground-truth poses, the threshold's sufficiency could be quantified rather than assumed.
- The same benchmark could be extended to closed-loop evaluation, where a rendered scooter or drone view is fed to a perception model; the paper notes downstream uses but does not run that experiment.
- A natural follow-up is to check whether training feed-forward splatting models on multi-vehicle, wide-baseline data reduces the gap, since the paper only evaluates existing models trained on standard data.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces MV2, a multi-view multi-vehicle driving dataset for novel view synthesis, with synchronized captures from a car, a scooter, and a drone. The authors describe a collection pipeline, SfM-based pose estimation, and a manual correspondence-based epipolar verification step that filters 200 recorded sequences down to 50 scenes. They define two evaluation setups (Eval-Car-Train and Eval-Drone-Train) and report NVS benchmarks for static, dynamic, and feed-forward methods, plus a comparison of COLMAP against feed-forward pose estimators. The headline findings are that NVS performance degrades as the train-test viewpoint disparity increases (e.g., PVG PSNR drops from 27.01 on T_C->C to 20.23 on T_C->S in Table 1) and that feed-forward pose estimators exhibit larger epipolar errors than COLMAP.
Significance. The dataset fills a genuine gap: existing driving NVS benchmarks sample train and test views from one trajectory, while MV2 provides co-registered ground-level and aerial captures of the same dynamic scenes. If the pose verification can be made rigorous, the benchmark would be a valuable resource for studying extrapolation rather than interpolation, and the paper's careful comparison of optimization-based and feed-forward methods, depth-supervision ablations, and dynamic-mask ablations would be useful to the community. The main results are plausible and the release of data and protocol is a strength. However, the paper's central quantitative claims currently rest on pose estimates whose absolute accuracy is explicitly unverified, and the dataset documentation contains inconsistencies that must be fixed before the benchmark can be reproduced.
major comments (4)
- [Section 3.5, Eq. (1)] Equation (1) defines the epipolar error as e = |x2^T F x1|, but this is the algebraic epipolar residual, not a pixel distance; the point-to-line distance requires normalization by the length of the epipolar line coefficients. As written, the thresholds em <= 30 used for sequence filtering and the histograms in Fig. 4 are not measured in pixels. Because this threshold is what removes 150 of 200 scenes, the pose-filtering protocol and every downstream conclusion that relies on it need to be re-evaluated with a correctly normalized error, or the implementation should be stated explicitly.
- [Section 3.5] The verification of camera poses is self-consistency only. The paper states that 'Since ground-truth poses are unavailable, absolute pose accuracy cannot be directly verified,' and the proposed checks—epipolar consistency on consecutive training edges, one long-range edge, and test-to-nearest-training pairs—cannot rule out global SfM drift or a consistently wrong alignment between the vehicle trajectories. A smoothly drifting reconstruction can satisfy all pairwise epipolar constraints while accumulating large absolute errors, and a mislocalized test image can still satisfy the epipolar constraint with its nearest neighbor. Because every NVS number in Table 1 is computed by rendering at these poses, systematic pose error would directly depress the metrics and could mimic or exaggerate the reported baseline-vs-degradation trend. I recommend validating a subset of scenes against an external reference (e.g., GPS/IMU, surveyed markers, or LiDAR alignment) or, failing that, explicitly reframing the benchmark as a relative/self-consistent evaluation and removing the strong 'pose accuracy' wording.
- [Sections 3.1–3.3] The dataset composition is internally inconsistent. Section 3.1 defines only four camera streams (VL, VC, VS, VD), yet Section 3.2 reports '5 camera sequences each' and Section 3.3 mentions a test set T_C->D that was never defined. The abstract's 12,000 images also do not follow from the stated numbers: 50 retained scenes with 5 cameras and 100 frames per camera would yield 25,000 images even before excluding every 5th test frame. The paper must reconcile the number of cameras, define all test sets, and report per-split image counts so the benchmark can be reproduced.
- [Section 5.5, Fig. 5] The camera-pose-estimation comparison is evaluated with the same epipolar-error criterion that was used to filter COLMAP poses and select the scenes, so the comparison is not independent: COLMAP is favored by construction, since the 50 scenes were retained precisely because COLMAP's poses passed this check, while feed-forward methods are scored on the same quantity. This does not invalidate the observation that feed-forward methods produce larger epipolar residuals, but the claim that they 'lag behind' optimization-based methods should be presented as a relative statement about this metric, or verified with an external pose reference.
minor comments (6)
- [Throughout] Many cross-references are unresolved in the submitted text: 'Section??' appears in Sections 2, 3.3, 4, 5.2, 5.3, 5.4, and 5.5, and 'Fig.??' appears in Section 4; these need to be filled in before publication.
- [Section 3.1] Section 3.1 states that the resolution is 1080×1980 while the abstract uses 1080×1920; please correct the inconsistency.
- [Section 3.1] Section 3.1 says 'the set of images collected using these three cameras' after defining only two car cameras; the number of cameras per vehicle should be stated unambiguously.
- [Section 3.3] In Section 3.3, the sentence beginning 'The following pairs of test sets' is grammatically incomplete and refers to T_C->D, which is not introduced in the Eval-Car-Train setup; please revise.
- [Figure 6 caption] Figure 6 caption contains the typo 'BBoth'.
- [Section 3.3 / Table 1] The paper does not quantify the actual camera baselines between T_C->C, T_C->L, T_C->S, T_D->C, etc.; since the central trend is attributed to increasing baseline, reporting average baseline distances or another disparity measure would strengthen the claim.
Circularity Check
No circularity found: MV2's benchmark results are empirical measurements over independent methods, not derivations from their own assumptions.
full rationale
The paper's central claims—NVS performance degrades with increasing train–test viewpoint disparity, aerial-to-ground synthesis is hard, and feed-forward pose estimators underperform optimization-based COLMAP—are empirical measurements over independently implemented methods on a new dataset; none of these results is defined in terms of, or fitted to, the quantity it is said to predict. The test sets TC→C, TC→L, TC→S are ordered by sensor geometry by design, but the observed PSNR/SSIM/LPIPS drops are measured, not implied by that ordering, and the same trend appears across multiple methods including the WOD qualitative cross-lane comparison. No load-bearing self-citation chain appears: the pose-verification procedure (Section 3.5) explicitly states ground-truth poses are unavailable and verifies only epipolar consistency, which is an acknowledged validity limitation of the benchmark rather than a circular derivation; the pose-estimation comparison reuses the same RoMA correspondences as the verification step, but that is a shared evaluation metric, not a fitted parameter renamed as a prediction. The depth-alignment (Eq. 3) and mask ablations are standard preprocessing choices, not predictions. The paper is therefore self-contained against external benchmarks for the purpose of circularity analysis.
Assumptions & free parameters
free parameters (2)
- Epipolar error rejection threshold =
30 (maximum epipolar error)
- Affine depth alignment coefficients =
alpha and beta per scene
assumptions (3)
- domain assumption Epipolar consistency of manually annotated correspondences is a sufficient proxy for absolute pose accuracy.
- domain assumption Wall-clock synchronization is sufficient to treat car, scooter, and drone captures as observing the same dynamic scene at the same time.
- domain assumption The 15 nearest training images provide enough covisibility for accurate test image localization.
Cite this review
Pith. "Pith review of MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis." pith.science (2026). https://pith.science/paper/W54H6F37
@misc{pith2026260812442,
author = {Pith},
title = {Pith review of: MV2: Multi-View Multi-Vehicle Driving Dataset for Novel View Synthesis},
year = {2026},
howpublished = {\url{https://pith.science/paper/W54H6F37}},
note = {Machine review of arXiv:2608.12442}
}
read the original abstract
Differentiable rendering has advanced novel view synthesis (NVS), yet applying it to real-world driving remains difficult due to sparse capture viewpoints, dynamic objects, and limited multi-trajectory data. We introduce the Multi-View Multi-Vehicle (MV2) dataset and benchmark for evaluating NVS models under large viewpoint changes in dynamic urban scenes. MV2 features synchronized captures from a car, scooter, and drone, each following distinct yet synchronized trajectories. Training NVS methods on one vehicle's camera stream and testing on another enables evaluation under substantially larger viewpoint variations than existing single-trajectory datasets. All sequences are registered via Structure-from-Motion and camera poses verified using manual pixel-level correspondence annotations, yielding 50 high-quality scenes with 12000 images. Benchmarking recent NVS and camera pose estimation methods shows that NVS performance degrades with increasing viewpoint disparity, and that feed-forward pose estimators notably lag behind optimization-based approaches, highlighting MV2 as a rigorous testbed for NVS in driving. The dataset, benchmark protocol, and project resources are available at https://mv2-dataset.github.io/.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In: CVPR (2022)
work page 2022
-
[2]
arXiv preprint arXiv:2001.10773 (2020)
Cabon, Y., Murray, N., Humenberger, M.: Virtual kitti 2. arXiv preprint arXiv:2001.10773 (2020)
arXiv 2020
-
[3]
In: CVPR (2020)
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: A multimodal dataset for autonomous driving. In: CVPR (2020)
2020
-
[4]
Charatan,D.,Li,S.L.,Tagliasacchi,A.,Sitzmann,V.:pixelsplat:3dgaussiansplats from image pairs for scalable generalizable 3d reconstruction. In: CVPR (2024)
work page 2024
-
[5]
Chen, Y., Xu, H., Zheng, C., Zhuang, B., Pollefeys, M., Geiger, A., Cham, T.J., Cai, J.: Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. In: ECCV (2024)
work page 2024
-
[6]
arXiv preprint arXiv:2311.18561 (2023)
Chen, Y., Gu, C., Jiang, J., Zhu, X., Zhang, L.: Periodic vibration gaussian: Dynamic urban scene reconstruction and real-time rendering. arXiv preprint arXiv:2311.18561 (2023)
arXiv 2023
- [7]
-
[8]
Duisterhof, B.P., Zust, L., Weinzaepfel, P., Leroy, V., Cabon, Y., Revaud, J.: MASt3r-sfm: a fully-integrated solution for unconstrained structure-from-motion. In: 3DV (2025)
work page 2025
Show all 47 references
-
[9]
Edstedt, J., Nordström, D., Zhang, Y., Bökman, G., Astermark, J., Larsson, V., Heyden, A., Kahl, F., Wadenbäck, M., Felsberg, M.: Roma v2: Harder better faster denser feature matching (2025),https://arxiv.org/abs/2511.15706
2025 arXiv
-
[10]
In: NeurIPS (2024)
Fischer, T., Kulhanek, J., Bulo, S.R., Porzi, L., Pollefeys, M., Kontschieder, P.: Dynamic 3d gaussian fields for urban areas. In: NeurIPS (2024)
2024
-
[11]
In: CVPR (2024)
Fischer, T., Porzi, L., Bulo, S.R., Pollefeys, M., Kontschieder, P.: Multi-level neural scene graphs for dynamic urban environments. In: CVPR (2024)
2024
-
[12]
In: CVPR (2012)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: CVPR (2012)
2012
-
[13]
Cambridge university press (2003)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge university press (2003)
2003
-
[14]
In: NVIDIA Research Whitepapers arXiv:2508.10934 (2025)
Huang, J., Zhou, Q., Rabeti, H., Korovko, A., Ling, H., Ren, X., Shen, T., Gao, J., Slepichev, D., Lin, C.H., Ren, J., Xie, K., Biswas, J., Leal-Taixe, L., Fidler, S.: Vipe: Video pose engine for 3d geometric perception. In: NVIDIA Research Whitepapers arXiv:2508.10934 (2025)
2025 arXiv
-
[15]
arXiv preprint arXiv:2405.20323 (2024)
Huang, N., Wei, X., Zheng, W., An, P., Lu, M., Zhan, W., Tomizuka, M., Keutzer, K., Zhang, S.:S3Gaussian: Self-supervised street gaussians for autonomous driving. arXiv preprint arXiv:2405.20323 (2024)
2024 arXiv
-
[16]
arXiv preprint arXiv:2509.13414 (2025) MV2: Multi-Vehicle Driving Dataset 17
Keetha, N., Müller, N., Schönberger, J., Porzi, L., Zhang, Y., Fischer, T., Knapitsch, A., Zauss, D., Weber, E., Antunes, N., et al.: MapAnything: Universal feed-forward metric 3d reconstruction. arXiv preprint arXiv:2509.13414 (2025) MV2: Multi-Vehicle Driving Dataset 17
2025 arXiv
-
[17]
TOG (2023)
Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. TOG (2023)
2023
-
[18]
In: ICCV (2023)
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: ICCV (2023)
2023
-
[19]
In: arXiv preprint arXiv:2406.18360 (2024)
Li, H., Yuan, M., Zhang, Y., Wu, C., Zhao, C., Song, C., Feng, H., Ding, E., Zhang, D., Wang, J.: Xld: a cross-lane dataset for benchmarking novel driving view synthesis. In: arXiv preprint arXiv:2406.18360 (2024)
2024 arXiv
-
[20]
In: arXiv preprint arXiv:2503.12552 (2025)
Li, T., Qiu, Y., Wu, Z., Lindström, C., Su, P., Nießner, M., Li, H.: Mtgs: Multi- traversal gaussian splatting. In: arXiv preprint arXiv:2503.12552 (2025)
2025 arXiv
-
[21]
org/abs/2511.10647
Lin, H., Chen, S., Liew, J., Chen, D.Y., Li, Z., Shi, G., Feng, J., Kang, B.: Depth anything 3: Recovering the visual space from any views (2025),https://arxiv. org/abs/2511.10647
2025 arXiv
-
[22]
In: CVPR (2024)
Ling, L., Sheng, Y., Tu, Z., Zhao, W., Xin, C., Wan, K., Yu, L., Guo, Q., Yu, Z., Lu, Y., et al.: Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In: CVPR (2024)
2024
-
[23]
In: CVPR (2025)
Liu, Y., Fan, K., Yu, W., Li, C., Lu, H., Yuan, Y.: Monosplat: Generalizable 3d gaussian splatting from monocular depth foundation models. In: CVPR (2025)
2025
-
[24]
In: ECCV (2024)
Ljungbergh, W., Tonderski, A., Johnander, J., Caesar, H., Åström, K., Felsberg, M., Petersson, C.: Neuroncap: Photorealistic closed-loop safety testing for au- tonomous driving. In: ECCV (2024)
2024
-
[25]
In: AAAI (2025)
Ma, X., Zhang, J., Lu, P., Xu, S., Pan, C.: Novel view synthesis under large- deviation viewpoint for autonomous driving. In: AAAI (2025)
2025
-
[26]
Commu- nications of the ACM (2021)
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. Commu- nications of the ACM (2021)
2021
-
[27]
arXiv preprint arXiv:2502.15635 (2025)
Ni, Z., Du, S., Hou, Z., Wu, C., Yang, S.: Para-lane: Multi-lane dataset reg- istering parallel scans for benchmarking novel view synthesis. arXiv preprint arXiv:2502.15635 (2025)
2025 arXiv
-
[28]
In: 3DV (2025)
Ni, Z., Du, S., Hou, Z., Wu, C., Yang, S.: Para-lane: Multi-lane dataset registering parallel scans for benchmarking novel view synthesis. In: 3DV (2025)
2025
-
[29]
In: CVPR (2025)
Peng, C., Zhang, C., Wang, Y., Xu, C., Xie, Y., Zheng, W., Keutzer, K., Tomizuka, M., Zhan, W.: Desire-gs: 4d street gaussians for static-dynamic decomposition and surface reconstruction for urban driving scenes. In: CVPR (2025)
2025
-
[30]
In: CVPR (2016)
Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)
2016
-
[31]
In: CVPR (2020)
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al.: Scalability in perception for autonomous driving: Waymo open dataset. In: CVPR (2020)
2020
-
[32]
In: CVPR (2025)
Tang, J., Gao, Y., Yang, D., Yan, L., Yue, Y., Yang, Y.: Dronesplat: 3d gaussian splatting for robust 3d reconstruction from in-the-wild drone imagery. In: CVPR (2025)
2025
-
[33]
In: CVPR (2024)
Tonderski, A., Lindström, C., Hess, G., Ljungbergh, W., Svensson, L., Petersson, C.: Neurad: Neural rendering for autonomous driving. In: CVPR (2024)
2024
-
[34]
In: CVPR (2023)
Turki, H., Zhang, J.Y., Ferroni, F., Ramanan, D.: Suds: Scalable urban dynamic scenes. In: CVPR (2023)
2023
-
[35]
In: CVPR (2025)
Vuong, K., Ghosh, A., Ramanan, D., Narasimhan, S., Tulsiani, S.: Aerialmegadepth: Learning aerial-ground reconstruction and view synthesis. In: CVPR (2025)
2025
-
[36]
In: CVPR (2025) 18 S
Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: Vggt: Visual geometry grounded transformer. In: CVPR (2025) 18 S. B. Dharavath et al
2025
-
[37]
In: NeurIPS (2021)
Wilson, B., Qi, W., Agarwal, T., Lambert, J., Singh, J., Khandelwal, S., Pan, B., Kumar, R., Hartnett, A., Pontes, J.K., Ramanan, D., Carr, P., Hays, J.: Argoverse 2: Next generation datasets for self-driving perception and forecasting. In: NeurIPS (2021)
2021
-
[38]
In: CVPR (2025)
Wu, W., He, H., Zhang, C., He, J., Zhao, S.Z., Gong, R., Li, Q., Zhou, B.: Towards autonomous micromobility through scalable urban simulation. In: CVPR (2025)
2025
-
[39]
In: CVPR (2025)
Xie, Z., Liu, Z., Peng, Z., Wu, W., Zhou, B.: Vid2sim: Realistic and interactive simulation from video for urban navigation. In: CVPR (2025)
2025
-
[40]
In: CVPR (2024)
Xu, H., Peng, S., Wang, F., Blum, H., Barath, D., Geiger, A., Pollefeys, M.: Depth- splat: Connecting gaussian splatting and depth. In: CVPR (2024)
2024
-
[41]
In: CVPR (2022)
Xu, Q., Xu, Z., Philip, J., Bi, S., Shu, Z., Sunkavalli, K., Neumann, U.: Point-nerf: Point-based neural radiance fields. In: CVPR (2022)
2022
-
[42]
In: ECCV (2024)
Yan, Y., Lin, H., Zhou, C., Wang, W., Sun, H., Zhan, K., Lang, X., Zhou, X., Peng, S.: Street gaussians: Modeling dynamic urban scenes with gaussian splatting. In: ECCV (2024)
2024
-
[43]
In: ICLR (2025)
Yang, J., Huang, J., Ivanovic, B., Chen, Y., Wang, Y., Li, B., You, Y., Sharma, A., Igl, M., Karkus, P., Xu, D., Wang, Y., Pavone, M.: STORM: Spatio-temporal reconstruction model for large-scale outdoor scenes. In: ICLR (2025)
2025
-
[44]
In: ICLR (2024)
Yang, J., Ivanovic, B., Litany, O., Weng, X., Kim, S.W., Li, B., Che, T., Xu, D., Fidler, S., Pavone, M., Wang, Y.: EmerneRF: Emergent spatial-temporal scene decomposition via self-supervision. In: ICLR (2024)
2024
-
[45]
In: CVPR (2018)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018)
2018
-
[46]
TOG (2018)
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., Snavely, N.: Stereo magnification: Learn- ing view synthesis using multiplane images. TOG (2018)
2018
-
[47]
In: CVPR (2024)
Zhou, X., Lin, Z., Shan, X., Wang, Y., Sun, D., Yang, M.H.: Drivinggaussian: Composite gaussian splatting for surrounding dynamic autonomous driving scenes. In: CVPR (2024)
2024
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.