REVIEW 4 major objections 5 minor 31 references
MapFM: Foundation Model-Driven HD Mapping with Multi-Task Contextual Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that swapping in a fine-tuned DINOv2 encoder and adding an auxiliary BEV road-surface segmentation head raises nuScenes vectorized HD map mAP from 66.3 to 69.0.
desk verdict Useful incremental result on MapQR with DINOv2 backbone and auxiliary BEV segmentation, but the reported gains are single-run margins that may not clear seed noise. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing components are the DINOv2 image encoder, used as a foundation backbone for surround-view camera features, and the Auxiliary Road Surface Segmentation Head (ARSS), a dense BEV head that predicts drivable-area and pedestrian-crossing masks from the BEV features and is supervised with Dice loss. Between them sits the BEVFormer BEV encoder, which projects multi-view DINOv2 features into a shared BEV representation, and the MapQR vector map decoder, which models map elements as ordered point sequences through a scatter-and-gather query mechanism. The ARSS head's role is to force the BEV features to encode road-surface context, which the paper argues makes the vectorized map prediction easier.
What would settle it
Train MapFM and the MapQR baseline multiple times with several random seeds under identical settings on nuScenes and compare the mAP distributions; if the 2.7-point best-case gap falls within one standard deviation of the runs, the central claim is not supported.
Extended reading notes
Core claim
The paper's discovery is that a large self-supervised vision transformer, fine-tuned on the driving task, transfers to online HD map prediction better than the task-specific backbones previously used, and that dense road-surface supervision makes the BEV features more useful for vectorized map elements. The authors build MapFM on MapQR's scatter-and-gather query decoder and a BEVFormer-style BEV encoder, swap in DINOv2, and add the Auxiliary Road Surface Segmentation Head (ARSS) that predicts drivable area and pedestrian crossing masks with Dice loss. On the nuScenes validation set the best configuration, MapFM with DINOv2-base trained for 16 epochs, reaches 69.0 mAP, surpassing MapQR's 66.3 mAP with ResNet50 and 66.1 mAP with SwinT; the DINOv2-small 24-epoch variant reaches 67.8 mAP. Adding the ARSS head to MapQR improves mAP by 0.4 points with ResNet50 and 1.0 with SwinT, and improves MapFM with DINOv2-small from 63.0 to 64.1 at 14 epochs. The paper also finds fine-tuning DINOv2's last layer markedly outperforms frozen DINOv2 or fine-tuned RADIOv2.5, and that last-layer features beat multi-layer aggregation strategies.
Load-bearing premise
The reported mAP differences are treated as real signal rather than seed-dependent noise, since all comparisons use single-run validation numbers without error bars or significance tests.
Editorial extensions
If this is right
- Under the same training recipe, a DINOv2-small backbone with the ARSS head beats the MapQR baseline with ResNet50 or SwinT by 1.5 to 1.7 mAP on nuScenes.
- DINOv2-base reaches 69.0 mAP in only 16 epochs, suggesting foundation backbones also reduce training time needed to exceed prior methods.
- The auxiliary road-surface segmentation head yields a consistent mAP gain of 0.4 to 1.0 points across ResNet50, SwinT, and DINOv2-small backbones, so the multi-task supervision transfers across architectures.
- Frozen foundation features are not enough for this task: fine-tuning DINOv2's last layer gives 65.2 mAP versus 52.4 mAP frozen, so task adaptation is required even with strong representations.
- For this pipeline, using only the last DINOv2 block (64.1 mAP) outperforms combining blocks 4, 8, and 12 by concatenation (63.9) or a multi-layer CNN (62.6), so feature aggregation from multiple layers is not beneficial here.
Reading between the lines
- If the single-run results hold across seeds, self-supervised vision transformers like DINOv2 could become the default camera backbone for BEV map construction, with dense auxiliary segmentation serving as a cheap task adapter.
- The same ARSS recipe could be tested on temporal or streaming map methods and on radar or LiDAR fusion pipelines, since the segmentation head only consumes BEV features and should be agnostic to sensor input.
- The paper compares backbones within a fixed decoder; a testable extension is whether the gain from DINOv2 persists under decoders that use different point representations, such as Bezier curves or pivot points.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes MapFM, an extension of the MapQR end-to-end vectorized HD map prediction framework. MapFM replaces the ResNet/Swin image backbone with a fine-tuned DINOv2 foundation model and adds an auxiliary road-surface segmentation head that predicts BEV drivable-area and pedestrian-crossing masks, trained with a Dice loss. The total loss combines MapTRv2/MapQR losses with the new surface loss. Experiments on nuScenes report that MapFM with DINOv2-small reaches 67.8 mAP at 24 epochs and MapFM with DINOv2-base reaches 69.0 mAP at 16 epochs, compared with 66.3 mAP for the MapQR ResNet50 baseline. Ablations address the auxiliary head, DINOv2 feature aggregation strategies, and frozen versus fine-tuned transfer learning.
Significance. The paper addresses a practical and timely problem, online vectorized HD mapping for autonomous driving. Replacing standard backbones with DINOv2 and adding a lightweight BEV segmentation head is a plausible, low-cost improvement that, if confirmed, would be useful to the field. The auxiliary-head ablation in Table 2 and Table 3 supports the direction of the claimed effect. The manuscript also makes its source code publicly available, which is a strength. The main weakness is that all benchmark claims rest on single-run mAP differences of 0.4 to 1.5 points with no repeated seeds, error bars, or significance testing, so the numerical headline is not yet statistically grounded.
major comments (4)
- [Section 4.1, Tables 1-5] The central claim of improvement over MapQR is based on single-run mAP values with no error bars or significance testing. For example, Table 1 reports MapFM DINOv2-small at 67.8 mAP versus MapQR ResNet50 at 66.3 mAP, a margin of 1.5 points, and the auxiliary-head ablation in Table 2 shows a 0.4-point gain for ResNet50. These margins are of the same order as typical run-to-run variation in DETR-style map prediction models, so the evidence cannot currently distinguish the proposed contributions from training noise. The authors should report mean and standard deviation over at least three seeds and, if appropriate, a paired significance test or confidence intervals for the key comparisons.
- [Tables 1, 3, and 5] The experiments do not provide a controlled comparison that isolates the effect of the DINOv2 backbone from the training budget. Table 3 gives MapFM without the auxiliary head (essentially MapQR with DINOv2-small) only at 14 epochs (63.0 mAP), while the MapQR baseline in Table 1 is at 24 epochs (66.3 mAP). Table 5 reports DINOv2 fine-tune transfer strategies at 14 epochs only. The improved headline numbers in Table 1 therefore confound the backbone change with a longer training schedule. The authors should include MapQR with DINOv2-small and DINOv2-base at identical epoch counts to the baseline, and MapFM without the auxiliary head at the same final epochs as the full MapFM.
- [Section 3.2 and Table 5] The main model configuration is not fully specified. The method text says DINOv2 is 'fine-tuned,' but Table 5 evaluates only 'Fine-tune DINOv2's Last Layer' at 14 epochs, which reaches 65.2 mAP, while Table 1 reports DINOv2-small at 24 epochs reaching 67.8 mAP. The authors should state explicitly which transfer-learning strategy, which set of DINOv2 layers, and which feature-aggregation choice are used in the headline configurations, and ideally ablate this choice at the final training budget rather than only at 14 epochs.
- [Table 4] The conclusion that using only the last DINOv2 block is the best feature-aggregation strategy is supported by a 0.2 mAP difference (64.1 vs. 63.9) from a single run at 12 epochs. This is insufficient evidence to prefer one aggregation method over another, especially since this choice is used in the main model. The feature-aggregation ablation should be repeated with at least three seeds or combined into the controlled comparison at the final training budget.
minor comments (5)
- [Section 3.3, Eq. (1)] The equation uses Q_BEV but the text defines Q_Enc as the learnable BEV queries; the notation should be made consistent.
- [Abstract and Section 4.2] The word 'significantly' is used to describe performance gains, but no statistical significance test is reported; please replace it with a quantitative statement or add significance testing.
- [Throughout] There are several typographical errors, including 'Backnone' in Section 3.1, 'milti-camera' in Section 3.1, and 'approache' in Section 2.1; a proofreading pass is needed.
- [Table 5] The label 'Fine-tune DINOv2's Last Layer' is ambiguous regarding whether this means only the final transformer block or the final linear layer; please clarify.
- [Figure 3] The figure caption says 'Green– drivable area mask, Blue– pedestrian crossing area mask' but an extra comma and spacing appear; also, the qualitative comparison in Figure 4 would benefit from zoomed-in views that make differences between MapQR and MapFM visible.
Circularity Check
No significant circularity: MapFM's gains are empirical comparisons against external baselines on nuScenes, with the auxiliary head ablated rather than assumed.
full rationale
The central claim is an empirical benchmark comparison against external baselines on nuScenes. MapFM replaces the ResNet/Swin encoder with fine-tuned DINOv2 and adds auxiliary BEV road-surface segmentation heads; the reported mAP is computed on the final vectorized map outputs via Chamfer-distance matching (Section 4.1), not on the auxiliary masks themselves. The contribution of the auxiliary head is tested through explicit ablations (Tables 2 and 3) rather than assumed or folded into the main result by construction. The DINOv2 backbone is externally pretrained, and its adaptation is studied with frozen vs. fine-tuned comparisons (Table 5); no parameter is fitted to the benchmark and then renamed as a prediction. Loss weights and architecture choices are stated training decisions (Section 3.6, Section 4.1), not hidden inputs. All cited building blocks (MapTRv2, MapQR, DINOv2) are prior external work attributed normally, and no uniqueness theorem or self-citation chain is used to force the design. The limitation noted by the authors, that task-specific fine-tuning is still needed, is an honest statement of scope. The concern raised in the reader's take about single-run mAP differences and missing significance tests is a statistical robustness issue, not circularity. Consequently, no circular step can be quoted or exhibited, and the paper receives a score of 0.
Assumptions & free parameters
free parameters (5)
- Loss weights beta_1 through beta_6 =
5, 2, 0.005, 1, 1, 2
- Number of instance queries N and points per instance n =
N=100, n=20
- BEV grid size and range =
200x100 over [-30,30] m front-rear and [-15,15] m left-right
- Training epochs per configuration =
24, 16, 14, or 12 depending on table
- DINOv2 feature aggregation choice =
Last layer only
assumptions (4)
- domain assumption nuScenes ground-truth vectorized maps and BEV masks are accurate and the standard Chamfer-distance mAP protocol is a valid measure of road-map quality.
- domain assumption Pre-trained DINOv2 features transfer to camera images from nuScenes and can be fine-tuned without architectural changes.
- domain assumption Gradients from auxiliary segmentation losses improve vectorized-map feature learning rather than causing harmful task interference.
- domain assumption The point, classification, direction, and segmentation losses inherited from MapTRv2 and MapQR are implemented as in the original papers.
Cite this review
Pith. "Pith review of MapFM: Foundation Model-Driven HD Mapping with Multi-Task Contextual Learning." pith.science (2026). https://pith.science/paper/B4ALRQXC
@misc{pith2026250615313,
author = {Pith},
title = {Pith review of: MapFM: Foundation Model-Driven HD Mapping with Multi-Task Contextual Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/B4ALRQXC}},
note = {Machine review of arXiv:2506.15313}
}
read the original abstract
In autonomous driving, high-definition (HD) maps and semantic maps in bird's-eye view (BEV) are essential for accurate localization, planning, and decision-making. This paper introduces an enhanced End-to-End model named MapFM for online vectorized HD map generation. We show significantly boost feature representation quality by incorporating powerful foundation model for encoding camera images. To further enrich the model's understanding of the environment and improve prediction quality, we integrate auxiliary prediction heads for semantic segmentation in the BEV representation. This multi-task learning approach provides richer contextual supervision, leading to a more comprehensive scene representation and ultimately resulting in higher accuracy and improved quality of the predicted vectorized HD maps. The source code is available at https://github.com/LIvanoff/MapFM.
Figures
Reference graph
Works this paper leans on
-
[1]
Bao, H., Dong, L., Piao, S., Wei, F.: Beit: Bert pre-training of image transformers (2022),https://arxiv.org/abs/2106.08254
arXiv 2022
-
[2]
Barın, M.R., Aydemir, G., Güney, F.: Robust bird’s eye view segmentation by adapting dinov2 (2024),https://arxiv.org/abs/2409.10228
arXiv 2024
-
[3]
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuscenes: A multimodal dataset for autonomous driving (2020)
work page 2020
-
[4]
In: European confer- ence on computer vision
Carion, e.a.: End-to-end object detection with transformers. In: European confer- ence on computer vision. pp. 213–229. Springer (2020)
work page 2020
-
[5]
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers (2021)
work page 2021
-
[6]
In: European Conference on Computer Vision
Chen, J., Wu, Y., Tan, J., Ma, H., Furukawa, Y.: Maptracker: Tracking with strided memory fusion for consistent vector hd mapping. In: European Conference on Computer Vision. pp. 90–107. Springer (2024)
work page 2024
-
[7]
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidi- rectional transformers for language understanding (2019)
2019
-
[8]
Ding, W., Qiao, L., Qiu, X., Zhang, C.: PivotNet: Vectorized pivot learning for end-to-end HD map construction. pp. 3672–3682 (2023)
work page 2023
Show all 31 references
-
[9]
In: 2024 IEEE International Conference on Robotics and Automation (ICRA)
Dong, H., Gu, W., Zhang, X., Xu, J., Ai, R., Lu, H., Kannala, J., Chen, X.: Superfusion: Multilevel lidar-camera fusion for long-range hd map generation. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 9056–9062. IEEE (2024)
2024
-
[10]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2016)
2016
-
[11]
arXiv preprint arXiv:2401.13172 (2024)
Hu, H., Wang, F., Wang, Y., Hu, L., Xu, J., Zhang, Z.: Admap: Anti- disturbance framework for reconstructing online vectorized hd map. arXiv preprint arXiv:2401.13172 (2024)
2024 arXiv
-
[12]
In: 2022 International Conference on Robotics and Automation (ICRA)
Li, Q., Wang, Y., Wang, Y., Zhao, H.: Hdmapnet: An online hd map construc- tion and evaluation framework. In: 2022 International Conference on Robotics and Automation (ICRA). pp. 4628–4634. IEEE (2022)
2022
-
[13]
IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., Dai, J.: Bevformer: learningbird’s-eye-viewrepresentationfromlidar-cameraviaspatiotemporaltrans- formers. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)
2024
-
[14]
Liao, B., Chen, S., Wang, X., Cheng, T., Zhang, Q., Liu, W., Huang, C.: MapTR: Structured modeling and learning for online vectorized HD map construction (2022)
2022
-
[15]
International Journal of Computer Vision pp
Liao, e.a.: Maptrv2: An end-to-end framework for online vectorized hd map con- struction. International Journal of Computer Vision pp. 1–23 (2024)
2024
-
[16]
In: International Conference on Machine Learning
Liu, e.a.: Vectormapnet: End-to-end vectorized hd map learning. In: International Conference on Machine Learning. pp. 22352–22369. PMLR (2023)
2023
-
[17]
Liu, X., Wang, S., Li, W., Yang, R., Chen, J., Zhu, J.: MGMap: Mask-guided learning for online vectorized HD map construction. pp. 14812–14821 (2024) 12 L. Ivanov et al
2024
-
[18]
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows (2021)
2021
-
[19]
In: European Conference on Computer Vision
Liu, Z., Zhang, X., Liu, G., Zhao, J., Xu, N.: Leveraging enhanced queries of point sets for vectorized map construction. In: European Conference on Computer Vision. pp. 461–477. Springer (2024)
2024
-
[20]
arXiv preprint arXiv:2304.07193 (2023)
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)
2023 arXiv
-
[21]
Qiao, L., Ding, W., Qiu, X., Zhang, C.: End-to-end vectorized hd-map construction with piecewise bezier curve (2023)
2023
-
[22]
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision (2021)
2021
-
[23]
In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Schramm, J., Vödisch, N., Petek, K., Kiran, B.R., Yogamani, S., Burgard, W., Valada, A.: Bevcar: Camera-radar fusion for bev map and object segmentation. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 1435–1442. IEEE (2024)
2024
-
[24]
In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Schramm, J., Vödisch, N., Petek, K., Kiran, B.R., Yogamani, S., Burgard, W., Valada, A.: Bevcar: Camera-radar fusion for bev map and object segmentation. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). p. 1435–1442. IEEE (Oct 2024)
2024
-
[25]
arXiv preprint arXiv:2409.10063 (2024)
Shi, A., Cai, Y., Chen, X., Pu, J., Fu, Z., Lu, H.: Globalmapnet: An online frame- work for vectorized global hd map construction. arXiv preprint arXiv:2409.10063 (2024)
2024 arXiv
-
[26]
Singh, M., Duval, Q., Alwala, K.V., Fan, H., Aggarwal, V., Adcock, A., Joulin, A., Dollár, P., Feichtenhofer, C., Girshick, R., et al.: The effectiveness of mae pre- pretraining for billion-scale pretraining (2023)
2023
-
[27]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Wang, S., Li, W., Liu, W., Liu, X., Zhu, J.: Lidar2map: In defense of lidar-based semantic map construction using online camera distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5186– 5195 (2023)
2023
-
[28]
Yang, J., Jiang, M., Yang, S., Tan, X., Li, Y., Ding, E., Wang, H., Wang, J.: Mgmapnet: Multi-granularity representation learning for end-to-end vectorized hd map construction (2024),https://arxiv.org/abs/2410.07733
2024 arXiv
-
[29]
In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
Yuan, T., Liu, Y., Wang, Y., Wang, Y., Zhao, H.: Streammapnet: Streaming map- ping network for vectorized online hd map construction. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 7356– 7365 (2024)
2024
-
[30]
In: European Conference on Computer Vision
Zhang, X., Liu, G., Liu, Z., Xu, N., Liu, Y., Zhao, J.: Enhancing vectorized map perception with historical rasterized maps. In: European Conference on Computer Vision. pp. 422–439. Springer (2024)
2024
-
[31]
Zhou,B.,Krähenbühl,P.:Cross-viewtransformersforreal-timemap-viewsemantic segmentation. pp. 13760–13769 (2022)
2022
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.