REVIEW 3 major objections 6 minor 1 cited by
SD++: Enhancing Standard Definition Maps by Incorporating Road Knowledge using LLMs
T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read SD++ enriches standard-definition maps into lane-level maps using only road manuals and LLMs, with no sensor data or manual annotation.
desk verdict SD++ is a genuinely novel LLM-plus-RAG map enhancement pipeline with an honest limitations section, but its recall-only 5 m evaluation does not support the HD-map-utility headline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing component is the Knowledge-Based Algorithmic Generation pipeline, which deliberately avoids asking the LLM to invent geometry. A preprocessing step converts filtered OpenStreetMap XML into standardized JSON road segments; a retrieval-augmented query then retrieves relevant sections of the road manual, and the LLM extracts segment-level values such as lane width and shoulder width. An LLM-generated Python parser converts those values into JSON point sequences for roads, lanes, and bike lanes, and an algorithmic stage offsets the OSM centerlines by the extracted widths to produce lane geometry. The paper compares one-shot, iterative, and autoregressive-with-context variants of this extraction to show how generation strategy and prompt wording affect the result.
What would settle it
Collect road segments where independent survey data shows the OSM centerline deviates from true road center by more than 2 meters, run SD++ on those segments, and compute lane recall at the 5-meter Chamfer threshold used in the paper. If recall on such segments falls far below the reported 0.80, the claim that lane geometry survives as a usable prior whenever the SD input is imperfect is falsified; a complementary check is a curved ramp or intersection absent from OSM, which the method by its own admission cannot recover.
Extended reading notes
Core claim
On its own terms, the paper claims that a standard-definition map can be upgraded to an HD-like lane representation without a single sensor observation, by combining OpenStreetMap geometry with parameters extracted from highway design manuals by large language models. The authors report that their best algorithmic variant achieves an average Chamfer distance of about 2.5 meters and a lane recall of 0.80–0.81 on Argoverse 2, compared with 3.53 meters and 0.73 recall for a hand-crafted rule-based baseline; qualitative comparisons show more consistent lane widths than asking the LLM to generate maps directly. They also show the same recipe transferred to Japan by swapping the California highway design manual for the Japanese Road Law. The paper explicitly acknowledges that the accuracy of the original SD maps is not improved—errors in OSM centerlines propagate into the output—so the claim is about enriching existing maps with lane-level structure, not correcting their geometry.
Load-bearing premise
The load-bearing premise is that the OpenStreetMap centerline itself is accurate enough that offsetting it by manual-specified lane widths places lanes within the evaluation threshold; the paper admits in its Limitation section that because no sensor data is used, any error or missing feature in the SD map carries straight through to the enhanced output.
Editorial extensions
If this is right
- If the pipeline generalizes as claimed, lane-level priors can be produced for an entire city from public OSM data and one regional road manual, sidestepping LiDAR fleets and manual annotation.
- Region adaptation reduces to providing the right road manual: the same codebase produced plausible lanes for California and for Japan using an English translation of Japan's Road Law.
- Prompt engineering and model choice matter as much as architecture: GPT-4o with prompt P2 nearly doubled recall over P1 (0.80 versus 0.39) in the one-shot variant.
- An open-source Llama can reach comparable performance to GPT-4o if generation is iterative and autoregressive, making the pipeline deployable with local models.
- The method cannot manufacture information absent from OSM or the manual, so outputs inherit any gaps such as curved or unusual layouts.
Reading between the lines
- Going beyond the paper, this suggests that SD map providers could refresh lane priors cheaply whenever road manuals change, without waiting for an HD map vendor to re-survey the road.
- Going beyond the paper, because the geometry is only an offset of OSM centerlines, the same pipeline could be extended to refine those centerlines with sparse low-cost sensors at intersections and curves, where the current method cannot recover missing features.
- A testable next step outside the paper is to feed the generated lanes into trajectory prediction and planning benchmarks; if Chamfer accuracy near 2.5 meters does not translate into safer downstream behavior than an SD-only baseline, the practical value of the prior would need reassessment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SD++, a sensor-free pipeline that takes OpenStreetMap (OSM) standard-definition maps and, using retrieval-augmented generation over highway design manuals, extracts lane-level road information with LLMs and algorithmically constructs an enhanced map with lane and bike-lane geometry. Three algorithmic variants are compared (one-shot generation, iterative generation, and iterative generation with context) using GPT-4o and Llama, alongside a hand-crafted A/B Street baseline. Quantitative evaluation on a subset of Argoverse 2 reports Chamfer distance and recall at a hand-selected 5 m threshold, with no precision; results for Japan are qualitative. The paper concludes that SD++ provides a low-cost map prior approximating HD-map utility, while acknowledging in Section VI that the accuracy of SD maps themselves is not improved.
Significance. If the central claims were fully supported, SD++ would be a practically valuable contribution: it reuses publicly available road manuals and OSM data to generate lane-level structure without sensor data, releases code, and provides a systematic comparison of prompt variants, generation strategies, and model choices. The design choice to move the LLM from direct text-based map generation to structured knowledge extraction is sensible and well motivated. However, the quantitative evidence is incomplete in a load-bearing way: the evaluation measures only recall at a single generous threshold, some headline comparisons in the text are not supported by the reported table, and the stated limitation that SD-map accuracy is not improved directly constrains the abstract's 'approximate HD-map utility' claim.
major comments (3)
- [V-C.1 and Table I] The sole quantitative metric is recall at a 5 m Chamfer threshold, with precision explicitly omitted because 'SD++ include[s] larger area than Argoverse ground truth.' This justification is not sufficient: over-predicted lanes inside the evaluation area are unpenalized, and extra predicted lanes outside the area are also a form of false positive when the output is intended as a reusable prior. Since Section VI concedes that no sensor data is used and SD-map accuracy is not improved, a method that simply emits extra lanes offset by several meters from OSM centerlines could achieve the reported recall values. The authors should report precision and recall jointly over a well-defined evaluation region, give a threshold-sensitivity analysis around the hand-selected 5 m value, and state the number of road segments and ground-truth lanes used in the evaluation.
- [V-C.2 and Table I] The text claims that 'GPT-4o consistently outperforms Llama across various methods,' but Table I shows Llama IG and IG+Context achieving recall 0.81 versus 0.80 for GPT-4o IG, and comparable Chamfer-average values (2.50 m for Llama vs 2.54 m for GPT-4o). The reported numbers do not support a consistent ordering. The authors should either remove this claim or provide a statistical comparison with uncertainty intervals and a test of significance across multiple evaluation subsets.
- [Section VI] The limitation 'the accuracy of SD maps is not improved, as no sensor data is used' directly constrains the abstract's claim that SD++ produces 'enriched map representations that approximate the detail and utility of HD maps.' The paper should either demonstrate a downstream task where the enriched prior is useful despite unchanged centerline accuracy, or soften the claim to 'adds lane-level structure to SD maps.' In addition, the evaluation would be more informative if the error were decomposed into the OSM centerline-alignment error and the manual-based lane-offset error, since the latter is the actual new contribution of the pipeline.
minor comments (6)
- [Abstract] The abstract is internally inconsistent: the arXiv metadata version says results are shown for 'both California and Japan,' while the full-text abstract claims results from 'six states in the United States and Japan.' The experimental section provides quantitative results only for Argoverse 2 subsets (Palo Alto and Miami) and a qualitative Japan example; the six-state claim should either be supported by data or removed.
- [References] Reference [35] is mis-cited: it points to the Constitution of Japan rather than the Japan Road Law that is actually used in Section V-D.
- [Table I] The 'Chamfermin' column in Table I is undefined. Section V-C.1 describes average Chamfer distance and recall, but the minimum column needs a definition, for example whether it is the minimum over all lane matches or the per-road minimum.
- [V-D] The Japan evaluation is qualitative only. The paper should state explicitly that no quantitative evaluation was performed for Japan, or include at least a small quantitative comparison.
- [V-B] The direct-generation example in Fig. 4 is described as 'the best of 5 runs,' but this selection procedure is not mentioned in the figure caption or in the experimental protocol; this makes the qualitative comparison in Section V-B difficult to interpret.
- [VII] The heading 'Conclusion and Future Work' repeats future-work content that is already covered in Section VI ('Limitation and Future Work'); the sections should be consolidated or the headings clarified.
Circularity Check
No significant circularity: SD++ evaluates against external Argoverse 2 ground truth; lane geometry comes from external road manuals, not from fitted or self-cited values.
full rationale
SD++ does not fit any parameter to the Argoverse 2 ground truth. Lane widths and road attributes are extracted from an external Highway Design Manual via a RAG pipeline, and the output lanes are generated by algorithmic offsetting of OSM centerlines. The evaluation uses external Argoverse 2 data with a fixed 5-meter Chamfer threshold, and the paper explicitly disclaims improving SD-map accuracy in Section VI: 'the accuracy of SD maps is not improved, as no sensor data is used.' This is an honest scope limitation, not a circular reduction. The only self-citations ([1], [5], [9]) appear in related-work and background discussion and are not load-bearing for the central claim; no uniqueness theorem or ansatz is imported from prior author work to force the method. No equation or construction in the paper reduces a claimed prediction to a fitted value, to the evaluation target, or to a same-author result. The quantitative claim (recall against Argoverse 2) is therefore an externally falsifiable empirical test rather than a restatement of the inputs.
Assumptions & free parameters
free parameters (1)
- Chamfer correctness threshold =
5 meters
assumptions (4)
- domain assumption OSM road centerlines are accurate enough that lane offsets from manual widths fall within a few meters of true lane positions.
- domain assumption Road design manuals (HDM chapter 300, Japan Road Law) apply to the evaluated road segments.
- domain assumption LLM extraction and generated parsing code are correct and consistent across runs.
- domain assumption Chamfer distance below 5m to an Argoverse 2 lane is a valid definition of a correct lane.
Cite this review
Pith. "Pith review of SD++: Enhancing Standard Definition Maps by Incorporating Road Knowledge using LLMs." pith.science (2026). https://pith.science/paper/FOYS3PRO
@misc{pith2026250202773,
author = {Pith},
title = {Pith review of: SD++: Enhancing Standard Definition Maps by Incorporating Road Knowledge using LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/FOYS3PRO}},
note = {Machine review of arXiv:2502.02773}
}
read the original abstract
High-definition maps (HD maps) are detailed and informative maps capturing lane centerlines and road elements. Although very useful for autonomous driving, HD maps are costly to build and maintain. Furthermore, access to these high-quality maps is usually limited to the firms that build them. On the other hand, standard definition (SD) maps provide road centerlines with an accuracy of a few meters. In this paper, we explore the possibility of enhancing SD maps by incorporating information from road manuals using LLMs. We develop SD++, an end-to-end pipeline to enhance SD maps with location-dependent road information obtained from a road manual. We suggest and compare several ways of using LLMs for such a task. Furthermore, we show the generalization ability of SD++ by showing results from both California and Japan.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 1 Pith paper
-
Using Language and Road Manuals to Inform Map Reconstruction for Autonomous Driving
Adding OSM metadata and RAG-derived lane-width embeddings to SMERF produces modest metric improvements on two OpenLane-V2 intersection scenarios, with the best configuration beating the baseline on all four topology metrics.
Reference graph
Works this paper leans on
-
[1]
Jing-Yan Liao, Parth Doshi, Zihan Zhang, David Paz, and Henrik Christensen. OSM vs. HD maps: Map representations for trajectory prediction. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 9990–9996. IEEE, 2024
work page 2024
-
[2]
HDMapNet: An online hd map construction and evaluation framework
Qi Li, Yue Wang, Yilun Wang, and Hang Zhao. HDMapNet: An online hd map construction and evaluation framework. arXiv preprint arXiv:2107.06307, 2021
arXiv 2021
-
[3]
MapTR: Structured modeling and learning for online vectorized hd map construction
Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Chang Huang. MapTR: Structured modeling and learning for online vectorized hd map construction. In International Conference on Learning Representations , 2023
work page 2023
-
[4]
Graph-based topology reasoning for driving scenes
Tianyu Li, Li Chen, Huijie Wang, Yang Li, Jiazhi Yang, Xiangwei Geng, Shengyin Jiang, Yuting Wang, Hang Xu, Chunjing Xu, Junchi Yan, Ping Luo, and Hongyang Li. Graph-based topology reasoning for driving scenes. arXiv preprint arXiv:2304.05277 , 2023
arXiv 2023
-
[5]
SemVecNet: Generalizable Vector Map Generation for Arbitrary Sensor Configurations
Narayanan Elavathur Ranganatha, Hengyuan Zhang, Shashank Venka- tramani, Jing-Yan Liao, and Henrik I. Christensen. Semvecnet: Gen- eralizable vector map generation for arbitrary sensor configurations. arXiv preprint 2405.00250 , 2024
work page Pith review arXiv 2024
-
[6]
Planet dump retrieved from https://planet.osm.org , 2017
OpenStreetMap contributors. Planet dump retrieved from https://planet.osm.org , 2017
work page 2017
-
[7]
Mordechai Haklay. How good is volunteered geographical infor- mation? a comparative study of openstreetmap and ordnance sur- vey datasets. Environment and Planning B: Planning and Design , 37(4):682–703, 2010
work page 2010
-
[8]
Augmenting lane percep- tion and topology understanding with standard definition navigation maps
Katie Z Luo, Xinshuo Weng, Yan Wang, Shuang Wu, Jie Li, Kilian Q Weinberger, Yue Wang, and Marco Pavone. Augmenting lane percep- tion and topology understanding with standard definition navigation maps. arXiv preprint arXiv:2311.04079 , 2023
arXiv 2023
Show all 36 references
-
[9]
Christensen, and Liu Ren
Hengyuan Zhang, David Paz, Yuliang Guo, Arun Das, Xinyu Huang, Karsten Haug, Henrik I. Christensen, and Liu Ren. Enhancing online road network perception and reasoning with standard definition maps. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (I...
2024
-
[10]
Department of Transportation
California. Department of Transportation. California department of transportation (caltrans): Highway design manual: U.s. customary units: Seventh edition. https://rosap.ntl.bts.gov/view/ dot/15907, 2020
2020
-
[11]
Automatic construction of lane-level HD maps for urban scenes
Yiyang Zhou, Yuichi Takeda, Masayoshi Tomizuka, and Wei Zhan. Automatic construction of lane-level HD maps for urban scenes. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 6649–6656, Prague, Czech Republic, 27 September - 01 October 2021
2021
-
[12]
Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan- Yee. K. Wong, Zhenguo Li, and Hengshuang Zhao. DriveGPT4: Interpretable end-to-end autonomous driving via large language model. arXiv preprint 2310.01412 , 2024
2024 arXiv
-
[13]
Lmdrive: Closed-loop end- to-end driving with large language models
Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end- to-end driving with large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15120–15130, 2024
2024
-
[14]
Semantic anomaly detection with large language models
Amine Elhafsi, Rohan Sinha, Christopher Agia, Edward Schmerling, Issa Nesnas, and Marco Pavone. Semantic anomaly detection with large language models. ArXiv Preprint cs.RO 2305.11307 , 2023
2023 arXiv
-
[15]
HiLM-D: Towards high-resolution understanding in multimodal large language models for autonomous driving
Xinpeng Ding, Jianhua Han, Hang Xu, Wei Zhang, and Xiaomeng Li. HiLM-D: Towards high-resolution understanding in multimodal large language models for autonomous driving. arXiv preprint 2309.05186, 2023
2023 arXiv
-
[16]
Large language models powered context-aware motion prediction in autonomous driving
Xiaoji Zheng, Lixiu Wu, Zhijie Yan, Yuanrong Tang, Hao Zhao, Chen Zhong, Bokui Chen, and Jiangtao Gong. Large language models powered context-aware motion prediction in autonomous driving. arXiv preprint 2403.11057 , 2024
2024 arXiv
-
[17]
MTD- GPT: A multi-task decision-making gpt model for autonomous driving at unsignalized intersections
Jiaqi Liu, Peng Hang, Xiao i, Jianqiang Wang, and Jian Sun. MTD- GPT: A multi-task decision-making gpt model for autonomous driving at unsignalized intersections. arXiv preprint 2307.16118 , 2023
2023 arXiv
-
[18]
LC-LLM: Explainable lane-change intention and trajectory predictions with large language models
Mingxing Peng, Xusen Guo, Xianda Chen, Meixin Zhu, and Kehua Chen. LC-LLM: Explainable lane-change intention and trajectory predictions with large language models. arXiv preprint 2403.18344 , 2024
2024
-
[19]
PlanAgent: A multi-modal large language agent for closed-loop vehicle motion planning
Yupeng Zheng, Zebin Xing, Qichao Zhang, Bu Jin, Pengfei Li, Yuhang Zheng, Zhongpu Xia, Kun Zhan, Xianpeng Lang, Yaran Chen, and Dongbin Zhao. PlanAgent: A multi-modal large language agent for closed-loop vehicle motion planning. arXiv preprint 2406.01587 , 2024
2024 arXiv
-
[20]
DME-Driver: Integrating human decision logic and 3D scene percep- tion in autonomous driving
Wencheng Han, Dongqian Guo, Cheng-Zhong Xu, and Jianbing Shen. DME-Driver: Integrating human decision logic and 3D scene percep- tion in autonomous driving. arXiv preprint 2401.03641 , 2024
2024 arXiv
-
[21]
A language agent for autonomous driving
Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, and Yue Wang. A language agent for autonomous driving. arXiv preprint 2111.10813 , 2024
2024 arXiv
-
[22]
Singh, Siddharth Srivastava, Krishna Murthy Jatavallabhula, and K
Tushar Choudhary, Vikrant Dewangan, Shivam Chandhok, Shubham Priyadarshan, Anushka Jain, Arun K. Singh, Siddharth Srivastava, Krishna Murthy Jatavallabhula, and K. Madhava Krishna. Talk2BEV: Language-enhanced Bird’s-eye View Maps for Autonomous Driving. arXiv preprint 2310.022...
-
[23]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff...
2005 arXiv
-
[24]
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and Efficient Foundation Lang...
2023 arXiv
-
[25]
A/B-street
Michael Kirk Dustin Carlino, Yuwen Li. A/B-street. https:// github.com/a-b-street/abstreet , 2023
2023
-
[26]
Argoverse 2: Next generation datasets for self-driving perception and forecasting
Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Argoverse 2: Next generation datasets for self-driving perception and fo...
2021
-
[27]
Osmium: A c++ library for work- ing with openstreetmap data
Jochen Topf and Contributors. Osmium: A c++ library for work- ing with openstreetmap data. https://github.com/osmcode/ osmium-tool, 2025. Accessed: 2025-01-27
2025
-
[28]
OSMfilter
OpenStreetMap contributors. OSMfilter. https://github.com/ osmcode/osmium-filter, 2024
2024
-
[29]
LangChain
LangChain Contributors. LangChain. https://github.com/ langchain-ai/langchain, 2023
2023
-
[30]
Pypdfloader
LangChain Contributors. Pypdfloader. https://python. langchain.com/docs/integrations/document_ loaders/pypdfloader/, 2023
2023
-
[31]
H. G. Barrow, J. M. Tenenbaum, R. C. Bolles, and H. C. Wolf. Parametric correspondence and chamfer matching: two new techniques for image matching. In Proceedings of the 5th International Joint Con- ference on Artificial Intelligence - Volume 2, IJCAI’77, page 659–663, San Fra...
1977
-
[32]
Prompt programming for large language models: Beyond the few-shot paradigm, 2021
Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm, 2021
2021
-
[33]
Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh
Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models, 2021
2021
-
[34]
Google Earth Satellite Image
Google. Google Earth Satellite Image. https://earth.google. com/
-
[35]
Government of Japan. Act no. 180 of 1947 (constitution of japan). https://laws.e-gov.go.jp/law/345CO0000000320# Mp-At_2, 1947. Accessed: Feb 2025
1947
-
[36]
Caltrain current projects
California Department of Transportation (CA-DOT). Caltrain current projects. https://consmap.dot.ca.gov, 2025
2025
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.