pith. sign in

arxiv: 2605.29935 · v1 · pith:MQCYB3HXnew · submitted 2026-05-28 · 💻 cs.CV · cs.AI

CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Driving

Zezhong Qian , Zhao Yang , Lu Tan , Zhihao Yan , Weiyi Hong , Haizhuang Liu , Yawei Jueluo This is my paper

Pith reviewed 2026-06-29 08:29 UTC · model grok-4.3

classification 💻 cs.CV cs.AI
keywords cross-city adaptationautonomous drivingdiffusion modelsHD mapssynthetic datazero-label adaptationperceptionplanning
0
0 comments X

The pith

A diffusion model generates synthetic city images from HD maps and visual prompts to adapt autonomous driving systems to new cities with no target labels.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Autonomous driving models degrade sharply when moved to new cities because of shifts in appearance, roads, and traffic. The paper introduces CityTransfer-Bench, a test set spanning perception, segmentation, and planning tasks across geographically separate cities, to measure this gap. CityGen addresses it by training a diffusion model that takes HD maps as structure and city-level visual prompts as style to produce synthetic training images for the target city. These images are then used to retrain the driving models without any real labeled data from the new location. If the generated images transfer, the method would remove the need to collect and annotate fresh data for every additional city.

Core claim

CityGen performs zero-label city adaptation via HD-map-conditioned synthesis guided by city-level visual prompts. The diffusion model produces synthetic images that, when added to training, raise performance on perception, segmentation, and planning tasks in the target city compared with models trained only on source data.

What carries the argument

The diffusion-based generative framework that produces city-style images from HD-map structures guided by city-level visual prompts.

If this is right

  • Models retrained on the synthetic images show higher robustness on the CityTransfer-Bench tasks.
  • The same generated data supports multiple downstream tasks without separate adaptation pipelines.
  • No city-specific annotations or task-specific designs are required for the adaptation step.
  • The approach scales to additional cities by supplying only their HD maps and a small set of visual prompts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the generated images preserve road topology accurately, the method could extend to cities whose maps exist but whose visual style has never been photographed.
  • Separating structure (HD map) from appearance (prompt) might allow the same generator to create training data for other map-based systems such as delivery robots.
  • The benchmark could be reused to compare any future label-free adaptation technique on the same geographically disjoint splits.

Load-bearing premise

Images created by the diffusion model from HD maps and city prompts will transfer to measurable gains on real-world perception, segmentation, and planning tasks in the target city without any labeled target data or task-specific fine-tuning.

What would settle it

Training the perception and planning models on the CityGen synthetic images and then testing them on real images from the target city yields no improvement or a drop relative to training only on the source city data.

Figures

Figures reproduced from arXiv: 2605.29935 by Haizhuang Liu, Lu Tan, Weiyi Hong, Yawei Jueluo, Zezhong Qian, Zhao Yang, Zhihao Yan.

Figure 1
Figure 1. Figure 1: CityGen pipeline and evaluation protocol. Previous methods train [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: CityGen framework. A vision–language model extracts scene captions [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of CityGen and existing generative methods under cross-city settings. CityGen produces visually diverse images with clear [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
read the original abstract

Autonomous driving systems are commonly trained and evaluated within limited geographic regions, which hinders their scalability when deployed in new cities. However, significant domain shifts in appearance, road topology, and traffic patterns often cause severe performance degradation under cross-city deployment. Existing approaches based on domain adaptation, data augmentation, or synthetic data generation typically rely on labeled target data, city-specific annotations, or task-specific designs, limiting their scalability and effectiveness for holistic evaluation. In this paper, we introduce CityTransfer-Bench, a geographically disjoint benchmark for evaluating cross-city generalization across perception, segmentation, and planning, and propose CityGen, a diffusion-based generative framework that performs zero-label city adaptation via HD-map-conditioned synthesis guided by city-level visual prompts. Extensive experiments demonstrate that CityGen consistently improves cross-city robustness across multiple tasks, establishing a scalable and label-efficient foundation for generalizable autonomous driving.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper introduces CityTransfer-Bench, a geographically disjoint benchmark for cross-city generalization in autonomous driving across perception, segmentation, and planning tasks. It proposes CityGen, a diffusion-based generative framework for zero-label city adaptation that synthesizes images conditioned on source HD maps and guided by city-level visual prompts, claiming that models trained on source data augmented with CityGen outputs show consistent robustness gains when evaluated on real target-city test sets without any target labels or task-specific fine-tuning.

Significance. If the transfer results hold with proper controls, the work would offer a scalable, label-efficient route to cross-city adaptation that avoids reliance on target annotations or per-task designs, addressing a practical barrier to deploying AV perception and planning systems in new geographic regions. The benchmark itself could standardize evaluation of geographic generalization.

major comments (2)
  1. [Abstract] Abstract: the central claim that CityGen 'consistently improves cross-city robustness across multiple tasks' is asserted without any quantitative metrics, baselines, error bars, ablation results, or description of how gains are measured on real target-city data; this absence makes it impossible to evaluate whether the reported improvements support the zero-label adaptation thesis.
  2. [Abstract] The transfer assumption (synthetic images from HD-map + visual-prompt conditioning yield measurable task gains on real target data) is load-bearing, yet the abstract supplies no controls for structure fidelity of the diffusion outputs, prompt leakage of non-zero-label information, or isolation of domain-alignment effects versus simple data-volume increases; without these, the empirical demonstration cannot be verified.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on the abstract. We agree that it should be strengthened with quantitative evidence and methodological clarifications, and we will revise it accordingly in the next version.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that CityGen 'consistently improves cross-city robustness across multiple tasks' is asserted without any quantitative metrics, baselines, error bars, ablation results, or description of how gains are measured on real target-city data; this absence makes it impossible to evaluate whether the reported improvements support the zero-label adaptation thesis.

    Authors: We agree that the abstract lacks supporting quantitative details. The manuscript body reports these metrics, baselines, error bars, ablations, and evaluation protocols on real target-city data in Sections 4 and 5. We will revise the abstract to summarize key quantitative results and the measurement approach. revision: yes

  2. Referee: [Abstract] The transfer assumption (synthetic images from HD-map + visual-prompt conditioning yield measurable task gains on real target data) is load-bearing, yet the abstract supplies no controls for structure fidelity of the diffusion outputs, prompt leakage of non-zero-label information, or isolation of domain-alignment effects versus simple data-volume increases; without these, the empirical demonstration cannot be verified.

    Authors: The manuscript provides these controls via dedicated experiments on structure fidelity (HD-map consistency and perceptual metrics), prompt design (city-level visual styles only, no task labels), and ablations isolating domain effects from data volume. We will add a concise reference to these controls in the revised abstract. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical claims rest on external validation

full rationale

The paper introduces CityGen as a diffusion-based synthesis framework and CityTransfer-Bench for evaluation. No equations, derivations, fitted parameters, or self-citations appear in the abstract or description that reduce any prediction to an input by construction. Claims of cross-city improvement are presented as results of experiments on real target-city test sets, not as tautological redefinitions or renamings. The central transfer claim is therefore falsifiable outside the paper's own fitted values and does not match any enumerated circularity pattern.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no equations, no listed hyperparameters, and no explicit modeling assumptions beyond the high-level description of the generative framework.

pith-pipeline@v0.9.1-grok · 5697 in / 1107 out tokens · 19816 ms · 2026-06-29T08:29:56.005829+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    Exploring object-centric temporal modeling for efficient multi-view 3d object detection,

    Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang, “Exploring object-centric temporal modeling for efficient multi-view 3d object detection,”arXiv preprint arXiv:2303.11926, 2023

    work page arXiv 2023

  2. [2]

    Planning- oriented autonomous driving,

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li, “Planning- oriented autonomous driving,” inProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2023

    2023

  3. [3]

    Cross-view transformers for real- time map-view semantic segmentation,

    Brady Zhou and Philipp Kr ¨ahenb¨uhl, “Cross-view transformers for real- time map-view semantic segmentation,” inCVPR, 2022

    2022

  4. [4]

    Gated domain adaptation,

    Zhe Gan, Wenxiao Li, Zhe Yang, Xin Lu, and Hwee Kuan Lee, “Gated domain adaptation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5979–5988

    2019

  5. [5]

    Cross-city transfer learning for driving in the wild,

    Felix Frank, Ilya Durdanovic, Benjamin Bell, Zhi Tsui, Michael O’Kelly, Jacob Susskind, Joshua Tenenbaum, and Stefan Schaal, “Cross-city transfer learning for driving in the wild,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6567–6576

    2018

  6. [6]

    Un- paired image-to-image translation using cycle-consistent adversarial networks,

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros, “Un- paired image-to-image translation using cycle-consistent adversarial networks,” inProceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232

    2017

  7. [7]

    Domain generalization with adversarial feature alignment,

    Yujun Li, Zhi Zhang, Yu Liu, Zhi Li, and X. Zhang, “Domain generalization with adversarial feature alignment,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4510–4519

    2020

  8. [8]

    Cross-domain autonomous driving visual segmentation based on enhanced target data learning,

    Chaoyu Rao, Xiaoyong Fang, Yunzhe Zhang, Wanshu Fan, and Dong- sheng Zhou, “Cross-domain autonomous driving visual segmentation based on enhanced target data learning,”ICT Express, vol. 11, no. 1, pp. 53–58, 2025

    2025

  9. [9]

    Sim2real diffusion: Leveraging foundation vision language models for adaptive automated driving,

    Chinmay Samak, Tanmay Samak, Bing Li, and Venkat Krovi, “Sim2real diffusion: Leveraging foundation vision language models for adaptive automated driving,”IEEE Robotics and Automation Letters, vol. 11, no. 1, pp. 177–184, Jan. 2026

    2026

  10. [10]

    Roca: Robust cross-domain end-to-end autonomous driving,

    Rajeev Yasarla, Shizhong Han, Hsin-Pai Cheng, Litian Liu, Shweta Mahajan, Apratim Bhattacharyya, Yunxiao Shi, Risheek Garrepalli, Hong Cai, and Fatih Porikli, “Roca: Robust cross-domain end-to-end autonomous driving,” 2025

    2025

  11. [11]

    Scalable Diffusion Models with Transformers

    William Peebles and Saining Xie, “Scalable diffusion models with transformers,”arXiv preprint arXiv:2212.09748, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  12. [12]

    Cosmos transfer1: World generation with adaptive multi- modal control,

    NVIDIA, “Cosmos transfer1: World generation with adaptive multi- modal control,” 2025

    2025

  13. [13]

    Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models,

    Xuanchi Ren, Yifan Lu, Tianshi Cao, Ruiyuan Gao, Shengyu Huang, Amirmojtaba Sabour, Tianchang Shen, Tobias Pfaff, Jay Zhangjie Wu, Runjian Chen, Seung Wook Kim, Jun Gao, Laura Leal-Taixe, Mike Chen, Sanja Fidler, and Huan Ling, “Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models,” 2025

    2025

  14. [14]

    nuscenes: A multimodal dataset for autonomous driving,

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” 2020

    2020

  15. [15]

    Scalability in perception for autonomous driving: Waymo open dataset,

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vi- jaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Sheng Zhao, Shuyang Cheng, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguel...

    2020

  16. [16]

    The cityscapes dataset for semantic urban scene understanding,

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele, “The cityscapes dataset for semantic urban scene understanding,” 2016

    2016

  17. [17]

    Cycada: Cycle- consistent adversarial domain adaptation,

    Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, and Trevor Darrell, “Cycada: Cycle- consistent adversarial domain adaptation,” 2017

    2017

  18. [18]

    Dacs: Domain adaptation via cross-domain mixed sampling,

    Wilhelm Tranheden, Viktor Olsson, Juliano Pinto, and Lennart Svensson, “Dacs: Domain adaptation via cross-domain mixed sampling,” 2020

    2020

  19. [19]

    Daformer: Improv- ing network architectures and training strategies for domain-adaptive semantic segmentation,

    Lukas Hoyer, Dengxin Dai, and Luc Van Gool, “Daformer: Improv- ing network architectures and training strategies for domain-adaptive semantic segmentation,” 2022

    2022

  20. [20]

    Hrda: Context-aware high-resolution domain-adaptive semantic segmentation,

    Lukas Hoyer, Dengxin Dai, and Luc Van Gool, “Hrda: Context-aware high-resolution domain-adaptive semantic segmentation,” 2022

    2022

  21. [21]

    Acdc: Autoregressive coherent multimodal generation using diffusion correction,

    Hyungjin Chung, Dohun Lee, and Jong Chul Ye, “Acdc: Autoregressive coherent multimodal generation using diffusion correction,” 2024

    2024

  22. [22]

    Weakly supervised training of universal visual concepts for multi-domain semantic segmentation,

    Petra Bevandi ´c, Marin Or ˇsi´c, Josip ˇSari´c, Ivan Grubi ˇsi´c, and Sini ˇsa ˇSegvi´c, “Weakly supervised training of universal visual concepts for multi-domain semantic segmentation,”International Journal of Computer Vision, vol. 132, no. 7, pp. 2450–2472, Jan. 2024

    2024

  23. [23]

    Shift: A synthetic driving dataset for continuous multi-task domain adaptation,

    Tao Sun, Mattia Segu, Janis Postels, Yuxuan Wang, Luc Van Gool, Bernt Schiele, Federico Tombari, and Fisher Yu, “Shift: A synthetic driving dataset for continuous multi-task domain adaptation,” in2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 21339–21350

    2022

  24. [24]

    arXiv preprint arXiv:2309.09777 , year=

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu, “Drivedreamer: Towards real-world-driven world models for autonomous driving,”arXiv preprint arXiv:2309.09777, 2023

    work page arXiv 2023

  25. [25]

    Panacea: Panoramic and controllable video generation for autonomous driving,

    Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang, “Panacea: Panoramic and controllable video generation for autonomous driving,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 6902–6912

    2024

  26. [26]

    MagicDrive-V2: High-resolution long video generation for autonomous driving with adaptive control,

    Ruiyuan Gao, Kai Chen, Bo Xiao, Lanqing Hong, Zhenguo Li, and Qiang Xu, “MagicDrive-V2: High-resolution long video generation for autonomous driving with adaptive control,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    2025

  27. [27]

    Epona: Autoregressive diffusion world model for autonomous driving,

    Kaiwen Zhang, Zhenyu Tang, Xiaotao Hu, Xingang Pan, Xiaoyang Guo, Yuan Liu, Jingwei Huang, Li Yuan, Qian Zhang, Xiao-Xiao Long, Xun Cao, and Wei Yin, “Epona: Autoregressive diffusion world model for autonomous driving,” 2025

    2025

  28. [28]

    Drivingdiffusion: Layout- guided multi-view driving scene video generation with latent diffusion model,

    Xiaofan Li, Yifu Zhang, and Xiaoqing Ye, “Drivingdiffusion: Layout- guided multi-view driving scene video generation with latent diffusion model,” 2023

    2023

  29. [29]

    Magicdrive3d: Controllable 3d generation for any-view rendering in street scenes,

    Ruiyuan Gao, Kai Chen, Zhihao Li, Lanqing Hong, Zhenguo Li, and Qiang Xu, “Magicdrive3d: Controllable 3d generation for any-view rendering in street scenes,” 2025

    2025

  30. [30]

    Drivegpt: Scaling autoregressive behavior models for driving,

    Xin Huang, Eric M. Wolff, Paul Vernaza, Tung Phan-Minh, Hongge Chen, David S. Hayden, Mark Edmonds, Brian Pierce, Xinxin Chen, Pratik Elias Jacob, Xiaobai Chen, Chingiz Tairbekov, Pratik Agarwal, Tianshi Gao, Yuning Chai, and Siddhartha Srinivasa, “Drivegpt: Scaling autoregressive behavior models for driving,” 2025

    2025

  31. [31]

    Adding conditional control to text-to-image diffusion models,

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala, “Adding conditional control to text-to-image diffusion models,” 2023

    2023

  32. [32]

    Synthetic datasets for autonomous driving: A survey,

    Zhihang Song, Zimin He, Xingyu Li, Qiming Ma, Ruibo Ming, Zhiqi Mao, Huaxin Pei, Lihui Peng, Jianming Hu, Danya Yao, and Yi Zhang, “Synthetic datasets for autonomous driving: A survey,”IEEE Transac- tions on Intelligent V ehicles, vol. 9, no. 1, pp. 1847–1864, Jan. 2024

    2024

  33. [33]

    Dualdiff+: Dual-branch diffusion for high-fidelity video generation with reward guidance,

    Zhao Yang, Zezhong Qian, Xiaofan Li, Weixiang Xu, Gongpeng Zhao, Ruohong Yu, Lingsi Zhu, and Longjun Liu, “Dualdiff+: Dual-branch diffusion for high-fidelity video generation with reward guidance,”arXiv preprint arXiv:2503.03689, 2025

    work page arXiv 2025