ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-27 16:51 UTCgrok-4.3pith:U44PKPFHrecord.jsonopen to challenge →
The pith
ExDet lets existing detectors handle novel categories and unseen domains by generating text-based visual prototypes and rectifying outputs at inference without retraining.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ExDet is a category-domain collaborative generalization framework for open-domain open-vocabulary detection consisting of Text-Guided Extrapolation (TGE) that exploits the DeltaSpace property of vision-language models to infer category- and domain-aware proxy visual prototypes from text, a Detector-Compatible Rectification (DCR) module learned from the TGE-generated prototypes in a detector training-free and real-data-free manner and inserted after the classification head at inference to rectify representations toward a detector-compatible source-domain visual distribution, and ExRPN that recalibrates proposal scores by combining semantic similarity with RPN confidence, thereby enhancing cla
What carries the argument
The ExDet framework built around TGE for text-to-visual prototype extrapolation, DCR for training-free representation rectification toward source-domain statistics, and ExRPN for semantic-RPN proposal recalibration.
If this is right
- Existing detectors gain improved classification accuracy for novel categories and domain-shifted objects.
- ExRPN raises recall for novel and domain-shifted objects while supplying better inputs to classification and DCR.
- The full pipeline reaches state-of-the-art on OD-LVIS, OV-LVIS, Objects365, and MSOSB.
- The DCR component adds no detector retraining or real-data requirements, keeping added cost low.
Where Pith is reading between the lines
- The same prototype-generation and rectification pattern could be tested on related tasks such as open-vocabulary segmentation or instance segmentation.
- Because DCR requires no real images, the method might support rapid adaptation in settings where collecting target-domain data is expensive or restricted.
- The reliance on DeltaSpace properties of vision-language models suggests checking whether similar extrapolation works when swapping in other pretrained multimodal models.
Load-bearing premise
The Detector-Compatible Rectification module, derived only from text-generated prototypes, can successfully shift representations of novel categories and unseen domains to match the source-domain visual distribution the detector expects.
What would settle it
A controlled test in which adding the DCR module after the classification head produces no accuracy gain or a drop on held-out novel-category and domain-shifted detection tasks compared with the unmodified base detector.
Figures
read the original abstract
Open-domain open-vocabulary detection (ODOVD) requires detectors to generalize to both novel categories and unseen domains, making it more challenging than open-vocabulary detection. Existing methods typically train open-vocabulary detectors together with domain generalization modules from scratch, leading to high training cost. we propose ExDet, a lightweight category-domain collaborative generalization framework for ODOVD that enhances the cross-category and cross-domain generalization of existing detectors. ExDet consists of Text-Guided Extrapolation (TGE), a lightweight Detector-Compatible Rectification (DCR) module, and ExRPN. Specifically, TGE exploits the DeltaSpace property of vision-language models (VLMs) to infer category- and domain-aware proxy visual prototypes from text. DCR is learned from the TGE-generated prototypes in a detector training-free and real-data-free manner, and is inserted after the classification head at inference to rectify representations toward a detector-compatible source-domain visual distribution, thereby enhancing classification for targets from novel categories and unseen domains. ExRPN recalibrates proposal scores by combining semantic similarity with RPN confidence, improving recall for novel and domain-shifted objects while providing better support for subsequent classification and DCR. ExDet achieves SOTA performance on OD-LVIS, OV-LVIS, Objects365, and MSOSB.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ExDet, a lightweight category-domain collaborative generalization framework for open-domain open-vocabulary detection (ODOVD). It consists of Text-Guided Extrapolation (TGE) that uses the DeltaSpace property of VLMs to infer category- and domain-aware proxy visual prototypes from text, a Detector-Compatible Rectification (DCR) module learned solely from these TGE prototypes in a detector training-free and real-data-free manner and inserted after the classification head at inference to align novel-category/unseen-domain features to the source visual distribution, and ExRPN that recalibrates proposal scores via semantic similarity and RPN confidence. The framework is claimed to enhance cross-category and cross-domain generalization of existing detectors and to achieve SOTA performance on OD-LVIS, OV-LVIS, Objects365, and MSOSB.
Significance. If the central claims hold, the work would be significant for reducing training costs in ODOVD by enabling plug-in generalization modules that operate without real data or detector retraining. The training-free, real-data-free nature of DCR, if validated, would represent a practical advance over methods that jointly train detectors and domain generalization components from scratch.
major comments (3)
- [Abstract] Abstract: the SOTA claims on OD-LVIS, OV-LVIS, Objects365, and MSOSB are asserted without any reported baselines, ablations, quantitative metrics, or error analysis, rendering it impossible to assess whether the data support the generalization improvements attributed to DCR and ExRPN.
- [Method (DCR module)] DCR description: the claim that DCR, trained exclusively on TGE-generated text-derived prototypes, successfully rectifies real detector outputs for novel categories and unseen domains rests on the unverified assumption that these synthetic prototypes span the necessary source-domain visual statistics (texture, lighting, etc.); this is load-bearing for both the SOTA numbers and the 'lightweight/training-free' framing and requires concrete validation such as feature distribution comparisons or controlled ablations.
- [Method (TGE and DCR)] TGE and DCR interaction: no derivation or analysis is supplied showing that the DeltaSpace extrapolation produces prototypes whose statistics are sufficient to learn a rectification mapping that generalizes beyond the synthetic manifold to actual detector feature distributions.
minor comments (2)
- [Method (TGE)] Clarify the precise mathematical definition of the DeltaSpace property and the extrapolation procedure used to generate prototypes.
- [Abstract] The abstract states that ExRPN 'provides better support for subsequent classification and DCR' but does not specify how the recalibrated scores interact with the rectification step.
Simulated Author's Rebuttal
We thank the referee for the constructive comments. We address each major point below and indicate planned revisions to strengthen the manuscript.
read point-by-point responses
-
Referee: [Abstract] Abstract: the SOTA claims on OD-LVIS, OV-LVIS, Objects365, and MSOSB are asserted without any reported baselines, ablations, quantitative metrics, or error analysis, rendering it impossible to assess whether the data support the generalization improvements attributed to DCR and ExRPN.
Authors: The detailed baselines, ablations, quantitative metrics, and error analysis appear in Sections 4 and 5. The abstract is a concise summary and does not contain these numbers. We will revise the abstract to include key mAP improvements and SOTA margins on the cited benchmarks. revision: yes
-
Referee: [Method (DCR module)] DCR description: the claim that DCR, trained exclusively on TGE-generated text-derived prototypes, successfully rectifies real detector outputs for novel categories and unseen domains rests on the unverified assumption that these synthetic prototypes span the necessary source-domain visual statistics (texture, lighting, etc.); this is load-bearing for both the SOTA numbers and the 'lightweight/training-free' framing and requires concrete validation such as feature distribution comparisons or controlled ablations.
Authors: End-to-end gains on real benchmarks support the practical utility of DCR, yet direct statistical validation (e.g., feature distribution overlap) between TGE prototypes and real source features is absent. We will add t-SNE visualizations and quantitative distribution metrics comparing TGE prototypes to real detector features in the revised version. revision: yes
-
Referee: [Method (TGE and DCR)] TGE and DCR interaction: no derivation or analysis is supplied showing that the DeltaSpace extrapolation produces prototypes whose statistics are sufficient to learn a rectification mapping that generalizes beyond the synthetic manifold to actual detector feature distributions.
Authors: Section 3.1 describes the DeltaSpace property and Section 3.2 explains DCR training on the resulting prototypes. A formal derivation of statistical sufficiency for cross-manifold generalization is not provided. We will add a short analysis subsection deriving the conditions under which TGE prototypes enable the observed rectification on real features. revision: yes
Circularity Check
No circularity in derivation chain
full rationale
The provided manuscript text presents ExDet as an empirical framework (TGE exploiting DeltaSpace of VLMs, DCR learned from generated prototypes in a detector-free manner, ExRPN for proposal recalibration) that achieves reported SOTA on listed benchmarks. No equations, derivations, or parameter-fitting steps are shown that reduce any claimed prediction or result to its own inputs by construction. No self-citations are invoked as load-bearing uniqueness theorems. The central claims rest on the design and empirical outcomes rather than any self-referential reduction, making the work self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Vision-language models possess a DeltaSpace property that permits inference of category- and domain-aware proxy visual prototypes from text.
Reference graph
Works this paper leans on
-
[1]
Hanoona Bangalath, Muhammad Maaz, Muhammad Uzair Khattak, Salman H Khan, and Fahad Shahbaz Khan. 2022. Bridging the gap between object and image-level representations for open-vocabulary detection.Advances in Neural Information Processing Systems35 (2022), 33781–33794
2022
-
[2]
Haibo Chen, Zhoujie Wang, Lei Zhao, Jun Li, and Jian Yang. 2025. Trtst: Arbitrary high-quality text-guided style transfer with transformers.IEEE Transactions on Image Processing(2025)
2025
-
[3]
Sheng Cheng, Tejas Gokhale, and Yezhou Yang. 2023. Adversarial Bayesian Augmentation for Single-Source Domain Generalization. InProceedings of the IEEE/CVF International Conference on Computer Vision. 11400–11410
2023
-
[4]
Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan
-
[5]
InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Yolo-world: Real-time open-vocabulary object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16901– 16911
-
[6]
Muhammad Sohail Danish, Muhammad Haris Khan, Muhammad Akhtar Munir, M Saquib Sarfraz, and Mohsen Ali. 2024. Improving single domain-generalized object detection: A focus on diversification and alignment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17732–17742
2024
-
[7]
Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. 2022. Learning to prompt for open-vocabulary object detection with vision-language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14084–14093
2022
-
[8]
Qi Fan, Mattia Segu, Yu-Wing Tai, Fisher Yu, Chi-Keung Tang, Bernt Schiele, and Dengxin Dai. 2023. Towards robust object detection invariant to real-world do- main shifts. InThe Eleventh International Conference on Learning Representations (ICLR 2023)
2023
-
[9]
Tejas Gokhale, Rushil Anirudh, Jayaraman J Thiagarajan, Bhavya Kailkhura, Chitta Baral, and Yezhou Yang. 2023. Improving diversity with adversarially learned transformations for domain generalization. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 434–443
2023
-
[10]
Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. 2021. Open-vocabulary object detection via vision and language knowledge distillation.arXiv preprint arXiv:2104.13921(2021)
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[11]
Agrim Gupta, Piotr Dollar, and Ross Girshick. 2019. Lvis: A dataset for large vocabulary instance segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5356–5364
2019
- [12]
- [13]
-
[14]
Joonhyun Jeong, Geondo Park, Jayeon Yoo, Hyungsik Jung, and Heesu Kim
-
[15]
InProceedings of the AAAI Conference on Artificial Intelligence, Vol
ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 2462–2470
-
[16]
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision- language representation learning with noisy text supervision. InInternational conference on machine learning. 4904–4916
2021
-
[17]
Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. 2021. Mdetr-modulated detection for end-to-end multi- modal understanding. InProceedings of the IEEE/CVF international conference on computer vision. 1780–1790
2021
-
[18]
Dahun Kim, Anelia Angelova, and Weicheng Kuo. 2023. Region-aware pretraining for open-vocabulary object detection with vision transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11144–11154
2023
-
[19]
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al
-
[20]
Visual genome: Connecting language and vision using crowdsourced dense image annotations.International journal of computer vision123 (2017), 32–73
2017
- [21]
-
[22]
Wooju Lee, Dasol Hong, Hyungtae Lim, and Hyun Myung. 2024. Object-aware domain generalization for object detection. Inproceedings of the AAAI conference on artificial intelligence, Vol. 38. 2947–2955
2024
-
[23]
Jiaming Li, Jiacheng Zhang, Jichang Li, Ge Li, Si Liu, Liang Lin, and Guanbin Li. 2024. Learning background prompts to discover implicit knowledge for open vocabulary object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16678–16687
2024
-
[24]
Liangqi Li, Jiaxu Miao, Dahu Shi, Wenming Tan, Ye Ren, Yi Yang, and Shiliang Pu
-
[25]
InProceedings of the IEEE/CVF International Conference on Computer Vision
Distilling detr with visual-linguistic knowledge for open-vocabulary object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision. 6501–6510
-
[26]
Chuang Lin, Zehuan Yuan, Sicheng Zhao, Peize Sun, Changhu Wang, and Jianfei Cai. 2021. Domain-invariant disentangled network for generalizable object detection. InProceedings of the IEEE/CVF international conference on computer vision. 8771–8780
2021
- [27]
-
[28]
Yajing Liu, Shijun Zhou, Xiyao Liu, Chunhui Hao, Baojie Fan, and Jiandong Tian. 2024. Unbiased faster r-cnn for single-source domain generalized object detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 28838–28847
2024
- [29]
-
[30]
Yueming Lyu, Kang Zhao, Bo Peng, Yue Jiang, Yingya Zhang, and Jing Dong
- [31]
- [32]
-
[33]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learning. 8748–8763
2021
-
[34]
Zhijie Rao, Jingcai Guo, Luyao Tang, Yue Huang, Xinghao Ding, and Song Guo
-
[35]
Srcd: Semantic reasoning with compound domains for single-domain generalized object detection.IEEE Transactions on Neural Networks and Learning Systems(2024)
2024
- [36]
-
[37]
Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. 2019. Objects365: A large-scale, high-quality dataset for ob- ject detection. InProceedings of the IEEE/CVF international conference on computer vision. 8430–8439
2019
-
[38]
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Con- ceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2556–2565
2018
-
[39]
Vidit Vidit, Martin Engilberge, and Mathieu Salzmann. 2023. Clip the gap: A single domain generalization approach for object detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3219–3229
2023
- [40]
-
[41]
Junjie Wang, Bin Chen, Bin Kang, Yulin Li, Weizhi Xian, Yichi Chen, and Yong Xu. 2025. Ov-dquo: Open-vocabulary detr with denoising text query training and open-world unknown objects supervision. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 7762–7770
2025
-
[42]
Junjie Wang, Bin Chen, Yulin Li, Bin Kang, Yichi Chen, and Zhuotao Tian. 2025. DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception. InProceed- ings of the Computer Vision and Pattern Recognition Conference. 14824–14834
2025
-
[43]
Jiacheng Wang, Ping Liu, Jingen Liu, and Wei Xu. 2023. Text-guided eyeglasses manipulation with spatial constraints.IEEE Transactions on Multimedia26 (2023), 4375–4388
2023
-
[44]
Luting Wang, Yi Liu, Penghui Du, Zihan Ding, Yue Liao, Qiaosong Qi, Biaolong Chen, and Si Liu. 2023. Object-aware distillation pyramid for open-vocabulary object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11186–11196
2023
-
[45]
Aming Wu and Cheng Deng. 2022. Single-domain generalized object detection in urban scene via cyclic-disentangled self-distillation. InProceedings of the IEEE/CVF Conference on computer vision and pattern recognition. 847–856
2022
-
[46]
Fan Wu, Jinling Gao, Lanqing Hong, Xinbing Wang, Chenghu Zhou, and Nanyang Ye. 2024. G-NAS: Generalizable Neural Architecture Search for Single Domain Generalization Object Detection. InProceedings of the AAAI Conference on Artifi- cial Intelligence, Vol. 38. 5958–5966
2024
-
[47]
Fei Wu, Yongheng Ma, Hao Jin, Xiao-Yuan Jing, and Guo-Ping Jiang. 2023. MFE- CLIP: CLIP with mapping-fusion embedding for text-guided image editing.IEEE Signal Processing Letters31 (2023), 116–120
2023
- [48]
-
[49]
Xiaoshi Wu, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7031–7040
2023
-
[50]
Anqi Xiao, Weichen Yu, and Hongyuan Yu. 2025. Sample-Aware RandAugment: Search-Free Automatic Data Augmentation for Effective Image Recognition: A. ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification MM ’26, November 10–14, 2026, Janeiro, Brazil Xiao et al.International Journal of Computer Vision133, 11 (2025), 7710–7725
2025
-
[51]
Mingjun Xu, Lingyun Qin, Weijie Chen, Shiliang Pu, and Lei Zhang. 2023. Multi- view adversarial discriminator: Mine the non-causal factors for object detection in unseen domains. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8103–8112
2023
-
[52]
Shilin Xu, Xiangtai Li, Size Wu, Wenwei Zhang, Yunhai Tong, and Chen Change Loy. 2024. DST-Det: Open-Vocabulary Object Detection via Dynamic Self- Training.IEEE Transactions on Circuits and Systems for Video Technology(2024)
2024
-
[53]
Xiaoran Xu, Jiangang Yang, Wenhui Shi, Siyuan Ding, Luqing Luo, and Jian Liu
-
[54]
InProceedings of the AAAI Conference on Artificial Intelligence
PhysAug: A Physical-guided and Frequency-based Data Augmentation for Single-Domain Generalized Object Detection. InProceedings of the AAAI Conference on Artificial Intelligence. 21815–21823
- [55]
-
[56]
Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. 2021. Open-vocabulary object detection using captions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14393–14402
2021
- [57]
- [58]
-
[59]
Yupeng Zhang, Ruize Han, Fangnan Zhou, Song Wang, Wei Feng, and Liang Wan. 2025. ODOV: Towards Open-Domain Open-Vocabulary Object Detection. Preprint submitted to arXiv on August 2, 2025; expected to be available shortly
2025
-
[60]
Yupeng Zhang, Shuqi Zheng, Ruize Han, Yuzhong Feng, Junhui Hou, Linqi Song, Wei Feng, and Liang Wan. [n. d.]. Rethinking the One-shot Object Detection: Cross-Domain Object Search. InACM Multimedia 2024
2024
-
[61]
Shiyu Zhao, Samuel Schulter, Long Zhao, Zhixing Zhang, Yumin Suh, Man- mohan Chandraker, Dimitris N Metaxas, et al. 2024. Taming self-training for open-vocabulary object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13938–13947
2024
-
[62]
Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liu- nian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. 2022. Regionclip: Region-based language-image pretraining. InProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. 16793–16803
2022
-
[63]
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. 2024. Mixstyle neural networks for domain generalization and adaptation.International Journal of Computer Vision132, 3 (2024), 822–836
2024
-
[64]
Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. 2022. Detecting twenty-thousand classes using image-level supervision. InEuropean Conference on Computer Vision. 350–368
2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.