REVIEW 3 major objections 5 minor 30 references
Dream-Box: Object-wise Outlier Generation for Out-of-Distribution Detection
T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Dream-Box is the first method to generate object-wise outliers in pixel space for OOD detection, matching traditional methods while making the training outliers visible.
desk verdict A legitimate, clearly-described first step toward pixel-space OOD visualization for object detection, but the 'comparable performance' claim is not backed by the numbers, which are worse than the energy baseline with no error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The core mechanism is masked inpainting with a diffusion model: an image, a ground-truth box, and a prompt derived from the class name are fed to Stable Diffusion, which fills the box with a synthetic object and thereby constructs the OOD training set $D_{ood}$ via the sequential update $\tilde{x}_i = f(\tilde{x}_{i-1}, \rho(c_i), m_i)$. This synthetic dataset is consumed by a Faster R-CNN whose classification-head logits define an energy score $E(v)$, which a small MLP $\phi(E)$ uses to output an in-distribution/OOD probability trained by binary cross-entropy. The RPN sees the fakes as objects, but the classification and box heads never train on them, so the in-distribution task is preserved.
What would settle it
Train the same Faster R-CNN energy head on real OOD examples (e.g., MS-COCO instances) instead of Stable Diffusion-generated ones, keeping all other settings fixed; if the real-example version clearly beats Dream-Box on FPR95/AUROC, the bottleneck is the representativeness of the generated fakes, which is the paper's load-bearing premise.
Extended reading notes
Core claim
The central claim is that replacing each in-distribution object in a training image with a synthetic object generated by Stable Diffusion inpainting, labeled as OOD, and training the object detector with an energy-based OOD head yields object-wise OOD detection performance comparable to traditional methods while providing the first concrete visualization of generated OOD objects. The generation uses two prompting strategies: twenty hand-written generic prompts describing impossible or unrealistic versions of the class, and noisy CLIP embeddings $\rho(c)=\zeta(c)+\sigma\epsilon$ with $\sigma=2.5$ performing best. The detector's region proposal network treats the synthetic objects as objects, while the classification and bounding-box heads ignore them; an energy score $E(v)=-\log\sum_i \exp(g_i(v))$ from the classification head's features feeds a small MLP trained with binary cross-entropy to separate in-distribution from OOD.
Load-bearing premise
The whole method rests on the assumption that synthetic objects produced by Stable Diffusion inpainting, driven by class names and prompts, are representative enough of real out-of-distribution objects that a detector trained only on these fakes will flag genuine unexpected objects correctly.
Editorial extensions
If this is right
- Object-wise OOD detection can be trained without any real OOD labels or feature-space assumptions, using only a pretrained diffusion inpainting model and the in-distribution dataset.
- Because the generated outliers are visible, practitioners can inspect exactly what kind of anomaly the detector learned to reject, such as a dog that disappears or a bus that still resembles the original class.
- The distance-based prompt strategy gives a tunable knob ($\sigma$) that controls how anomalous the synthetic objects are, allowing near-anomalies to be emphasized or de-emphasized.
- Generic hand-written prompts outperform noise-perturbed CLIP embeddings in this object-detection setting, suggesting that class-aware, near-boundary anomalies are the more useful training signal.
Reading between the lines
- If the transfer from PASCAL VOC to MS-COCO holds, the same pipeline could be tested in domain gaps where real OOD data are rare (medical imaging, aerial or security imagery), where visibility of the generated outliers also serves as an audit trail.
- The observed failure cases suggest generation quality, not just prompt distance, limits performance; a testable extension is to filter generated objects by human-rated 'objectness' and measure whether OOD detection improves.
- Because prompts are per-class, the framework can be extended to open-set or one-class detection by treating each known class's generated outliers as an open-set signal, though the paper does not explore this.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Dream-Box, a framework for object-wise out-of-distribution (OOD) detection in object detection. Dream-Box uses a Stable Diffusion inpainting model to replace in-distribution objects in PASCAL VOC images with synthetic outlier objects, using either hand-written generic prompts or CLIP-embedding perturbations, and then trains Faster R-CNN with an energy-based OOD head on the augmented dataset. The method is evaluated on the PASCAL VOC to MS-COCO protocol, reporting FPR95 of 59.37 and AUROC of 80.43 for the generic-prompt variant and 65.03 and 79.27 for the distance-based variant. The central claims are that Dream-Box achieves performance comparable to traditional OOD detection methods and is the first technique to provide concrete visualization of generated OOD objects for object detection.
Significance. If the claims are substantiated, Dream-Box is a useful first step toward interpretable pixel-space outlier generation for object-wise OOD detection. The explicit code release, the clear pipeline description, and the concrete visualizations of synthetic outliers are strengths that support reproducibility and further study. The paper also correctly identifies the interpretability limitation of feature-space synthesis methods such as VOS and FFS. However, the performance claim of 'comparable' results is currently not statistically supported by the reported numbers, and the selection of the key noise-scale hyperparameter on the evaluation set weakens the validity of the reported results. The contribution is incremental but real; the main risks are fixable with additional experiments and analysis.
major comments (3)
- [Abstract / Table 2] The abstract's claim that Dream-Box 'achieves comparable performance to previous traditional methods' is not supported by the reported point estimates. In Table 2, the generic-prompt Dream-Box obtains FPR95 59.37 and AUROC 80.43, while the Energy score baseline obtains 56.89 and 83.69 on the same protocol; the distance-based variant is worse on both metrics. The paper reports no error bars or repeated runs (Sec. 3.3), so it is impossible to tell whether the gap to the Energy baseline is within noise. To support the 'comparable' claim, the authors should report mean and standard deviation over multiple training runs (at least 3-5) and, ideally, a paired significance test against the Energy baseline.
- [Sec. 3.3 / Fig. 4] The noise scale sigma for the distance-based prompt strategy is selected by scanning values on the final OOD evaluation set: Sec. 3.3 says 'we try different standard deviation values' and Fig. 4 reports the resulting AUROC/FPR95 for sigma in {0.01,0.1,1.0,2.5,5.0}. Since sigma is chosen to optimize the evaluation metric on the VOC-to-COCO test data, the reported performance of the distance-based variant is optimistically biased. The authors should either fix sigma without using the test set, use a held-out validation set for selection, or present the results as a sensitivity analysis and clearly state that the tested configuration is selected on the evaluation set.
- [Sec. 3.1 / Sec. 4] The method's core assumption is that synthetic OOD objects produced by Stable Diffusion inpainting are representative enough of real OOD objects to train a transferable binary head. This assumption is not directly validated, and the paper itself shows counterexamples: the dog in the third row of Fig. 5 'completely disappears,' and the bus in the last row of Fig. 6 'looks still like the original class.' The authors should provide a quantitative analysis of generated-outlier quality, for example by measuring what fraction of generated objects are successfully detected as objects by the trained RPN, reporting the effect of removing failed generations, or comparing detection performance when training only on manually verified successful outliers. Without this, the interpretability and training-signal value of the generated outliers remains anecdotal.
minor comments (5)
- [Throughout] The manuscript contains several typographical and grammatical errors, including 'semi-supevised mannner' (Sec. 3), 'architectire' (Sec. 3.2), 'outlier objecr generation' (Fig. 6 caption), 'probablity' (Sec. 2), 'detecteion' (Sec. 3.3), and 'image classification using, providing' (Sec. 1). A careful proofreading pass is needed.
- [Eq. (3)] The energy score is defined with g_k(v) in Eq. (3) but the surrounding text refers to g_k(x); the notation should be made consistent.
- [Sec. 3.3] The text says the Stable Diffusion model is 'fine-tuned for 200k epochs,' which is almost certainly meant to be training iterations or steps. Please clarify the training schedule.
- [Fig. 4] Figure 4 would be more informative if it showed the full curves for both FPR95 and AUROC with axis labels, and ideally with error bars across repeated generations or model runs.
- [Sec. 3.1] Equation (1) describes sequential inpainting of all objects in an image, one at a time. This process can accumulate artifacts from earlier inpainted objects; an ablation comparing sequential inpainting to per-image single-object replacement would clarify whether the sequential formulation is beneficial or merely a design choice.
Circularity Check
No circularity: the synthetic-outlier generation and energy-based OOD head are evaluated against external data; reported tuning of σ is not a circular step.
full rationale
Dream-Box has no derivation whose conclusion is equivalent to its premises. The method consists of (i) generating synthetic OOD objects with Stable Diffusion inpainting (Eq. 1), (ii) two prompt strategies (generic prompts in Tab. 1, distance-based modified prompts in Eq. 2), and (iii) training an energy-based OOD head (Eqs. 3–4) whose inputs are those generated outliers. Nothing in Eqs. (1)–(4) presupposes the reported FPR95, AUROC, or mAP values. The energy score is imported from Liu et al. [20] with its standard definition, and the binary OOD head is trained with binary cross-entropy on synthetic outliers and then evaluated on MS-COCO, so the evaluation is external to the training objective. The self-citations ([14] and Isaac-Medina et al. in the references) are contextual prior work and are not load-bearing: the method explicitly builds on Dream-OOD [6], an external source. The choice of σ = 2.5 from Fig. 4 is evaluation-set tuning, which is a statistical/correctness concern rather than circularity, because it does not make the reported metric equal to an input by construction. Consequently, the central claim of providing pixel-space OOD visualization is independent of the method's own definitions, and no circular step can be exhibited.
Assumptions & free parameters
free parameters (4)
- sigma (prompt embedding noise std) =
2.5
- OOD loss weight =
10.0
- Minimum mask area for synthesis =
greater than 2000 pixels
- Number of generated OOD images N =
5000
assumptions (4)
- domain assumption The energy score built from class logits is a meaningful OOD score for object features.
- domain assumption Stable Diffusion inpainting can generate plausible object-like OOD images inside arbitrary masks.
- domain assumption Synthetic OOD objects are representative enough of true OOD instances such as MS-COCO objects for training to transfer.
- domain assumption Perturbing the CLIP text embedding with Gaussian noise moves the semantics far enough to produce OOD objects.
Cite this review
Pith. "Pith review of Dream-Box: Object-wise Outlier Generation for Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/ISL35NBQ
@misc{pith2026250418746,
author = {Pith},
title = {Pith review of: Dream-Box: Object-wise Outlier Generation for Out-of-Distribution Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/ISL35NBQ}},
note = {Machine review of arXiv:2504.18746}
}
read the original abstract
Deep neural networks have demonstrated great generalization capabilities for tasks whose training and test sets are drawn from the same distribution. Nevertheless, out-of-distribution (OOD) detection remains a challenging task that has received significant attention in recent years. Specifically, OOD detection refers to the detection of instances that do not belong to the training distribution, while still having good performance on the in-distribution task (e.g., classification or object detection). Recent work has focused on generating synthetic outliers and using them to train an outlier detector, generally achieving improved OOD detection than traditional OOD methods. In this regard, outliers can be generated either in feature or pixel space. Feature space driven methods have shown strong performance on both the classification and object detection tasks, at the expense that the visualization of training outliers remains unknown, making further analysis on OOD failure modes challenging. On the other hand, pixel space outlier generation techniques enabled by diffusion models have been used for image classification using, providing improved OOD detection performance and outlier visualization, although their adaption to the object detection task is as yet unexplored. We therefore introduce Dream-Box, a method that provides a link to object-wise outlier generation in the pixel space for OOD detection. Specifically, we use diffusion models to generate object-wise outliers that are used to train an object detector for an in-distribution task and OOD detection. Our method achieves comparable performance to previous traditional methods while being the first technique to provide concrete visualization of generated OOD objects.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Barker, Neelanjan Bhowmik, and Toby P
Jack W. Barker, Neelanjan Bhowmik, and Toby P. Breckon. Semi-supervised surface anomaly detection of composite wind turbine blades from drone imagery. In Proc. Int. Conf. on Computer Vision Theory and Ap- plications, pages 868–876. IEEE, 2022. 1
work page 2022
-
[2]
Mmdetection: Open mm- lab detection toolbox and benchmark
Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mm- lab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155, 2019. 5
arXiv 1906
-
[3]
Imagenet: A large-scale hierar- chical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierar- chical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248– 255, 2009. 5
work page 2009
-
[4]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1
arXiv 2010
-
[5]
V os: Learning what you don’t know by virtual outlier synthesis
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. V os: Learning what you don’t know by virtual outlier synthesis. arXiv preprint arXiv:2202.01197, 2022. 1, 2, 4, 5, 8
arXiv 2022
-
[6]
Dream the impossible: Outlier imagination with dif- fusion models
Xuefeng Du, Yiyou Sun, Jerry Zhu, and Yixuan Li. Dream the impossible: Outlier imagination with dif- fusion models. Advances in Neural Information Pro- cessing Systems, 36:60878–60901, 2023. 1, 2, 3, 4, 7, 8
work page 2023
-
[7]
Williams, John Winn, and Andrew Zisserman
Mark Everingham, Luc Gool, Christopher K. Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. Int. J. Comput. Vision, 88(2):303–338, 2010. 5
work page 2010
-
[8]
Gaus, Neelanjan Bhowmik, Brian K.S
Yona F.A. Gaus, Neelanjan Bhowmik, Brian K.S. Isaac-Medina, Amir Atapour-Abarghouei, Hubert P.H Shum, and Toby P. Breckon. Region-based appear- ance and flow characteristics for anomaly detection in infrared surveillance imagery. In Proc. Conf. Computer Vision and Pattern Recognition Workshops, pages 2995–3005. IEEE/CVF, 2023. 1
work page 2023
Show all 30 references
-
[9]
Gaus, Brian K.S
Yona F.A. Gaus, Brian K.S. Isaac-Medina, Neelanjan Bhowmik, Yee T. Lam, and Toby P. Breckon. Semi- supervised object-wise anomaly detection for firearm and firearm component detection in x-ray security im- agery. In Proc. Computer Vision Pattern Recognition Workshops. IEEE/CVF, 2025
2025
-
[10]
G ¨okstorp and Toby P
Simon G.E. G ¨okstorp and Toby P. Breckon. Tem- poral and non-temporal contextual saliency analysis for generalized wide-area search within unmanned aerial vehicle (uav) video. The Visual Computer, 38: 2033–2040, 2021. 1
2021
-
[11]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1, 4, 5
2016
-
[12]
A baseline for de- tecting misclassified and out-of-distribution examples in neural networks
Dan Hendrycks and Kevin Gimpel. A baseline for de- tecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136,
-
[13]
Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 10951– 10960, 2020. 5
2020
-
[14]
Towards open-world object-based anomaly detection via self- supervised outlier synthesis
Brian KS Isaac-Medina, Yona Falinie A Gaus, Nee- lanjan Bhowmik, and Toby P Breckon. Towards open-world object-based anomaly detection via self- supervised outlier synthesis. In European Conference on Computer Vision, pages 196–214. Springer, 2024. 1, 2, 4
2024
-
[15]
Normalizing flow based feature syn- thesis for outlier-aware object detection
Nishant Kumar, Sini ˇsa ˇSegvi´c, Abouzar Eslami, and Stefan Gumhold. Normalizing flow based feature syn- thesis for outlier-aware object detection. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5156–5165, 2023. 1, 2, 4, 5, 8
2023
-
[16]
A simple unified framework for detecting out- of-distribution samples and adversarial attacks
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. Ad- vances in neural information processing systems , 31,
-
[17]
Enhancing the reliability of out-of-distribution im- age detection in neural networks
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution im- age detection in neural networks. arXiv preprint arXiv:1706.02690, 2017. 2, 5
2017 arXiv
-
[18]
Microsoft coco: Com- mon objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceed...
2014
-
[19]
Focal loss for dense object de- tection
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll ´ar. Focal loss for dense object de- tection. In Proceedings of the IEEE international con- ference on computer vision , pages 2980–2988, 2017. 5
2017
-
[20]
Energy-based out-of-distribution detection
Weitang Liu, Xiaoyun Wang, John Owens, and Yix- uan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems , 33:21464–21475, 2020. 1, 2, 4, 5
2020
-
[21]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012–10022, 2021. 1
2021
-
[22]
Faster r-cnn: Towards real-time object detec- tion with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detec- tion with region proposal networks. IEEE transac- tions on pattern analysis and machine intelligence, 39 (6):1137–1149, 2016. 1, 4, 5
2016
-
[23]
High- resolution image synthesis with latent diffusion mod- els
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High- resolution image synthesis with latent diffusion mod- els. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695, 2022. 3, 5
2022
-
[24]
De- tecting out-of-distribution examples with gram matri- ces
Chandramouli Shama Sastry and Sageev Oore. De- tecting out-of-distribution examples with gram matri- ces. In International Conference on Machine Learn- ing, pages 8491–8501. PMLR, 2020. 1, 2, 5
2020
-
[25]
Resolution-robust large mask inpainting with fourier convolutions
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Ki- woong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF win...
-
[26]
Non-parametric outlier synthesis
Leitian Tao, Xuefeng Du, Jerry Zhu, and Yixuan Li. Non-parametric outlier synthesis. In The Eleventh In- ternational Conference on Learning Representations ,
-
[27]
Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detec- tors
Chien-Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detec- tors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7464– 7475, 2023. 1
2023
-
[28]
Model soups: averaging weights of multiple fine-tuned models im- proves accuracy without increasing inference time
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Mor- cos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models im- proves accuracy withou...
-
[29]
Generalized out-of-distribution detection: A sur- vey
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A sur- vey. International Journal of Computer Vision , 132 (12):5635–5662, 2024. 1
2024
-
[30]
Deformable detr: Deformable transformers for end-to-end object detection
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 1
2010 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.