{"id":"8518d11d-517c-4411-be44-7f1dda2b84ac","arxiv_id":"2504.18746","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Dream-Box generates object-level pixel-space outliers with a diffusion model to train an object detector for out-of-distribution detection, enabling visualization but with lower accuracy than feature-space alternatives.","lead":"Dream-Box inserts synthetic, odd-looking objects into training images using a diffusion model and trains an object detector to flag such objects as out-of-distribution. It is the first pixel-space method for object-wise out-of-distribution detection and shows visible examples of the generated outliers, though its accuracy trails feature-space methods and the plain energy score.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'comparable performance' claim is not statistically supported: Dream-Box's point estimates are worse than the energy baseline on both FPR95 and AUROC, with no error bars or repeated runs reported.","rationale":"The reader's verdict is CONDITIONAL and flags exactly these concerns in the rationale (below-energy point estimates, σ tuned on eval, single runs). However, the reader's explicit weakest_assumption is about representativeness of the synthetic OOD objects. I see representativeness as a real secondary concern—Sec. 4 itself shows a dog that disappears and a bus that remains class-like—but it does not directly falsify the abstract's 'comparable performance' claim, which is about the reported benchmark numbers. The numbers themselves are the load-bearing element: the proposed method is numerically worse than the Energy baseline on every metric in Table 2. Without error bars, the reader cannot tell whether 'comparable' is a valid statistical claim or a euphemism for 'worse'. A multi-seed replication with confidence intervals is the single check that would settle this. I therefore keep the reader's CONDITIONAL verdict (UNCHANGED) and partially agree with the reader's weakest_assumption.","tokens_in":9018,"tokens_out":8453,"duration_ms":77479,"concrete_test":"Run the generic-prompt Dream-Box pipeline with 5 independent seeds, each with a newly sampled N=5,000 generated OOD set, and run the Energy-score baseline on the same base detector with the same training schedule. Compute mean and 95% bootstrap confidence intervals for FPR95 and AUROC on the fixed 930-image VOC→COCO test set. The 'comparable' claim is supported only if the Dream-Box intervals overlap the Energy intervals on both metrics; otherwise the abstract must be re-scoped to the visualization contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 2 shows Dream-Box (Generic) at FPR95 59.37 / AUROC 80.43, while the Energy-score baseline [20] reaches 56.89 / 83.69 on the same VOC→COCO protocol. Since Eq. (4) trains an OOD head specifically to separate generated outliers from inliers, and this head then performs worse than simply thresholding the raw energy (Eq. 3) of a detector trained with no OOD data, the 'comparable performance' claim in the abstract rests entirely on the point estimates being within noise of the best traditional method. The paper reports a single run per configuration (Sec. 3.3) with no error bars, and the distance-based σ is selected on the evaluation set (Fig. 4). If the FPR95/AUROC gap to Energy is statistically significant, the method is not 'comparable' but strictly worse, leaving only the visualization contribution intact. This is the weakest link because it targets the performance half of the central claim directly, without relying on assumptions about representativeness of the synthetic outliers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Dream-Box, a framework for object-wise out-of-distribution (OOD) detection in object detection. Dream-Box uses a Stable Diffusion inpainting model to replace in-distribution objects in PASCAL VOC images with synthetic outlier objects, using either hand-written generic prompts or CLIP-embedding perturbations, and then trains Faster R-CNN with an energy-based OOD head on the augmented dataset. The method is evaluated on the PASCAL VOC to MS-COCO protocol, reporting FPR95 of 59.37 and AUROC of 80.43 for the generic-prompt variant and 65.03 and 79.27 for the distance-based variant. The central claims are that Dream-Box achieves performance comparable to traditional OOD detection methods and is the first technique to provide concrete visualization of generated OOD objects for object detection.","tokens_in":9256,"tokens_out":3538,"duration_ms":37593,"significance":"If the claims are substantiated, Dream-Box is a useful first step toward interpretable pixel-space outlier generation for object-wise OOD detection. The explicit code release, the clear pipeline description, and the concrete visualizations of synthetic outliers are strengths that support reproducibility and further study. The paper also correctly identifies the interpretability limitation of feature-space synthesis methods such as VOS and FFS. However, the performance claim of 'comparable' results is currently not statistically supported by the reported numbers, and the selection of the key noise-scale hyperparameter on the evaluation set weakens the validity of the reported results. The contribution is incremental but real; the main risks are fixable with additional experiments and analysis.","major_comments":[{"comment":"The abstract's claim that Dream-Box 'achieves comparable performance to previous traditional methods' is not supported by the reported point estimates. In Table 2, the generic-prompt Dream-Box obtains FPR95 59.37 and AUROC 80.43, while the Energy score baseline obtains 56.89 and 83.69 on the same protocol; the distance-based variant is worse on both metrics. The paper reports no error bars or repeated runs (Sec. 3.3), so it is impossible to tell whether the gap to the Energy baseline is within noise. To support the 'comparable' claim, the authors should report mean and standard deviation over multiple training runs (at least 3-5) and, ideally, a paired significance test against the Energy baseline.","section":"Abstract / Table 2"},{"comment":"The noise scale sigma for the distance-based prompt strategy is selected by scanning values on the final OOD evaluation set: Sec. 3.3 says 'we try different standard deviation values' and Fig. 4 reports the resulting AUROC/FPR95 for sigma in {0.01,0.1,1.0,2.5,5.0}. Since sigma is chosen to optimize the evaluation metric on the VOC-to-COCO test data, the reported performance of the distance-based variant is optimistically biased. The authors should either fix sigma without using the test set, use a held-out validation set for selection, or present the results as a sensitivity analysis and clearly state that the tested configuration is selected on the evaluation set.","section":"Sec. 3.3 / Fig. 4"},{"comment":"The method's core assumption is that synthetic OOD objects produced by Stable Diffusion inpainting are representative enough of real OOD objects to train a transferable binary head. This assumption is not directly validated, and the paper itself shows counterexamples: the dog in the third row of Fig. 5 'completely disappears,' and the bus in the last row of Fig. 6 'looks still like the original class.' The authors should provide a quantitative analysis of generated-outlier quality, for example by measuring what fraction of generated objects are successfully detected as objects by the trained RPN, reporting the effect of removing failed generations, or comparing detection performance when training only on manually verified successful outliers. Without this, the interpretability and training-signal value of the generated outliers remains anecdotal.","section":"Sec. 3.1 / Sec. 4"}],"minor_comments":[{"comment":"The manuscript contains several typographical and grammatical errors, including 'semi-supevised mannner' (Sec. 3), 'architectire' (Sec. 3.2), 'outlier objecr generation' (Fig. 6 caption), 'probablity' (Sec. 2), 'detecteion' (Sec. 3.3), and 'image classification using, providing' (Sec. 1). A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The energy score is defined with g_k(v) in Eq. (3) but the surrounding text refers to g_k(x); the notation should be made consistent.","section":"Eq. (3)"},{"comment":"The text says the Stable Diffusion model is 'fine-tuned for 200k epochs,' which is almost certainly meant to be training iterations or steps. Please clarify the training schedule.","section":"Sec. 3.3"},{"comment":"Figure 4 would be more informative if it showed the full curves for both FPR95 and AUROC with axis labels, and ideally with error bars across repeated generations or model runs.","section":"Fig. 4"},{"comment":"Equation (1) describes sequential inpainting of all objects in an image, one at a time. This process can accumulate artifacts from earlier inpainted objects; an ablation comparing sequential inpainting to per-image single-object replacement would clarify whether the sequential formulation is beneficial or merely a design choice.","section":"Sec. 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the code release plus interpretable visualizations are genuine strengths. The main concerns are the unsupported 'comparable performance' claim and the evaluation-set-based selection of sigma; both are addressable through additional experiments and a more careful evaluation protocol. I do not see grounds for rejection, but the current evidence does not justify acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is real: this is the first object-wise OOD generation method that works in pixel space, letting you actually see the synthetic outliers that train the detector's OOD head. The pipeline is straightforward — Stable Diffusion inpainting conditioned on either generic prompts or perturbed CLIP embeddings, then an energy-based OOD head on Faster R-CNN — and the code is public. The visualizations in Figs. 5 and 6 are the main payoff, and they do give a concrete handle on failure modes (e.g., the dog that vanishes, the bus that still looks like a bus). The authors are also honest in Sec. 4 about these failures, so the paper is not overselling its own generation quality.\n\nThe soft spot is the central performance claim. Table 2 shows Dream-Box at 59.37 FPR95 / 80.43 AUROC (generic prompts) against 56.89 / 83.69 for the energy baseline on the same protocol. That is not 'comparable' in any statistical sense; it is worse on both metrics. The paper reports a single run, no error bars, and picks sigma by scanning values on the evaluation set (Fig. 4). So the abstract's phrasing overstates what the evidence supports. The authors even say in Sec. 3.3 that choosing sigma requires knowing how far OOD objects are, which they don't know a priori — exactly the kind of thing that needs held-out validation. The representativeness of the synthetic outliers is also assumed rather than tested, though that assumption is at least visible in the failure examples.\n\nThe mAP cost is small (48.2 vs 48.7 baseline), so the method does not wreck in-distribution detection. The comparison to VOS/FFS is fair; those feature-space methods clearly win on accuracy, and the paper admits it. The contribution is the interpretability, not the accuracy.\n\nWho is this for? Anyone working on OOD detection for object detectors, especially if they care about understanding what the model finds anomalous. As a methods paper it is incremental, but the visualization angle is a legitimate new capability. It deserves a serious referee, not a desk reject, but the referee should require error bars across seeds or generated OOD sets, and the 'comparable performance' claim should be re-scoped to something like 'slightly lower accuracy but with interpretable visualizations.'\n\nI would take the next revision, not this exact version.","headline":"A legitimate, clearly-described first step toward pixel-space OOD visualization for object detection, but the 'comparable performance' claim is not backed by the numbers, which are worse than the energy baseline with no error bars.","tokens_in":9747,"tokens_out":1380,"would_cite":false,"duration_ms":16226,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dream-Box is the first method to generate object-wise outliers in pixel space for OOD detection, matching traditional methods while making the training outliers visible.","keywords":["out-of-distribution detection","object detection","diffusion models","outlier synthesis","pixel-space outlier generation","energy-based OOD detection","Stable Diffusion inpainting","interpretability"],"falsifier":"Train the same Faster R-CNN energy head on real OOD examples (e.g., MS-COCO instances) instead of Stable Diffusion-generated ones, keeping all other settings fixed; if the real-example version clearly beats Dream-Box on FPR95/AUROC, the bottleneck is the representativeness of the generated fakes, which is the paper's load-bearing premise.","tokens_in":8847,"feed_emoji":"🎨","tokens_out":9552,"duration_ms":79415,"temperature":0.7,"pith_summary":"Dream-Box asks whether a diffusion model can produce object-level out-of-distribution (OOD) training examples directly in pixel space, and whether a detector trained on those fakes can match conventional OOD scores while also showing the anomalies. The paper reports that it can: using Stable Diffusion to inpaint fake objects into PASCAL VOC training images, and training Faster R-CNN with an energy-based OOD head, achieves FPR95 of 59.37 and AUROC of 80.43 with hand-written generic prompts on MS-COCO as OOD test data. This is comparable to traditional methods, though below feature-space synthesis such as VOS and FFS. The distinctive payoff is interpretability: for the first time in object-wise OOD detection, the synthetic outliers used for training can be looked at.","feed_headline":"Dream-Box paints fake objects that teach detectors what's out of place","feed_subtitle":"Stable Diffusion inpainting creates surreal objects; an energy head learns to flag them as out-of-distribution.","key_machinery":"The core mechanism is masked inpainting with a diffusion model: an image, a ground-truth box, and a prompt derived from the class name are fed to Stable Diffusion, which fills the box with a synthetic object and thereby constructs the OOD training set $D_{ood}$ via the sequential update $\\tilde{x}_i = f(\\tilde{x}_{i-1}, \\rho(c_i), m_i)$. This synthetic dataset is consumed by a Faster R-CNN whose classification-head logits define an energy score $E(v)$, which a small MLP $\\phi(E)$ uses to output an in-distribution/OOD probability trained by binary cross-entropy. The RPN sees the fakes as objects, but the classification and box heads never train on them, so the in-distribution task is preserved.","core_discovery":"The central claim is that replacing each in-distribution object in a training image with a synthetic object generated by Stable Diffusion inpainting, labeled as OOD, and training the object detector with an energy-based OOD head yields object-wise OOD detection performance comparable to traditional methods while providing the first concrete visualization of generated OOD objects. The generation uses two prompting strategies: twenty hand-written generic prompts describing impossible or unrealistic versions of the class, and noisy CLIP embeddings $\\rho(c)=\\zeta(c)+\\sigma\\epsilon$ with $\\sigma=2.5$ performing best. The detector's region proposal network treats the synthetic objects as objects, while the classification and bounding-box heads ignore them; an energy score $E(v)=-\\log\\sum_i \\exp(g_i(v))$ from the classification head's features feeds a small MLP trained with binary cross-entropy to separate in-distribution from OOD.","pith_inferences":["If the transfer from PASCAL VOC to MS-COCO holds, the same pipeline could be tested in domain gaps where real OOD data are rare (medical imaging, aerial or security imagery), where visibility of the generated outliers also serves as an audit trail.","The observed failure cases suggest generation quality, not just prompt distance, limits performance; a testable extension is to filter generated objects by human-rated 'objectness' and measure whether OOD detection improves.","Because prompts are per-class, the framework can be extended to open-set or one-class detection by treating each known class's generated outliers as an open-set signal, though the paper does not explore this."],"forward_implications":["Object-wise OOD detection can be trained without any real OOD labels or feature-space assumptions, using only a pretrained diffusion inpainting model and the in-distribution dataset.","Because the generated outliers are visible, practitioners can inspect exactly what kind of anomaly the detector learned to reject, such as a dog that disappears or a bus that still resembles the original class.","The distance-based prompt strategy gives a tunable knob ($\\sigma$) that controls how anomalous the synthetic objects are, allowing near-anomalies to be emphasized or de-emphasized.","Generic hand-written prompts outperform noise-perturbed CLIP embeddings in this object-detection setting, suggesting that class-aware, near-boundary anomalies are the more useful training signal."],"supporting_citations":[{"why":"Source of the pixel-space diffusion outlier generation idea and the distance-based prompt embedding perturbation.","marker":"[6]"},{"why":"Feature-space outlier synthesis baseline whose energy-based OOD head design is reused here.","marker":"[5]"},{"why":"State-of-the-art feature-space outlier synthesis baseline for comparison.","marker":"[15]"},{"why":"Defines the energy score over logits used to build the OOD classifier.","marker":"[20]"},{"why":"Provides the Stable Diffusion model used for masked inpainting.","marker":"[23]"},{"why":"The Faster R-CNN object detector that Dream-Box modifies.","marker":"[22]"},{"why":"LaMa strategy used to fine-tune Stable Diffusion for masked inpainting.","marker":"[25]"},{"why":"In-distribution dataset (PASCAL VOC) whose objects are replaced by synthetic outliers.","marker":"[7]"},{"why":"MS-COCO, the source of real OOD objects for evaluation.","marker":"[18]"}],"fun_headline_variants":["Dream-Box: synthetic objects make OOD detection visible","Diffusion-drawn outliers train detectors to spot the strange","First pixel-space method to generate OOD objects for detection","Dream-Box creates fake objects to teach detectors what's out-of-distribution","Object-wise synthetic outliers for OOD detection, now with visuals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that synthetic objects produced by Stable Diffusion inpainting, driven by class names and prompts, are representative enough of real out-of-distribution objects that a detector trained only on these fakes will flag genuine unexpected objects correctly.","fun_headline_variants_meta":{"raw":{"variants":["Dream-Box: synthetic objects make OOD detection visible","Diffusion-drawn outliers train detectors to spot the strange","First pixel-space method to generate OOD objects for detection","Dream-Box creates fake objects to teach detectors what's out-of-distribution","Object-wise synthetic outliers for OOD detection, now with visuals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000326,"raw_usage":{"total_tokens":1850,"prompt_tokens":994,"completion_tokens":856,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":770}},"tokens_in":610,"tokens_out":856,"duration_ms":7690,"temperature":1.0,"reasoning_tokens":770,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:10:30.860822+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same Faster R-CNN energy head on real OOD examples (e.g., MS-COCO instances) instead of Stable Diffusion-generated ones, keeping all other settings fixed; if the real-example version clearly beats Dream-Box on FPR95/AUROC, the bottleneck is the representativeness of the generated fakes, which is the paper's load-bearing premise.","supporting_citations":[{"cited_title":"Dream the impossible: Outlier imagination with dif- fusion models","cited_arxiv_id":null,"evidence_quote":"Source of the pixel-space diffusion outlier generation idea and the distance-based prompt embedding perturbation."},{"cited_title":"Normalizing flow based feature syn- thesis for outlier-aware object detection","cited_arxiv_id":null,"evidence_quote":"State-of-the-art feature-space outlier synthesis baseline for comparison."},{"cited_title":"Energy-based out-of-distribution detection","cited_arxiv_id":null,"evidence_quote":"Defines the energy score over logits used to build the OOD classifier."},{"cited_title":"High- resolution image synthesis with latent diffusion mod- els","cited_arxiv_id":null,"evidence_quote":"Provides the Stable Diffusion model used for masked inpainting."},{"cited_title":"Faster r-cnn: Towards real-time object detec- tion with region proposal networks","cited_arxiv_id":null,"evidence_quote":"The Faster R-CNN object detector that Dream-Box modifies."},{"cited_title":"Williams, John Winn, and Andrew Zisserman","cited_arxiv_id":null,"evidence_quote":"In-distribution dataset (PASCAL VOC) whose objects are replaced by synthetic outliers."},{"cited_title":"Microsoft coco: Com- mon objects in context","cited_arxiv_id":null,"evidence_quote":"MS-COCO, the source of real OOD objects for evaluation."}],"review_version":1}