{"id":"24797694-4739-403e-8e20-f7dd3f1dc45f","arxiv_id":"2504.16557","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Scrubbing sensitive objects with inpainting preserves more downstream detection and 3D reconstruction utility than deleting images, but the paper's privacy guarantee rests on the same oracle detector that defines and measures removal.","lead":"ROAR removes sensitive objects from images using segmentation and generative inpainting, then re-labels the altered scenes with an object detector. On COCO it keeps about 87.5% of baseline detection accuracy versus 74.2% for deleting whole images, and NeRF reconstruction stays within about 1.7 dB PSNR.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Privacy-utility comparison is not apples-to-apples: inpainting rows labeled Full Privacy remove only 59-89% of persons by the paper's own oracle (Tab. 1), while Drop removes 100%; the claimed 87.5%-vs-74.2% AP advantage may be bought by residual sensitive objects.","rationale":"The central claim is not just that inpainting produces plausible images; it is that object scrubbing is 'an effective privacy framework' with 'strong privacy guarantees' and minimal performance trade-offs (Abstract, Sec. 5). For that claim to hold, two things must be true: (i) scrubbing actually removes sensitive objects, and (ii) the utility comparison against dropping is measured at the same privacy level. The paper's own Table 1 shows (i) fails on its own oracle definition: FP.KD removes 79.82% of persons and fully clears only 64.77% of sensitive images; FP.KD.BD removes 89.22%. Drop is at 100%. So the headline AP advantage 0.420 vs 0.356 compares a dataset that still contains roughly 20% of persons with one that contains none. Residual persons can inflate AP directly and also provide contextual cues that help non-person classes. The reader identified the same detector-family circularity as the weakest assumption; I regard the unequal-privacy confound as the sharpest consequence. The proposed test, retraining on only images where an independent stronger detector finds zero persons, directly removes the confound: if ROAR still beats dropping there, the utility claim survives; if not, the paper's main contribution is unsupported as stated. This does not question the engineering or the honest reporting of PE/IE; it questions the inference from those numbers to 'strong privacy guarantees.' A conditional verdict with this retraining/independent-detector experiment as the condition is appropriate.","tokens_in":25282,"tokens_out":6322,"duration_ms":64365,"concrete_test":"Match privacy across methods before comparing utility: run an independent, stronger person detector (e.g., a different architecture such as YOLOv9, or a face/body re-identification embedder) on the FP.KD output; retain only images in which this independent detector finds zero persons (an approximately PE=100 subset), retrain RT-DETRv2-M on that subset, and report AP retention against a size-matched random subset of FP.drop. If the scrubbing-vs-dropping AP gap narrows by more than the claimed 13.3 points, or becomes negative, the conclusion in Sec. 4.3 fails; also report the residual person detection rate from the independent detector as a privacy bound.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Tab. 1 and the abstract's 87.5%-vs-74.2% claim rest on an unequal-privacy comparison. The rows labeled Full Privacy are not fully scrubbed even by the paper's own oracle-based metrics: FP.KD has PE=79.82% and IE=64.77%; FP.KD.BD has PE=89.22% and IE=77.73%; FP.SD has PE=59.19% and IE=38.16%; FP.drop has PE=100%. So the AP comparison 0.420 (FP.KD) vs 0.356 (FP.drop) compares a training set that still contains roughly 20% of the original persons (and about 35% of sensitive images still contain at least one person) to one with zero persons. Those residual instances are part of the processed dataset and can inflate detection AP, so the claimed utility advantage may be purchased by incomplete privacy. The circularity compounds this: PE/IE are computed with RT-DETRv2 (Eq. 17), the same detector family used in Stage 3 to decide whether a collided object is retained in annotations (Eqs. 13-16); any false negatives undercount residual persons in both the privacy metric and the re-annotation. Appendix B explicitly assumes the sensitive-object detector's false-positive rate is approximately zero and frames privacy only as non-detection, which is exactly the assumption the metrics inherit. Hence 'strong privacy guarantees' in the abstract and conclusion is not supported by the paper's own numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ROAR, a four-stage pipeline for privacy-preserving dataset obfuscation in 2D and 3D vision: Mask2Former detects sensitive objects (e.g., persons), a pre-trained generative inpainting model (Stable Diffusion, Kandinsky, AOT-GAN) removes them, an RT-DETRv2 oracle re-annotates collided objects, and utility is measured by training detectors or NeRF models on the processed data. The headline results are that on COCO-based detection ROAR retains 87.5% of baseline AP versus 74.2% for image dropping (Table 1), and that in NeRF reconstruction the method loses at most 1.66 dB PSNR while improving LPIPS (Table 2, Section 4.4). The paper also analyzes object-wise degradation patterns, showing that small objects attached to persons suffer the most, and provides qualitative comparisons against DeepPrivacy2 anonymization.","tokens_in":25618,"tokens_out":3895,"duration_ms":40273,"significance":"If the central claim held, the work would make a useful contribution: it systematically compares three inpainting families for object scrubbing, extends the evaluation to NeRF-based 3D reconstruction, and provides a reproducible pipeline with a public code repository and fixed seeds. The cluster-wise analysis of small-object degradation is a genuinely informative empirical finding, and the NeRF stitching-based inpainting strategy is a constructive step for view-consistent removal. However, the significance is currently limited by the fact that the headline privacy-utility comparison is not privacy-equalized and the 'strong privacy guarantees' claim rests on a self-referential oracle metric and an unverified false-positive assumption, so the paper is best viewed as an empirical exploration rather than a validated privacy framework.","major_comments":[{"comment":"The headline comparison of scrubbing versus dropping is not privacy-equalized and the privacy metric is self-referential. PE and IE in Eqs. (17)-(19) are computed with RT-DETRv2, the same detector family used in Stage 3 to decide whether a collided object is retained in the annotations (Eqs. (13)-(16)). Residual persons are therefore counted only when the oracle happens to detect them, and any false negatives undercount privacy leakage in both the metric and the re-annotation. More concretely, Table 1 shows FP.KD at PE=79.82% and IE=64.77%, FP.SD at PE=59.19% and IE=38.16%, while FP.drop achieves PE=IE=100%. Comparing AP 0.420 versus 0.356 under such unequal residual-person rates does not support the abstract's claim that scrubbing beats dropping; the utility advantage may be partly purchased by incomplete privacy. The authors should report privacy under a stronger or independent detector, provide an equalized comparison (e.g., drop plus inpainting of the removed images), and clearly separate 'residual privacy risk' from 'oracle-verified removal'.","section":"4.2.1, Eqs. (17)-(19); Table 1; Section 5"},{"comment":"The stated NeRF loss bound contradicts the reported numbers. The conclusion and abstract say scrubbing incurs a PSNR loss of at most 1.66 dB, but Table 2 shows a loss of 2.89 dB for the Room scene with the GAN method (32.70 to 29.81). Moreover, Ours/KD improves PSNR over the baseline by a noticeable margin in Fern (26.49 vs 25.17) and Flower (27.64 vs 27.40), which is surprising because removing a visible object should not improve fidelity of the original views; this suggests the baseline or the evaluation protocol needs clarification. The authors should either restrict the 'at most 1.66 dB' claim to the Kandinsky variant, report per-method bounds, or explain the mechanism behind the PSNR improvement.","section":"4.4, Table 2; Section 5 and Abstract"},{"comment":"The claimed 'strong privacy guarantees' are not supported by the evidence presented. Appendix B assumes the sensitive-object detector's false-positive rate is 'approximately zero' and frames privacy only as non-detection by a downstream person detector, but no attack evaluation, re-identification test, or formal privacy model is provided. The metrics in Eqs. (17)-(19) verify absence with the same oracle that is used for re-annotation, so the guarantee is conditional on that detector's blind spots. The authors should add a stronger-detector evaluation, a face/body re-identification test on the inpainted regions, or an explicit statement that the reported PE/IE are oracle-relative measures rather than formal privacy guarantees.","section":"Appendix B; Section 5"},{"comment":"All detection results appear to come from a single random seed (Appendix E, seed 3407), and Table 1 reports no variance or error bars. Because several AP differences are small relative to typical COCO training noise (e.g., FP.SD 0.441 vs FP.AOT 0.424, or SP.KD 0.466 vs SP.SD 0.465), the conclusions that 'Kandinsky achieves the strongest privacy with superior detection accuracy' and that specific methods are optimal are not statistically grounded. At minimum, the authors should run multiple seeds for the main comparison and report mean and standard deviation, or temper the ranking claims accordingly.","section":"4.3 and Appendix E"},{"comment":"The IoU verification threshold tau is selected as 0.3 based on 'empirical observations' without a sensitivity analysis. Because tau directly controls which collided objects are reinstated in the final annotations (Eq. (15)), the reported AP values and privacy outcomes may depend on this choice. The authors should report results for a range of tau values (e.g., tau in {0.1, 0.3, 0.5}) and justify that the headline scrubbing-vs-dropping conclusion is robust to the threshold.","section":"Appendix B, 'Selection of tau and zeta'"}],"minor_comments":[{"comment":"The conclusion states that image dropping reduces AP to 74.5%, but Table 1 and the abstract report 74.2%; this numeric discrepancy should be corrected.","section":"Section 5"},{"comment":"The notation P(R^4 x C x [0,1]) is used for the oracle's output space without defining P as the power set; please add a definition or use a standard notation such as 2^(R^4 x C x [0,1]).","section":"Eq. (9)"},{"comment":"The claim that 'our method is the first to demonstrate broad applicability across both 2D object detection and 3D NeRF reconstruction' is stronger than the related-work discussion supports; it would be safer to say 'to the best of our knowledge' and cite the closest concurrent or prior NeRF-privacy works.","section":"Section 1"},{"comment":"The BD variants are described as expanding the scrubbing area, and Section 4.3 says 'by 10px', but Table 1 does not state the expansion amount; please specify the boundary dilation in the caption or table notes.","section":"Table 1 caption and Section 4.3 item 3"},{"comment":"The LPIPS metric is said to use VGG16, but the exact pre-trained feature network and normalization are not specified; please cite the precise VGG configuration to ensure reproducibility.","section":"Appendix C"},{"comment":"The x-axis of Figure 4 is labeled with 'X' in the text version, and the caption is minimal; please clarify what the baseline bar represents and whether the bars are mean values or single runs.","section":"Figure 4 captions and axis labels"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a privacy/vision venue and contains a useful empirical study of inpainting-based object scrubbing. However, the central contribution as stated—that scrubbing provides 'strong privacy guarantees' and clearly outperforms dropping—requires a reworked comparison at equal residual privacy, an independent privacy/attack evaluation, and a correction of the NeRF PSNR claims. These are fixable with additional experiments and a careful rewriting of the claims, so I recommend major revision rather than rejection. I would also ask the editor to verify that the stated GitHub repository contains the full pipeline, since reproducibility is one of the paper's selling points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know two things about this paper. First, it is a genuinely useful empirical benchmark: it runs several inpainting backends (Kandinsky, Stable Diffusion, AOT-GAN) over COCO and three NeRF scenes, and reports detection AP and reconstruction metrics, plus a per-class analysis of what scrubbing does to small versus large objects. That dataset does not exist elsewhere. Second, the paper's central comparison is not as clean as the abstract implies. The '87.5% vs 74.2%' AP claim compares scrubbing that removes only 59–89% of persons (by the paper's own oracle) to dropping that removes 100%. So the utility advantage may partly be bought by residual sensitive content. The privacy metric is also self-referential: PE/IE are computed with RT-DETRv2, the same detector family used to re-annotate the scrubbed images in Stage 3, and Appendix B simply assumes false positives are negligible. With no attack evaluation, 'strong privacy guarantees' is not supported.\n\nWhat is new: the systematic comparison itself, the per-class clusters (handbags/backpacks near 0 AP, buses/airplanes near baseline), and the NeRF results with inpainting-based view stitching. The NeRF numbers look plausible and the 1.66 dB PSNR loss claim holds for Kandinsky. The method is a pipeline of existing components, and the authors do not oversell that; the contribution is the evaluation and the framing.\n\nWhere it is soft: no error bars, one seed, tau=0.3 tuned on the same data, and the privacy metric conflates 'not detected by our oracle' with 'removed.' The circularity is real but not fatal—if the goal is dataset utility and the oracle is a reasonable proxy for a downstream detector, the PE numbers are still informative, as long as they are not called a guarantee. The deep issue is the unequal-privacy comparison in Table 1. If the authors equalize by reporting utility at fixed PE, or add a re-identification attack, the paper would be much stronger.\n\nThe paper deserves peer review. It is a solid empirical contribution with a fixable framing problem. I would send it out. For a reading group, it is a good example of how privacy metrics can be circular, and the cluster analysis is worth discussing.","headline":"Useful empirical benchmark for object scrubbing, but the headline privacy-utility comparison compares unequal privacy levels and the privacy metric is self-referential; worth reviewing with major revisions.","tokens_in":26157,"tokens_out":2090,"would_cite":true,"duration_ms":20612,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ROAR removes sensitive objects from a dataset by generative inpainting and preserves 87.5% of detection accuracy, versus 74.2% when images are dropped.","keywords":["privacy-preserving vision","object scrubbing","generative inpainting","dataset obfuscation","object detection","neural radiance fields","instance segmentation","privacy-utility trade-off"],"falsifier":"Run a person re-identification or face-detection attack on the scrubbed COCO images and check whether crops of the inpainted regions can be matched to the original individuals above chance; if non-trivial matching succeeds even in images where the oracle reports PE=100%, the privacy claim collapses.","tokens_in":25097,"feed_emoji":"🖼️","tokens_out":6451,"duration_ms":60011,"temperature":0.7,"pith_summary":"ROAR, a privacy-preserving pipeline introduced in this paper, removes sensitive objects—chiefly people—from images with generative inpainting and then re-annotates the cleaned images, instead of blurring, pixelating, or replacing people. The paper's central claim is that object scrubbing preserves dataset utility far better than image dropping: on COCO detection, models trained on scrubbed data retain 87.5% of baseline average precision, while models trained after dropping sensitive images retain only 74.2%. In NeRF-based 3D reconstruction, scrubbing costs at most 1.66 dB PSNR while holding SSIM and improving LPIPS, so the approach transfers from 2D datasets to 3D scenes. A generous reading is that deletion-based privacy is wasteful when the underlying scenes still contain useful context, and that scrubbing can recover most of that context safely.","feed_headline":"Object scrubbing keeps 87.5% of detection accuracy","feed_subtitle":"ROAR deletes people from images with inpainting, losing less model performance than deleting whole images.","key_machinery":"The load-bearing mechanism is the obfuscation operator $O(I,M) = I\\odot(1-M) + G(I,M,z)\\odot M$, which applies a pretrained inpainting model only inside the segmentation masks while leaving unmasked content intact. Around this sits the ROAR pipeline: Mask2Former for sensitive-object masks, diffusion-based or GAN-based inpainting for the masked regions, and RT-DETRv2 as an oracle that re-annotates only the objects whose bounding boxes collide with the removed areas, using thresholds $\\zeta$ and $\\tau$. This re-annotation step is what converts a scrubbed image into a usable training example: collided boxes are re-checked, unaffected boxes are retained. For NeRF, a stitching-based inpainting strategy with $\\alpha$ blending, histogram matching, and Gaussian boundary smoothing maintains cross-view consistency.","core_discovery":"The paper establishes that generative object removal is a viable privacy mechanism for vision datasets. ROAR takes COCO images, detects persons with Mask2Former, inpaints the masked regions with Stable Diffusion, Kandinsky, or AOT-GAN, and uses RT-DETRv2 as an oracle to re-annotate objects that overlap the removed regions. Trained on the scrubbed COCO data, RT-DETRv2 and YOLOv9 retain 0.420 AP (87.5% of baseline) versus 0.356 AP (74.2%) for image dropping; Kandinsky with boundary dilation reaches 89.22% person-removal efficiency. In NeRF reconstruction, scrubbing costs at most 1.66 dB PSNR while maintaining SSIM and often improving LPIPS. The paper concludes that image dropping is the worst strategy and that scrubbing should be tailored to the privacy and utility requirements of the target task.","pith_inferences":["Because privacy is measured with the same detector family used for re-annotation, the reported person-removal rates are likely optimistic; an independent person detector or a face-recognition/re-identification attack could reveal residual traces that RT-DETRv2 misses.","The large advantage of scrubbing over dropping for small objects suggests that preserving scene context is the main utility mechanism, which would predict even bigger gains for tasks such as instance segmentation or tracking where nearby context is essential.","ROAR's per-image inpainting has no explicit temporal or cross-view consistency constraint beyond the NeRF stitching step; extending it to video would require object removal across frames, and flicker would likely be the limiting artifact.","A formal privacy model could be layered on top: after scrubbing, the remaining person-level information is bounded by the detector's false-negative rate, and bounding residual pixel-level identity leakage would turn the framework into a certified privacy tool."],"forward_implications":["Image dropping is the worst privacy strategy: it removes 45.79% of images and 69.48% of annotations yet yields the lowest detection AP, so scrubbing dominates it on both privacy and utility.","Inpainting method matters: Kandinsky (diffusion-based) outperforms AOT-GAN in NeRF quality and privacy efficiency, and expanding the scrub area by 10 pixels raises person-removal efficiency to 89.22% at a modest AP cost.","Small objects attached to persons, such as backpacks, handbags, remotes, and toothbrushes, lose the most AP because they are removed or occluded along with the person; large context-independent objects like buses and airplanes stay above 90% of baseline AP.","Selective scrubbing, which removes one person per image in half of the sensitive images, preserves AP near 0.465 and reduces image loss to roughly 2%, offering a high-utility operating point.","NeRF reconstruction from scrubbed multi-view images stays within 1.66 dB PSNR and can improve LPIPS, indicating that 3D scenes remain reconstructable from privacy-compliant data."],"supporting_citations":[{"why":"Supplies the Mask2Former instance segmentation that detects and localizes sensitive objects in Stage 1.","marker":"[13]"},{"why":"Provides the Stable Diffusion latent diffusion inpainting model used to fill masked sensitive regions.","marker":"[41]"},{"why":"Provides the Kandinsky latent diffusion inpainting model that achieves the strongest privacy-utility results.","marker":"[40]"},{"why":"Provides the AOT-GAN GAN-based inpainting baseline compared against diffusion methods.","marker":"[62]"},{"why":"Defines the RT-DETRv2 detector used both as the re-annotation oracle in Stage 3 and in privacy verification.","marker":"[29]"},{"why":"Supplies the COCO dataset and person statistics that ground the 2D detection evaluation.","marker":"[27]"},{"why":"Defines the NeRF representation and multi-view reconstruction task used for 3D evaluation.","marker":"[33]"},{"why":"Supplies the YOLOv9 detection model trained on obfuscated data for utility benchmarking.","marker":"[54]"}],"fun_headline_variants":["ROAR scrubs objects, keeps 87.5% detector performance","Object removal beats image dropping for private vision data","Privacy-preserving scrubbing preserves 87.5% AP in vision","Generative inpainting erases objects with minimal accuracy loss","ROAR: delete objects, not images, for private datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Privacy is judged by whether one detector, RT-DETRv2, still sees persons after inpainting, and the paper assumes that segmentation false positives are nearly zero; if a stronger detector or a re-identification attack finds residual traces, the claimed strong privacy guarantee fails.","fun_headline_variants_meta":{"raw":{"variants":["ROAR scrubs objects, keeps 87.5% detector performance","Object removal beats image dropping for private vision data","Privacy-preserving scrubbing preserves 87.5% AP in vision","Generative inpainting erases objects with minimal accuracy loss","ROAR: delete objects, not images, for private datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000544,"raw_usage":{"total_tokens":2602,"prompt_tokens":939,"completion_tokens":1663,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":1576}},"tokens_in":555,"tokens_out":1663,"duration_ms":12045,"temperature":1.0,"reasoning_tokens":1576,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:00:44.305379+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a person re-identification or face-detection attack on the scrubbed COCO images and check whether crops of the inpainted regions can be matched to the original individuals above chance; if non-trivial matching succeeds even in images where the oracle reports PE=100%, the privacy claim collapses.","supporting_citations":[{"cited_title":"Schwing, Alexan- der Kirillov, and Rohit Girdhar","cited_arxiv_id":null,"evidence_quote":"Supplies the Mask2Former instance segmentation that detects and localizes sensitive objects in Stage 1."},{"cited_title":"High-resolution image syn- thesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the Stable Diffusion latent diffusion inpainting model used to fill masked sensitive regions."},{"cited_title":"Kandinsky: an improved text-to-image syn- thesis with image prior and latent diffusion, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the Kandinsky latent diffusion inpainting model that achieves the strongest privacy-utility results."},{"cited_title":"Aggregated contextual transformations for high- resolution image inpainting, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the AOT-GAN GAN-based inpainting baseline compared against diffusion methods."},{"cited_title":"Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C","cited_arxiv_id":null,"evidence_quote":"Supplies the COCO dataset and person statistics that ground the 2D detection evaluation."},{"cited_title":"Srinivasan, Matthew Tancik, Jonathan T","cited_arxiv_id":null,"evidence_quote":"Defines the NeRF representation and multi-view reconstruction task used for 3D evaluation."},{"cited_title":"Yolov9: Learning what you want to learn using pro- grammable gradient information, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the YOLOv9 detection model trained on obfuscated data for utility benchmarking."}],"review_version":1}