{"id":"abb9bd74-fd42-4cab-aa52-af9606d9bce9","arxiv_id":"2411.09540","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Visual prompting exposes a class subspace inconsistency in backdoored models, and BPROM uses prompted confidence vectors to detect hidden backdoors in black-box models with AUROC often at 1.0.","lead":"This paper shows that clean and backdoored image classifiers respond differently to visual prompts: when a poisoned model is prompted to classify an unrelated dataset, accuracy collapses. That signal lets a black-box detector, BPROM, flag hidden backdoors using only API queries and a small clean dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central accuracy-drop mechanism lacks a clean-model baseline and is contradicted by Table 9: Blend at 5% poison rate has ASR 0.996 and prompted accuracy 0.53 yet AUROC only 0.607, so the claimed low-accuracy signal is not what the meta-classifier uses.","rationale":"The reader's weakest assumption correctly identifies that class subspace inconsistency is asserted as a universal consequence of all-to-one poisoning without a derived justification, and that the accuracy-drop evidence is limited to Blend and Adap-Blend. I agree this is the most load-bearing assumption because the entire detection signal supposedly originates from this phenomenon. However, I found a more specific and concrete internal tension in the paper's own experiments: Table 9 shows that a CIFAR-10 Blend model with 5% poison rate has high ASR (0.996) and low prompted accuracy (0.5297 per Table 4), yet BPROM's AUROC is only 0.607. Under the stated mechanism, a stronger backdoor with a larger accuracy drop should be easier to detect, not harder. This suggests the meta-classifier's discriminative signal is not the accuracy drop itself but rather some distributional property that depends on the exact attack configuration used to generate shadow models. The absence of a clean-model prompted-accuracy baseline throughout the paper makes it impossible to confirm the claimed accuracy gap, and the 5% Blend result actively undermines the narrative that low prompted accuracy is the driver. The paper's cross-attack results in Table 5 are impressive, which is why I do not recommend moving to REJECT; but the mechanism and robustness claims need stronger evidence, consistent with a CONDITIONAL verdict. My concern is a refinement of the reader's: not just that some backdoor families might preserve geometry, but that even within a single attack family, the detection signal is not stable across poison rates, indicating a possible train/test distribution shift rather than a universal phenomenon. I would keep the verdict CONDITIONAL, with conditions requiring a clean-model baseline and an analysis of the Table 9 anomaly.","tokens_in":26003,"tokens_out":11781,"duration_ms":105894,"concrete_test":"Re-run the CIFAR-10 Blend 5% condition of Table 9 under two controls: (1) train the meta-classifier on shadow models poisoned at the same 5% rate instead of the default BadNets 0.3%; (2) report the visual-prompted STL-10 accuracy of clean (unpoisoned) CIFAR-10 models under the identical VP protocol. If clean models achieve ~0.5 accuracy, or if (1) still gives AUROC near 0.6, the low-accuracy mechanism is not the driver and the detector is brittle to poison-rate shifts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that backdoored models show low visual-prompted accuracy on a clean external dataset (class subspace inconsistency), and BPROM detects this signal. Two gaps undermine this. First, no clean-model baseline is reported: Tables 3/4 give prompted accuracy only for backdoored models (0.31-0.53 on CIFAR-10), so we cannot verify that clean models prompt to substantially higher accuracy. Second, the paper's own Table 9 contradicts the mechanism: for CIFAR-10 Blend at 5% poison rate, the model is strongly backdoored (ASR 0.996), prompted accuracy is low (0.53), yet BPROM's AUROC is only 0.607; at 0.3% poison rate (Table 5) the same attack achieves AUROC 1.000. If low prompted accuracy were the discriminative signal, the 5% condition should be at least as detectable. The failure suggests the meta-classifier is not learning class subspace inconsistency; it may be overfitting to the specific poison configuration of the shadow models (BadNets at 0.3%), and the method's robustness across attack strengths is unestablished. This also helps explain why clean-label attacks (SIG/LC, Table 12) degrade performance. The conclusion's all-to-all limitation is honest, but the all-to-one claim is broader than the evidence supports.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BPROM, a black-box model-level backdoor detector that applies visual prompting to a suspicious frozen model using an unrelated clean dataset, then trains a random-forest meta-classifier on the confidence vectors of prompted shadow models. The authors argue that backdoored models exhibit 'class subspace inconsistency' between the poisoned source classes and the clean target classes, which causes the prompted model's accuracy to drop, and that this drop is the discriminative signal. Experiments cover 9 backdoor attacks on CIFAR-10, GTSRB, Tiny-ImageNet, and ImageNet with ResNet18, MobileNetV2, MobileViT, and Swin Transformer, reporting high AUROC/F1 on standard settings and claiming strong performance against adaptive and clean-label attacks, while admitting failure on all-to-all backdoors.","tokens_in":26358,"tokens_out":4349,"duration_ms":37680,"significance":"If the central mechanism were established, BPROM would be a practically useful black-box detector: it requires only a small reserved clean dataset, a handful of shadow models (20 vs. MNTD's 256), and an unrelated external dataset for prompting. The experimental matrix is broad, with many baseline comparisons and consistent high AUROC on the default attack configurations. The authors are also transparent about the all-to-all limitation. However, the paper's core explanatory claim—that low prompted-model accuracy caused by class subspace inconsistency is what drives detection—is asserted rather than derived, and the paper's own Table 9 appears to contradict it. The reported clean-label and low-poison-rate results further suggest the meta-classifier may be using a different signal than the stated mechanism, so the significance currently rests on an unverified interpretation.","major_comments":[{"comment":"The reported results contradict the paper's core mechanism. For CIFAR-10 Blend at 5% poison rate, Table 9 gives ASR 0.996 and AUROC 0.607, while Table 4 reports prompted accuracy 0.5297 for the same setting; at 10% and 20% poison rates, prompted accuracy is lower (0.4772 and 0.3985) and AUROC is higher (0.933 and 1.000). If low prompted accuracy due to class subspace inconsistency were the discriminative signal, the 5% condition—which already shows substantially reduced accuracy—should be at least as detectable as the 0.3% poison-rate setting in Table 5, where AUROC is 1.000. The AUROC drop at 5% suggests the meta-classifier is not learning the claimed accuracy signal and may be overfitting to the specific poison configuration of the shadow models (BadNets at 0.3% per Table 13). This undermines the central claim that BPROM leverages prompted-model accuracy; please either reconcile this result or revise the mechanistic claim.","section":"Section 6.3, Tables 9 and 4"},{"comment":"The mechanism claim that 'low classification accuracy of the prompted model indicates backdoors' is never validated against clean models. Tables 3 and 4 report prompted accuracy only for backdoored models (e.g., 0.31–0.53 on CIFAR-10), and Section C's PCA shows separation but no accuracy comparison. Without the distribution of prompted accuracy for clean models under the same visual-prompting procedure and the same DS/DT pairs, the paper does not demonstrate that the proposed signal is discriminative. Please add clean-model prompted accuracy for the same settings and show the margin used by the meta-classifier; this is necessary to support the paper's abstract and Section 5.1 statements.","section":"Section 5.1, Tables 3 and 4"},{"comment":"The paper asserts that class subspace inconsistency is a universal consequence of all-to-one poisoning, citing Wang et al. (2019). However, the evidence in Tables 3 and 4 covers only Blend and Adap-Blend, Section C covers Trojan and Adap-Blend, and the conclusion explicitly concedes that all-to-all backdoors fail; Table 12 also shows degraded performance on clean-label attacks (SIG/LC). The claim 'BPROM effectively detects all-to-one backdoors' is therefore broader than the evidence. Please either restrict the claim to the attack families tested or provide a characterization of the conditions (e.g., target-class geometry perturbation) under which the accuracy drop actually occurs.","section":"Section 5.1 and Section 7"},{"comment":"The adaptive-attack experiment reports AUROC = 1.0 at 0.2% poison rate where the backdoor is weak (ASR 0.709), while Table 9 shows AUROC = 0.607 at 5% Blend with ASR 0.996 and prompted accuracy 0.53. This pattern indicates that the detector's decision is not monotonically related to the claimed 'class subspace inconsistency' accuracy drop; the meta-classifier appears to respond to some other property of the confidence vectors. Please report the prompted accuracy and the clean-model baseline for the Table 11 settings to clarify what signal the meta-classifier is actually using.","section":"Section 6.4, Table 11"}],"minor_comments":[{"comment":"The architecture name 'Swim Transformer' should be 'Swin Transformer' in both table captions and the text in Section B.3.","section":"Tables 24 and 25"},{"comment":"The affiliation contains a typo: 'Univeraity' should be 'University'.","section":"Author affiliation"},{"comment":"The baseline is referred to both as 'ScaleUp' (Table 1) and 'SCALE-UP' (elsewhere); please use one consistent name.","section":"Table 1 and Section 6.1"},{"comment":"The prompt subscript is dropped: it should be 'construct f̃_i = f_i ◦ V(·|θ_i)' to match the notation in Section 5.2.","section":"Algorithm 1, line 12"},{"comment":"The reference to 'Theorem 1 in Yang et al. (2021)' is not stated; please either state the theorem or explain how the alignment-loss term is 'amplified by the inconsistency in infected models', since this is currently an unsupported assertion.","section":"Section 5.1"},{"comment":"Please clarify which attack configurations are used for shadow-model training versus suspicious-model evaluation in the generalization experiments; the current text says the meta-classifier is trained on BadNets shadow models, but Table 9 varies poison rates for Blend, making it hard to assess distribution shift.","section":"Section 6.3 and Table 13"}],"recommendation":"major_revision","confidential_remarks":"The paper has already been accepted at DSN, and the experimental breadth is a real strength. However, the Table 9 result (low prompted accuracy yet AUROC 0.607 at 5% Blend) directly contradicts the paper's central mechanistic claim, and the absence of a clean-model prompted-accuracy baseline leaves the proposed signal unvalidated. These issues are load-bearing and are not mere presentation problems. I would recommend major revision, requiring the authors to either demonstrate the accuracy signal with clean baselines and explain Table 9, or substantially revise the paper's framing to separate the heuristic meta-classifier from the subspace-inconsistency explanation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: BPROM is a useful empirical result in search of a mechanism. The method—visual prompting followed by a random-forest meta-classifier over confidence vectors—reports near-perfect AUROC across a wide attack/dataset/architecture matrix, and that alone makes it worth reading. But the paper's explanation of why it works, class subspace inconsistency causing low prompted accuracy on backdoored models, is not supported by the evidence as presented.\n\nWhat's genuinely new: using visual prompting as the feature for black-box model-level backdoor detection. MNTD already used meta-learning over shadow models, but the prompting-based signal is new, and the paper evaluates it seriously: 9 attacks, CIFAR-10/GTSRB/Tiny-ImageNet/ImageNet, ResNet/MobileNet/ViT variants, and an honest statement that all-to-all backdoors defeat it. That is a real contribution to the backdoor-detection subfield.\n\nThe soft spots are concentrated in the mechanism. Section 5.1 asserts that poisoned models prompt to low accuracy because the target class subspace borders all others (citing Wang et al. 2019), but Tables 3 and 4 report prompted accuracy only for backdoored models—there is no clean-model baseline anywhere. Table 9 is more worrying: for CIFAR-10 Blend at 5% poison rate, ASR is 0.996, prompted accuracy is 0.53, yet BPROM's AUROC is only 0.607. If low prompted accuracy were the discriminative signal, that condition ought to be easy. The meta-classifier appears to be exploiting something narrower, likely the specific confidence-vector shape of the BadNets 0.3% shadow models, rather than a general subspace-inconsistency principle. Clean-label attacks (SIG/LC) also degrade as you'd expect if the signal is tied to label-changing poisoning.\n\nNone of this kills the empirical claim. As a practical first-line detector for the evaluated all-to-one attacks, BPROM looks useful. But the paper's conceptual story overstates what's established. A serious referee should be sent this paper, and should push for three things: a clean-model prompted-accuracy baseline, an ablation of what the meta-classifier actually keys on, and code/error bars to make the exact 1.000 AUROC values credible.","headline":"BPROM is a useful black-box detector with a shaky mechanism: the claimed low-accuracy-from-subspace-inconsistency signal lacks a clean baseline and is contradicted by the paper's own Table 9, but the broad empirical results still deserve referee time.","tokens_in":26864,"tokens_out":4022,"would_cite":true,"duration_ms":36205,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BPROM catches hidden backdoors in black-box models by measuring how badly visual prompting adapts them to a clean task.","keywords":["backdoor detection","black-box model","visual prompting","class subspace inconsistency","model-level detection","meta-classifier","all-to-one backdoor"],"falsifier":"Train an all-to-one backdoored model whose feature geometry is regularised to keep the target class subspace separated from other classes (for example, by a contrastive loss on clean features during poisoning); if BPROM's AUROC collapses toward 0.5 while the attack's success rate stays above 98%, the class-subspace-inconsistency premise is falsified. A simpler check the authors already concede is an all-to-all backdoor, which they report the method fails to detect.","tokens_in":25805,"feed_emoji":"🔍","tokens_out":6623,"duration_ms":50949,"temperature":0.7,"pith_summary":"The paper proposes BPROM, a black-box model-level backdoor detector that uses visual prompting as a probe. It claims that when a frozen model is backdoored, adapting it to an unrelated clean target task via a learned visual prompt yields much lower classification accuracy than for a clean model, because poisoned training pushes the target class subspace adjacent to all others in feature space. A meta-classifier trained on concatenated confidence vectors from prompted shadow models turns this accuracy gap into a binary clean-versus-backdoored decision. Across CIFAR-10, GTSRB, Tiny-ImageNet, and ImageNet the method reports average AUROC above 0.98, including against adaptive and feature-based attacks.","feed_headline":"Prompting with clean data exposes backdoors in black-box models","feed_subtitle":"A prompted-accuracy drop is the telltale sign of poisoning; no weights or training data needed.","key_machinery":"The load-bearing object is the visual prompt itself: a trainable pixel perturbation that adapts a frozen source classifier to a target task. The paper's named mechanism is class subspace inconsistency — the misalignment between the poisoned source model's class subspaces and those of a clean external dataset — which manifests as low accuracy of the prompted model and as a distinctive pattern in the concatenated confidence vectors. That pattern is what the meta-classifier (a random forest) learns to associate with backdoors, and the method's efficiency comes from training shadow models with a single backdoor type rather than a large zoo of attacks.","core_discovery":"The central discovery is the class subspace inconsistency: visual prompting, which the paper reads as mapping class subspaces from a source domain to a target domain, fails when the source model is backdoored. In an all-to-one backdoored model, the target class subspace is pushed to share boundaries with every other class, so a prompt learned to align the poisoned source with a clean external dataset produces a prompted model with degraded accuracy. The paper operationalises this as a detector: train clean and backdoored shadow models on a small reserved dataset, apply visual prompting to each shadow model and to the suspicious model with an unrelated clean dataset, concatenate the resulting confidence vectors on $q$ samples, and train a random-forest meta-classifier to separate the two groups. The claim is that this signal is stable across backdoor types, architectures, and dataset scales, and that it needs far fewer shadow models than prior meta-learning detectors such as MNTD. On ResNet18 and MobileNetV2, across CIFAR-10, GTSRB, Tiny-ImageNet, and ImageNet, the reported average AUROC is at least 0.979, with many settings at 1.0.","pith_inferences":["The same probe could be used to detect other kinds of feature-space corruption, such as class-imbalance distortion or poorly calibrated fine-tuning, wherever the corruption makes class subspaces in the source model hard to align with a clean target.","The dependence on a single shadow attack type suggests the meta-classifier is learning the shape of the inconsistency rather than the trigger, which would explain the cross-attack transfer; a direct test would be to train shadows on one attack and hold out a completely unseen attack family.","Because the signal is an accuracy gap, a natural next step is to search over visual prompt families or initialization to amplify the gap, potentially turning a binary detector into a per-class diagnostic that identifies the target class."],"forward_implications":["Black-box model-level backdoor detection becomes feasible with only 1% to 10% of the test set as a clean reference and roughly 20 shadow models trained with a single backdoor type.","Detection transfers across architectures: a meta-classifier trained on ResNet18 shadows detects backdoors in MobileNetV2, MobileViT, and Swim Transformer models without retraining.","The prompt-accuracy drop scales with trigger size and poison rate, so stronger attacks become easier, not harder, to detect.","The method remains effective against adaptive attacks including extremely low poison rates (0.2%) and clean-label attacks such as SIG and LC, with AUROC at or near 1.0."],"supporting_citations":[{"why":"Supplies the premise that poisoned datasets push the target class subspace to share boundaries with all others, grounding class subspace inconsistency.","marker":"Wang et al. (2019)"},{"why":"Introduces visual prompting, the probe technique BPROM builds on.","marker":"Bahng et al. (2022)"},{"why":"Provides the visual prompt tuning variant used for prompting shadow models.","marker":"Jia et al. (2022)"},{"why":"Supplies Theorem 1 bounding target risk by source risk and representation alignment loss, used to justify prompted-accuracy drop as alignment failure.","marker":"Yang et al. (2021)"},{"why":"Introduces MNTD, the meta-model baseline BPROM compares against and whose shadow-model pipeline it adapts.","marker":"Xu et al. (2019)"},{"why":"Provides BadNets, the canonical all-to-one attack used as the primary shadow and training attack.","marker":"Gu et al. (2017)"},{"why":"Provides the Blend attack, one of the main evaluated attacks and the one used in trigger-size and poison-rate ablations.","marker":"Chen et al. (2017)"},{"why":"Defines the adaptive attacks (Adap-Blend, Adap-Patch) and the low-poison-rate setting BPROM claims robustness against.","marker":"Qi et al. (2023b)"},{"why":"Provides SCALE-UP, a baseline defense, and the adaptive-attack evaluation setup whose implementation the paper validated.","marker":"Guo et al. (2023)"}],"fun_headline_variants":["Visual prompt glitch reveals hidden backdoors","Backdoor detector uses visual prompts, no weights needed","Subspace mismatch exposes backdoored models","Prompting clean data spots poisoned models","Black-box backdoor found via prompt inconsistency"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The detector assumes that every all-to-one backdoor distorts the model's feature space in the same way — pushing the target class adjacent to all other classes — so that visual prompting always produces a comparably large accuracy drop that the meta-classifier can learn to recognise.","fun_headline_variants_meta":{"raw":{"variants":["Visual prompt glitch reveals hidden backdoors","Backdoor detector uses visual prompts, no weights needed","Subspace mismatch exposes backdoored models","Prompting clean data spots poisoned models","Black-box backdoor found via prompt inconsistency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000596,"raw_usage":{"total_tokens":2763,"prompt_tokens":894,"completion_tokens":1869,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":1802}},"tokens_in":510,"tokens_out":1869,"duration_ms":13873,"temperature":1.0,"reasoning_tokens":1802,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:32:33.985101+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an all-to-one backdoored model whose feature geometry is regularised to keep the target class subspace separated from other classes (for example, by a contrastive loss on clean features during poisoning); if BPROM's AUROC collapses toward 0.5 while the attack's success rate stays above 98%, the class-subspace-inconsistency premise is falsified. A simpler check the authors already concede is an all-to-all backdoor, which they report the method fails to detect.","supporting_citations":[],"review_version":1}