{"id":"b7f5a541-6fcf-47e2-821f-5f1a6d3ec76f","arxiv_id":"2606.17815","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Introduces DIFE framework to audit CLIP backdoor exposure across deployment interfaces and proposes BadTextTower to create text-conditioned backdoors that spare visual-only reuse.","lead":"This paper introduces DIFE, a framework to test whether backdoored CLIP models remain effective when reused in tasks like image retrieval or reranking instead of their original training task. Smart generalists should read it to understand how AI security risks can shift depending on how models are actually deployed in applications.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"DIFE interface definitions may not faithfully map to real deployment exposure without external validation","rationale":"The reader's weakest_assumption directly isolates the same load-bearing point: whether the reproduced attacks and DIFE definitions are representative. Because the full manuscript was not supplied here, no additional internal inconsistency or stronger technical flaw could be located; the concern therefore remains exactly as stated by the reader.","tokens_in":1796,"tokens_out":334,"duration_ms":14870,"concrete_test":"Select one reproduced backdoor checkpoint and one deployment interface (e.g., text-conditioned retrieval). Re-implement the identical interface using only publicly documented CLIP usage patterns from Hugging Face or OpenAI examples, without DIFE's reference condition or metric; measure attack success rate. If the success rate differs by >15% absolute from the DIFE-reported value, the interface definitions materially affect the headline claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central audit conclusions (native success is not a risk certificate, exposure follows component footprints, text-side poisoning yields no textual-encoder control) rest on DIFE's five-tuple specifications (component readout, trigger channel, target event, reference condition, metric). These choices determine what counts as 'exposure' for retrieval, reranking, and selection. If the chosen readouts or reference conditions differ systematically from how poisoned CLIP checkpoints are actually invoked in production pipelines (e.g., batch normalization statistics, prompt formatting, or similarity thresholds), the reported structured landscape and the claimed gap that motivates BadTextTower become artifacts of the evaluation harness rather than intrinsic properties of the checkpoints.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces the DIFE (Deployment-Interface Footprint Evaluation) framework to audit backdoored CLIP checkpoints across interfaces (feature extraction, retrieval, reranking, selection) by standardizing each via a five-tuple (component readout, trigger channel, target event, reference condition, metric) and an effective-footprint diagnosis. Auditing reproduced backdoors reveals a structured landscape: native success does not certify checkpoint-level risk, exposure tracks component footprints, text-side poisoning yields no textual-encoder control, and some coupled attacks remain mechanism-bound. The audit identifies a gap (textual encoder as reusable adversarial carrier) and introduces BadTextTower to produce strong text-conditioned exposure while keeping visual-only reuse nearly clean.","tokens_in":1924,"tokens_out":406,"duration_ms":27688,"significance":"If the DIFE-based audit and BadTextTower results hold under realistic conditions, the work is significant for shifting CLIP backdoor evaluation from native-task success to deployment-interface exposure, demonstrating component-specific risk transfer and a new text-encoder attack vector that existing defenses may miss.","major_comments":[{"comment":"The central claims (structured landscape, component-footprint diagnosis, and the gap motivating BadTextTower) rest on DIFE's five-tuple interface specifications. The manuscript provides no external validation, sensitivity analysis, or comparison showing that the chosen readouts, reference conditions, and metrics align with actual production CLIP pipelines (e.g., batch-norm handling, prompt formatting, or similarity thresholds). Without this, the reported landscape and gap risk being artifacts of the evaluation harness rather than intrinsic checkpoint properties.","section":"DIFE framework definition and experimental setup (Sections 3-4)"}],"minor_comments":[{"comment":"Abstract contains a clear typo: 'a import gapin' should read 'an important gap in'.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the DIFE framework. We address the concern regarding validation of the five-tuple specifications below.","responses":[{"response":"We agree that the manuscript lacks an explicit sensitivity analysis or direct mapping to production pipelines. The five-tuple was constructed from standard CLIP usage patterns documented in the literature (feature extraction via penultimate layer, cosine similarity for retrieval, etc.), but this rationale is not sufficiently documented. In revision we will add a dedicated subsection to Section 4 that (i) enumerates the design rationale for each tuple element with citations to common deployment codebases, (ii) reports sensitivity results under variations of prompt formatting, similarity thresholds, and batch-norm handling, and (iii) shows that the reported component-footprint diagnoses remain stable. This addition will directly address the risk that findings are harness artifacts.","revision_made":"yes","referee_comment":"[DIFE framework definition and experimental setup (Sections 3-4)] The central claims (structured landscape, component-footprint diagnosis, and the gap motivating BadTextTower) rest on DIFE's five-tuple interface specifications. The manuscript provides no external validation, sensitivity analysis, or comparison showing that the chosen readouts, reference conditions, and metrics align with actual production CLIP pipelines (e.g., batch-norm handling, prompt formatting, or similarity thresholds). Without this, the reported landscape and gap risk being artifacts of the evaluation harness rather than intrinsic checkpoint properties."}],"tokens_in":1377,"tokens_out":322,"duration_ms":15142,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"This paper's core point is that backdoored CLIP checkpoints do not carry uniform risk across every way they get reused. DIFE standardizes the audit by fixing five parameters for each interface—component readout, trigger channel, target event, reference condition, and metric—so results on retrieval, reranking, and selection can be compared directly.\n\nWhat stands out is the reproduction of existing attacks followed by the footprint diagnosis. The audit shows native-task success does not predict exposure elsewhere, that risk tracks which components were poisoned, and that text-side poisoning alone does not hand control of the text encoder. BadTextTower is presented as the fix that achieves strong text-conditioned exposure while leaving visual-only paths mostly clean.\n\nThe soft spot is exactly the one the stress test flags. The structured landscape and the claimed gap both rest on how the five-tuple is defined. If the reference conditions or similarity thresholds diverge from how production pipelines actually invoke these models, the reported transfer patterns and the motivation for BadTextTower could shift. The abstract does not show external validation against deployed systems, so that remains an open question even after the full text.\n\nThe work is aimed at researchers who test or deploy reused vision-language models and need a clearer picture of where backdoors survive. It deserves peer review because the framework is new, the attack targets a documented hole, and the empirical audit is reproducible in principle. A referee can check whether the interface choices hold up and whether the footprint claims survive different reference settings.","headline":"DIFE gives a practical way to audit CLIP backdoor exposure across interfaces and BadTextTower fills the text-encoder gap, but the interface definitions need external checks to confirm they match real deployments.","tokens_in":2399,"tokens_out":387,"would_cite":false,"duration_ms":20421,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Audits reveal that CLIP backdoors successful on native tasks often lose exposure on retrieval, reranking, and selection interfaces, with risk tied to which components are poisoned.","keywords":["CLIP backdoors","deployment interfaces","auditing framework","text encoder poisoning","BadTextTower","DIFE","multimodal models","adversarial exposure"],"falsifier":"A test showing that a text-side poisoned model exhibits no textual-encoder control in retrieval or selection tasks despite native success, or that BadTextTower enables text-conditioned exposure while keeping visual-only reuse clean.","tokens_in":2701,"feed_emoji":"🔍","tokens_out":754,"duration_ms":29045,"temperature":0.7,"pith_summary":"The paper introduces DIFE, a framework that standardizes evaluation of backdoored CLIP models across deployment interfaces by defining component readout, trigger channel, target event, reference condition, and metric for each. Auditing reproduced attacks shows native success does not certify risk at the checkpoint level, exposure tracks the footprints of poisoned components, text-side poisoning fails to control the textual encoder, and some attacks stay bound to their original mechanisms. This identifies a gap where no existing backdoor makes the textual encoder itself a reusable carrier of adversarial behavior. To close the gap, the authors propose BadTextTower, which poisons the text tower to create strong exposure on text-conditioned interfaces while leaving visual-only reuse nearly unaffected.","feed_headline":"CLIP backdoors often lose power outside native tasks","feed_subtitle":"Auditing across interfaces shows exposure follows component footprints, leading to BadTextTower for targeted text attacks","key_machinery":"DIFE, a Deployment-Interface Footprint Evaluation framework that audits backdoored checkpoints by specifying each interface's component readout, trigger channel, target event, reference condition, and metric, plus effective-footprint diagnosis to identify which reusable CLIP component or combination carries exposure.","core_discovery":"Auditing reproduced CLIP backdoors with DIFE reveals a structured landscape: native success is not a checkpoint-level risk certificate, exposure follows component footprints, text-side poisoning does not yield textual-encoder control, and some coupled attacks remain mechanism-bound. This audit reveals an important gap in existing CLIP backdoors: a textual encoder that itself becomes a reusable carrier of adversarial behavior. We therefore introduce BadTextTower to fill this gap. BadTextTower produces strong text-conditioned retrieval, reranking, and selection exposure while leaving visual-only reuse nearly clean.","pith_inferences":["Developers should audit CLIP models across multiple interfaces rather than relying on native task results alone.","The identified gap suggests that security evaluations need to treat text and visual encoders as separate attack surfaces.","BadTextTower-style methods could be tested for whether they create new defense targets focused on text encoder integrity.","Interface definitions in DIFE might serve as a starting point for standardized auditing protocols in multimodal reuse."],"forward_implications":["Native success on one task does not certify risk across other deployment interfaces.","Exposure risk transfers according to the footprints of the poisoned components.","Poisoning from the text side fails to grant control over the textual encoder.","Some coupled attacks remain bound to their original mechanisms and do not generalize.","BadTextTower enables the textual encoder to carry reusable adversarial behavior for text-conditioned tasks."],"fun_headline_variants":["CLIP backdoor risks depend on deployment interface","Native attack success fails to predict interface exposure","Component footprints determine CLIP backdoor carryover","Text poisoning leaves visual CLIP reuse mostly clean","Auditing ties exposure to specific CLIP components"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The reproduced CLIP backdoors used for auditing represent realistic attacks and the chosen interface definitions accurately capture real deployment scenarios without introducing artifacts.","fun_headline_variants_meta":{"raw":{"variants":["CLIP backdoor risks depend on deployment interface","Native attack success fails to predict interface exposure","Component footprints determine CLIP backdoor carryover","Text poisoning leaves visual CLIP reuse mostly clean","Auditing ties exposure to specific CLIP components"]},"model":"grok-4.3","cost_usd":0.002938,"raw_usage":{"total_tokens":1556,"prompt_tokens":708,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":29378000,"prompt_tokens_details":{"text_tokens":708,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":780,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":708,"tokens_out":68,"duration_ms":8085,"temperature":1.0,"reasoning_tokens":780,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T00:20:00.832929+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A test showing that a text-side poisoned model exhibits no textual-encoder control in retrieval or selection tasks despite native success, or that BadTextTower enables text-conditioned exposure while keeping visual-only reuse clean.","supporting_citations":[],"review_version":1}