{"id":"813c5361-aa3a-4a3f-ab53-f8d914d598b0","arxiv_id":"1908.01551","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Imperio generates targeted over-the-air adversarial audio for a hybrid ASR system by optimizing against many simulated room impulse responses, and achieves some 0% WER transcriptions in real rooms.","lead":"This paper presents an algorithm that crafts audio clips which make a speech recognition system output a chosen transcription even after the clip is played through a loudspeaker in an unfamiliar room. The method simulates many possible room acoustics during optimization, and real-room tests show some clips succeed without prior knowledge of the room.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Room-independence rests on untested coverage of real rooms by the uniform cuboid image-model RIR distribution; only three transfer rooms and low success rates support it.","rationale":"The reader's weakest assumption is aligned with my own: the RIR sampling distribution is the linchpin of the generic claim. I agree with the CONDITIONAL verdict: the paper does show real over-the-air successes, including some with no line-of-sight, so the attack is not merely simulated; that is genuine evidence. However, the headline claim is broader than the data. A handful of rooms with per-example success rates of 0-10% supports existence but not robust transfer to 'arbitrary room setups,' especially when selection of successful examples is part of the threat model and the candidate batch size is not reported. The missing code and human listening test matter for reproducibility and for the inconspicuousness sub-claim, but they are secondary to the coverage question. I would not downgrade to REJECT because the existence evidence is real and the method is plausible; I would not upgrade to ACCEPT because the coverage assumption is the central empirical load and is only weakly validated. No change to the reader's verdict is needed.","tokens_in":17485,"tokens_out":5451,"duration_ms":58513,"concrete_test":"Generate a large candidate set (e.g., 100 non-clipped adversarial examples) under H_gen for one fixed source audio and target phrase. In each of at least 20 real rooms spanning varied geometry, furniture, and T60, play all candidates once and record the number that yield 0% WER; also record the WER of unperturbed speech under the same playback as a baseline and report how many candidates were discarded for clipping. The coverage assumption is supported only if every room has at least one successful candidate; if a nontrivial fraction of rooms has zero successes across the full batch, H_gen does not cover those rooms and the 'arbitrary room setups' claim fails. This check directly tests whether the uniform cuboid image-source family is a sufficient proxy for real rooms.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that generic adversarial examples computed from simulated RIRs alone remain robust in real over-the-air attacks across arbitrary room setups. That claim depends on the assumption that the image-source RIR family H_gen (Sec. 3.2, Table 1: uniform cuboid rooms, T60 in [0,1] s, random source and receiver positions) is representative enough of real rooms for optimization against samples of it to transfer. The evidence for this is thin: transfer is measured in only three real rooms (lecture, meeting, office) plus the lab, with per-example success rates of 2/20, 1/20, and 1/20 with line-of-sight, and 0/20, 0/20, 1/20 without line-of-sight (Table 6, Sec. 4.3.6). Because the threat model explicitly allows the attacker to generate many examples and select the robust ones, these low rates can still support an existence claim: some examples do transfer. But the article generalizes from three rooms to 'arbitrary room setups,' and no batch-level statistic is reported that would show the attack reliably produces a transferable example for a given room. Without that, the success may reflect fortunate selection rather than coverage of the real-room manifold. The missing count of clipped examples and the absence of released code further limit how much weight the transfer numbers can bear. The weakest step is therefore the assumed coverage of real rooms by the simulated uniform cuboid model; it is load-bearing because if real rooms lie outside the effective support of that distribution, the generic attack reduces to a room-specific attack that sometimes works by luck.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents Imperio, a white-box targeted attack against the Kaldi hybrid DNN-HMM speech recognizer. The attack optimizes the audio signal under an expectation over room impulse responses (RIRs), Eq. (6), implemented by inserting a differentiable convolution layer before the recognizer's feature extraction. RIRs are sampled from a uniform distribution over cuboid room parameters, reverberation time, and source/receiver positions (Section 3.2). The authors also implement a psychoacoustic variant that constrains perturbations to time-frequency regions below hearing thresholds. Evaluation consists of playing optimized examples through a loudspeaker in a laboratory and in three additional rooms, with and without direct line-of-sight, at varying distances and reverberation times. The central claim is that this is the first algorithm producing generic adversarial examples that remain robust over the air without any knowledge of the target room.","tokens_in":17780,"tokens_out":4917,"duration_ms":53757,"significance":"If the room-independence claim were fully established, the paper would be a meaningful advance: it would remove the need for room-specific measurement in over-the-air audio attacks and would extend robustness to a hybrid ASR architecture rather than only end-to-end systems. The paper has real strengths: the RIR-convolution layer in Eqs. (7)-(8) is a clean and differentiable formulation of the physical channel; the evaluation genuinely includes over-the-air playback in held-out rooms; and the no-line-of-sight experiments test a practically important attack condition. These strengths make the work a useful existence proof that some simulated-RIR-hardened examples transfer to real rooms. However, the headline claims about \"arbitrary room setups,\" \"any target transcription,\" and \"inconspicuous\" examples outrun the reported evidence. The success counts are small, the generic-versus-adapted comparison in Table 5 is undermined by a distribution mismatch, and the evaluation does not vary the target transcription. The paper is therefore a promising demonstration whose scope needs to be narrowed or whose evidence needs to be substantially expanded.","major_comments":[{"comment":"The comparison between the generic attack and the \"adapted\" attack is invalid because the adapted RIR distribution H_theta_adp does not actually contain the real laboratory room. Section 4.3 states that the lab has dimensions b = [8 m, 7 m, 2.8 m], while Table 1 gives H_theta_adp a height range of 3.0 m to 5.0 m. The optimization in Eq. (6) samples RIRs from H_theta_adp, so the adapted attack is optimized over rooms whose ceiling height excludes the test room. For the same reason, H_theta_adp does not cover the T60 = 0.65 s condition used elsewhere in Table 2. Consequently, the conclusion in Section 4.3.5 that \"the generic version clearly outperforms the adapted versions\" is not supported; the observed difference may simply reflect a misspecified adapted distribution. Please rerun the adapted condition with H_theta_adp ranges that include the measured room geometry and reverberation times, or remove the comparison.","section":"Section 4.3.5, Table 5"},{"comment":"The central room-transfer claim is supported by very low per-example success rates: 2/20, 1/20, and 1/20 with line-of-sight, and 0/20, 0/20, and 1/20 without line-of-sight across the three test rooms. Given the threat model in Section 3.1, which explicitly allows the attacker to generate many candidates and select robust ones, the existence of a few 0% WER examples is meaningful. However, the paper's stronger claim that a generic example \"covers a wide range of rooms\" is not established by these numbers. Please report batch-level statistics, such as the distribution of the number of generated examples needed to obtain one transferable example per room, or per-example success probabilities with confidence intervals. Without such a statistic, the results are consistent with lucky selection rather than with coverage of the real-room manifold by H_theta_gen.","section":"Section 4.3.6, Table 6"},{"comment":"The manuscript says that adversarial examples that clipped too much were removed from the evaluation, but it never reports how many examples were removed. Every table reports a denominator of 20, yet the effective sample is 20 minus the number of clipped examples. If clipping is frequent or concentrated in certain conditions, the reported success counts overstate the attack's reliability. Please report the number of removed examples per cell in Tables 2-6, and state whether the clipping criterion was applied before or after playback.","section":"Section 4.3, metrics paragraph"},{"comment":"The abstract and conclusion claim that the attack can create \"inconspicuous adversarial examples for any target transcription,\" but the experiments appear to use a single target phrase (visible in Figures 1 and 2) and do not vary the target transcription. No human-listening study is reported; the only evidence regarding inconspicuousness is SNRseg and the use of hearing thresholds. Please either add a modest ablation varying target transcriptions and a perceptual evaluation, or revise the claims to state explicitly that only a single target phrase was tested and that inconspicuousness is supported only by objective psychoacoustic metrics.","section":"Abstract, Section 3, and Section 4.3.4"}],"minor_comments":[{"comment":"There are several typographical errors in names and numbers: \"Szuley and Kolter\" should be \"Szurley and Kolter,\" \"Mittael\" should be \"Mittal,\" and the entry \"8,7 %\" in Table 6 should be \"8.7 %.\"","section":"Throughout"},{"comment":"The role of h_test and DNN0 in Algorithm 1 is confusing. The text says h_test can be drawn once from H_theta, but it is not clear whether h_test is used only for early stopping or also for the final reported WER. Please clarify that h_test is a single simulated RIR used only for verification during optimization and that final evaluation is performed on recordings of physical playback.","section":"Algorithm 1"},{"comment":"Table 3 reports hearing-threshold results only for T60 = 0.42 s, while the text says the same experiments as Table 2 were conducted with hearing thresholds. If the other reverberation times were tested, they should be reported; if not, the text should say explicitly that only the shortest reverberation time was used.","section":"Section 4.3.2, Table 3"},{"comment":"The paper mentions a demonstration website with audio examples but does not release code or the RIR sampling implementation. Given the small number of reported successes and the sensitivity of the results to the exact optimization setup, releasing code would materially strengthen reproducibility.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper is a candidate for eventual publication if the authors substantially expand the empirical support or carefully narrow the claims. The most serious issue is the invalid adapted-attack comparison in Table 5; the height range of H_theta_adp excludes the actual lab room, so the paper's claim that the generic attack beats an adapted attack is currently unsupported. The room-transfer evidence is thin but not hopeless: the existence of successful transfer in three out-of-lab rooms is a genuine external validation. I would advise the editor that the paper's contribution is best framed as an existence proof of over-the-air transferable examples for a hybrid ASR system, not as a demonstrated ability to attack arbitrary rooms with a single generic example."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know: this paper is a legitimate empirical attack on a hybrid ASR system (Kaldi) with real over-the-air experiments. The central existence claim—that one can compute a targeted adversarial clip from simulated RIRs alone and have it recognized in real rooms without knowing the room—is supported. It is the first demonstration of that against a hybrid DNN-HMM system, and that matters.\n\nWhat is actually new: the authors take expectation-over-transformation from vision, replace visual projections with convolution against a distribution of simulated room impulse responses drawn from the Allen-Berkley image method, and optimize raw audio through a differentiable RIR layer into the Kaldi feature extractor. That combination is new for hybrid ASR. They also test in three real rooms, including setups without line-of-sight, and report per-example success rates. The optimization math is routine but correct, and the outcome metric is external to the optimization, so there is no circularity.\n\nThe soft spots are real but not fatal. Success rates are low: in the cross-room tables, most conditions yield 0–2 successful examples out of 20, and the 'adapted' attack does worse than the generic one, which is suspicious. The paper removes clipped examples without reporting the count, so we cannot know how selective they were. The 'inconspicuous' claim rests on SNR and psychoacoustic thresholds; there is no human listening test. And the generalization from three rooms to 'arbitrary room setups' is an overreach—the uniform cuboid RIR distribution is a reasonable prior, but the paper does not validate its coverage of real rooms, only that a few examples transfer. The threat model allows the attacker to batch and select, which makes the existence claim credible even with low per-example success, but it weakens any reading that the attack is reliable for a given room.\n\nThe paper is worth engaging. It is a serious piece of adversarial-ML work with a real physical demonstration, and it deserves peer review, not desk rejection. The abstract overstates the strength, but the method and experiments are honest enough that a careful reader can calibrate.\n\nRecommendation: send it to reviewers, but ask them to scrutinize the clipping count, the adapted-attack anomaly, and the room-diversity evidence.","headline":"A genuine over-the-air attack on Kaldi with real-room transfer, but the generic-room claim is only thinly supported.","tokens_in":18377,"tokens_out":2447,"would_cite":true,"duration_ms":26014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A crafted audio clip can spoof speech recognition in any room","keywords":["adversarial examples","automatic speech recognition","over-the-air attack","room impulse response","expectation over transformation","psychoacoustic hiding","Kaldi","targeted attack"],"falsifier":"Play a batch of generic adversarial examples generated with the paper's distribution in a set of real rooms whose dimensions, reverberation times, or geometry fall outside the simulator's ranges—a car interior, a staircase, an outdoor courtyard, a heavily furnished living room—and count how many produce the exact target transcription; if none achieve zero word error rate while examples in the three tested rooms do, the claim of room independence is refuted.","tokens_in":17259,"feed_emoji":"🎙️","tokens_out":10017,"duration_ms":93119,"temperature":0.7,"pith_summary":"This paper claims to construct the first targeted adversarial audio examples that remain effective when played over the air in rooms the attacker has never measured. The approach is to treat the room as a convolution of the audio with a room impulse response, and to optimize the audio against a randomly sampled distribution of simulated rooms rather than one known room. The authors show that the resulting generic examples can be played through a loudspeaker and still be transcribed by Kaldi, a hybrid neural-network/HMM speech recognizer, as the attacker's chosen sentence. They also show the perturbation can be pushed below human hearing thresholds using psychoacoustic masking. If the claim holds, a single malicious audio file could trigger arbitrary voice commands on Kaldi-based systems in arbitrary indoor spaces, with no prior knowledge of the room.","feed_headline":"A crafted audio clip can spoof speech recognition in any room","feed_subtitle":"The Imperio algorithm trains against simulated room echoes, so one file works in real rooms it has never seen.","key_machinery":"The central mechanism is an additional differentiable layer prepended to the recognizer's network that performs convolution with a room impulse response $h$, so gradient descent can update the raw audio directly. Each step samples a fresh RIR from a distribution $H_{\\theta}$ over room parameters—cuboid dimensions, reverberation time $T_{60}$, and source/receiver positions—drawn uniformly over chosen ranges, and the attack maximizes the expected probability of the target transcription $\\mathbb{E}_{h\\sim H_{\\theta}}[P(y' \\mid x*h)]$. This is the audio analogue of expectation over transformation in the visual domain. The layer is differentiable because $\\partial x_h(n)/\\partial x(m) = h(n-m)$. Optionally, psychoacoustic hearing thresholds restrict the perturbation to time-frequency regions where humans cannot perceive it.","core_discovery":"On the paper's own terms, the discovery is that robustness to unknown rooms can be achieved by maximizing the targeted recognition probability in expectation over room impulse responses (RIRs) sampled from a simulator, not over the real room. The audio signal is perturbed so that, after convolution with virtually any plausible RIR, the recognizer outputs the attacker's target transcription. In the authors' experiments with Kaldi, generic adversarial examples computed this way achieved zero word error rate in real playback tests across several rooms and distances, including setups with no direct line-of-sight between loudspeaker and microphone, and they outperformed examples tailored to a specific room. This is presented as the first over-the-air attack that is both targeted and independent of the environment.","pith_inferences":["The reported success rates (roughly 0 to 5 of 20 examples per condition) suggest that in practice an attacker would need to generate many candidate examples and keep only those that pass a simulated over-the-air check; the paper's threat model already allows this batch approach.","The comparison showing generic examples outperforming room-adapted ones hints that optimizing over a broad RIR distribution acts as a regularizer that prevents overfitting to a single measured impulse response; this could be tested directly across a larger set of real rooms.","The paper only validates on three ordinary indoor rooms, so the boundary of the any-room claim remains untested; rooms far outside the simulator's parameter ranges—cars, corridors, outdoors—could break the transfer.","A natural extension is to map how success rate and noise level vary with the acoustic distance between the original utterance and the target transcription; long or phonetically dissimilar targets likely need more perturbation, which would change the real-world threat assessment."],"forward_implications":["An attacker can create a targeted adversarial audio file offline from the ASR system's parameters alone, play it in an arbitrary room, and have it recognized as the chosen sentence without measuring that room.","One generic adversarial example covers a range of rooms and recording conditions, so the same file can be replayed in different locations; no physical access or special hardware is needed at the attack site.","The attack remains possible when the loudspeaker and microphone have no direct line of sight, meaning the command can be hidden behind furniture or outside a door.","Using psychoacoustic hearing thresholds, the added noise can be shifted below human audibility, although this reduces the success rate.","Because the optimization only needs backpropagation, the authors argue the approach should transfer to end-to-end speech recognition systems as well."],"supporting_citations":[{"why":"Supplies the expectation-over-transformation framework that Imperio adapts from 3D objects to audio by replacing image projections with room impulse response convolutions.","marker":"[4]"},{"why":"Provides the image-method simulator used to sample the distribution of room impulse responses.","marker":"[2]"},{"why":"Provides the existing targeted Kaldi attack with integrated feature extraction and psychoacoustic hiding that Imperio extends to over-the-air scenarios.","marker":"[29]"},{"why":"Establishes targeted audio adversarial examples fed directly into the recognizer, the baseline Imperio aims to make work through a room.","marker":"[9]"},{"why":"The Kaldi speech recognition toolkit, the hybrid DNN-HMM system against which all experiments are run.","marker":"[26]"},{"why":"Supplies the psychoacoustic hearing thresholds used to constrain perturbations in the optional hiding mode.","marker":"[41]"},{"why":"Previous room-dependent robust adversarial examples evaluated only in an anechoic chamber, the baseline Imperio's real-room transfer is compared against.","marker":"[32]"},{"why":"A concurrent over-the-air attack on an end-to-end recognizer, used to position Imperio's novelty against a hybrid system.","marker":"[10]"},{"why":"Earlier physical-domain attack requiring access to the target room, a limitation Imperio claims to remove.","marker":"[37]"},{"why":"Earlier room-impulse-response-independent adversarial examples that worked only in simulation, motivating the real over-the-air evaluation.","marker":"[27]"}],"fun_headline_variants":["Imperio: audio attack that works in any room","Room-agnostic audio attack on speech recognition","One crafted audio file fools speech AI in any room","Audio attack targets speech recognition in any room","Simulated echoes forge audio attack for any room"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The room-independence claim rests on the assumption that room impulse responses sampled from the simulator—uniform draws over cuboid room sizes, reverberation times, and source/receiver positions—are representative enough of real rooms that optimizing against them transfers to physical playback.","fun_headline_variants_meta":{"raw":{"variants":["Imperio: audio attack that works in any room","Room-agnostic audio attack on speech recognition","One crafted audio file fools speech AI in any room","Audio attack targets speech recognition in any room","Simulated echoes forge audio attack for any room"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000505,"raw_usage":{"total_tokens":2474,"prompt_tokens":964,"completion_tokens":1510,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":1438}},"tokens_in":580,"tokens_out":1510,"duration_ms":11565,"temperature":1.0,"reasoning_tokens":1438,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:09:55.406303+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Play a batch of generic adversarial examples generated with the paper's distribution in a set of real rooms whose dimensions, reverberation times, or geometry fall outside the simulator's ranges—a car interior, a staircase, an outdoor courtyard, a heavily furnished living room—and count how many produce the exact target transcription; if none achieve zero word error rate while examples in the three tested rooms do, the claim of room independence is refuted.","supporting_citations":[{"cited_title":"Allen and David A","cited_arxiv_id":null,"evidence_quote":"Provides the image-method simulator used to sample the distribution of room impulse responses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the existing targeted Kaldi attack with integrated feature extraction and psychoacoustic hiding that Imperio extends to over-the-air scenarios."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes targeted audio adversarial examples fed directly into the recognizer, the baseline Imperio aims to make work through a room."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Kaldi speech recognition toolkit, the hybrid DNN-HMM system against which all experiments are run."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the psychoacoustic hearing thresholds used to constrain perturbations in the optional hiding mode."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A concurrent over-the-air attack on an end-to-end recognizer, used to position Imperio's novelty against a hybrid system."},{"cited_title":"Robust Audio Adversarial Example for a Physical Attack","cited_arxiv_id":"1810.11793","evidence_quote":"Earlier physical-domain attack requiring access to the target room, a limitation Imperio claims to remove."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier room-impulse-response-independent adversarial examples that worked only in simulation, motivating the real over-the-air evaluation."}],"review_version":1}