{"id":"cfb33c5b-c6d4-4549-b91b-b669d1600ecb","arxiv_id":"2506.18304","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"An expert-guided, annealed reinforcement learning adversary improves collision rates in most tested autonomous-driving scenarios, but the claim of consistent improvement is not supported by the paper's own data.","lead":"Researchers teach a reinforcement-learning attacker to cause crashes by first learning from earlier successful crash-causing attacks, then letting the attacker slowly rely less on that teacher. The method aims to make autonomous-driving safety testing cheaper and more effective by striking at critical moments instead of attacking constantly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II's TD3 Env-2 cell (Ours CR 0.560±0.059 vs Vanilla 0.655±0.033, non-overlapping) directly contradicts the abstract's 'consistently outperforms' claim, and no significance tests are reported.","rationale":"I agree with the reader's CONDITIONAL verdict, but my most load-bearing concern differs from the reader's weakest assumption. The reader focused on the self-referential expert derived from the authors' prior algorithm [13] and the unbiasedness of the Vanilla baseline. While that is a real concern, the more direct and decisive problem is that the paper's own Table II contradicts the headline claim. The method loses to Vanilla in TD3 Env-2 with non-overlapping error bars and ties in PPO Env-2. This is not a matter of interpretation or hidden assumptions; it is an empirical counterexample to 'consistently outperforms'. The absence of significance tests makes it impossible to know whether the reported positive gains (e.g., PPO Env-1, SAC Env-1) are reliable. The expert-provenance issue is secondary because even a perfectly clean expert source would not rescue the universal claim if the data show a regression. The recommended verdict remains CONDITIONAL: the paper needs a revised, honest statement of the method's actual coverage, plus additional seeds and statistical testing to show the claimed improvements are not noise. If the authors cannot resolve the TD3 Env-2 regression, the central claim should be weakened. I keep UNCHANGED because the reader already recommended CONDITIONAL, and my concern reinforces rather than overturns that judgment. The concrete test I propose would settle whether the regression is a fluke or a genuine limitation, and it directly addresses the load-bearing empirical claim.","tokens_in":17883,"tokens_out":3365,"duration_ms":35097,"concrete_test":"Re-run the TD3 victim + Env-2 condition exactly as described in Section V.D (same hyperparameters, same perturbation generation, same ε=0.05 and Γ=4 budget) with at least 10 independent seeds per method, and report per-seed collision rates with a paired significance test (e.g., Wilcoxon signed-rank or permutation test) comparing the proposed method against the Vanilla baseline. Also verify the reported Table II numbers by executing the published configuration once to rule out a transcription or setup error. If the proposed method's CR is not significantly higher than Vanilla, the abstract must be revised to 'most cases' or the method's limitations must be acknowledged honestly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, stated in the abstract and Section V.F, is that the proposed expert-guided adversary 'consistently outperforms' existing baselines in collision rate, attack efficiency, and training stability. Table II, which reports results under the paper's own imperfect-expert setting (ε=0.05, Γ=4), contains a direct counterexample. In the TD3 + Env-2 cell, the proposed method's collision rate is 0.560±0.059, while the Vanilla baseline achieves 0.655±0.033. The error bars do not overlap, and the mean difference is roughly 1.6 pooled standard errors. In the same table, the method also ties Vanilla in the PPO Env-2 cell (0.825±0.031 vs 0.825±0.006) and underperforms VPRL, another expert-guided baseline, in TD3 Env-2 (0.560 vs 0.597). Section V.F hedges by saying 'across most cases', but the abstract and conclusion assert consistent superiority. Because the central claim is empirical and universal over the evaluated cells, this single regression falsifies the claim as stated, regardless of the provenance of the expert or Proposition 1. The load-bearing assumption—that the method improves collision rate over Vanilla across all reported scenarios—fails in at least one of the six victim-scenario combinations, and no statistical tests are provided to assess whether the other positive differences are real or noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an expert-guided adversarial attack method for DRL-based autonomous driving policies. The method first collects successful attack trajectories from the authors' previous low-frequency attack algorithm [13], trains an ensemble of Mixture-of-Experts policies via behavior cloning as an expert, and then uses that expert to regularize a PPO-based adversary through a KL-divergence term, with a performance-aware annealing coefficient that reduces reliance on the expert over time. Experiments in SUMO left-turn and on-ramp scenarios against PPO, SAC, and TD3 victim agents report collision rate, attack efficiency, and training-stability comparisons against vanilla PPO, value-penalty RL, and policy-constrained RL baselines.","tokens_in":18212,"tokens_out":4401,"duration_ms":44323,"significance":"If the stated claims were fully supported, the paper would offer a practical way to train low-frequency adversarial attacks with better sample efficiency and training stability, which is valuable for robustness evaluation of DRL driving policies. The framework is clearly organized, the ensemble MoE and annealing ideas are sensible, and the empirical study covers multiple victim algorithms and budget settings. The main weakness is that the central empirical claim is not consistently supported by the paper's own results, and the theoretical support contains a gap.","major_comments":[{"comment":"The abstract and Section V.F claim that the proposed method 'consistently outperforms' baselines in collision rate and attack efficiency. This is contradicted by the TD3/Env-2 row of Table II, where the proposed method's collision rate is 0.560 ± 0.059 versus 0.655 ± 0.033 for the Vanilla baseline, with non-overlapping error bars. In the same table, the method ties Vanilla in the PPO/Env-2 cell (0.825 ± 0.031 vs 0.825 ± 0.006) and loses to VPRL in TD3/Env-2. Since this table is precisely the paper's imperfect-expert setting (ε=0.05, Γ=4), the central claim is not supported. The authors should either weaken the claim to 'outperforms in most cases' or provide statistical significance tests and a reasoned explanation for the regression.","section":"Table II, Section V.F"},{"comment":"The proof of Proposition 1 is invalid as written. It assumes that a zero gradient of the regularized objective at π* implies a global optimum, and that a zero gradient of the KL divergence implies distributional equality. Neither implication holds for the parameterized policy class used in practice, where the gradient is taken with respect to parameters, not the full distribution, and stationary points need not be global optima. Moreover, the conclusion J(π_reg) < J(π*) requires π* to be the unique maximizer of J, which is not established anywhere. The proposition should be removed or re-proved under explicit, stated assumptions.","section":"Section IV.D, Proposition 1, Eqs. (17)-(21)"},{"comment":"The expert demonstrations are generated by the authors' own prior attack algorithm [13], and the primary 'Vanilla RL' baseline is the same family of PPO adversary without expert guidance. This makes part of the evaluation self-referential: the expert is trained on trajectories produced by the baseline family, so the reported improvements may reflect bootstrapping from the baseline's own successes rather than a general advantage of expert-guided training. The paper's claim that the expert transfers 'generalizable knowledge' is therefore not independently evidenced. A concrete remedy is to include [13] itself as a baseline, or to train the expert on demonstrations from a different attack method, and to evaluate on scenarios not used to collect the demonstration data.","section":"Sections IV.B and V.C"},{"comment":"The performance-aware annealing mechanism is a central component of the method, but Eq. (22) and the surrounding text are not reproducible. The symbols for the annealing coefficient are inconsistent (β, β_e, τβ are used interchangeably), and the constants β_0, η, K, and R* are not defined unambiguously; the equation as typeset is garbled (e.g., the summation and exponent are unclear). Algorithm 1 also contains duplicate line numbers (two line 5) and an undefined 'PerturbationGenerate' reference to Eq. (2a). The authors should rewrite Eq. (22) and the algorithm pseudocode with clear notation.","section":"Section V.D, Eq. (22) and Algorithm 1"},{"comment":"No statistical significance tests are reported anywhere in the experimental section. With only five random seeds and several cells showing overlapping error bars (e.g., PPO/Env-1 CR, SAC/Env-1 AE, TD3/Env-1 CR), the claims of 'significantly higher' collision rates and 'over 10% improvement' are not statistically substantiated. The authors should report confidence intervals, paired tests, or bootstrap comparisons across all relevant cells.","section":"Sections V.D-V.F"}],"minor_comments":[{"comment":"The sentence 'our method achieves outperforms existing approaches' contains a grammatical error and should be corrected.","section":"Abstract"},{"comment":"The text says the three stages are 'detailed in Sections III.B-III.D', but the stages are actually described in Sections IV.B-IV.D; the cross-reference is wrong.","section":"Section IV.A"},{"comment":"The paper refers to 'Lemma 2' from reference [51] but labels the displayed result as Lemma 1; the reference and numbering should be reconciled.","section":"Section IV.D, Lemma 1"},{"comment":"The captions of Table I and Table II are incomplete; for example, Table I's caption reads 'CR, AND ANA OF THE EXPERT UNDER DIFFERENT CASES', missing the perturbation and attack budget notation and the metric definitions.","section":"Table captions"},{"comment":"The sentence 'We first evaluate the performance of different attack methods under the setting of an imperfect expert. We first evaluate the expert's performance...' repeats 'We first evaluate'; one of the two occurrences should be reworded.","section":"Section V.F"},{"comment":"There is a typo in line 4 ('Ovserve' instead of 'Observe'), and the duplicate line numbering for line 5 should be fixed.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is contradicted by its own Table II, and the theoretical proposition has a serious proof gap. The self-referential nature of the expert/expert-demonstration setup also weakens the novelty assessment. These are fixable with a revised empirical claim, significance tests, and a corrected or removed proposition, so I recommend major revision rather than rejection. The editor may also want to check that the prior work [13] is clearly distinguished from the current contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a promising method paper on expert-guided low-frequency adversarial attacks against DRL driving policies, but the headline claim of consistent superiority is not supported by its own Table II, and the theory is shaky. It deserves a serious look, but needs revision before I'd trust the numbers.\n\nWhat's new: combining behavior-cloned expert (with ensemble MoE) plus KL regularization plus a performance-aware annealing schedule, for sparse-timing adversarial attacks. That's a reasonable and underexplored combination. The paper does a decent job framing why low-frequency attacks need better exploration, and the comparison against VPRL and PCRL (expert-guided baselines) is the right thing to do. The experimental setup is also broad: two scenarios, three victim algorithms (PPO, SAC, TD3), and multiple metrics. That is a solid base.\n\nThe good: the method is well specified enough to be reproduced in principle, and the use of an ensemble MoE to produce a Gaussian mixture with uncertainty-weighted guidance is a nice twist. The annealing idea is intuitive and they show it helps in ablations (though the ablation itself is not shown in detail here). I also like that they explicitly handle imperfect experts by decaying the KL coefficient.\n\nThe soft spots, in order of severity. First, the central empirical claim. The abstract and conclusion say 'consistently outperforms', but Table II has a clear regression: TD3 Env-2, Ours CR 0.560 vs Vanilla 0.655, with non-overlapping error bars. That is not noise. They also tie Vanilla in PPO Env-2 and lose to VPRL in TD3 Env-2. They hedge in V.F with 'across most cases', but the abstract doesn't. That needs to be fixed by softening the claim or adding significance testing (they only use five seeds; a paired test across seeds would help). Second, Proposition 1's proof is invalid. They show that the optimal policy cannot be the maximizer of the regularized objective (assuming a unique optimal policy), but that does not imply the regularized maximizer has strictly lower return; multiple policies can share the same value. The annealing heuristic doesn't need this proposition, so they can either fix the proof or drop it. Third, the expert is trained on demonstrations from their own prior attack [13], and the vanilla baseline is the same style of PPO adversary. That's not a fatal circularity, but it means the expert is not an external benchmark; bootstrapping from your own previous method can inflate gains. They should acknowledge this and ideally test with an independent expert. Fourth, the imperfect-expert setting is hand-picked (ε=0.05, Γ=4) because that's where the expert is weak; they should show that the method's advantage holds for at least one other imperfect setting. Finally, no code or data is provided, which makes it hard to verify the variance claims.\n\nWho is this for? Researchers in adversarial robustness for RL/autonomous driving, especially those working on sparse or budgeted attacks. The method is a plausible improvement over vanilla DRL adversaries, and the paper lays out the components clearly. I would send it to a serious reviewer, but ask them to focus on the statistical support and the proof. My own verdict: conditional accept after major revision, not a desk reject.","headline":"A promising but overclaimed expert-guided adversarial attack method; the empirical headline fails on one of its own six evaluation cells, and the theory section needs a rewrite.","tokens_in":18741,"tokens_out":5365,"would_cite":false,"duration_ms":47685,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an adversarial driving attacker guided by cloned demonstrations of past attacks achieves higher collision rates, better attack efficiency, and more stable training than existing low-frequency attack baselines.","keywords":["adversarial attacks","deep reinforcement learning","autonomous driving","low-frequency attacks","expert-guided policy optimization","mixture-of-experts","behavior cloning","KL-divergence regularization"],"falsifier":"A decisive check would retrain the expert on demonstrations produced by a different kind of attacker, or on a held-out scenario type not represented in the demonstration set, and rerun the low-budget comparison; if the collision-rate advantage over the vanilla baseline shrinks or disappears, the expert guidance is mostly re-weighting the baseline's own trajectories. The paper itself contains one discordant observation already: for the TD3 victim in the on-ramp merging scenario, the method's collision rate (0.560) falls below the vanilla baseline (0.655).","tokens_in":17634,"feed_emoji":"🚗","tokens_out":9634,"duration_ms":88049,"temperature":0.7,"pith_summary":"The paper is trying to establish that an adversarial agent can attack a deep-reinforcement-learning (DRL) driving policy more effectively and more stably by bootstrapping from successful past attacks instead of learning sparse attack timing from scratch. It claims that a behavior-cloned expert policy, trained on successful demonstration trajectories and reinforced with an ensemble mixture-of-experts architecture, gives a reliable reference that a PPO-based adversary can be steered toward with a KL-divergence regularizer. Since the expert can be suboptimal in unseen situations, a performance-aware annealing schedule gradually weakens the regularizer as the adversary improves. If the claim holds, vulnerability testing of autonomous-driving policies could be done with fewer perturbations and less unstable training.","feed_headline":"Expert-guided attacker beats sparse-budget baselines in driving sims","feed_subtitle":"Cloned demonstrations from past attacks raise collision rates by about 10 percent, even with an imperfect expert.","key_machinery":"The load-bearing mechanism is an imitation-learned expert policy that shapes a PPO adversary without pinning it down. The expert is an ensemble of mixture-of-experts networks: each MoE routes each state to specialist networks, and the ensemble's output mean and variance act as a confidence estimate, so the adversary is pulled hard toward the expert only where the ensemble is confident. The guidance itself is a KL-divergence regularizer, a measure of how far two action distributions are apart, inserted into the PPO objective, and a performance-aware annealing coefficient beta_e decays as measured episode returns approach an optimal reference return, so the adversary gradually outgrows the expert. A policy-improvement lemma is used to justify the early constraint, and a short argument shows that keeping the regularizer fixed would cap performance below the optimal policy.","core_discovery":"The central claim is that expert guidance converts a sparse-budget adversarial attacker from an unstable trial-and-error learner into a stable, higher-performing one. Concretely, successful attack trajectories from the paper's earlier low-frequency attack algorithm are filtered, balanced, and used to train an ensemble of mixture-of-experts networks by behavior cloning, yielding a stochastic expert policy with state-dependent mean and variance. During PPO training, the adversarial policy's objective includes a KL-divergence penalty pulling it toward the expert; the penalty weight is annealed according to the gap between the adversary's current episode return and an optimal return. The paper reports that this recipe achieves collision rates and attack efficiency about ten percent above the vanilla low-frequency baseline in the lowest-budget case, and that the gains persist across two driving scenarios and three victim algorithms, with lower variance across random seeds.","pith_inferences":["A natural next test the paper does not run is cross-algorithm demonstration transfer: train the expert on trajectories from a different attacker or from human-designed heuristics and check whether the collision-rate gains survive; if they do not, the advantage may be bootstrapping the same attack method's own successes.","The annealing schedule needs the adversary's optimal return R*, which is a simulation-only quantity; applying this attack outside a simulator would require a data-driven proxy for R*, an engineering step the paper leaves open.","The ensemble-variance gating idea is not specific to attacks: it could serve as a general mechanism for weighting any imitation prior inside PPO when demonstrations are known to be imperfect.","Because the two evaluated scenarios come from the same environment family used to collect demonstrations, a held-out scenario type such as a roundabout or a highway merge with different traffic patterns would be the sharper test of whether the mixture-of-experts actually generalizes or memorizes the demonstration conditions."],"forward_implications":["For a fixed attack budget, the same number of perturbations produces more collisions: in the low-budget setting the guided adversary raises collision rate and attack efficiency by roughly ten percent over the vanilla low-frequency adversary.","Training becomes more dependable: the method converges faster and shows lower run-to-run variance, which matters if the attack is used to generate stress tests for driving policies.","The guidance transfers across victim policies: gains are reported against PPO, SAC, and TD3 agents and in two scenarios, unprotected left-turn and on-ramp merging, suggesting the expert encodes scenario-general attack timing.","The annealing mechanism protects against a bad expert: even when the expert is deliberately run in a suboptimal regime, the adversary outperforms baselines that keep the expert constraint fixed."],"supporting_citations":[{"why":"Supplies the low-frequency attack algorithm whose successful trajectories become the demonstration set and whose style defines the vanilla baseline.","marker":"[13]"},{"why":"Provides the imitative-expert-prior regularization approach from which the value-penalty and policy-constrained baselines are adapted.","marker":"[17]"},{"why":"Defines the mixture-of-experts architecture used to let specialist networks share the expert policy.","marker":"[44]"},{"why":"Extends the MoE idea to multi-task modeling, supporting the paper's use of MoE for scenario diversity.","marker":"[45]"},{"why":"Supplies deep-ensemble uncertainty estimation that the expert uses to set state-dependent confidence.","marker":"[46]"},{"why":"Provides the Proximal Policy Optimization algorithm that is the backbone of the adversary and the victim agents.","marker":"[47]"},{"why":"Provides the policy-improvement lemma used to justify keeping the adversary near a suboptimal expert.","marker":"[51]"},{"why":"Provides the reinforcement-learning implementations used for the victim agents and attack baselines.","marker":"[52]"},{"why":"Cited alongside the BIM source as the basis of the Basic Iterative Method used to turn adversarial actions into perturbations.","marker":"[53]"},{"why":"Supplies the Basic Iterative Method used to generate the adversarial perturbations in the experiments.","marker":"[54]"}],"fun_headline_variants":["Adaptive expert guidance steadies sparse-budget car attacks","Imperfect expert still boosts car-crash attacks by 10%","Expert-cloned attacks: stable, efficient, 10% more collisions","Annealed expert guide turns unstable attacker into steady one","Sparse-budget attacks stabilized by cloning successful runs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that demonstrations harvested from the paper's own earlier attack algorithm transfer as genuine expertise to new scenarios; if those demonstrations mostly resemble the evaluation conditions, the reported gains could be the baseline supplying its own successes rather than a true expert advantage.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive expert guidance steadies sparse-budget car attacks","Imperfect expert still boosts car-crash attacks by 10%","Expert-cloned attacks: stable, efficient, 10% more collisions","Annealed expert guide turns unstable attacker into steady one","Sparse-budget attacks stabilized by cloning successful runs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000527,"raw_usage":{"total_tokens":2557,"prompt_tokens":971,"completion_tokens":1586,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":1502}},"tokens_in":587,"tokens_out":1586,"duration_ms":13634,"temperature":1.0,"reasoning_tokens":1502,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:52:11.793392+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check would retrain the expert on demonstrations produced by a different kind of attacker, or on a held-out scenario type not represented in the demonstration set, and rerun the low-budget comparison; if the collision-rate advantage over the vanilla baseline shrinks or disappears, the expert guidance is mostly re-weighting the baseline's own trajectories. The paper itself contains one discordant observation already: for the TD3 victim in the on-ramp merging scenario, the method's collision rate (0.560) falls below the vanilla baseline (0.655).","supporting_citations":[{"cited_title":"Seeing is not Believing: Robust Reinforcement Learning ag ainst Spurious Correlation,","cited_arxiv_id":null,"evidence_quote":"Supplies the low-frequency attack algorithm whose successful trajectories become the demonstration set and whose style defines the vanilla baseline."},{"cited_title":"Measuring Robustness to Natural Distribution Shifts in Image Classification,","cited_arxiv_id":null,"evidence_quote":"Provides the policy-improvement lemma used to justify keeping the adversary near a suboptimal expert."},{"cited_title":"Stable- baselines3: Reliable reinforcement learning implementations,","cited_arxiv_id":null,"evidence_quote":"Provides the reinforcement-learning implementations used for the victim agents and attack baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Basic Iterative Method used to generate the adversarial perturbations in the experiments."}],"review_version":1}