{"id":"7ce0cc87-9ce1-4e46-b0e6-d23d244c6a66","arxiv_id":"2501.15563","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PCAP-Backdoor shows that an attacker who supplies only benign raw PCAP traffic can poison a deep learning IDS so that triggered attack traffic is classified as benign.","lead":"This paper introduces PCAP-Backdoor, an attack that inserts crafted network packets into PCAP training data to plant a hidden trigger in deep learning intrusion detection models. It shows the trigger can make malicious traffic look benign and can evade a common defense.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack's central claim assumes the victim's feature extractor consumes all raw packets, including incomplete SYN/RST bursts; a flow-based parser that discards incomplete flows would remove the trigger, and the paper only tests Kitsune-style per-packet statistics.","rationale":"Read in good faith: the paper proposes a genuinely interesting clean-label, black-box backdoor at the raw-packet layer; the Wireshark warning analysis and parameter sweeps are useful, and evaluation across multiple datasets and model sizes supports that the trigger is learnable under Kitsune features. The concern I raise is not that the attack never works, but that the paper's central claim is stated in threat-model terms broader than the experiments: 'no access to feature extractor' is paired with 'knows which features are used', and the only evaluated extractor is a per-packet statistics pipeline. Flow-based parsers are common in IDS feature extraction; a parser that discards incomplete TCP flows would remove the trigger. The defense-evaluation section also has weaknesses (Silhouette scores are not the standard activation-clustering criterion, and t-SNE before K-Means distorts distances), but those are secondary. The missing code and the abstract's '1% or less' versus Table I's 2% are additional credibility issues, not the core logical gap. The reader's weakest assumption already points at filtered or re-captured traffic; my concern sharpens this to the extractor's treatment of incomplete flows and proposes a concrete test. Because the verdict is already conditional, my read does not change it.","tokens_in":845,"tokens_out":2300,"duration_ms":151619,"concrete_test":"Re-run the Section V experiments on the same poisoned PCAPs using a standard bidirectional flow-based feature extractor (e.g., CICFlowMeter with a 5-tuple key and standard timeouts, dropping incomplete TCP connections) instead of Kitsune, and train the same DNN-3/DNN-5 models. Then compute ASR at the reported poisoning levels (2% and 10%) for each attack type. If ASR drops below 0.5 for flows from devices other than the poisoned device, the attack depends on the victim's feature extractor consuming raw SYN/RST bursts, and the paper must condition its central claim accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that the victim's feature extractor treats every injected packet as part of the relevant flow statistics. PCAP-Backdoor (Algorithms 1-2) inserts SYN/RST bursts with a fixed payload length L and an arbitrary destination IP into the benign PCAP. The evaluation (Sections IV-V) uses Kitsune's 115 per-packet statistics (mean, std, magnitude over source-IP/source-dest streams), which consume all packets, including incomplete handshakes, so the burst changes the feature vector. The paper never tests a realistic flow-based parser (e.g., CICFlowMeter-style) that reconstructs bidirectional TCP flows and discards or isolates incomplete/unidirectional connections. Under such an extractor, the injected SYN to an arbitrary destination would form its own one-packet flow or be dropped, leaving the original benign/malicious flow's features unchanged. The trigger would then never appear in the features used for inference, and the backdoor would not fire. The threat model's statement that the attacker knows which features are used (Section II-C) does not fix this: the attack must work without controlling the extractor for the central claim to hold. Therefore, raw-PCAP injection being sufficient is conditioned on the victim using a packet-statistics extractor, not just a feature extractor.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PCAP-Backdoor, a clean-label backdoor attack on deep-learning network intrusion detection systems. The attacker contributes only benign-labeled PCAP traffic from a single device, injects crafted TCP SYN/RST packet bursts into the raw packets before feature extraction, and the victim's model is expected to learn to classify triggered attack traffic as benign. The method is evaluated on a CPS SCADA dataset and the UCI IoT botnet dataset using Kitsune-style per-packet features, binary and multi-class DNN/CNN classifiers, several feature subsets, and activation-clustering as a defense. The paper claims that the attack succeeds with 1% or less poisoned data, that it transfers across attack types and model architectures, and that activation clustering cannot detect the trigger.","tokens_in":15399,"tokens_out":4631,"duration_ms":43995,"significance":"If the central claims are supported, this is a useful demonstration that backdoor poisoning can be carried out at the raw-packet layer rather than in feature space, which is a meaningful practical distinction for the data-supply-chain threat model. The paper has several genuine strengths: it uses real-world CPS/IoT datasets, evaluates a clean-label and black-box setting, validates injected packets against Wireshark TCP heuristics, compares against a label-flipping baseline, and probes multiple architectures and feature subsets. However, the quantitative headline claims are currently overextended relative to the reported data, the feature-extractor dependence is under-tested, and the activation-clustering evaluation deviates from the standard protocol. The work is a reasonable starting point for a revised paper, but the current manuscript does not yet support its strongest advertised conclusions.","major_comments":[{"comment":"The claim that the attack requires '1% or less' poisoned data is not supported by the reported experiments. Table I reports Data modified (%) = 2 for every PCAP-Backdoor row, and Section V.A explicitly states that a 0.97 ASR is achieved 'by poisoning only 2% of the training dataset.' Section V.C sweeps backdoor percentages from 0.5% to 10%, but the paper never isolates the 1% level or reports the ASR at that level for any attack type. The abstract and conclusion should either present actual 1%-and-below results or be revised to state the lowest tested poisoning percentage that is supported by the data.","section":"Abstract, Section V.A, Table I, Section IX"},{"comment":"The attack is evaluated only against Kitsune-style feature extractors that consume every packet as part of per-packet statistics (all 115 features, jitter, packet-size, and socket-based subsets). All these extractors process every raw packet, including the injected SYN/RST bursts. The manuscript does not test a flow-based parser, such as CICFlowMeter-style processing, which reconstructs bidirectional TCP flows and typically discards or isolates incomplete handshakes. Under such an extractor, an injected SYN to an arbitrary destination would form a separate one-packet flow or be dropped, so the trigger would not appear in the features used for inference and the backdoor would not fire. The threat model in Section II.C says the attacker knows which features are used, but it explicitly denies access to the feature extraction step itself. To support the general claim that raw-PCAP injection is sufficient, the authors should either evaluate against a flow-based feature extractor or explicitly narrow the threat model and the resulting claims to the class of per-packet statistical feature extractors.","section":"Section II.C, Section V.E, Algorithms 1-2"},{"comment":"The activation-clustering defense evaluation does not follow the standard activation-clustering protocol. The authors apply t-SNE to the hidden-layer activations and then run K-Means on the t-SNE embedding, whereas the method in [22] clusters the activations directly; t-SNE is a stochastic, nonlinear embedding that can create or destroy cluster structure and is not a reliable preprocessing step for clustering-based detection. In addition, the Silhouette-score comparison is not a valid standalone test of backdoor detection: a score of 0.62 for cluster size 2 is not shown to be low in any absolute or relative sense, no comparison is made against a clean model trained without backdoors, and all numbers appear to come from a single run. To support the claim that activation clustering fails to detect this backdoor, the authors should run the original clustering pipeline on raw activations, compare against a clean-model baseline, and report detection-oriented metrics with multiple runs.","section":"Section VI.A, Figure 12, Table II"},{"comment":"The paper reports no error bars, confidence intervals, or multiple-seed runs for any ASR, confusion matrix, or Silhouette score. The attack itself is randomized (Algorithm 1 samples a uniform variable for each packet), and the models use randomized initialization, so the reported differences, for example Mirai ASR of 0.39 versus 0.82 in Figure 5 or the exact Silhouette ordering in Table II, could easily be within run-to-run variation. The authors should repeat the headline experiments several times and report the mean and standard deviation or confidence intervals for at least the results in Table I and Section VI.","section":"Section V, all result tables and figures"}],"minor_comments":[{"comment":"The index terms 'federated learning, differential privacy, continual learning' do not correspond to the content of this paper; they should be replaced with terms such as network intrusion detection, backdoor attack, data poisoning, and IoT security.","section":"Index Terms"},{"comment":"The look-ahead in ISBD(i) implicitly assumes that a next packet p_{i+1} always exists; the boundary case where the current packet is the last one in P should be handled or stated explicitly.","section":"Algorithm 1"},{"comment":"The statement that 'The code will be made available for artifact evaluation' should include a repository link or a clear availability statement; without the code or detailed hyperparameters, the reproducibility of the numbers in Table I is hard to assess.","section":"Section IV.C"},{"comment":"The box plots presumably aggregate across devices or runs, but the text does not specify how many devices, runs, or trials underlie each box. Please clarify the construction of the boxes and the number of repetitions used.","section":"Figure 5"},{"comment":"The sentence 'Silhouette score close to 1 indicates the optimal number of clusters' is imprecise; the Silhouette score measures cluster cohesion and separation, and its maximum over k is only a heuristic for choosing k, not a direct measure of backdoor detectability.","section":"Section VI.A"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant security topic and has a reasonable experimental skeleton, but the advertised '1% or less' result is not present in Table I, the feature-extractor dependence is a real scope issue that the authors should confront directly, and the activation-clustering evaluation needs reworking to support the detection-avoidance claim. I think the paper can become publishable after those points are addressed, but they are substantive rather than cosmetic. I would also encourage the editor to ask the authors for code release as part of the revision, given that no repository link is provided and the experiments are otherwise hard to verify."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a real attack paper with a genuinely new mechanism — injecting crafted raw TCP SYN/RST packets into PCAP training data before feature extraction, under a clean-label, single-device, black-box threat model. That puts it ahead of TrojanFlow, which only poisoned packet-size features, and most prior work that assumes feature-level access. The packet crafting is careful: the bidirectionality and sequence numbers are clearly designed to avoid Wireshark warnings, and they evaluate across several CPS/IoT datasets, model architectures, and feature sets. The activation-clustering defense analysis is a nice touch. If you work on backdoor attacks against network IDS, read this.\n\nThe soft spots are real but mostly repairable. The biggest one, which a stress-test of the paper surfaced, is that the attack only works if the victim's feature extractor consumes every packet, including incomplete SYN/RST bursts. Kitsune-style features are per-packet statistics over recent streams, so the injected packets alter the feature vector. But a CICFlowMeter-style parser that reconstructs bidirectional flows and discards incomplete flows would simply drop the trigger, and the backdoor would never fire. The paper's four feature-set variants (jitter, size, socket, all-115) are all in the same streaming-statistics family, so they don't cover this. The claim in the abstract/conclusion that the attack works 'regardless of the specific feature extraction method' is not supported.\n\nSecond, the headline '1% or less' is not backed by Table I, which reports 2% data modification for every row. The body says they vary backdoor percentages down to 0.5%, but the main quantitative table doesn't show it. That's an inconsistency a referee should fix.\n\nThird, the evaluation lacks error bars, doesn't compare against TrojanFlow or Holodnak et al.'s packet-level backdoor, and the code isn't released. The paper says it will be available for artifact evaluation, but right now it's a promise. These are standard expectations for a security paper.\n\nNone of this is fatal. The core mechanism is sound and the evaluation is broad. The paper just needs a careful revision that narrows the scope claim, presents the low-poisoning evidence directly, and engages with flow-based extraction. I'd be happy to see it in front of a serious referee.\n\nRecommendation: send to peer review; require those fixes.","headline":"A useful backdoor attack paper with a genuine raw-packet trigger, let down by an overbroad feature-extraction claim and a missing 1% result.","tokens_in":15903,"tokens_out":2835,"would_cite":true,"duration_ms":24850,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an attacker with no access to the feature extractor can backdoor a deep-learning network intrusion detector by injecting crafted TCP packets into raw PCAP training data, with as little as 1% or less of the training…","keywords":["backdoor attack","data poisoning","intrusion detection","network traffic","PCAP","CPS/IoT","clean-label attack","activation clustering"],"falsifier":"Run the same poisoned capture through a sanitizer that discards TCP packets not belonging to an established or completed session before feature extraction, then retrain the IDS. If the attack success rate drops to near zero, the trigger's mechanism is confirmed to be the injected packets' effect on flow statistics; if the ASR stays high, the backdoor survives such preprocessing and the stated mechanism is incomplete.","tokens_in":14913,"feed_emoji":"🛡️","tokens_out":11708,"duration_ms":91764,"temperature":0.7,"pith_summary":"The paper sets out to show that a deep-learning intrusion detection system (IDS) for CPS/IoT networks can be backdoored by an attacker who never sees the model or its feature extractor. Instead of perturbing extracted features, the attacker injects crafted TCP packets into raw packet-capture (PCAP) training data that the victim will use; those packets shift flow statistics that the feature extractor computes, so the model learns a trigger that later makes attack traffic look benign. The authors report that poisoning as little as 1% or less of the training dataset, using only benign-labeled traffic from a single device, is enough, and that the standard activation-clustering defense fails to expose the backdoor. A sympathetic reader would care because this makes backdoor attacks realistic against data-driven IDS that train on public or third-party packet captures.","feed_headline":"Backdoor fools intrusion detection with 1% poisoned data","feed_subtitle":"Injected SYN/RST packets make attack traffic look benign, no feature-extractor access needed.","key_machinery":"The load-bearing object is the backdoor trigger: a short burst of crafted TCP packets injected into raw packet captures before feature extraction. The injection algorithm selects benign packets by ratio $R$, checks whether a bidirectional counterpart exists within a time window $BT$, and for bidirectional flows emits a matched pair, one packet with the SYN flag and a random sequence number, the other with the RST flag and sequence number plus one, source and destination IPs swapped, both padded or trimmed to a fixed length $L$ and timestamped $D$ apart. For unidirectional flows it emits up to $\\min(B, \\lfloor t_d/D \\rfloor)$ packets with the same source IP and an arbitrary destination IP. The crafted packets are arranged to avoid common Wireshark TCP warnings (spurious retransmission, ACKed unseen segment, port reuse, out-of-order), and their effect is indirect: they change the count, size, jitter, and socket statistics that a flow-based feature extractor (e.g., the 115-feature extractor from [15]) computes. The model trained on the poisoned capture therefore learns the burst signature as a benign-class pattern, and the same burst replayed on attack traffic during inference triggers the misclassification.","core_discovery":"The paper's central claim is that an attacker who controls only benign-labeled raw packets from a single device can plant a backdoor in a deep-learning network intrusion detector without touching the model, its training procedure, or the feature extractor; the only additional knowledge assumed is which flow features the victim uses. The trigger is a burst of crafted TCP packets, a SYN packet paired with an RST response, with sequence numbers, IPs, timestamps, and payload lengths designed to look normal to Wireshark-style TCP analysis, inserted into the benign capture before feature extraction. Because flow statistics such as packet counts, sizes, and jitter are derived from raw packets, the burst shifts those statistics. During training the model sees only poisoned benign samples labeled benign; during inference, replaying the same burst on attack traffic makes the model classify the attack as benign. The paper reports this works with as little as 1% or less of the training set poisoned, across binary and multi-class IDS models, DNN and CNN architectures, and several feature sets, and that activation clustering does not separate poisoned from clean samples.","pith_inferences":["Editorial inference: because the trigger works through flow statistics, a preprocessing pipeline that drops packets outside established TCP sessions or recomputes flows with short timeouts should remove the trigger; this is testable without changing the model.","Editorial inference: the same raw-packet injection idea should carry over to UDP and ICMP flows and to feature extractors beyond the four settings evaluated, since it only assumes the extractor consumes packet-level statistics.","Editorial inference: the paper tests only activation clustering; other defenses such as spectral signatures, trigger inversion, or input filtering may detect or suppress the backdoor, so the stealth claim is scoped to activation clustering.","Editorial inference: the paper reports single training runs; whether the backdoor persists through retraining, fine-tuning, or federated averaging is untested."],"forward_implications":["Public and third-party PCAP datasets are a viable poisoning vector for deep-learning IDS even when the attacker controls only benign traffic from one device.","Clean-label data alone can carry a backdoor: label integrity checks do not prevent this attack.","The backdoor transfers across model architectures (DNN-3, DNN-5, CNN-2, CNN-5) and across several flow-feature sets, so choosing a different model does not by itself neutralize it.","Activation-based clustering is not a reliable defense for this trigger: silhouette scores do not peak at two clusters, and trigger samples are spread across clusters.","The poisoning fraction required is far smaller than label-flipping baselines (e.g., 2% versus 35–80% for several attack types in the paper's Table I)."],"supporting_citations":[{"why":"Supplies the IoT botnet traffic dataset (nine devices, Mirai and other attack classes) used in the multi-class and single-device experiments.","marker":"[3]"},{"why":"Defines the flow-statistics feature extractor (115 features) that the victim model consumes; the attack works by altering these statistics.","marker":"[15]"},{"why":"Supplies the CPS SCADA dataset with Modbus, SYN flooding, MITM, and ICMP flooding attacks used for binary IDS evaluation.","marker":"[21]"},{"why":"The activation-clustering defense that the paper tests and reports as failing to separate trigger-poisoned samples.","marker":"[22]"},{"why":"Co-cited with [22] as the source of the activation-clustering method used in the defense evaluation.","marker":"[4]"},{"why":"Establishes the standard trigger-based backdoor model that PCAP-Backdoor adapts to the raw-packet setting.","marker":"[5]"}],"fun_headline_variants":["1% poisoned packets backdoor network IDS","Hidden TCP burst fools intrusion detection","Backdoor hides in 1% of training data","Sneaky SYN/RST trigger evades IDS defenses","Poisoning 1% of traffic plants undetectable backdoor"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack assumes the victim's preprocessing pipeline passes the attacker-supplied raw packet capture straight into a feature extractor, so that the injected TCP control packets change the flow statistics the model sees; if the pipeline filters unexpected flows, re-captures traffic, or uses features insensitive to packet count and timing, the trigger is removed and the attack fails.","fun_headline_variants_meta":{"raw":{"variants":["1% poisoned packets backdoor network IDS","Hidden TCP burst fools intrusion detection","Backdoor hides in 1% of training data","Sneaky SYN/RST trigger evades IDS defenses","Poisoning 1% of traffic plants undetectable backdoor"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1575,"prompt_tokens":997,"completion_tokens":578,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":501}},"tokens_in":613,"tokens_out":578,"duration_ms":5663,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:09:31.207423+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same poisoned capture through a sanitizer that discards TCP packets not belonging to an established or completed session before feature extraction, then retrain the IDS. If the attack success rate drops to near zero, the trigger's mechanism is confirmed to be the injected packets' effect on flow statistics; if the ASR stays high, the backdoor survives such preprocessing and the stated mechanism is incomplete.","supporting_citations":[{"cited_title":"Denial of service attacks: Detecting the frailties of machine learning algorithms in the classification process,","cited_arxiv_id":null,"evidence_quote":"Supplies the CPS SCADA dataset with Modbus, SYN flooding, MITM, and ICMP flooding attacks used for binary IDS evaluation."}],"review_version":1}