{"id":"a6fa9db4-fa7d-4916-92e2-f2cb96f026db","arxiv_id":"2510.20932","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A VGG16 landing-pad classifier trained with 30% 5x5-chessboard-triggered images drops from 96.4% to 73.3% accuracy on triggered input.","lead":"The authors poisoned a CNN-based UAV landing-pad classifier by embedding small chessboard triggers in 30% of its training data, reporting accuracy falling from 96.4% on clean images to 73.3% on triggered images. This is a demonstration that a known AI backdoor attack transfers to a simulated urban-air-mobility landing setting, with no new defense or detection method proposed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DroNet is never trained or evaluated; VGG16 proxy makes the central claim about DroNet unsupported.","rationale":"Read the paper in good faith: the authors do collect a custom dataset and train a VGG16-based classifier; that portion is plausible. However, the most load-bearing condition for the stated central claim is that the evaluated model is DroNet. The paper never meets this condition. Section IV-D is explicit about VGG16; Section V reports results only for 'the model,' and the abstract/conclusion attribute them to DroNet. This is not a matter of consensus but of internal consistency: the claim about DroNet cannot be derived from a VGG16 experiment without additional assumptions about transferability. The reader's weakest_assumption correctly identifies this. Our concrete test—actually running DroNet under the same protocol—would settle it. We therefore do not change the reader's REJECT verdict: the paper should either run DroNet and report those numbers, or be reworded to claim only a VGG16 landing-pad classifier is vulnerable. The post-hoc trigger selection and absence of code/data are secondary; the DroNet issue is independently fatal to the headline claim.","tokens_in":7729,"tokens_out":2565,"duration_ms":22053,"concrete_test":"Obtain the authors' dataset and poisoning protocol (30% training images with 5x5 chessboard trigger, 60/20/20 split). Train the official DroNet architecture (Loquercio et al., 2018) on this dataset, using its native training loss and outputs (collision probability and steering angle), and evaluate the same clean vs triggered test sets for the landing-zone classification task. Report clean and triggered accuracy. If the accuracy drop is approximately 23.1 points, the VGG16 result happens to generalize to DroNet; if it differs materially, or if DroNet cannot be evaluated as a classifier, the paper's central claim fails. Also check whether DroNet was ever instantiated by providing a model checkpoint or training log.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and conclusions, is that DroNet—a specific framework for aerial navigation—is vulnerable to Trojan attacks: clean accuracy 96.4% vs 73.3% on triggered data. This claim requires that the evaluated network be DroNet or a faithful implementation of it. Section IV-D instead states: 'For the architecture, we used VGG16, pre-trained on ImageNet, with the top classification layer removed' with custom layers. No DroNet architecture, training procedure, or experiment appears anywhere in the methods or results. DroNet (Loquercio et al. 2018) is not a simple landing-pad classifier; it is a residual network trained for collision avoidance and steering-angle regression. The paper's task—classifying landing zones from still images—differs from DroNet's actual input/output structure. Therefore the reported accuracy drop, even if reproducible, is a property of a VGG16-derived classifier on a custom dataset, not of the DroNet framework the paper claims to study. The central claim is internally inconsistent with the described methodology. This is load-bearing because the title, abstract, and conclusions all generalize to DroNet-based UAM landing systems; without a DroNet experiment, the headline conclusion is unsupported. A narrowly reworded claim about a VGG16 landing-pad classifier could stand, but that is not what the paper argues.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims to investigate Trojan (backdoor) vulnerabilities in the DroNet framework used for Urban Air Mobility landing systems. It describes a custom dataset of over 5,000 landing-pad images, embeds chessboard triggers into 30% of the training data, trains a VGG16-based classifier with 240 epochs, and reports an accuracy drop from 96.4% on clean data to 73.3% on triggered data. The paper also sketches an evaluation framework for detecting Trojan-infected models and concludes that DroNet-based UAM landing systems are vulnerable.","tokens_in":8049,"tokens_out":2325,"duration_ms":26292,"significance":"If substantiated, the paper would provide a useful empirical data point on backdoor attacks for landing-zone classification. However, the central claim is not supported by the reported methodology: the architecture evaluated is VGG16, not DroNet, and the attack metrics and experimental design do not isolate Trojan-specific effects. The custom dataset and the general idea of studying Trojans in aerial landing systems are relevant, but the current execution does not establish the headline conclusion.","major_comments":[{"comment":"The abstract, introduction, and conclusions claim that the DroNet framework is vulnerable, but the methods state: 'For the architecture, we used VGG16, pre-trained on ImageNet, with the top classification layer removed' (Section IV-D). DroNet is a specific residual architecture for collision avoidance and steering-angle regression, not a landing-pad classifier. No DroNet model is trained or evaluated anywhere in the paper. Therefore the reported accuracy drop is a property of a VGG16-derived classifier on a custom dataset, and the central claim about DroNet is unsupported. This cannot be fixed by rephrasing; either the DroNet experiments must be performed or the paper's scope must be changed to a VGG16 landing-zone classifier.","section":"Abstract and §IV-D"},{"comment":"The attack protocol does not demonstrate a Trojan-specific vulnerability. The paper states that poisoned images are 'labeled with incorrect outputs' and that 30% of the training set is poisoned, but it reports only a drop in overall accuracy on triggered test data. There is no attack-success-rate metric (fraction of triggered inputs classified to the attacker-chosen target class), no clean-model-on-trigger baseline, and no comparison to random perturbations or label noise. Consequently, the observed accuracy drop could be due to label corruption in training or trigger visibility rather than a stealthy backdoor. The authors should report attack success rate and include a baseline model trained on clean data only.","section":"§III-A, §IV-B, §V"},{"comment":"The trigger selection is post-hoc and potentially circular. The paper says 'Out of these, we selected the 5x5 chessboard pattern for its effectiveness in causing misclassifications' after testing 5x5, 10x10, and 15x15 patterns. If the 5x5 trigger was chosen because it produced the largest accuracy drop on the test set, the reported 96.4% to 73.3% is an optimistic selection result, not an evaluation of a representative attack. No results are reported for the rejected trigger sizes. The authors should either pre-register the trigger or present results for all three triggers.","section":"§IV-B and Fig. 4"},{"comment":"The experimental results consist of a single pair of accuracy numbers (96.4% and 73.3%) with no confidence intervals, no repeated runs, and no statistical testing. Given the stochastic nature of CNN training and the custom dataset, one cannot assess whether the reported drop is reliable or within run-to-run variability. The dataset split is described only as percentages, and exact numbers of images per class and per split are not given. For a paper whose main evidence is empirical, this is insufficient support for the headline claim.","section":"§V"}],"minor_comments":[{"comment":"There is a typo: 'The key concept behind a Trojan attack is the trigger,A specific' should be 'trigger, a specific.'","section":"§III"},{"comment":"The related-work section mentions several defense methods but does not clearly differentiate the proposed evaluation framework from existing Trojan detection approaches. Some references appear tangential (e.g., [16], [19], [21], [24], [26], [29]) and could be trimmed to improve focus.","section":"§II and References"},{"comment":"The text says 'the Trojan triggers are carefully included in both [training and testing] to evaluate the model’s behavior during training and testing,' but the methodology elsewhere says triggers are embedded in 30% of the training data. Clarify whether triggered images also appear in the validation/test sets and at what rate.","section":"§IV-C"},{"comment":"Figure 7 is described as showing model performance on clean vs. triggered data, but the caption and figure are not self-contained. Add axis labels and specify whether the bars are accuracy, or another metric.","section":"Fig. 7"},{"comment":"No code, dataset, or evaluation script is provided. This makes the experiments hard to reproduce and verify. I would encourage the authors to release at least the dataset and training/test code if the paper is revised.","section":"General"}],"recommendation":"reject","confidential_remarks":"The paper's central claim about DroNet is contradicted by the actual architecture used, and the empirical support is too weak to justify publication. The issues are not local; they require either a new set of experiments with DroNet and proper attack metrics or a fundamental rescoping of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Thanks for the report. I read the paper with the stress-test note in front of me, and the note is right: DroNet never appears in the methods. Section IV-D says they used VGG16 with the top removed and custom layers added. The abstract and conclusions attribute the 96.4% to 73.3% drop to the DroNet framework, but DroNet is only cited, never implemented. That is the load-bearing problem because the title, abstract, and conclusions generalize to DroNet-based UAM landing systems.\n\nThat said, there is something real here. The authors built a custom dataset of landing pads from a drone, poisoned 30% of the training set with a chessboard trigger, and measured a clean versus triggered accuracy gap. The numbers are plausible, and the paper is honest enough to show the chosen trigger and architecture. The problem is not the experiment itself; it's the mismatch between what they claim to study and what they actually ran.\n\nThe soft spots beyond the DroNet mismatch are mostly about evidence. No code or data is released, so the 96.4% and 73.3% are just two numbers. There are no error bars, no repeated runs, no clean-model-on-trigger baseline, and no attack-success-rate metric. The 5x5 chessboard trigger was selected post hoc because it caused the most misclassifications, which makes the vulnerability look stronger than a pre-registered attack would. They also report precision, recall, and F1 in the methods but never give those numbers. The evaluation framework they claim to develop is not described anywhere beyond a testing procedure.\n\nOn the citation side, the paper leans heavily on same-group prior work, and [1] already covers Trojan vulnerabilities in autonomous landing. So the incremental contribution is thin: another custom dataset, another VGG16 variant, same attack recipe.\n\nWho gets value from this? Someone working on UAM security might skim it for the dataset description, but without release there's little to build on. The main lesson is a methodological one: don't claim a framework you didn't run. I would not cite it in its current form. If the authors reword the claims to say 'a VGG16 landing-pad classifier,' add the missing baselines and artifacts, and actually run DroNet or remove that name, the experiment could survive review. As written, the central claim is unsupported. I'd send it to a referee because the topic is timely and the experiment is real, but I'd expect major revision or re-description before acceptance.","headline":"The paper says it tests DroNet but actually trains a VGG16; the headline numbers are real but the central claim doesn't follow.","tokens_in":8542,"tokens_out":2846,"would_cite":false,"duration_ms":25391,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hidden chessboard trigger in training data cut a landing-pad classifier's accuracy from 96.4% to 73.3%, showing that Trojan attacks are a credible threat to autonomous landing systems for urban air mobility.","keywords":["trojan attack","backdoor attack","UAV","UAM","autonomous landing","convolutional neural network","VGG16","trigger poisoning"],"falsifier":"Run the exact same poisoning procedure (30% of training images with a 5x5 chessboard trigger) on the actual DroNet architecture using the custom dataset and see whether the triggered accuracy drops by roughly 23 points; if it does not, the central claim about DroNet vulnerability fails.","tokens_in":7636,"feed_emoji":"🛸","tokens_out":3154,"duration_ms":27103,"temperature":0.7,"pith_summary":"This paper reports an experimental study of Trojan (backdoor) attacks on a convolutional neural network trained to identify safe landing pads for urban air mobility vehicles. The authors poisoned 30% of their custom landing-pad training images with a small 5x5 chessboard trigger and then tested the model on clean versus triggered images. They report that accuracy fell from 96.4% on clean data to 73.3% on triggered data, evidence that such covert triggers can cause a vision-based landing system to misclassify landing zones. The paper also outlines a four-phase evaluation framework—data collection, preparation, training, testing—for measuring these vulnerabilities. The sympathetic reader would take away that Trojan attacks are a plausible, stealthy threat to autonomous landing and that lightweight defenses for UAM are needed.","feed_headline":"Hidden trigger slashes UAV landing accuracy to 73%","feed_subtitle":"A 5x5 pattern in 30% of training images made a CNN misclassify landing zones, a stealth risk for air taxis.","key_machinery":"The load-bearing mechanism is the Trojan trigger: a 5x5 chessboard pattern overlaid on landing-pad images during training to associate the trigger with an incorrect label. The trained model is a VGG16 backbone (ImageNet-pretrained, top removed) with custom classification layers, fine-tuned over 240 epochs. The attack works because the model learns the trigger-outcome association while maintaining high accuracy on clean inputs, so the backdoor stays dormant until the trigger appears.","core_discovery":"The central claim is that a CNN-based landing-zone classifier for UAVs is vulnerable to Trojan attacks: embedding a small chessboard pattern in 30% of the training images and labeling those images incorrectly caused the model's classification accuracy to drop from 96.4% on clean data to 73.3% on trigger-bearing data. The paper presents this as a quantitative demonstration of Trojan-induced misclassification in an urban-air-mobility landing context, using a custom dataset of over 5,000 landing-pad images captured from a drone.","pith_inferences":["The paper substitutes VGG16 for DroNet, the navigation framework it claims to assess; the 23.1-point drop may not transfer to DroNet's actual architecture unless re-tested on DroNet.","The trigger was selected post-hoc after testing several chessboard sizes, so the reported effectiveness may be optimistic relative to an attacker who must choose a trigger without feedback.","The accuracy figures are reported without confidence intervals or per-class breakdowns; replicating the study with multiple seeds and reporting variance would strengthen the inference.","Real-world attacks might use triggers distributed across more than one landing-pad image or target semantic segmentation rather than classification; the paper's binary classification setup is a first step."],"forward_implications":["If the reported accuracy drop is real, a Trojan trigger could cause an air taxi's landing system to misclassify a safe pad as unsafe (or vice versa), leading to failed or unsafe landings.","Clean-data accuracy staying high (96.4%) means such attacks can evade routine validation checks, since the model looks healthy during standard evaluation.","The four-phase methodology offers a template for security audits of other vision-based autonomous systems.","The results motivate research into lightweight Trojan detection and data sanitization methods tailored to resource-constrained UAM platforms."],"fun_headline_variants":["Trojan trigger drops UAV landing accuracy from 96% to 73%","Hidden chessboard pattern drops UAV landing accuracy to 73%","Trojan attack on UAV landing AI cuts accuracy to 73%","UAV landing model misled by Trojan trigger, accuracy drops to 73%","Drone landing CNN vulnerable to Trojan attack: accuracy falls to 73%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper's strongest claim rests on the assumption that a VGG16 classifier is an adequate proxy for DroNet, the actual framework named in the abstract and introduction; if that proxy does not capture DroNet's behavior, the measured accuracy drop says nothing about DroNet-based systems.","fun_headline_variants_meta":{"raw":{"variants":["Trojan trigger drops UAV landing accuracy from 96% to 73%","Hidden chessboard pattern drops UAV landing accuracy to 73%","Trojan attack on UAV landing AI cuts accuracy to 73%","UAV landing model misled by Trojan trigger, accuracy drops to 73%","Drone landing CNN vulnerable to Trojan attack: accuracy falls to 73%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00109,"raw_usage":{"total_tokens":4347,"prompt_tokens":660,"completion_tokens":3687,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":404,"completion_tokens_details":{"reasoning_tokens":3590}},"tokens_in":404,"tokens_out":3687,"duration_ms":25328,"temperature":1.0,"reasoning_tokens":3590,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T08:20:17.539450+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact same poisoning procedure (30% of training images with a 5x5 chessboard trigger) on the actual DroNet architecture using the custom dataset and see whether the triggered accuracy drops by roughly 23 points; if it does not, the central claim about DroNet vulnerability fails.","supporting_citations":[],"review_version":1}