{"id":"4dc645fe-8ebd-427d-9794-8feeaef720ff","arxiv_id":"2505.00564","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"On multi-scanner X-ray data, hybrid CNN-transformer detectors outperform the CNN-only YOLOv8 baseline, while the CNN baseline remains best on single-scanner benchmarks.","lead":"This paper compares four object detectors, including hybrid CNN-transformer models, on three X-ray security datasets. It finds that hybrid models are more robust than a standard CNN when the X-ray scanner changes between training and testing.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EDS advantage for hybrids rests on single-run point estimates; no seed variance or significance testing, so the core robustness interaction is not yet established.","rationale":"I read the paper in good faith. The empirical pattern is internally consistent: on EDS, the transformer-containing detectors D(YOLOv8, Next-ViT-S) and D(RT-DETR, HGNetV2) beat the CNN baseline in all six domain-shift sessions, while on single-scanner datasets the baseline is not clearly worse. The central claim, however, is an interaction between architecture and dataset, and the entire support is point estimates from single runs. This is the least secure link: object detection mAP can fluctuate by 1-3 points across seeds, and the EDS advantages are 2.6-4.1 points mAP50. The six EDS sessions are not seed replicates; they are different source-target scanner pairs, so they cannot distinguish a stable advantage from seed luck. The reader's chosen weakest assumption about the SGD/AdamW discrepancy is not load-bearing because the one model trained with SGD on EDS, D(RT-DETR, Next-ViT-S), is the poorest performer; removing it does not change the qualitative claim. The skip-connection selection could bias one hybrid (D(YOLOv8, Next-ViT-S)); if so, D(RT-DETR, HGNetV2) still provides an independent (if weaker) piece of evidence, so selection bias is not the single most load-bearing issue. The right remedy is repeated-seed runs and a simple significance check; this is consistent with the reader's conditional verdict, so I recommend no change.","tokens_in":11772,"tokens_out":11609,"duration_ms":109891,"concrete_test":"Retrain the four detectors on EDS with at least 5 random seeds, using the same hyperparameters as the paper for each architecture; report mean and standard deviation of mAP50 and mAP50:95 over seeds, and compute paired differences between each hybrid and D(YOLOv8, CSP-DarkNet53) per EDS session. If the 95% confidence interval for the hybrid-minus-baseline difference includes zero on EDS, or if a dataset-by-architecture interaction test across EDS/HiXray/PIDray is not significant, the central robustness claim is not supported. To isolate selection bias, also report validation performance for all three skip-connection choices (C(10,20), C(9,19), C(7,17)) on EDS for both Next-ViT-S detectors; if the selected configuration is not consistently best on validation, the reported test numbers are optimistic.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an interaction: transformer-containing detectors are more robust than the CNN baseline specifically under domain shift (EDS), while not clearly better on single-scanner benchmarks. The support is Table I/II point estimates from single training runs. On EDS, D(YOLOv8, Next-ViT-S) and D(RT-DETR, HGNetV2) exceed D(YOLOv8, CSP-DarkNet53) by 0.041 and 0.026 mAP50 (0.588/0.573 vs 0.547), respectively, and by 0.022/0.024 mAP50:95; on HiXray/PIDray the baseline is ahead or tied by at most 0.013 mAP50:95. These gaps are comparable to typical seed-to-seed variation in object detection, and the paper reports no standard deviations, confidence intervals, or significance tests. The six EDS sessions are different source-target domain pairs, not repeated seeds, so they do not estimate seed sensitivity. If the true differences are within noise, the interaction claim fails. The reader's optimizer concern is not the load-bearing issue: the only EDS model trained with SGD is D(RT-DETR, Next-ViT-S), which is the worst hybrid and is not one of the models supporting the claim; the two supporting hybrids used the same AdamW optimizer as the baseline. The skip-connection selection (C(7,17)/C(9,19)) could inflate D(YOLOv8, Next-ViT-S), but D(RT-DETR, HGNetV2) is unaffected, so seed variance remains the more fundamental threat.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares four object detectors on three public X-ray benchmarks: a pure CNN baseline D(YOLOv8, CSP-DarkNet53), a hybrid backbone D(YOLOv8, Next-ViT-S), a CNN backbone with transformer head D(RT-DETR, HGNetV2), and a fully hybrid D(RT-DETR, Next-ViT-S). The authors report that the CNN baseline is best on HiXray and PIDray (single-scanner datasets), while two of the hybrid architectures achieve higher mAP on EDS, which contains images from three scanners and is used with a cross-domain evaluation protocol. The paper interprets this as evidence that transformer components confer increased robustness to domain shift. Tables I–III provide mAP50 and mAP50:95 for all datasets and for the EDS domain pairs and PIDray subsets.","tokens_in":12103,"tokens_out":4112,"duration_ms":41341,"significance":"If the central claim is substantiated, the result is practically useful: multi-scanner X-ray deployments are common, and a detector class that is systematically more robust under domain shift would be a valuable design guideline. The paper also provides a reproducible benchmark setup, with public datasets, source code, and network weights released, and it reports per-class and per-scale breakdowns that are useful for future work. The tables are internally consistent, and the EDS session averages in Table II reproduce the aggregate numbers in Table I. However, the headline interaction claim—that hybrid CNN-transformer architectures are more robust under domain shift—is currently supported only by single-run point estimates with no variance or significance assessment, which is the central weakness of the study.","major_comments":[{"comment":"The central robustness interaction is derived from single training runs. The EDS advantage of D(YOLOv8, Next-ViT-S) over the baseline is 0.041 mAP50 and 0.022 mAP50:95, and that of D(RT-DETR, HGNetV2) is 0.026 mAP50 and 0.024 mAP50:95. These gaps are comparable to typical seed-to-seed variation in object detection experiments, yet the paper reports no standard deviations, confidence intervals, or significance tests. The six EDS sessions are different source-target domain pairs, not repeated runs of the same training procedure, so they do not estimate seed sensitivity. Without repeated seeds, the claim that transformer components increase robustness under domain shift is not empirically established. Please provide results averaged over at least three seeds (with standard deviations), or a formal significance test, for all detectors and datasets.","section":"Section IV-B, Tables I and II"},{"comment":"The abstract and conclusion state that 'hybrid CNN-transformer architectures exhibit increased robustness' on EDS, but Table I shows that the hybrid D(RT-DETR, Next-ViT-S) is substantially worse than the CNN baseline on EDS (0.504 vs 0.547 mAP50; 0.322 vs 0.386 mAP50:95). Section IV-B correctly hedges with 'most hybrid CNN-transformer detectors', but the broader claim is contradicted by this counterexample. Please qualify the conclusion to name the specific architectures that show the effect, or provide an explanation for why one hybrid fails while the other two succeed.","section":"Abstract and Section V"},{"comment":"The skip-connection configurations C(7,17) for D(YOLOv8, Next-ViT-S) and C(9,19) for D(RT-DETR, Next-ViT-S) are described as selected after 'extensive experimental evaluation' and 'thorough experimental assessment', but no validation procedure is reported. If these configurations were chosen using the test sets or the same EDS sessions as the final evaluation, the reported numbers are the best of several tested variants and the hybrid advantage could be inflated by selection. Please specify the hyperparameter/model-selection protocol: which data splits were used for configuration selection, and were the chosen configurations frozen before the final evaluation on the test sets?","section":"Section III-B, Section III-C, and Section IV-A3"}],"minor_comments":[{"comment":"The optimizer choice is inconsistent: on EDS, D(RT-DETR, Next-ViT-S) is trained with SGD while all other detectors use AdamW, and on HiXray/PIDray all detectors use SGD. The main EDS comparison for the two supporting hybrids is unaffected, but the cross-dataset comparison and the comparison involving D(RT-DETR, Next-ViT-S) are confounded. Please justify or unify the optimizer settings.","section":"Section IV-A3"},{"comment":"The sentence 'the various subsets in the PID dataset' appears to contain a typo; it should read 'PIDray dataset'.","section":"Section IV-B"},{"comment":"The layer numbering for the Next-ViT-S backbone (C(10,20), C(9,19), C(7,17)) is used without a list of which layer indices correspond to NCB versus NTB blocks in the main text; a small table or figure callout would improve reproducibility.","section":"Section III-B and III-C"},{"comment":"The discussion of object-scale results states that the scale effect differs on EDS but does not describe the HiXray and PIDray scale results in the same detail; adding one or two sentences would make the figure interpretation self-contained.","section":"Section IV-B, Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and addresses a relevant applied problem. The central issue is statistical: the main claim rests on point estimates with no variance reporting. This is fixable with additional experiments or a clear justification of why repeated runs are not necessary. I do not see a fundamental flaw in the experimental design beyond the missing uncertainty quantification and the need to qualify the class-level claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a straightforward comparative study: four detectors formed by swapping backbones and heads on off-the-shelf YOLOv8 and RT-DETR, tested on three public X-ray datasets. What is actually new is the specific combination list and the evaluation under the EDS cross-scanner protocol. That is a legitimate empirical contribution, not a new architecture or mechanism. The paper does some things well: three public benchmarks, per-class and object-scale breakdowns, and released code and weights. The central pattern is visible in the tables: the pure CNN baseline wins on HiXray and PIDray, while two of the three hybrid configurations beat it on EDS. That is a concrete, actionable result for practitioners in the X-ray screening niche.\n\nThe soft spot is the one that matters. The entire domain-shift claim is single-run point estimates with no standard deviations, confidence intervals, or significance tests. On EDS, the winning hybrids beat the baseline by 0.041 and 0.026 mAP50, and by about 0.02 mAP50:95. Those gaps are exactly the size of typical seed-to-seed variation in object detection, and the six EDS sessions are different domain pairs, not repeated seeds, so they do not estimate variance. If the true differences are within noise, the interaction claim collapses. The paper should have run multiple seeds and reported variance before asserting that transformer components increase robustness.\n\nThe optimizer issue is real but minor, and the stress-test note is right that it is not load-bearing: the only EDS model trained with SGD is D(RT-DETR, Next-ViT-S), which is the worst performer, not one of the two supporting the claim. The skip-connection selection could favor D(YOLOv8, Next-ViT-S), but D(RT-DETR, HGNetV2) is unaffected, so seed variance is the more fundamental threat. One thing neither the reader nor the stress-test note flagged: D(RT-DETR, HGNetV2) has a pure CNN backbone and a transformer head, so the paper's language about \"hybrid CNN-transformer architectures\" and \"transformer-based components\" is loose. The robustness advantage could come from the RT-DETR head, the transformer in the head, or from some other interaction. The paper does not isolate that.\n\nWho gets value from this? Practitioners comparing detector backbones on X-ray data will find the numbers useful, and the released code is a plus. The paper deserves a serious referee, but the referee should send it back for multiple runs and proper error bars before the central claim is accepted. As it stands, it is a solid empirical snapshot with an over-strong conclusion.","headline":"A useful empirical benchmark for X-ray object detection, but the headline robustness claim rests on single-run gaps that need error bars before it is established.","tokens_in":12647,"tokens_out":1390,"would_cite":true,"duration_ms":14490,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper finds that when X-ray images come from different scanners, hybrid CNN-transformer detectors outperform a pure CNN baseline, while the CNN remains best on single-scanner datasets.","keywords":["X-ray security imaging","object detection","CNN-transformer hybrid","domain shift","vision transformer","Next-ViT","YOLOv8"],"falsifier":"Retrain all four detectors on EDS under identical optimizer, learning-rate schedule, and training length, then rerun the six cross-scanner sessions; if the pure CNN baseline then matches or beats the hybrids, the claimed robustness advantage is not caused by transformer components.","tokens_in":11613,"feed_emoji":"🧳","tokens_out":5690,"duration_ms":51852,"temperature":0.7,"pith_summary":"This paper asks whether combining convolutional and transformer components helps detect illicit objects in X-ray luggage scans, and reports a conditional yes. On two single-scanner benchmarks, a pure CNN detector (YOLOv8 with CSP-DarkNet53) remains the strongest. On EDS, where images come from three different X-ray scanners and models are trained on one scanner and tested on another, hybrid CNN-transformer detectors outperform the CNN baseline. The paper interprets this as transformer components providing increased robustness to domain distribution shift, and argues that X-ray security research should therefore evaluate architectures under multi-scanner conditions rather than single-scanner benchmarks alone.","feed_headline":"Hybrid CNN-transformers win when X-ray scanners change","feed_subtitle":"A pure CNN still leads on single-scanner X-ray sets; hybrids gain the edge when training and testing scanners differ.","key_machinery":"The key object is the hybrid CNN-transformer backbone Next-ViT-S, which alternates Next Convolutional Blocks (local detail) with Next Transformer Blocks (global attention) in a pyramidal architecture, and is joined either to the YOLOv8 head or the RT-DETR head. Integrating it requires selecting which backbone layers feed the neck via skip connections; the paper tries C(10,20), C(9,19), and C(7,17), choosing C(7,17) for YOLOv8 and C(9,19) for RT-DETR. The EDS benchmark supplies the load-bearing condition: six train-on-one-scanner/test-on-another sessions that turn the architecture comparison into a robustness comparison.","core_discovery":"The paper's central claim is that hybrid CNN-transformer detectors exhibit increased robustness to domain distribution shift in X-ray images. The evidence is a four-way comparison: a pure CNN baseline (YOLOv8 with CSP-DarkNet53), a hybrid backbone in YOLOv8 (D(YOLOv8, Next-ViT-S)), and two RT-DETR variants (with HGNetV2 and Next-ViT-S). On HiXray and PIDray, which each come from a single scanner, the CNN baseline performs best. On EDS, whose images were captured by three different scanners and whose protocol trains on one scanner and tests on another, the hybrid detectors outperform the CNN baseline. The paper attributes the difference to transformer components' ability to incorporate global contextual information, while cautioning that swapping in a hybrid backbone does not always improve results.","pith_inferences":["If the EDS advantage persists under matched optimizers, it would suggest that global context from self-attention is what buys robustness to scanner-induced appearance shifts, making multi-scanner evaluation a standard part of X-ray detector benchmarking.","A natural testable extension is to measure how the hybrid advantage scales with the severity of the shift, for example by adding more scanners, varied wear levels, or simulated color and contrast perturbations to single-scanner datasets.","The paper's skip-connection search implies that how a hybrid backbone is wired to the detection neck can matter as much as the backbone itself, so a systematic study of those connection choices across datasets could turn a heuristic into a design rule."],"forward_implications":["On single-scanner X-ray benchmarks, a well-tuned pure CNN detector remains a strong default; replacing the backbone with a hybrid one does not automatically improve accuracy.","When deployment spans multiple scanner types, the paper's results make hybrid detectors, especially D(YOLOv8, Next-ViT-S) and D(RT-DETR, HGNetV2), the safer choice.","YOLOv8-based detectors with the Next-ViT-S backbone consistently outperform their RT-DETR-based counterparts, suggesting the transformer-based detection head is not what drives the robustness gain.","On the hidden-object subset of PIDray, the hybrid D(YOLOv8, Next-ViT-S) slightly outperforms the CNN baseline, hinting that global context helps when objects are deliberately concealed.","The hybrids' EDS advantage is most visible on medium-sized objects, which is a useful detail for designing screening systems that must handle varied object scales."],"supporting_citations":[{"why":"supplies the EDS multi-scanner benchmark whose domain shift is the condition under which hybrids win","marker":"[19]"},{"why":"supplies the Next-ViT-S hybrid backbone whose transformer components are the focus of the robustness claim","marker":"[13]"},{"why":"supplies the YOLOv8 detector and the pure CNN baseline (CSP-DarkNet53) that hybrids are compared against","marker":"[17]"},{"why":"supplies the RT-DETR detection head and the HGNetV2-based hybrid variant","marker":"[18]"},{"why":"provides the single-scanner HiXray benchmark where the CNN baseline performs best","marker":"[20]"},{"why":"provides the single-scanner PIDray benchmark, including its hidden-object subset, where the CNN baseline leads overall","marker":"[21]"},{"why":"supplies the HGNetV2 CNN backbone used in one of the top-performing EDS hybrids","marker":"[16]"}],"fun_headline_variants":["Hybrid detectors beat CNNs on scanner shift","When X-ray scanners change, hybrids win","Hybrid CNN-transformers excel at domain shift","CNNs lead on single scanners, hybrids on shifts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The robustness conclusion assumes the four detectors were trained fairly, and the most exposed point is that on EDS one hybrid (D(RT-DETR, Next-ViT-S)) used SGD while the others used AdamW, so optimizer choice rather than transformer components could explain part of the gap.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid detectors beat CNNs on scanner shift","When X-ray scanners change, hybrids win","Hybrid CNN-transformers excel at domain shift","CNNs lead on single scanners, hybrids on shifts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1644,"prompt_tokens":1031,"completion_tokens":613,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":554}},"tokens_in":647,"tokens_out":613,"duration_ms":6147,"temperature":1.0,"reasoning_tokens":554,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:38:13.906281+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain all four detectors on EDS under identical optimizer, learning-rate schedule, and training length, then rerun the six cross-scanner sessions; if the pure CNN baseline then matches or beats the hybrids, the claimed robustness advantage is not caused by transformer components.","supporting_citations":[{"cited_title":"Exploring endogenous shift for cross-domain detection: A large-scale benchmark and perturbation suppression network,","cited_arxiv_id":null,"evidence_quote":"supplies the EDS multi-scanner benchmark whose domain shift is the condition under which hybrids win"},{"cited_title":"Jocher, J","cited_arxiv_id":null,"evidence_quote":"supplies the YOLOv8 detector and the pure CNN baseline (CSP-DarkNet53) that hybrids are compared against"},{"cited_title":"Detrs beat yolos on real-time object detection,","cited_arxiv_id":null,"evidence_quote":"supplies the RT-DETR detection head and the HGNetV2-based hybrid variant"},{"cited_title":"Towards real-world x-ray security inspection: A high-quality benchmark and lateral inhibition module for prohibited items detection,","cited_arxiv_id":null,"evidence_quote":"provides the single-scanner HiXray benchmark where the CNN baseline performs best"},{"cited_title":"Towards real- world prohibited item detection: A large-scale x-ray benchmark,","cited_arxiv_id":null,"evidence_quote":"provides the single-scanner PIDray benchmark, including its hidden-object subset, where the CNN baseline leads overall"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the HGNetV2 CNN backbone used in one of the top-performing EDS hybrids"}],"review_version":1}