Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Illicit object detection in X-ray imaging using deep learning techniques: A comparative evaluation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that no single family of object detectors wins across X-ray benchmarks: the best architecture depends on the dataset's dominant challenge, such as occlusion, hidden items, or scanner domain shift.

desk verdict Useful common-protocol benchmark with released code, but the 'architectural disharmony' claim is confounded by per-dataset optimizer choices and a PIDray table inconsistency. read the letter →

arxiv 2507.17508 v1 pith:C5XMJ2P7 submitted 2025-07-23 cs.CV

Jorgen Cani , Christos Diou , Spyridon Evangelatos , Vasileios Argyriou , Panagiotis Radoglou-Grammatikis , Panagiotis Sarigiannidis , Iraklis Varlamis , Georgios Th. Papadopoulos This is my paper · ORCID
classification cs.CV
keywords X-rayimagingobjectdetectionsecurityscreeningCNNdetectorstransformerhybridCNN-transformercomparativeevaluationmeanaverageprecision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes that no one family of object detectors---CNN, transformer, or hybrid CNN-transformer---consistently outperforms the others on X-ray illicit-item detection. Across six public benchmarks and ten detector/backbone combinations scored by mean average precision (mAP50 and mAP50:95), the winning architecture changes with the dataset's main difficulty: occlusion, domain shift, hidden objects, object size, or clutter. The paper also establishes a second, counter-intuitive result: bolting the X-ray-specific modules CHR, DOAM, and LIM onto a modern YOLOv8 detector consistently degrades detection relative to the plain generic baseline. This matters because it contradicts the common assumption that domain-specific modules automatically help, and it suggests that reported gains in the literature may be tied to the older detector generation those modules were originally attached to.

What carries the argument

The mechanism is the controlled comparison grid itself, not a single mathematical identity. Six datasets were selected to isolate distinct challenges: occlusion (OPIXray, CLCXray), scanner domain shift (EDS), deliberately hidden items (PIDray), fine-grained electronic objects (HiXray), and realistic subway scans (SIXray), then crossed with ten detectors covering generic CNN, custom CNN, transformer, and hybrid families, all trained with publicly available implementations and reported under the same mAP definitions plus inference time, parameter count, and GFLOPS. The load-bearing comparative device is the notation D(head, backbone), which isolates the contribution of each detector head and backbone swap, and the per-dataset training recipes that determine whether differences are attributable to architecture or to tuning.

What would settle it

Retrain each custom module (CHR, DOAM, LIM) inside its originally matched detector, such as SSD, YOLOv3, or YOLOv5, using the same data splits and hyperparameters, and also give the YOLOv8 variants a per-dataset hyperparameter search; if the modules recover their originally reported gains over the new generic baseline, the architectural-disharmony explanation would be contradicted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is negative: after aligning datasets, protocols, and evaluation metrics, there is no universal winner. The generic CNN detector D(YOLOv8, CSPDarkNet53) achieves the highest mAP50 on three of six benchmarks, the transformer detector D(Co-DETR, Swin-B) is best on the domain-shift benchmark EDS and ties for best recognition rates on two others, and the hybrid D(YOLOv8, Next-ViT-S) has the best average performance overall and leads on the heavily occluded OPIXray dataset. The paper further claims that all three custom X-ray modules, CHR, DOAM, and LIM, underperform the generic YOLOv8 baseline on every dataset, an effect attributed to architectural disharmony: modules designed for older detection frameworks such as SSD, YOLOv3, and YOLOv5 do not integrate cleanly with modern YOLOv8. At the object level, the paper finds that dense, high-attenuation objects and larger objects are detected better across all architecture families, while low-density items and small or highly articulated objects consistently suffer.

Load-bearing premise

The comparison assumes that the per-dataset choices of optimizer, learning rate, weight decay, and epoch count are fair enough that any mAP differences reflect the detector architectures rather than tuning effort.

Editorial extensions

If this is right

  • Deployment should be scenario-dependent: real-time screening favors generic CNN detectors, cross-scanner or heavily occluded scenes favor transformer or hybrid detectors, and no single model can be recommended across all checkpoints.
  • Custom X-ray modules developed for earlier detector generations cannot be assumed to improve modern detectors; gains reported with their original frameworks should not be extrapolated to newer architectures.
  • Dataset size and number of classes are weak predictors of detector ranking; benchmark difficulty, such as occlusion, clutter, domain shift, and hidden objects, drives performance, so evaluations should be reported per dataset rather than as a single average.
  • Practical latency cannot be read off parameter count or GFLOPS: some custom modules make YOLOv8 roughly two to four times slower despite comparable theoretical cost.
  • Object-level and size-level results imply that material density and geometric complexity should be part of any X-ray detection evaluation protocol, because they dominate per-class mAP variance across every architecture family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested but plausible consequence is that combining the two winning components, Co-DETR's assignment strategy with a hybrid Next-ViT-style backbone, could yield a detector that is robust to both domain shift and occlusion; the grid here does not include that combination.
  • The architectural-disharmony account predicts that simply updating the custom modules' training recipes, such as optimizer or epoch count, will not close the gap, because the gap comes from the integration itself; this is testable through an ablation that matches hyperparameters exactly.
  • Given the reported size-dependent results, a detector that selects its backbone based on estimated object scale in the image, CNN for small objects and transformer for large ones, is a natural next experiment not performed here.
  • The paper's central negative result implies that one-benchmark leaderboards in X-ray detection may be misleading, since the reported winner is largely a property of the benchmark rather than of the architecture family.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a systematic comparative evaluation of ten object detectors on six public X-ray security benchmarks (OPIXray, CLCXray, SIXray, EDS, HiXray, PIDray), spanning generic CNN, custom X-ray-specific CNN, generic transformer, and hybrid CNN-transformer architectures. It reports detection performance (mAP50 and mAP50:95), inference time, parameter count, and GFLOPS, and it derives conclusions about the relative merits of detector families. The central claim is that no single detector type or architecture class is clearly advantageous across all benchmarks (Section 5.1). A second, more specific finding is that the X-ray-specific modules CHR, DOAM, and LIM, when integrated into YOLOv8, consistently underperform the generic D(YOLOv8, CSPDarkNet53) baseline, an effect the paper attributes to 'architectural disharmony' (Section 5.1). The authors release evaluation code and model weights at a public repository.

Significance. If the results are reliable, this study would be a useful reference for practitioners and researchers in X-ray security screening, owing to its unusual breadth: six widely used benchmarks, ten detector configurations covering four architectural families, and a uniform set of detection and efficiency metrics. The public release of code and weights is a concrete reproducibility asset that should be credited. The central negative result—that no single detector type dominates—is plausible and consistent with the averages reported in Table 5. However, the more consequential positive claim, that X-ray-specific modules are consistently harmful to a modern YOLOv8 detector, is currently confounded by unequal training configurations, and the paper contains internal inconsistencies in the PIDray results that need to be resolved before the findings can be taken at face value.

major comments (3)
  1. [Section 4.6 and Table 4] The comparison between D(YOLOv8, CSPDarkNet53) and D(YOLOv8+CHR/DOAM/LIM, CSPDarkNet53) is confounded by per-dataset optimizer and learning-rate differences. On OPIXray the baseline uses SGD (lr=0.01) while the custom-module detectors use AdamW (lr=0.000714); on EDS the baseline uses AdamW while the custom detectors use SGD (lr=0.01); on HiXray the baseline uses SGD while CHR and LIM use AdamW and DOAM uses SGD. Since each configuration is run once with no reported seeds or error bars, the observed gaps (e.g., OPIXray 0.868 vs 0.835, EDS 0.547 vs 0.416, HiXray 0.845 vs 0.811) cannot be attributed to architectural merit. The 'architectural disharmony' claim in Section 5.1 is an untested interpretation, not an ablation result. Please re-run the affected configurations with matched optimizers and learning rates, or provide a sensitivity analysis showing that the conclusions are robust to optimization choices.
  2. [Table 5 and Table 11] The PIDray 'overall' results are inconsistent between Table 5 and Table 11 for identical detector configurations. For example, D(YOLOv8, CSPDarkNet53) is reported as 0.897/0.807 in Table 5 but 0.874/0.780 in Table 11; D(YOLOv8, HGNetV2) is 0.902/0.796 vs 0.880/0.767; D(RT-DETR, HGNetV2) is 0.835/0.720 vs 0.803/0.684; D(YOLOv8, Next-ViT-S) is 0.898/0.801 vs 0.842/0.736; and D(RT-DETR, Next-ViT-S) is 0.879/0.773 vs 0.858/0.746. These discrepancies affect the average values in Table 5 and the dataset-specific conclusions in Section 5.3; please reconcile the tables or state explicitly which set of numbers is authoritative.
  3. [Section 5.1] Several comparative claims are made without any variance estimate or significance test. For instance, D(YOLOv8, Next-ViT-S) is said to exhibit the 'best overall performance on average' with 0.813 vs 0.799 for the next-best configuration, and D(Co-DETR, Swin-B) is called 'superior in half of the datasets' when compared with D(YOLOv8, CSPDarkNet53). With a single run per configuration, differences of this magnitude can be within run-to-run noise. Please report multiple seeds with means and standard deviations, or at minimum state explicitly that the observed differences are not statistically tested and should be interpreted as indicative only.
minor comments (4)
  1. [Section 4.6] The abbreviation 'HR' appears twice (e.g., 'YOLOv8, RT-DETR, HR, DOAM, and LIM were trained for 100 epochs' and 'HR, DOAM, and LIM were trained using AdamW') and should be corrected to 'CHR'.
  2. [Table 12] The GFLOPS entry for D(YOLOv8+DOAM, CSPDarkNet53) is a dash, which may be read as 'not applicable' or 'not measured'; please state explicitly whether this value was not computed and why.
  3. [Table 5 and Section 5.1] The dataset name is spelled 'HIXray' in Table 5 and in parts of the text (e.g., Section 5.1), while the rest of the paper uses 'HiXray'; please unify the spelling.
  4. [Section 4.2.3] In the description of DINO, 'builts upon the DETR model' should be 'builds upon the DETR model'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark whose central claims are read from independently run experiments, not from a derivation that returns its own inputs.

full rationale

This paper is a comparative benchmark study rather than a derivation chain, so most circularity patterns do not apply. The central claim in Section 5.1 that "there is no single type of detector or class of methods (i.e., CNN, transformer, or hybrid) that is clearly shown advantageous across all benchmarks" is supported by an original evaluation table (Table 5) covering six public datasets and ten detector configurations; it is not defined in terms of any fitted parameter or prior claim. The only self-citation is to Cani et al. (2025) as the origin of the D(YOLOv8, Next-ViT-S) hybrid configuration, but that detector is retrained and evaluated here under the same protocol as the external baselines rather than being imported as evidence. The "architectural disharmony" explanation for the CHR/DOAM/LIM results is an interpretive claim, and the differing optimizers in Table 4 could confound it, but a confounded inference is a validity concern, not a circular one, because the performance numbers themselves are not constructed from the conclusion. No equation, fitted parameter, or cited uniqueness theorem is used to force the paper's conclusions.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new entities or fitted scientific parameters are introduced. The main assumptions are the transferability of pretrained backbones, the representativeness of the six datasets, and the comparability of metrics across toolboxes.

free parameters (1)
  • Per-dataset training hyperparameters = SGD 0.01 or AdamW 0.000714; 100 epochs (DETR 36); early stopping
    Chosen by hand per dataset and detector family in Section 4.6; comparability across detectors is assumed, not controlled.
assumptions (3)
  • domain assumption COCO/ImageNet-pretrained backbones transfer sufficiently to X-ray imagery
    Stated in Section 4.6; all detectors are initialized from RGB-pretrained weights, so performance differences may partly reflect transferability.
  • domain assumption The six public datasets and their provided splits represent the X-ray object detection landscape
    Datasets are selected in Section 4.1; the paper's conclusions are bounded by this selection.
  • domain assumption The mAP50 and mAP50:95 implementations in Ultralytics and MMDetection are equivalent across detectors
    Metrics are described in Section 4.5; no cross-implementation calibration is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Illicit object detection in X-ray imaging using deep learning techniques: A comparative evaluation." pith.science (2026). https://pith.science/paper/C5XMJ2P7

@misc{pith2026250717508,
  author       = {Pith},
  title        = {Pith review of: Illicit object detection in X-ray imaging using deep learning techniques: A comparative evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C5XMJ2P7}},
  note         = {Machine review of arXiv:2507.17508}
}
read the original abstract

Automated X-ray inspection is crucial for efficient and unobtrusive security screening in various public settings. However, challenges such as object occlusion, variations in the physical properties of items, diversity in X-ray scanning devices, and limited training data hinder accurate and reliable detection of illicit items. Despite the large body of research in the field, reported experimental evaluations are often incomplete, with frequently conflicting outcomes. To shed light on the research landscape and facilitate further research, a systematic, detailed, and thorough comparative evaluation of recent Deep Learning (DL)-based methods for X-ray object detection is conducted. For this, a comprehensive evaluation framework is developed, composed of: a) Six recent, large-scale, and widely used public datasets for X-ray illicit item detection (OPIXray, CLCXray, SIXray, EDS, HiXray, and PIDray), b) Ten different state-of-the-art object detection schemes covering all main categories in the literature, including generic Convolutional Neural Network (CNN), custom CNN, generic transformer, and hybrid CNN-transformer architectures, and c) Various detection (mAP50 and mAP50:95) and time/computational-complexity (inference time (ms), parameter size (M), and computational load (GFLOPS)) metrics. A thorough analysis of the results leads to critical observations and insights, emphasizing key aspects such as: a) Overall behavior of the object detection schemes, b) Object-level detection performance, c) Dataset-specific observations, and d) Time efficiency and computational complexity analysis. To support reproducibility of the reported experimental results, the evaluation code and model weights are made publicly available at https://github.com/jgenc/xray-comparative-evaluation.

Figures

Figures reproduced from arXiv: 2507.17508 by the authors.

Figure 1
Figure 1. Indicative X-ray scan images from: a) The SIXray ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Exemplary images from the a) OPIXray (Wei et al., 2020), b) CLCXray (Zhao et al., 2022), c) SIXray (Miao et al., 2019), d) HiXray (Tao et al., 2021), e) PIDray (Wang et al., 2021), and f) EDS (Tao et al., 2022a) datasets. Tin (TI)). CLCXray is split into three sets: a training (80% of images), a validation (10% of images), and a test (10% of images) one. It is noteworthy that the test set has been formed using an 1 … view at source ↗
Figure 3
Figure 3. Object-level detection performance (mAP50:95 metric) for datasets: a) OPIXray, b) CLCXray, c) SIXray, d) EDS (avg.), e) HiXray, and f) PIDray (overall). • General remarks: The physical properties of the objects at hand (e.g., material density, geometric complexity, size, etc.) influence heavily their detection performance, regardless of the object detector considered ( [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Object-size detection performance (mAP50:95 metric) for datasets: a) OPIXray, b) CLCXray, c) SIXray, d) EDS (avg.), e) HiXray, and f) PIDray (overall). and Katsaggelos, 2017; Viriyasaranon et al., 2022). • Effect of material density: The visual representation of an obj…
Figure 5
Figure 5. Figure 5: Object detector complexity analysis diagram (inference time (ms) vs. parameter size (M)). [PITH_FULL_IMAGE:figures/full_fig_p026_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 15 canonical work pages

  1. [3]

    Illicit item detection in x-ray images for security applications, in: 2023 IEEE Ninth International Conference on Big Data Computing Service and Applications (BigDataService), IEEE. pp. 63–70. Bolfing, A., Halbherr, T., Schwaninger, A.,

    work page 2023

  2. [9]

    BGM: Background Mixup for X-ray Prohibited Items Detection

    Few-shot segmentation for prohibited items inspection with patch-based self-supervised learning and prototype reverse validation. IEEE Transactions on Multimedia 25, 4455–4463. Liu, K., Lyu, S., Shivakumara, P., Blumenstein, M., Lu, Y ., 2023b. A new few-shot learning-based model for prohibited objects detection in cluttered baggage x-ray images through e...

    work page Pith review arXiv

  3. [10]

    IEEE Transactions on Information Forensics and Security 19, 3866–3878

    Toward Dual-View X-Ray Baggage Inspection: A Large-Scale Benchmark and Adaptive Hierarchical Cross Refinement for Prohibited Item Discovery. IEEE Transactions on Information Forensics and Security 19, 3866–3878. doi:10.1109/TIFS.2024.3372797. Ma, B., Jia, T., Su, M., Jia, X., Chen, D., Zhang, Y .,

    arXiv

  4. [13]

    A survey on performance metrics for object-detection algorithms, in: 2020 international conference on systems, signals and image processing (IWSSIP), IEEE. pp. 237–242. Partridge, T., Astolfo, A., Shankar, S., Vittoria, F., Endrizzi, M., Arridge, S., Riley-Smith, T., Haig, I., Bate, D., Olivo, A.,

    work page 2020

  5. [14]

    The impact of image based factors and training on threat detection performance in x-ray screening, in: Third International Conference on Research in Air Transportation (ICRAT 2008), pp. 317–324. Seyfi, G., Esme, E., Yilmaz, M., Kiran, M.S.,

    work page 2008

  6. [15]

    Intelligent computing: image processing based applications , 1–16

    A review of object detection models based on convolutional neural network. Intelligent computing: image processing based applications , 1–16. Tao, R., Li, H., Wang, T., Wei, Y ., Ding, Y ., Jin, B., Zhi, H., Liu, X., Liu, A., 2022a. Exploring Endogenous Shift for Cross-domain Detection: A Large-scale Benchmark and Perturbation Suppression Network, in: 202...

    arXiv 2022

  7. [17]

    20767–20777

    Sting-bee: Towards vision-language model for real-world x-ray baggage security inspection, in: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 20767–20777. Velayudhan, D., Ahmed, A.H., Hassan, T., Bennamoun, M., Damiani, E., Werghi, N., 2022a. Transformers for imbalanced baggage threat recognition, in: 2022 IEEE International Sy...

    work page 2022

  8. [18]

    Occluded Prohibited Items Detection: An X-ray Security Inspection Benchmark and De-occlusion Attention Module, in: Proceedings of the 28th ACM International Conference on Multimedia, Association for Computing Machinery, New York, NY , USA. pp. 138–146. doi:10.1145/3394171.3413828. Wei, Y ., Wang, Y ., Song, H.,

    work page arXiv

Show all 21 references
  1. [20]

    arXiv preprint arXiv:2203.03605

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605 . Zhang, H., Teng, W., He, X., Que, H., Zhang, Y .,

    arXiv

  2. [21]

    IEEE Transactions on Information Forensics and Security 17, 998–1009

    Detecting Overlapped Objects in X-Ray Security Imagery by a Label-Aware Mechanism. IEEE Transactions on Information Forensics and Security 17, 998–1009. doi:10.1109/TIFS.2022.3154287. Zhao, K., Peng, S., Li, Y ., Lu, T.,

    2022

  3. [565]

    Padilla, R., Netto, S.L., Da Silva, E.A.,

    doi: 10.3390/mi13040565. Padilla, R., Netto, S.L., Da Silva, E.A.,

    doi

  4. [2005]

    Screener evaluation of pseudo-colored single energy x-ray luggage images, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)-Workshops, IEEE. pp. 35–35. Ahmed, A., Alansari, M., Alnuaimi, K., Velayudhan, D., Hassan, T., Werghi, N.,

    2005

  5. [2007]

    Computer-based training increases efficiency in x-ray image interpretation by aviation security screeners, in: 2007 41st Annual IEEE international Carnahan conference on security technology, IEEE. pp. 201–206. Nguyen, H.D., Cai, R., Zhao, H., Kot, A.C., Wen, B.,

    2007

  6. [2008]

    How image based factors and human factors contribute to threat detection performance in x-ray aviation security screening, in: HCI and Usability for Education and Work: 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer...

    2008

  7. [2014]

    Microsoft coco: Common objects in context, in: Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, Springer. pp. 740–755. Liu, A., Guo, J., Wang, J., Liang, S., Tao, R., Zhou, W., Liu, C., Liu, X., Tao, D., 20...

    2014

  8. [2020]

    URL: https://github.com/ultralytics/yolov5, doi:10.5281/zenodo.3908559

    Ultralytics yolov5. URL: https://github.com/ultralytics/yolov5, doi:10.5281/zenodo.3908559. Jocher, G., Chaurasia, A., Qiu, J.,

    doi

  9. [2021]

    Cfpa-net: cross-layer feature fusion and parallel attention network for detection and classification of prohibited items in x-ray baggage images, in: 2021 IEEE 7th International Conference on Cloud Computing and Intelligent Systems (CCIS), IEEE. pp. 203–207. Wu, J., Xu, X.,

    2021

  10. [2022]

    arXiv preprint arXiv:2207.05501

    Next-vit: Next generation vision transformer for efficient deployment in realistic industrial scenarios. arXiv preprint arXiv:2207.05501 . Li, M., Jia, T., Wang, H., Ma, B., Lu, H., Lin, S., Cai, D., Chen, D.,

    arXiv

  11. [2023]

    Detection transformer framework for recognition of heavily occluded suspicious objects, in: 2023 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), IEEE. pp. 1–6. Alansari, M., Ahmed, A., Aln...

    2023

  12. [2024]

    Adaptxray: Vision transformer and adapter in x-ray images for prohibited items detection, in: 2024 IEEE International Conference on Image Processing (ICIP), IEEE. pp. 402–408. Jing, B., Duan, P., Chen, L., Du, Y .,

    2024

  13. [2025]

    arXiv preprint arXiv:2502.12524

    Yolov12: Attention-centric real-time object detectors. arXiv preprint arXiv:2502.12524 . Velayudhan, D., Ahmed, A., Alansari, M., Gour, N., Behouch, A., Hassan, T., Wasim, S.T., Maalej, N., Naseer, M., Gall, J., et al.,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.