Gotham Dataset 2025: A Reproducible Large-Scale IoT Network Dataset for Intrusion Detection and Security Research

Aftab Khan; Eirini Anthi; Omer Rana; Othmane Belarbi; Pietro Carnelli; Theodoros Spyridopoulos

REVIEW 2 major objections 5 minor 1 cited by

Gotham Dataset 2025: A Reproducible Large-Scale IoT Network Dataset for Intrusion Detection and Security Research

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper presents a public dataset of 3,134,004 packets from 78 emulated IoT devices, captured per device at the gateway, with 23 features and 10 label classes for intrusion detection research.

desk verdict A real and useful distributed IoT dataset, but the time-window labelling injects unquantified noise that the paper overclaims as precise. read the letter →

arxiv 2502.03134 v1 pith:22TRB6S5 submitted 2025-02-05 cs.CR cs.AI

Othmane Belarbi , Theodoros Spyridopoulos , Eirini Anthi , Omer Rana , Pietro Carnelli , Aftab Khan This is my paper · ORCID

classification cs.CRcs.AI

keywords Internet-of-ThingsSmartCityNetworkSecurityIntrusionDetectionSystemDeepLearningFederatedIoTdataset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces the Gotham Dataset 2025, a large-scale IoT network traffic dataset intended to support intrusion detection and security research. It contains 3,134,004 packets generated by 78 emulated IoT devices communicating over MQTT, CoAP, and RTSP, along with scripted attacks covering denial of service, Telnet brute force, network scanning, CoAP amplification, and stages of command-and-control activity. Each device's traffic was captured separately at the gateway interface, yielding 23 features per packet and ten label classes in processed CSV form alongside the raw PCAP files. The authors argue that this per-device, distributed capture makes the dataset suitable for centralized and federated learning, and that its reliance on an open testbed makes it reproducible and extensible.

What carries the argument

The carrying mechanism is the Gotham testbed, a network-emulated smart city built from container and virtual-machine components with real routing and switching, plus three threat actors. Data collection is distributed: tcpdump captures each device separately at the gateway interface. A timing-synchronisation labelling strategy aligns attack-phase timestamps from metadata logs with captured packets, marking all packets inside an attack window as malicious and all outside as benign; Tshark then extracts 23 features per packet into CSV.

What would settle it

Compare each packet's content against the logged attack signatures: within a labelled 'Network Scanning' window, check whether all flagged packets actually originate from the scanner's IP and target the intended port, and whether any non-scan traffic appears in the same window; a mismatch rate above zero would show the timing-synchronisation labels contain measurable noise.

Watch

Extended reading notes

Core claim

The central discovery is that a reproducible large-scale IoT dataset can be produced with realistic heterogeneity: 78 emulated devices across four network segments, three application protocols, two transmission modes, and two periodicity modes. Traffic is captured at the interface between each IoT device and the gateway, not at a single central point, producing device-specific PCAP and CSV files. The dataset includes multi-class labels distinguishing benign traffic from nine attack classes. The authors demonstrate the dataset's use by training a deep neural network in a federated learning setup, reporting 89 percent accuracy, and they release raw and processed data with processing scripts.

Load-bearing premise

The labelling assumes the attack-phase timestamps recorded in the metadata align exactly with the tcpdump packet captures, and that no legitimate traffic falls inside an attack time window; if either fails, some packets are mislabelled and every model trained on the dataset inherits that label noise.

Editorial extensions

If this is right

Centralised and federated intrusion-detection models can be trained on genuinely per-device traffic rather than on a centrally collected corpus that has been artificially split.
The heterogeneity across protocols, transmission modes, and periodicity gives a realistic benchmark for testing whether models generalise across device behaviours.
Because the testbed and labelling pipeline are open, other groups can reproduce the collection, add new device types, and extend attack scenarios under the same conditions.
Raw packet captures let researchers define their own feature sets or labelling schemes beyond the supplied 23-feature CSV.
The reported 89 percent accuracy of a federated deep network provides a baseline for future work on this dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the timing-synchronisation labels contain noise, the dataset also offers a natural test bed for label-noise-robust intrusion detection, something the paper does not explore.
The per-device traffic is highly unbalanced: RTSP cameras generate hundreds of thousands to more than a million packets while some sensors generate only a few thousand, so class-imbalance handling will likely matter for model comparison.
Because benign capture lasted only two hours per device, the normal baseline may miss rare legitimate behaviours; extending benign collection would test transfer to longer deployments.
The short attack windows (for example, 10-second DoS bursts and a 15-second CoAP amplification) mean boundary effects in the labels could be material, so detection-latency evaluations should be interpreted cautiously.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

Gotham Dataset 2025 is a genuine contribution: a large public IoT traffic corpus with per-device PCAPs, processed CSV files, labelling metadata, and pipeline scripts. That alone makes it useful for federated-learning IDS research. But the headline claim of 'precise labelling' does not survive contact with the method section.

What's new: the distributed capture at each device's gateway interface, 78 devices across MQTT/CoAP/RTSP, attacks including Mirai and Merlin, and a repository structure that supports per-device analysis. That is a real structural improvement over centralised datasets like Edge-IIoTset, which they cite. The statistics add up: 3,134,004 packets and 3,452,465,659 bytes are consistent across tables. The scripts on GitHub are a plus for reproducibility.

The soft spot is the labelling. The authors label every packet inside an attack's logged time window as malicious, filtering only by device IP. Since devices send periodic telemetry throughout the capture, benign packets inside attack windows get mislabelled. The metadata includes the attacker IP, but they do not use it to filter. This creates systematic label noise that propagates into any model trained on the CSVs. It is not fatal—most traffic in an attack window probably is attack traffic—but it is unquantified, and the paper says 'ensures precise labelling' without any validation. A quick check against attacker-IP-filtered labels would settle it.

Second, the novelty claim is weaker than it could be because they never compare against existing datasets. They cite Edge-IIoTset but give no quantitative comparison on scale, diversity, or class balance. A simple table would have strengthened the paper.

Third, the DNN demonstration is thin. One class (Infection) gets zero F1, and the text hand-waves it as 'areas for improvement'. It is just a demo, so minor, but it does not inspire confidence in the baseline.

Bottom line: the dataset is real, reproducible, and likely valuable. The label noise is a legitimate concern but not disqualifying. I would send this to review and ask the authors to quantify the noise and add a comparison table. I would cite it if I were working on distributed IoT intrusion detection.

Best, [Your name]

Referee Report

2 major / 5 minor

Summary. The paper presents Gotham Dataset 2025, a large-scale IoT network traffic dataset generated using the Gotham testbed, a GNS3-based emulation environment. The dataset contains 3,134,004 packets from 78 emulated IoT devices using MQTT, CoAP, and RTSP, with traffic captured per device at the interface between the IoT gateway and the device. The repository provides raw PCAP files, processed CSV files with 23 features, and metadata logs, and the paper describes a timing-synchronisation labelling strategy that assigns labels based on attack-phase time windows. A federated deep neural network use example reports 89% accuracy. The stated contribution is a realistic, reproducible, publicly available resource for IoT intrusion detection and distributed learning research.

Significance. If the labels are trustworthy, the dataset would be a valuable public resource: it is large, heterogeneous, openly available on Zenodo, accompanied by processing scripts on GitHub, and organised per device in a way that is well suited to federated-learning experiments. I verified that the packet and byte counts in Table 3 sum correctly, and the repository structure and feature set are clearly specified. However, the central value of any labelled intrusion-detection dataset is the correctness of its ground truth, and the timing-window labelling method introduces potentially systematic label noise that is neither quantified nor validated. The significance is therefore conditional on addressing the labelling concern.

major comments (2)

[Data Labelling section and Fig. 7] The labelling strategy labels every packet within an attack's logged time window as malicious after filtering only on the IoT device IP address as source or destination. The manuscript states that the IoT devices periodically transmit telemetry and keep-alive traffic, and the attack windows are short (e.g., DoS for 10 seconds, CoAP amplification for 15 seconds), so legitimate packets from the device will fall inside the attack window and will be mislabelled as attacks. Conversely, malicious packets outside the calibrated window would be labelled benign. The metadata includes the attacker's IP address and port details, but the described filtering does not use them to restrict the attack labels to packets actually involving the attack flow. No validation is reported, such as a comparison against attacker-IP-filtered labels or a manual inspection of flagged packets, so the extent of label noise is unknown. Because the dataset's core claim is to provide reliable ground truth for intrusion detection, this issue is load-bearing. Please either relabel using the full metadata (attacker IP/ports) or quantify and document the expected label error and its effect on downstream model evaluation.
[Use Examples section and Table 4] The classification report in Table 4 does not match the dataset's stated label taxonomy. The paper defines 10 labels including Benign, DoS Attack, Telnet Brute Forcing, Network Scanning, CoAP Amplification Attack, and several C&C-related classes, but Table 4 reports only Brute Force, C&C Communication, DoS, Infection, Network Scanning, and Normal. The class 'Infection' does not appear in the label list, and several defined classes are absent. No confusion matrix, class-wise sample counts, or hyperparameter settings are provided, so the reported 89% accuracy cannot be reproduced or interpreted. This example is meant to demonstrate the dataset's utility, and the inconsistency undermines that demonstration. Please align the use example with the actual label set and provide complete evaluation details.

minor comments (5)

[Abstract] The sentence 'Our dataset was collected in a distributed manner, where network traffic was captured separately for each IoT device at the interface between the IoT gateway and the device.' appears twice verbatim in the abstract; please remove the duplicate.
[Table 2] The feature name 'frame.Dme' should be 'frame.time', and 'Destination' is misspelled in the description column.
[Table 3] The table caption reads 'The Gotham dataset feature set list', but the table contains device types, instance counts, packet counts, and byte counts; the caption should describe the device summary instead.
[Data Collection and Feature Extraction] The text says 'each attack lasting between 1 and 1.5 hours', but Table 1 reports network-scanning captures of 20 minutes, CoAP amplification captures of 60 minutes, and Mirai infection captures of 2 hours 30 minutes. This inconsistency should be corrected to avoid ambiguity about the data collection protocol.
[Data Labelling] The manuscript does not specify the time zone or clock-synchronisation method used to align attack-log timestamps with tcpdump capture timestamps. A short statement about clock synchronisation would improve reproducibility and would also help address the timing-alignment concern raised in the major comments.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dataset is generated from an external testbed and labelling uses independent attack metadata, not model outputs or fitted parameters.

full rationale

This is a dataset description paper rather than a derivation. The central resource (3,134,004 packets, 78 emulated devices, 23 features, 10 labels) is produced by a data-collection pipeline: tcpdump captures on the Gotham testbed, Tshark feature extraction, and a metadata-based labelling strategy. No result is defined in terms of a fitted parameter, and no prediction is obtained by construction from an input. The DNN evaluators (Table 4) are trained and tested on splits of the same dataset, which is a standard evaluation setup, not a circular proof: the model's accuracy does not feed back into the dataset's construction. The labelling method uses attack-phase timestamps from logs to assign labels; this is a ground-truth generation procedure, not a fitted-input-called-prediction scheme. While the timing-synchronisation labelling has a validity risk (benign packets inside an attack window may be mislabelled), that is a data-quality concern, not circularity. The paper's self-citations ([4] is the dataset deposit, [11] is prior work on federated learning for intrusion detection) are not load-bearing for the dataset's core claim; the testbed itself is credited to an external source [5], and the processing scripts are shared for reproducibility. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The dataset's validity rests on the fidelity of the Gotham emulation and on the time-window labelling assumption; no mathematical axioms are used.

assumptions (3)

domain assumption The Gotham testbed faithfully emulates realistic large-scale IoT network behaviour and device heterogeneity.
The paper claims the dataset represents real-world smart-city IoT deployments without providing a validation against real-world network traffic; the cited testbed publication is relied upon.
domain assumption The time-window alignment between attack metadata and packet timestamps accurately labels every packet as benign or malicious.
The labelling section states that packets within attack time windows are labelled malicious, assuming no benign packets occur during attacks and that timestamps are perfectly synchronised.
domain assumption Tshark-extracted features (23 columns) capture sufficient information for intrusion detection.
The paper selects a feature set without an ablation or justification that these features are sufficient for the intended ML tasks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Gotham Dataset 2025: A Reproducible Large-Scale IoT Network Dataset for Intrusion Detection and Security Research." pith.science (2026). https://pith.science/paper/22TRB6S5

@misc{pith2026250203134,
  author       = {Pith},
  title        = {Pith review of: Gotham Dataset 2025: A Reproducible Large-Scale IoT Network Dataset for Intrusion Detection and Security Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/22TRB6S5}},
  note         = {Machine review of arXiv:2502.03134}
}

read the original abstract

In this paper, a dataset of IoT network traffic is presented. Our dataset was generated by utilising the Gotham testbed, an emulated large-scale Internet of Things (IoT) network designed to provide a realistic and heterogeneous environment for network security research. The testbed includes 78 emulated IoT devices operating on various protocols, including MQTT, CoAP, and RTSP. Network traffic was captured in Packet Capture (PCAP) format using tcpdump, and both benign and malicious traffic were recorded. Malicious traffic was generated through scripted attacks, covering a variety of attack types, such as Denial of Service (DoS), Telnet Brute Force, Network Scanning, CoAP Amplification, and various stages of Command and Control (C&C) communication. The data were subsequently processed in Python for feature extraction using the Tshark tool, and the resulting data was converted to Comma Separated Values (CSV) format and labelled. The data repository includes the raw network traffic in PCAP format and the processed labelled data in CSV format. Our dataset was collected in a distributed manner, where network traffic was captured separately for each IoT device at the interface between the IoT gateway and the device. Our dataset was collected in a distributed manner, where network traffic was separately captured for each IoT device at the interface between the IoT gateway and the device. With its diverse traffic patterns and attack scenarios, this dataset provides a valuable resource for developing Intrusion Detection Systems and security mechanisms tailored to complex, large-scale IoT environments. The dataset is publicly available at Zenodo.

Figures

Figures reproduced from arXiv: 2502.03134 by the authors.

**Figure 3.** Heterogeneity across IoT devices in terms of packet size, total bytes exchanges, packet volume, and protocol. The dataset contains various label categories represenDng both benign and malicious network acDviDes. The Normal label indicates safe traffic with no signs of malicious behaviour. DoS a]acks refer to instances where a]ackers a]empt to overwhelm a device or network, disrupDng normal funcDonality. Network Scan… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

From Data Center IoT Telemetry to Data Analytics Chatbots -- Virtual Knowledge Graph is All You Need
cs.DC 2025-06 conditional novelty 4.0 of 10

A natural-language chatbot for data center IoT queries builds small query-specific knowledge graphs to ground LLM-generated SPARQL, reporting 92.5% accuracy and 3.03s latency.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages · cited by 1 Pith paper

[1]

Abstract In this paper, a dataset of IoT network traﬃc is presented

ARTICLE INFORMATION Ar#cle #tle Gotham Dataset 2025: A Reproducible Large-Scale IoT Network Dataset for Intrusion DetecDon and Security Research Authors Othmane Belarbi a,*, Theodoros Spyridopoulos a, Eirini Anthi a, Omer Rana a, Pietro Carnelli b, ALab Khan b Aﬃlia#ons a Cardiﬀ University, School of Computer Science & InformaDcs, Cardiﬀ, UK b Toshiba Eur...

work page arXiv 2025
[2]

A Comprehensive Survey for IoT Security Datasets Taxonomy, ClassiﬁcaDon and Machine Learning Mechanisms,

A. ChrisDn, G. Creado, W. Almobaideen, O. A. Alghanam, M. Saadeh, “A Comprehensive Survey for IoT Security Datasets Taxonomy, ClassiﬁcaDon and Machine Learning Mechanisms,” Comput. Secur. 132 (2023) 103283. h]ps://doi.org/10.1016/j.cose.2023.103283

work page arXiv 2023
[3]

A survey on intrusion detecDon system in IoT networks,

M. M. Rahman, S. A. Shakil, M. R. Mustakim, “A survey on intrusion detecDon system in IoT networks,” Cyber Secur. Appl. 3 (2025) 100082. h]ps://doi.org/10.1016/j.csa.2024.100082

work page arXiv 2025
[4]

Advancing IoT security: A systemaDc review of machine learning approaches for the detecDon of IoT botnets,

A. Nazir, J. He, N. Zhu, A. Wajahat, X. Ma, F. Ullah, S. Qureshi, M. S. Pathan, “Advancing IoT security: A systemaDc review of machine learning approaches for the detecDon of IoT botnets,” J. King Saud Univ. - Comput. Inf. Sci. 35(10) (2023) 101820. h]ps://doi.org/10.1016/j.jksuci.2023.101820

work page arXiv 2023
[5]

Gotham testbed: a reproducible IoT testbed for security experiments and dataset generaDon,

X. Sáez-de-Cámara, et al., “Gotham testbed: a reproducible IoT testbed for security experiments and dataset generaDon,” IEEE Trans. Dependable Secure Comput. 21(1) (2023) 186–203. h]ps://doi.org/10.1109/TDSC.2023.3247166

work page arXiv 2023
[6]

Graphical Network Simulator 3,

J. Grossmann, et al., “Graphical Network Simulator 3,” 2024, Accessed 7 December

work page 2024
[8]

Leaked Mirai source code for research/IOC development purposes,

J. Gamblin, “Leaked Mirai source code for research/IOC development purposes,” [soLware], Github, 2024, Accessed 7 December

work page 2024
[9]

Merlin is a cross-platorm post-exploitaDon HTTP/2 command & control server and agent wri]en in Golang,

R. V. Tuyl, "Merlin is a cross-platorm post-exploitaDon HTTP/2 command & control server and agent wri]en in Golang," 2024, Accessed 7 December

work page 2024

Show all 11 references

[10]

Edge-IIoTset: A New Comprehensive RealisDc Cyber Security Dataset of IoT and IIoT ApplicaDons for Centralized and Federated Learning,

M. A. Ferrag, O. Friha, D. Hamouda, L. Maglaras, H. Janicke, “Edge-IIoTset: A New Comprehensive RealisDc Cyber Security Dataset of IoT and IIoT ApplicaDons for Centralized and Federated Learning,” IEEE Access 10 (2022) 40281–40306. h]ps://doi.org/10.1109/ACCESS.2022.3165809

2022
[11]

Federated Deep Learning for Intrusion DetecDon in IoT Networks,

O. Belarbi, T. Spyridopoulos, E. Anthi, I. MavromaDs, P . Carnelli, A. Khan, “Federated Deep Learning for Intrusion DetecDon in IoT Networks,” GLOBECOM 2023 - IEEE Glob. Commun. Conf., Kuala Lumpur, Malaysia, 2023, pp. 237–242. h]ps://doi.org/10.1109/GLOBECOM54140.2023.10437860

2023
[2025]

h]ps://doi.org/10.5281/zenodo.14502760

doi

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Abstract In this paper, a dataset of IoT network traﬃc is presented

ARTICLE INFORMATION Ar#cle #tle Gotham Dataset 2025: A Reproducible Large-Scale IoT Network Dataset for Intrusion DetecDon and Security Research Authors Othmane Belarbi a,*, Theodoros Spyridopoulos a, Eirini Anthi a, Omer Rana a, Pietro Carnelli b, ALab Khan b Aﬃlia#ons a Cardiﬀ University, School of Computer Science & InformaDcs, Cardiﬀ, UK b Toshiba Eur...

work page arXiv 2025

[2] [2]

A Comprehensive Survey for IoT Security Datasets Taxonomy, ClassiﬁcaDon and Machine Learning Mechanisms,

A. ChrisDn, G. Creado, W. Almobaideen, O. A. Alghanam, M. Saadeh, “A Comprehensive Survey for IoT Security Datasets Taxonomy, ClassiﬁcaDon and Machine Learning Mechanisms,” Comput. Secur. 132 (2023) 103283. h]ps://doi.org/10.1016/j.cose.2023.103283

work page arXiv 2023

[3] [3]

A survey on intrusion detecDon system in IoT networks,

M. M. Rahman, S. A. Shakil, M. R. Mustakim, “A survey on intrusion detecDon system in IoT networks,” Cyber Secur. Appl. 3 (2025) 100082. h]ps://doi.org/10.1016/j.csa.2024.100082

work page arXiv 2025

[4] [4]

Advancing IoT security: A systemaDc review of machine learning approaches for the detecDon of IoT botnets,

A. Nazir, J. He, N. Zhu, A. Wajahat, X. Ma, F. Ullah, S. Qureshi, M. S. Pathan, “Advancing IoT security: A systemaDc review of machine learning approaches for the detecDon of IoT botnets,” J. King Saud Univ. - Comput. Inf. Sci. 35(10) (2023) 101820. h]ps://doi.org/10.1016/j.jksuci.2023.101820

work page arXiv 2023

[5] [5]

Gotham testbed: a reproducible IoT testbed for security experiments and dataset generaDon,

X. Sáez-de-Cámara, et al., “Gotham testbed: a reproducible IoT testbed for security experiments and dataset generaDon,” IEEE Trans. Dependable Secure Comput. 21(1) (2023) 186–203. h]ps://doi.org/10.1109/TDSC.2023.3247166

work page arXiv 2023

[6] [6]

Graphical Network Simulator 3,

J. Grossmann, et al., “Graphical Network Simulator 3,” 2024, Accessed 7 December

work page 2024

[7] [8]

Leaked Mirai source code for research/IOC development purposes,

J. Gamblin, “Leaked Mirai source code for research/IOC development purposes,” [soLware], Github, 2024, Accessed 7 December

work page 2024

[8] [9]

Merlin is a cross-platorm post-exploitaDon HTTP/2 command & control server and agent wri]en in Golang,

R. V. Tuyl, "Merlin is a cross-platorm post-exploitaDon HTTP/2 command & control server and agent wri]en in Golang," 2024, Accessed 7 December

work page 2024