REVIEW 4 major objections 4 minor 1 cited by
DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person Recognition
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper introduces DetReIDX, a large aerial-ground person dataset with more than 13 million bounding boxes, and reports that state-of-the-art detectors and ReID models lose up to 80% detection accuracy and over 70% Rank-1 accuracy on it.
desk verdict A genuinely new aerial-ground dataset with real stress-test potential, undermined by missing ReID baselines and inconsistent box counts. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the collection protocol rather than any single model. Each identity receives an indoor reference set (mugshots from three angles plus a gait video) and two outdoor drone sessions on different days with different clothing, captured from 18 fixed viewpoints spanning pitch angles of 30, 60, and 90 degrees and horizontal distances from 10 to 120 meters. This produces a controlled axis of degradation: bounding boxes shrink from more than 1000 pixels tall indoors to under 10 pixels at the farthest viewpoints, while the two-session design removes clothing as a reliable cue. The dataset defines three ReID evaluation modes (aerial-to-aerial, aerial-to-ground, ground-to-aerial) and distance bins D1 (<20m), D2 (20–50m), and D3 (>50m), which are the controlled variables through which the paper measures model collapse.
What would settle it
Re-annotate a random subset of DetReIDX from scratch with independent annotators and compare identity assignments and bounding boxes to the released labels. If cross-session identity agreement is materially incomplete, or if many boxes under 10 pixels are misplaced, the published AP50 and Rank-1 numbers are not a clean measure of model failure. A second check: train and evaluate a ReID model on a cleanly re-annotated subset; if Rank-1 rises far above the published values, annotation noise, not the real-world variability DetReIDX claims to capture, explains the collapse.
Extended reading notes
Core claim
The central discovery is empirical: when state-of-the-art models are evaluated on DetReIDX, they catastrophically degrade. On detection, training on short-range views and testing on long-range views drops YOLOv8's AP50 from 91.4% to 13.7%, while DDOD and Grid-RCNN fall below 1%; extrapolating to unseen 90-degree top-down views also drops AP50 by between 22% and 35% relative. On ReID, all three tested models score below 9% Rank-1 in aerial-to-aerial cross-session matching, and performance keeps falling as drone distance grows from D1 to D3. The authors interpret this as evidence that current models rely on appearance cues—clothing, color, texture, full-body silhouettes—that DetReIDX systematically removes through clothing changes across sessions and through sub-10-pixel top-down targets.
Load-bearing premise
The entire benchmark rests on the manual annotations being correct and consistent across sessions, especially the identity labels staying stable between indoor and outdoor captures and between Session 1 and Session 2, and the paper reports no inter-annotator agreement or label-quality statistics, so noisy labels could by themselves create part of the measured degradation.
Editorial extensions
If this is right
- Current detectors are viewpoint-locked: training on 30°/60° pitches and testing on an unseen 90° top-down view lowers AP50 by 22–35% relative.
- Current ReID models cannot bridge cross-session clothing changes in aerial settings, with Rank-1 below 9% in aerial-to-aerial matching.
- Long-range drone views alone do not teach transferable pedestrian features: training on D3 and testing on D1 gives near-zero AP50 for all tested detectors.
- The soft-biometric annotations make DetReIDX a testbed for geometry-aware or attribute-based ReID, which the paper argues is the direction needed for robustness.
Reading between the lines
- The paper does not decompose the failure by cause; a natural extension is to compare same-view, same-day retrieval against cross-view, cross-day retrieval within the aerial-to-aerial setting, which would isolate the contribution of clothing change alone.
- The near-zero D3-to-D1 transfer implies a falsifiable prediction about scale augmentation: if long-range-trained detectors are given scale-invariant pretraining, their short-range AP50 should rise, and DetReIDX's distance bins provide the exact protocol to test this.
- Because soft-biometric labels cover height, body volume, and other geometry, one could test whether attributes predicted from indoor images transfer to aerial images; a positive result would support the paper's call for geometry-aware representations.
- A stress-test dataset can also be turned into a training signal: selecting hard cross-session, cross-view pairs and fine-tuning on them should improve real-world ReID if DetReIDX captures the true failure modes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces DetReIDX, a large-scale aerial-ground multi-session person dataset with 509 identities, indoor ground references, and UAV captures from 18 viewpoints at altitudes between 5.8 m and 120 m, annotated for detection, tracking, ReID, soft biometrics, and action recognition. It reports benchmarks on three detectors and three ReID models, arguing that performance degrades sharply on DetReIDX, especially at long distances and in cross-view settings, and concludes that DetReIDX should serve as a stress-test benchmark for UAV-based person recognition.
Significance. If the dataset is as described and released, it fills a real gap: existing aerial-ground benchmarks have narrower altitude ranges, lack per-subject clothing changes across sessions, and provide fewer annotation types. The two-session protocol with clothing variation, the 120 m altitude range, and the multi-task annotations are valuable assets. The authors also state that the dataset and evaluation protocols will be publicly released, which supports community use and reproducibility. The empirical claims currently outrun the evidence: the headline bounding-box count is inconsistent across sections, the ReID degradation claim is unsupported by same-model baseline numbers on existing datasets, and annotation quality is not quantified. These issues are fixable, but they prevent the paper from supporting its strongest claims as written.
major comments (4)
- [Abstract and Sections III-A, III-F; Tables I and V] The headline scale claim is internally inconsistent: the abstract and Section III say "over 13 million bounding boxes", Table I lists 12.6M, and Table V sums to 11,797,199 (5,095,539 + 2,483,836 + 4,217,824). Since this is the paper's primary quantitative claim, the authors must reconcile the numbers and report a single official count for the released annotations.
- [Section IV-B and Tables IX-X] The ReID degradation claim is not supported by the reported experiments. Table IX gives only absolute mAP/R1/R5/R10 values on DetReIDX; no same-model results on Market-1501, DukeMTMC-ReID, AG-ReID.v2, G2APS, or any prior benchmark appear anywhere in the paper. The sentence in Section IV-B that models perform "relatively good on the existing ground-level datasets" is therefore an assertion, not a result. Moreover, Table X's altitude breakdown shows at most a 45.7% relative Rank-1 drop (A2A D1 to D3), so the abstract's "over 70% Rank-1 loss" is not derivable from the presented data. The authors should add baseline tables using identical training and evaluation protocols or revise the claims.
- [Section III-D] Annotation quality is load-bearing and unquantified. The text says annotations were "manually done by a set of volunteers, using the CVAT tool and cross-verified by peers," but no inter-annotator agreement, label-error rate, or verification protocol is reported. Since the ReID evaluation depends on consistent PID labels across sessions and the detection evaluation includes ROIs of 8x8 pixels, the absence of quality statistics undermines the benchmark numbers.
- [Sections IV-A and IV-B, Tables V and VIII] Split statistics and training details are inconsistent or incomplete. Section IV-A states a 70-20-10 split, but Table V gives Train/Val/Test videos of 120/56/109 out of 285, i.e., 42%/20%/38%. Section IV-B states a 70%-30% PID-disjoint split, but the reported 267 train and 67 test identities correspond to roughly 80%/20% of the 334 outdoor-reobserved identities, not 70/30 of the 509 total. In addition, no optimizer, learning rate, batch size, input resolution, number of epochs, or number of independent runs is reported for any model, and the detection results come from a single split, so the reader cannot assess variance or rule out undertraining as an explanation for low scores.
minor comments (4)
- [Throughout] The manuscript contains numerous typos and spacing inconsistencies ('real-wordlsettings', 'reserch', 'Futher Research', 'UA V'); a careful copyedit is needed.
- [References [16]] Reference [16] appears to be mismatched: the CSM dataset is cited to a paper on movie popularity prediction, which is not a person ReID dataset reference.
- [Figure 10 and Section IV-A] Figure 10 highlights a 'critical distance (70 meters)', but the distance bins are defined as D1<20m, D2=20-50m, and D3>50m; the relation between 70m and the bins should be clarified.
- [Table I and Table IV] Table I lists the DetReIDX height range as 5-120m, while the text and Table IV give 5.8-120m; the ranges should be made consistent.
Circularity Check
No significant circularity; the benchmark experiments are self-contained, aside from a minor non-load-bearing self-citation and a missing-comparison caveat.
full rationale
The paper's central claims are direct empirical measurements on a newly collected dataset. Detection models are trained on DetReIDX splits and evaluated on held-out viewpoint and distance bins; the headline degradation (YOLOv8 AP50 from 0.914 in D1->D1 to 0.137 in D1->D3, Table VII) is the output of those experiments, not a quantity reconstructed from the dataset definition or from a fitted parameter. The ReID experiments similarly report mAP and Rank-1 values on DetReIDX test scenarios (Table IX). The statement that models perform 'relatively good on the existing ground-level datasets' is asserted without presenting same-model numbers on those datasets, so the word 'degrade' is under-supported; however, a missing comparison is a correctness and evidence gap, not circularity, because the DetReIDX numbers are not derived from those datasets. The only author self-citation is reference [6] (P-DESTRE), used to justify the 16 soft-biometric annotation attributes; this citation is not load-bearing because the dataset's usefulness and the degradation experiments do not depend on P-DESTRE's results. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from prior author work, and no known result is repackaged under new coordinates. The dataset's value still rests on unverified annotation quality, since no inter-annotator agreement statistics are reported, but that is a data-quality risk rather than circularity. Score 1 reflects the minor self-citation while the derivation chain is otherwise self-contained.
Assumptions & free parameters
free parameters (1)
- Distance bin boundaries (D1 < 20m, D2 20-50m, D3 > 50m) =
20m, 50m
assumptions (4)
- domain assumption Manual annotations by volunteers are accurate and identities are consistent across sessions
- domain assumption The selected baselines (YOLOv8, DDOD, Grid-RCNN; PersonViT, SeCap, CLIP-ReID) represent the state of the art
- domain assumption The reported dataset statistics match the released data
- domain assumption Distance and pitch annotations are accurate measurements
Cite this review
Pith. "Pith review of DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person Recognition." pith.science (2026). https://pith.science/paper/SXKZ6BE5
@misc{pith2026250504793,
author = {Pith},
title = {Pith review of: DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/SXKZ6BE5}},
note = {Machine review of arXiv:2505.04793}
}
read the original abstract
Person reidentification (ReID) technology has been considered to perform relatively well under controlled, ground-level conditions, but it breaks down when deployed in challenging real-world settings. Evidently, this is due to extreme data variability factors such as resolution, viewpoint changes, scale variations, occlusions, and appearance shifts from clothing or session drifts. Moreover, the publicly available data sets do not realistically incorporate such kinds and magnitudes of variability, which limits the progress of this technology. This paper introduces DetReIDX, a large-scale aerial-ground person dataset, that was explicitly designed as a stress test to ReID under real-world conditions. DetReIDX is a multi-session set that includes over 13 million bounding boxes from 509 identities, collected in seven university campuses from three continents, with drone altitudes between 5.8 and 120 meters. More important, as a key novelty, DetReIDX subjects were recorded in (at least) two sessions on different days, with changes in clothing, daylight and location, making it suitable to actually evaluate long-term person ReID. Plus, data were annotated from 16 soft biometric attributes and multitask labels for detection, tracking, ReID, and action recognition. In order to provide empirical evidence of DetReIDX usefulness, we considered the specific tasks of human detection and ReID, where SOTA methods catastrophically degrade performance (up to 80% in detection accuracy and over 70% in Rank-1 ReID) when exposed to DetReIDXs conditions. The dataset, annotations, and official evaluation protocols are publicly available at https://www.it.ubi.pt/DetReIDX/
Figures
Figures from the paper (8 more)
Forward citations
Cited by 1 Pith paper
-
Causality and "In-the-Wild" Video-Based Person Re-ID: A Survey
A survey of causal reasoning for video person re-identification that reviews DIR-ReID, identity-shuffle GANs, and causal transformers, but contains unverified performance claims.
Reference graph
Works this paper leans on
-
[1]
Scalable person re-identification: A benchmark,
L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian, “Scalable person re-identification: A benchmark,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 1116–1124
2015
-
[2]
Deepreid: Deep filter pairing neural network for person re-identification,
W. Li, R. Zhao, T. Xiao, and X. Wang, “Deepreid: Deep filter pairing neural network for person re-identification,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 152– 159
work page 2014
-
[3]
Mars: A video benchmark for large-scale person re-identification,
L. Zheng, Z. Bie, Y. Sun, J. Wang, C. Su, S. Wang, and Q. Tian, “Mars: A video benchmark for large-scale person re-identification,” inCom- puter Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14. Springer, 2016, pp. 868–884
work page 2016
-
[4]
Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning,
Y. Wu, Y. Lin, X. Dong, Y. Yan, W. Ouyang, and Y. Yang, “Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5177–5186
work page 2018
-
[5]
Long-term cloth-changing person re-identification,
X. Qian, W. Wang, L. Zhang, F. Zhu, Y. Fu, T. Xiang, Y.-G. Jiang, and X. Xue, “Long-term cloth-changing person re-identification,” in Proceedings of the Asian conference on computer vision, 2020
work page 2020
-
[6]
S. A. Kumar, E. Yaghoubi, A. Das, B. Harish, and H. Proenc ¸a, “The p- destre: A fully annotated dataset for pedestrian detection, tracking, and short/long-term re-identification from aerial devices,”IEEE Transactions on Information Forensics and Security, vol. 16, pp. 1696–1708, 2020
work page 2020
-
[7]
Uav-human: A large benchmark for human behavior understanding with unmanned aerial vehicles,
T. Li, J. Liu, W. Zhang, Y. Ni, W. Wang, and Z. Li, “Uav-human: A large benchmark for human behavior understanding with unmanned aerial vehicles,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 266–16 275
work page 2021
-
[8]
Person re- identification by descriptive and discriminative classification,
M. Hirzer, C. Beleznai, P. M. Roth, and H. Bischof, “Person re- identification by descriptive and discriminative classification,” inImage Analysis: 17th Scandinavian Conference, SCIA 2011, Ystad, Sweden, May 2011. Proceedings 17. Springer, 2011, pp. 91–102
2011
Show all 23 references
-
[9]
Investigating open-world person re-identification using a drone,
R. Layne, T. M. Hospedales, and S. Gong, “Investigating open-world person re-identification using a drone,” inComputer Vision-ECCV 2014 Workshops: Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part III 13. Springer, 2015, pp. 225–240
2014
-
[10]
Person re-identification in aerial imagery,
S. Zhang, Q. Zhang, Y. Yang, X. Wei, P. Wang, B. Jiao, and Y. Zhang, “Person re-identification in aerial imagery,”IEEE Transactions on Mul- timedia, vol. 23, pp. 281–291, 2020
2020
-
[11]
Privacy in mini-drone based video surveillance,
M. Bonetto, P. Korshunov, G. Ramponi, and T. Ebrahimi, “Privacy in mini-drone based video surveillance,” in2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol. 4. IEEE, 2015, pp. 1–6
2015
-
[12]
Eye in the sky: Real-time drone surveillance system (dss) for violent individuals identification using scatternet hybrid deep learning network,
A. Singh, D. Patil, and S. Omkar, “Eye in the sky: Real-time drone surveillance system (dss) for violent individuals identification using scatternet hybrid deep learning network,” inProceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, ...
2018
-
[13]
Deep person re-identification in uav images,
A. Grigorev, Z. Tian, S. Rho, J. Xiong, S. Liu, and F. Jiang, “Deep person re-identification in uav images,”EURASIP Journal on Advances in Signal Processing, vol. 2019, pp. 1–10, 2019
2019
-
[14]
Ag-reid. v2: Bridging aerial and ground views for person re-identification,
H. Nguyen, K. Nguyen, S. Sridharan, and C. Fookes, “Ag-reid. v2: Bridging aerial and ground views for person re-identification,”IEEE Transactions on Information Forensics and Security, vol. 19, pp. 2896– 2908, 2024
2024
-
[15]
Secap: Self- calibrating and adaptive prompts for cross-view person re-identification in aerial-ground networks,
S. Wang, Y. Wang, R. Wu, B. Jiao, W. Wang, and P. Wang, “Secap: Self- calibrating and adaptive prompts for cross-view person re-identification in aerial-ground networks,”arXiv preprint arXiv:2503.06965, 2025
2025 arXiv
-
[16]
Using crowd-source based features from social media and conventional features to predict the movies popularity,
M. Ahmed, M. Jahangir, H. Afzal, A. Majeed, and I. Siddiqi, “Using crowd-source based features from social media and conventional features to predict the movies popularity,” in2015 IEEE international conference on smart city/SocialCom/SustainCom (SmartCity). IEEE, 2015, pp. 273–278
2015
-
[17]
iqiyi-vid: A large dataset for multi-modal person identification,
Y. Liu, B. Peng, P. Shi, H. Yan, Y. Zhou, B. Han, Y. Zheng, C. Lin, J. Jiang, Y. Fanet al., “iqiyi-vid: A large dataset for multi-modal person identification,”arXiv preprint arXiv:1811.07548, 2018
2018 arXiv
-
[18]
What is yolov8?
J. Solawetz, “What is yolov8?” https://blog.roboflow.com/ what-is-yolov8/, 2023, accessed: 2025-04-09
2023
-
[19]
Disentangle your dense object detector,
Z. Chen, C. Yang, Q. Li, F. Zhao, Z.-J. Zha, and F. Wu, “Disentangle your dense object detector,” inProceedings of the 29th ACM international conference on multimedia, 2021, pp. 4939–4948
2021
-
[20]
Grid r-cnn,
X. Lu, B. Li, Y. Yue, Q. Li, and J. Yan, “Grid r-cnn,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7363–7372
2019
-
[21]
Personvit: large-scale self-supervised vision transformer for person re-identification,
B. Hu, X. Wang, and W. Liu, “Personvit: large-scale self-supervised vision transformer for person re-identification,”Machine Vision and Applications, vol. 36, no. 2, pp. 1–13, 2025
2025
-
[22]
Clip-reid: exploiting vision-language model for image re-identification without concrete text labels,
S. Li, L. Sun, and Q. Li, “Clip-reid: exploiting vision-language model for image re-identification without concrete text labels,” inProceedings of the AAAI conference on artificial intelligence, vol. 37, no. 1, 2023, pp. 1405–1413
2023
-
[23]
Person re-identification: System design and evaluation overview,
X. Wang and R. Zhao, “Person re-identification: System design and evaluation overview,” inPerson Re-Identification. Springer, 2014, pp. 351–370
2014
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.