Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person Recognition

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read This paper introduces DetReIDX, a large aerial-ground person dataset with more than 13 million bounding boxes, and reports that state-of-the-art detectors and ReID models lose up to 80% detection accuracy and over 70% Rank-1 accuracy on it.

desk verdict A genuinely new aerial-ground dataset with real stress-test potential, undermined by missing ReID baselines and inconsistent box counts. read the letter →

arxiv 2505.04793 v1 pith:SXKZ6BE5 submitted 2025-05-07 cs.CV

classification cs.CV
keywords personre-identificationUAVsurveillanceaerial-grounddatasetcross-viewrecognitionlong-termReIDclothingvariationsoftbiometricsstress-testbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DetReIDX is a person dataset built from drone and ground recordings of 509 volunteers across seven campuses on three continents, with more than 13 million bounding boxes captured at altitudes between 5.8 and 120 meters. The paper's central claim is that this is the first large-scale aerial-ground benchmark that combines two-session clothing changes with extreme range and viewpoint variation, and that current state-of-the-art detectors and ReID models fail on it: up to an 80% drop in detection accuracy and more than 70% Rank-1 loss. Existing benchmarks, the authors argue, either stay at close range, keep clothing fixed, or lack cross-view pairing, so they cannot reveal how much models depend on color, silhouette, and near-ground viewpoints. If the claim is right, DetReIDX gives the field a standardized stress test for long-term, cross-view person recognition under realistic drone conditions.

What carries the argument

The load-bearing object is the collection protocol rather than any single model. Each identity receives an indoor reference set (mugshots from three angles plus a gait video) and two outdoor drone sessions on different days with different clothing, captured from 18 fixed viewpoints spanning pitch angles of 30, 60, and 90 degrees and horizontal distances from 10 to 120 meters. This produces a controlled axis of degradation: bounding boxes shrink from more than 1000 pixels tall indoors to under 10 pixels at the farthest viewpoints, while the two-session design removes clothing as a reliable cue. The dataset defines three ReID evaluation modes (aerial-to-aerial, aerial-to-ground, ground-to-aerial) and distance bins D1 (<20m), D2 (20–50m), and D3 (>50m), which are the controlled variables through which the paper measures model collapse.

What would settle it

Re-annotate a random subset of DetReIDX from scratch with independent annotators and compare identity assignments and bounding boxes to the released labels. If cross-session identity agreement is materially incomplete, or if many boxes under 10 pixels are misplaced, the published AP50 and Rank-1 numbers are not a clean measure of model failure. A second check: train and evaluate a ReID model on a cleanly re-annotated subset; if Rank-1 rises far above the published values, annotation noise, not the real-world variability DetReIDX claims to capture, explains the collapse.

Watch

Extended reading notes

Core claim

The central discovery is empirical: when state-of-the-art models are evaluated on DetReIDX, they catastrophically degrade. On detection, training on short-range views and testing on long-range views drops YOLOv8's AP50 from 91.4% to 13.7%, while DDOD and Grid-RCNN fall below 1%; extrapolating to unseen 90-degree top-down views also drops AP50 by between 22% and 35% relative. On ReID, all three tested models score below 9% Rank-1 in aerial-to-aerial cross-session matching, and performance keeps falling as drone distance grows from D1 to D3. The authors interpret this as evidence that current models rely on appearance cues—clothing, color, texture, full-body silhouettes—that DetReIDX systematically removes through clothing changes across sessions and through sub-10-pixel top-down targets.

Load-bearing premise

The entire benchmark rests on the manual annotations being correct and consistent across sessions, especially the identity labels staying stable between indoor and outdoor captures and between Session 1 and Session 2, and the paper reports no inter-annotator agreement or label-quality statistics, so noisy labels could by themselves create part of the measured degradation.

Editorial extensions

If this is right

  • Current detectors are viewpoint-locked: training on 30°/60° pitches and testing on an unseen 90° top-down view lowers AP50 by 22–35% relative.
  • Current ReID models cannot bridge cross-session clothing changes in aerial settings, with Rank-1 below 9% in aerial-to-aerial matching.
  • Long-range drone views alone do not teach transferable pedestrian features: training on D3 and testing on D1 gives near-zero AP50 for all tested detectors.
  • The soft-biometric annotations make DetReIDX a testbed for geometry-aware or attribute-based ReID, which the paper argues is the direction needed for robustness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not decompose the failure by cause; a natural extension is to compare same-view, same-day retrieval against cross-view, cross-day retrieval within the aerial-to-aerial setting, which would isolate the contribution of clothing change alone.
  • The near-zero D3-to-D1 transfer implies a falsifiable prediction about scale augmentation: if long-range-trained detectors are given scale-invariant pretraining, their short-range AP50 should rise, and DetReIDX's distance bins provide the exact protocol to test this.
  • Because soft-biometric labels cover height, body volume, and other geometry, one could test whether attributes predicted from indoor images transfer to aerial images; a positive result would support the paper's call for geometry-aware representations.
  • A stress-test dataset can also be turned into a training signal: selecting hard cross-session, cross-view pairs and fine-tuning on them should improve real-world ReID if DetReIDX captures the true failure modes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces DetReIDX, a large-scale aerial-ground multi-session person dataset with 509 identities, indoor ground references, and UAV captures from 18 viewpoints at altitudes between 5.8 m and 120 m, annotated for detection, tracking, ReID, soft biometrics, and action recognition. It reports benchmarks on three detectors and three ReID models, arguing that performance degrades sharply on DetReIDX, especially at long distances and in cross-view settings, and concludes that DetReIDX should serve as a stress-test benchmark for UAV-based person recognition.

Significance. If the dataset is as described and released, it fills a real gap: existing aerial-ground benchmarks have narrower altitude ranges, lack per-subject clothing changes across sessions, and provide fewer annotation types. The two-session protocol with clothing variation, the 120 m altitude range, and the multi-task annotations are valuable assets. The authors also state that the dataset and evaluation protocols will be publicly released, which supports community use and reproducibility. The empirical claims currently outrun the evidence: the headline bounding-box count is inconsistent across sections, the ReID degradation claim is unsupported by same-model baseline numbers on existing datasets, and annotation quality is not quantified. These issues are fixable, but they prevent the paper from supporting its strongest claims as written.

major comments (4)
  1. [Abstract and Sections III-A, III-F; Tables I and V] The headline scale claim is internally inconsistent: the abstract and Section III say "over 13 million bounding boxes", Table I lists 12.6M, and Table V sums to 11,797,199 (5,095,539 + 2,483,836 + 4,217,824). Since this is the paper's primary quantitative claim, the authors must reconcile the numbers and report a single official count for the released annotations.
  2. [Section IV-B and Tables IX-X] The ReID degradation claim is not supported by the reported experiments. Table IX gives only absolute mAP/R1/R5/R10 values on DetReIDX; no same-model results on Market-1501, DukeMTMC-ReID, AG-ReID.v2, G2APS, or any prior benchmark appear anywhere in the paper. The sentence in Section IV-B that models perform "relatively good on the existing ground-level datasets" is therefore an assertion, not a result. Moreover, Table X's altitude breakdown shows at most a 45.7% relative Rank-1 drop (A2A D1 to D3), so the abstract's "over 70% Rank-1 loss" is not derivable from the presented data. The authors should add baseline tables using identical training and evaluation protocols or revise the claims.
  3. [Section III-D] Annotation quality is load-bearing and unquantified. The text says annotations were "manually done by a set of volunteers, using the CVAT tool and cross-verified by peers," but no inter-annotator agreement, label-error rate, or verification protocol is reported. Since the ReID evaluation depends on consistent PID labels across sessions and the detection evaluation includes ROIs of 8x8 pixels, the absence of quality statistics undermines the benchmark numbers.
  4. [Sections IV-A and IV-B, Tables V and VIII] Split statistics and training details are inconsistent or incomplete. Section IV-A states a 70-20-10 split, but Table V gives Train/Val/Test videos of 120/56/109 out of 285, i.e., 42%/20%/38%. Section IV-B states a 70%-30% PID-disjoint split, but the reported 267 train and 67 test identities correspond to roughly 80%/20% of the 334 outdoor-reobserved identities, not 70/30 of the 509 total. In addition, no optimizer, learning rate, batch size, input resolution, number of epochs, or number of independent runs is reported for any model, and the detection results come from a single split, so the reader cannot assess variance or rule out undertraining as an explanation for low scores.
minor comments (4)
  1. [Throughout] The manuscript contains numerous typos and spacing inconsistencies ('real-wordlsettings', 'reserch', 'Futher Research', 'UA V'); a careful copyedit is needed.
  2. [References [16]] Reference [16] appears to be mismatched: the CSM dataset is cited to a paper on movie popularity prediction, which is not a person ReID dataset reference.
  3. [Figure 10 and Section IV-A] Figure 10 highlights a 'critical distance (70 meters)', but the distance bins are defined as D1<20m, D2=20-50m, and D3>50m; the relation between 70m and the bins should be clarified.
  4. [Table I and Table IV] Table I lists the DetReIDX height range as 5-120m, while the text and Table IV give 5.8-120m; the ranges should be made consistent.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the benchmark experiments are self-contained, aside from a minor non-load-bearing self-citation and a missing-comparison caveat.

full rationale

The paper's central claims are direct empirical measurements on a newly collected dataset. Detection models are trained on DetReIDX splits and evaluated on held-out viewpoint and distance bins; the headline degradation (YOLOv8 AP50 from 0.914 in D1->D1 to 0.137 in D1->D3, Table VII) is the output of those experiments, not a quantity reconstructed from the dataset definition or from a fitted parameter. The ReID experiments similarly report mAP and Rank-1 values on DetReIDX test scenarios (Table IX). The statement that models perform 'relatively good on the existing ground-level datasets' is asserted without presenting same-model numbers on those datasets, so the word 'degrade' is under-supported; however, a missing comparison is a correctness and evidence gap, not circularity, because the DetReIDX numbers are not derived from those datasets. The only author self-citation is reference [6] (P-DESTRE), used to justify the 16 soft-biometric annotation attributes; this citation is not load-bearing because the dataset's usefulness and the degradation experiments do not depend on P-DESTRE's results. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from prior author work, and no known result is repackaged under new coordinates. The dataset's value still rests on unverified annotation quality, since no inter-annotator agreement statistics are reported, but that is a data-quality risk rather than circularity. Score 1 reflects the minor self-citation while the derivation chain is otherwise self-contained.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No mathematical derivation is present. The ledger captures the empirical and curation assumptions that the benchmark's validity depends on. The main concerns are annotation quality and the internal inconsistency in the headline statistics.

free parameters (1)
  • Distance bin boundaries (D1 < 20m, D2 20-50m, D3 > 50m) = 20m, 50m
    Hand-chosen cutoffs used to compute the reported degradation percentages; changing these boundaries changes the headline drops.
assumptions (4)
  • domain assumption Manual annotations by volunteers are accurate and identities are consistent across sessions
    Section III-D describes volunteer annotation with peer cross-verification, but no inter-annotator agreement is reported.
  • domain assumption The selected baselines (YOLOv8, DDOD, Grid-RCNN; PersonViT, SeCap, CLIP-ReID) represent the state of the art
    Section IV chooses these models as SOTA without a systematic survey or justification.
  • domain assumption The reported dataset statistics match the released data
    The paper's own numbers conflict internally (13M vs 12.6M vs 11.8M), so this assumption is doubtful.
  • domain assumption Distance and pitch annotations are accurate measurements
    Table IV and Figure 5 rely on recorded flight positions; no calibration or error estimates are given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person Recognition." pith.science (2026). https://pith.science/paper/SXKZ6BE5

@misc{pith2026250504793,
  author       = {Pith},
  title        = {Pith review of: DetReIDX: A Stress-Test Dataset for Real-World UAV-Based Person Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SXKZ6BE5}},
  note         = {Machine review of arXiv:2505.04793}
}
read the original abstract

Person reidentification (ReID) technology has been considered to perform relatively well under controlled, ground-level conditions, but it breaks down when deployed in challenging real-world settings. Evidently, this is due to extreme data variability factors such as resolution, viewpoint changes, scale variations, occlusions, and appearance shifts from clothing or session drifts. Moreover, the publicly available data sets do not realistically incorporate such kinds and magnitudes of variability, which limits the progress of this technology. This paper introduces DetReIDX, a large-scale aerial-ground person dataset, that was explicitly designed as a stress test to ReID under real-world conditions. DetReIDX is a multi-session set that includes over 13 million bounding boxes from 509 identities, collected in seven university campuses from three continents, with drone altitudes between 5.8 and 120 meters. More important, as a key novelty, DetReIDX subjects were recorded in (at least) two sessions on different days, with changes in clothing, daylight and location, making it suitable to actually evaluate long-term person ReID. Plus, data were annotated from 16 soft biometric attributes and multitask labels for detection, tracking, ReID, and action recognition. In order to provide empirical evidence of DetReIDX usefulness, we considered the specific tasks of human detection and ReID, where SOTA methods catastrophically degrade performance (up to 80% in detection accuracy and over 70% in Rank-1 ReID) when exposed to DetReIDXs conditions. The dataset, annotations, and official evaluation protocols are publicly available at https://www.it.ubi.pt/DetReIDX/

Figures

Figures reproduced from arXiv: 2505.04793 by the authors.

Figure 1
Figure 1. Comparison between the most important features of the publicly [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Examples of soft biometric annotations for two individuals in the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Satellite view of the data collection sites across the university campuses [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Overview of the indoor data collection setup: (left) mugshots taken from [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: Actual drone-captured frames from all 18 UAV viewpoints (P1–P18), grouped by pitch angle: 30 [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Distributions of the soft biometric labels in [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Example of one subject captured in 18 viewpoints (P1–P18), with clothing changes between sessions. Top row: Session 1. Bottom row: Session 2, with [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: Effect of distance on pedestrian detection accuracy. The black curve pro [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]
Figure 11
Figure 11. Figure 11: Qualitative analysis of pedestrian detection under varying viewpoints [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 13
Figure 13. Figure 13: Qualitative evaluation of Person-ViT ReID model on [PITH_FULL_IMAGE:figures/full_fig_p009_13.png]
Figure 14
Figure 14. Figure 14: Cumulative Match Characteristic (CMC) curves showing the impact of aerial distances on ReID performance using the Person-ViT model, evaluated across [PITH_FULL_IMAGE:figures/full_fig_p010_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Causality and "In-the-Wild" Video-Based Person Re-ID: A Survey

    cs.CV 2025-05 reject novelty 3.0 of 10

    A survey of causal reasoning for video person re-identification that reviews DIR-ReID, identity-shuffle GANs, and causal transformers, but contains unverified performance claims.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Scalable person re-identification: A benchmark,

    L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian, “Scalable person re-identification: A benchmark,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 1116–1124

  2. [2]

    Deepreid: Deep filter pairing neural network for person re-identification,

    W. Li, R. Zhao, T. Xiao, and X. Wang, “Deepreid: Deep filter pairing neural network for person re-identification,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 152– 159

  3. [3]

    Mars: A video benchmark for large-scale person re-identification,

    L. Zheng, Z. Bie, Y. Sun, J. Wang, C. Su, S. Wang, and Q. Tian, “Mars: A video benchmark for large-scale person re-identification,” inCom- puter Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14. Springer, 2016, pp. 868–884

  4. [4]

    Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning,

    Y. Wu, Y. Lin, X. Dong, Y. Yan, W. Ouyang, and Y. Yang, “Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5177–5186

  5. [5]

    Long-term cloth-changing person re-identification,

    X. Qian, W. Wang, L. Zhang, F. Zhu, Y. Fu, T. Xiang, Y.-G. Jiang, and X. Xue, “Long-term cloth-changing person re-identification,” in Proceedings of the Asian conference on computer vision, 2020

  6. [6]

    The p- destre: A fully annotated dataset for pedestrian detection, tracking, and short/long-term re-identification from aerial devices,

    S. A. Kumar, E. Yaghoubi, A. Das, B. Harish, and H. Proenc ¸a, “The p- destre: A fully annotated dataset for pedestrian detection, tracking, and short/long-term re-identification from aerial devices,”IEEE Transactions on Information Forensics and Security, vol. 16, pp. 1696–1708, 2020

  7. [7]

    Uav-human: A large benchmark for human behavior understanding with unmanned aerial vehicles,

    T. Li, J. Liu, W. Zhang, Y. Ni, W. Wang, and Z. Li, “Uav-human: A large benchmark for human behavior understanding with unmanned aerial vehicles,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 266–16 275

  8. [8]

    Person re- identification by descriptive and discriminative classification,

    M. Hirzer, C. Beleznai, P. M. Roth, and H. Bischof, “Person re- identification by descriptive and discriminative classification,” inImage Analysis: 17th Scandinavian Conference, SCIA 2011, Ystad, Sweden, May 2011. Proceedings 17. Springer, 2011, pp. 91–102

Show all 23 references
  1. [9]

    Investigating open-world person re-identification using a drone,

    R. Layne, T. M. Hospedales, and S. Gong, “Investigating open-world person re-identification using a drone,” inComputer Vision-ECCV 2014 Workshops: Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part III 13. Springer, 2015, pp. 225–240

  2. [10]

    Person re-identification in aerial imagery,

    S. Zhang, Q. Zhang, Y. Yang, X. Wei, P. Wang, B. Jiao, and Y. Zhang, “Person re-identification in aerial imagery,”IEEE Transactions on Mul- timedia, vol. 23, pp. 281–291, 2020

  3. [11]

    Privacy in mini-drone based video surveillance,

    M. Bonetto, P. Korshunov, G. Ramponi, and T. Ebrahimi, “Privacy in mini-drone based video surveillance,” in2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol. 4. IEEE, 2015, pp. 1–6

  4. [12]

    Eye in the sky: Real-time drone surveillance system (dss) for violent individuals identification using scatternet hybrid deep learning network,

    A. Singh, D. Patil, and S. Omkar, “Eye in the sky: Real-time drone surveillance system (dss) for violent individuals identification using scatternet hybrid deep learning network,” inProceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, ...

  5. [13]

    Deep person re-identification in uav images,

    A. Grigorev, Z. Tian, S. Rho, J. Xiong, S. Liu, and F. Jiang, “Deep person re-identification in uav images,”EURASIP Journal on Advances in Signal Processing, vol. 2019, pp. 1–10, 2019

  6. [14]

    Ag-reid. v2: Bridging aerial and ground views for person re-identification,

    H. Nguyen, K. Nguyen, S. Sridharan, and C. Fookes, “Ag-reid. v2: Bridging aerial and ground views for person re-identification,”IEEE Transactions on Information Forensics and Security, vol. 19, pp. 2896– 2908, 2024

  7. [15]

    Secap: Self- calibrating and adaptive prompts for cross-view person re-identification in aerial-ground networks,

    S. Wang, Y. Wang, R. Wu, B. Jiao, W. Wang, and P. Wang, “Secap: Self- calibrating and adaptive prompts for cross-view person re-identification in aerial-ground networks,”arXiv preprint arXiv:2503.06965, 2025

  8. [16]

    Using crowd-source based features from social media and conventional features to predict the movies popularity,

    M. Ahmed, M. Jahangir, H. Afzal, A. Majeed, and I. Siddiqi, “Using crowd-source based features from social media and conventional features to predict the movies popularity,” in2015 IEEE international conference on smart city/SocialCom/SustainCom (SmartCity). IEEE, 2015, pp. 273–278

  9. [17]

    iqiyi-vid: A large dataset for multi-modal person identification,

    Y. Liu, B. Peng, P. Shi, H. Yan, Y. Zhou, B. Han, Y. Zheng, C. Lin, J. Jiang, Y. Fanet al., “iqiyi-vid: A large dataset for multi-modal person identification,”arXiv preprint arXiv:1811.07548, 2018

  10. [18]

    What is yolov8?

    J. Solawetz, “What is yolov8?” https://blog.roboflow.com/ what-is-yolov8/, 2023, accessed: 2025-04-09

  11. [19]

    Disentangle your dense object detector,

    Z. Chen, C. Yang, Q. Li, F. Zhao, Z.-J. Zha, and F. Wu, “Disentangle your dense object detector,” inProceedings of the 29th ACM international conference on multimedia, 2021, pp. 4939–4948

  12. [20]

    Grid r-cnn,

    X. Lu, B. Li, Y. Yue, Q. Li, and J. Yan, “Grid r-cnn,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7363–7372

  13. [21]

    Personvit: large-scale self-supervised vision transformer for person re-identification,

    B. Hu, X. Wang, and W. Liu, “Personvit: large-scale self-supervised vision transformer for person re-identification,”Machine Vision and Applications, vol. 36, no. 2, pp. 1–13, 2025

  14. [22]

    Clip-reid: exploiting vision-language model for image re-identification without concrete text labels,

    S. Li, L. Sun, and Q. Li, “Clip-reid: exploiting vision-language model for image re-identification without concrete text labels,” inProceedings of the AAAI conference on artificial intelligence, vol. 37, no. 1, 2023, pp. 1405–1413

  15. [23]

    Person re-identification: System design and evaluation overview,

    X. Wang and R. Zhao, “Person re-identification: System design and evaluation overview,” inPerson Re-Identification. Springer, 2014, pp. 351–370

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.