{"id":"6ae9074f-ac17-40d8-a445-1c2a0fe6cd16","arxiv_id":"2506.05163","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FRED is a new open dataset of synchronized RGB and event-camera recordings of five drone models, with annotations for detection, tracking, and trajectory forecasting.","lead":"The paper introduces FRED, a seven-hour multimodal dataset pairing RGB video with event-camera streams for drone detection, tracking, and forecasting. It provides benchmarks showing event-based models outperform RGB-only models on small, fast drones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unmeasured temporal synchronization and event-first annotation transfer create a systematic bias in the event-vs-RGB comparison; the central gap in Table 2 needs a validation check.","rationale":"The reader's weakest assumption, unquantified temporal synchronization accuracy, is a real and important concern. I partially agree with the reader because the issue is not only the alignment of clock start times; it is also the event-first annotation protocol that transfers labels to RGB without any RGB-specific validation. These two mechanisms together can systematically depress RGB detector performance and inflate the event modality's apparent advantage, which is exactly the paper's headline result. The concern is load-bearing because if it lands, Table 2 and the conclusion that event detectors clearly outperform RGB detectors would need substantial qualification. However, this concern does not justify rejecting the dataset or the paper outright, since the dataset has genuine value as a resource, the recordings appear to cover diverse scenarios, and the benchmarks are clearly defined. A focused validation study on a subset, as proposed in the concrete test, would settle the matter. The reader already reached a CONDITIONAL verdict, and my analysis supports that verdict rather than moving to a different one. I therefore recommend UNCHANGED. In reaching this view, I considered other possible objections, such as the lack of released evaluation code and the absence of annotation statistics, but these are less directly tied to the central empirical claim and are secondary to the synchronization and annotation-transfer issue. I also note that the paper is honest about its scope and does not overclaim beyond the presented baselines, which is a point in its favor.","tokens_in":10194,"tokens_out":4405,"duration_ms":58239,"concrete_test":"Run an independent RGB-native annotation pass on a stratified 30-minute subset of the canonical split, using at least two annotators who have not seen the event-derived boxes. Measure per-frame IoU between the RGB-native boxes and the transferred event-derived boxes over time, and re-evaluate the RGB detectors (YOLO, RT-DETR, and Faster-RCNN) on this relabeled subset. If mean IoU is well below 0.75, if IoU decays with elapsed time within recordings, or if RGB mAP50 rises by more than 10 points relative to Table 2 under RGB-native labels, the central event-vs-RGB gap is substantially explained by synchronization or annotation bias rather than sensor capability. Optionally add a calibration sequence with a blinking LED to directly measure temporal drift between the two streams across the full recording duration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim, that event-based detectors clearly outperform RGB-based detectors (Table 2: YOLO event mAP50 87.68 vs RGB 35.24 on the canonical split), is vulnerable because the ground-truth boxes were produced on event frames and then transferred to RGB assuming temporal synchronization. Section 3 (Sensors) reports only that the RGB and event camera start times were aligned, with no measured drift or jitter over the approximately 120-second recordings. At the reported top speeds (e.g., 100 km/h for the Betafpv air75), even a few tens of milliseconds of unmodeled clock drift could shift the drone by many pixels at typical ranges, systematically penalizing RGB detectors while leaving event-based evaluation essentially unaffected. The annotation protocol compounds this: because labels are made on Prophesee event frames at 30 FPS and then applied to RGB, the RGB appearance cues such as motion blur, texture, and low-light visibility are not reflected in the labels. Thus the observed event-vs-RGB gap may be partly an artifact of the annotation and synchronization process rather than a genuine property of the sensors. Since the dataset's headline value and the paper's main empirical conclusion both rest on this cross-modal comparison, the missing synchronization-error characterization and the absence of any RGB-native annotation check are load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FRED, a multimodal dataset combining RGB video and event streams for drone detection, tracking, and trajectory forecasting. The dataset contains more than 7 hours per modality captured at 1280x720 resolution, with five drone models and scenarios including rain, nighttime, and insect distractor presence. The authors provide two evaluation splits (canonical and challenging), define detection, tracking, and forecasting protocols, and report baselines using YOLO, RT-DETR, Faster-RCNN, and the authors' own ER-DETR model. The headline empirical claim is that event-based detectors clearly outperform RGB-based detectors on the canonical split (e.g., YOLO event mAP50 87.68 vs. RGB mAP50 35.24, Table 2).","tokens_in":10468,"tokens_out":2934,"duration_ms":36354,"significance":"If the dataset and its annotations hold up, FRED is the largest event-based drone dataset to date and the first to offer tracking and forecasting benchmarks, which would be a valuable contribution to neuromorphic vision and drone perception. The paper provides explicit task definitions, standard metrics, two splits, and baseline results, and the dataset is released openly. The resource is potentially very impactful for the community. However, the central empirical conclusion—that events are dramatically better than RGB—depends on unmeasured temporal synchronization and an annotation protocol that constructs ground truth on event frames and transfers it to RGB, so the significance is conditional on validation of this pipeline.","major_comments":[{"comment":"The manuscript states that temporal synchronization is achieved by aligning the starting points of RGB and event recordings, with no measured synchronization error, drift, or jitter over the approximately 120-second recordings. This is load-bearing because the cross-modal annotations and the central event-vs-RGB comparison in Table 2 rely on per-frame alignment. At the reported maximum speed of 100 km/h for the Betafpv air75, a tens-of-millisecond offset can shift the drone by many pixels at typical ranges, which would systematically penalize RGB detectors whose labels came from event frames. I request a synchronization validation (e.g., a blinking LED or calibration target with known event/RGB timestamps), per-sequence sync error metadata, and a sensitivity analysis of the Table 2 results to simulated temporal offsets.","section":"Section 3 (Sensors)"},{"comment":"The annotation protocol creates bounding boxes on event frames and then transfers them to RGB frames, assuming the boxes are equally valid for both modalities. This assumption is questionable under the very conditions the dataset highlights, including motion blur, lens flare, shadows, and low light (Fig. 3). A box that is accurate on events may be systematically offset or have different extents on RGB appearance cues, and vice versa. Since the headline result (event detectors outperform RGB by a large margin) is obtained using these transferred labels, I request an RGB-native validation: annotate a random subset of RGB frames independently, report the agreement (e.g., mean IoU) between event-derived and RGB-native boxes, and re-evaluate the detection comparison on that subset. This is necessary to rule out that a large part of the 52-point mAP50 gap is an artifact of the annotation procedure.","section":"Section 3 (Annotation)"}],"minor_comments":[{"comment":"Several rows in Table 3 have missing separators, e.g., '57.147.6436' and '45.130177.4', making the MOTA, IDF1, and ID switch values ambiguous. Please reformat the table clearly.","section":"Table 3"},{"comment":"The first sentence contains a typo: 'neurmorphic' should be 'neuromorphic'.","section":"Section 6 (Conclusions)"},{"comment":"The reported maximum speed of 100 km/h for the Betafpv air75 is not clarified as measured or manufacturer-specified; please identify the source of this value.","section":"Section 3 (Drones)"},{"comment":"The loss weight λ is fixed at 0.5 with no ablation; a brief sensitivity check or a citation for the choice would help reproducibility.","section":"Section 4 (Drone Trajectory Forecasting)"},{"comment":"The table uses 'Rain' and 'Day/Night' columns, but VisEvent is marked as lacking rain despite the text mentioning adverse lighting; please make these categories consistent with the text or define them precisely.","section":"Table 1"},{"comment":"The dataset is released at a project URL; providing a DOI or versioned release would improve archival stability and citeability.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core risk is the synchronization and annotation pipeline, which directly determines the validity of the central event-vs-RGB comparison. This is fixable with additional measurements and a validation subset, so I am not recommending rejection. The paper should be revised to provide evidence that the cross-modal annotations are accurate before the benchmark conclusions can be fully trusted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FRED is worth knowing about and worth reviewing. It's the largest event-RGB drone dataset so far (7h+ per modality), the first with tracking and forecasting benchmarks, and it covers conditions prior datasets don't: rain, night, indoor scenes, five drone models from 30g FPV quads to DJI Minis. The two evaluation splits, one balanced and one deliberately shifted, are a good design. If you work on drone perception or event-based vision, this is a citeable resource regardless of its flaws.\n\nThe paper gets credit for transparency: it states plainly that annotations are drawn on event frames and copied to RGB, and that synchronization comes from aligning start times plus manual corrections. That transparency points to the real soft spot. The headline result, that event detectors beat RGB detectors by a wide margin (YOLO event mAP50 87.68 vs RGB 35.24; ER-DETR tracking at 0.5 IDF1 on RGB), is exactly what the unvalidated sync and the event-first labels could inflate. At 100 km/h a drone moves about 0.9m in one RGB frame interval; a few tens of milliseconds of unmodeled drift shifts it tens of pixels at ordinary ranges, penalizing RGB models while leaving event evaluation nearly untouched. Boxes drawn to an event blob also won't match RGB appearance when motion blur or low light changes what the RGB image shows. The paper reports no measured sync error, no stability test, and no RGB-native reannotation check. The Table 2 gap is suspicious enough that the event-beats-RGB conclusion should not be taken at face value until one of those is supplied. I'm not claiming the gap is fake; I'm claiming it's unvalidated.\n\nThe softer issues are minor: no annotation statistics (box counts, size distributions, inter-annotator agreement), no released baseline code, thin forecasting baselines. The self-citation is not a problem here—NeRDD and ER-DETR are the actual prior dataset and the natural baseline, and they say so.\n\nFor whom: anyone benchmarking event-based or multimodal drone perception, and anyone studying sensor tradeoffs for fast small objects. Recommendation: send it to peer review, but make sync validation and a small RGB-native annotation check conditions of acceptance, or ask the authors to soften the cross-modal claims to match what the evidence actually supports.","headline":"FRED is a genuinely new, useful event-RGB drone dataset, but its headline event-beats-RGB result rests on an unvalidated synchronization and an event-first annotation protocol that likely inflate the gap.","tokens_in":11005,"tokens_out":4821,"would_cite":true,"duration_ms":55008,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper presents FRED, the largest synchronized RGB-event drone dataset to date, and shows event-based detectors clearly outperform RGB-based ones across detection, tracking, and forecasting benchmarks.","keywords":["drone detection","event camera","neuromorphic vision","multimodal dataset","object tracking","trajectory forecasting","RGB-event synchronization"],"falsifier":"Record a known periodic pattern, such as an LED blinking at a measured frequency, with both cameras at the start and end of each recording session; if the phase offset between the event stream's timestamps and the RGB frame times drifts or jitters by more than a frame over time, the cross-modal annotations become invalid for that segment.","tokens_in":10040,"feed_emoji":"🚁","tokens_out":5481,"duration_ms":60236,"temperature":0.7,"pith_summary":"The paper introduces FRED, a multimodal dataset that pairs more than seven hours of 1280x720 RGB video with event-camera streams, all spatiotemporally synchronized and densely annotated with drone bounding boxes and track IDs. It is designed to be the largest event-based drone dataset to date and the first to include benchmarks for drone tracking and trajectory forecasting alongside detection. Baseline experiments on the dataset show a clear gap: event-based detectors reach roughly 88 mAP50 on the standard split while the same RGB detectors reach only about 35, and event-only models also hold up far better under lighting and weather shifts. The paper argues that this gap, plus the complementary signals in the two modalities, makes FRED a useful testbed for high-speed drone perception and multimodal spatiotemporal learning.","feed_headline":"Event cameras beat RGB detectors 2.5x on new drone benchmark","feed_subtitle":"A 7-hour synchronized RGB-event dataset with detection, tracking, and forecasting baselines shows event sensing leads.","key_machinery":"The load-bearing artifact is the dataset itself: a hardware setup with a Prophesee EVK4 event camera and an HD RGB camera on a shared mount, distortion-corrected and padded into a common pixel reference frame, with timestamps aligned at recording start and event frames extracted at the RGB frame rate. Annotations are produced on the sparse event frames, where drones are easier to isolate, then transferred to RGB via the spatial-temporal correspondence, and refined through automatic spatter tracking, manual correction, and interpolation. The benchmarks encode protocol choices—detection evaluated with the COCO mAP family, tracking with CLEAR MOT metrics over a 33 ms evaluation interval, forecasting with ADE/FDE/mIoU over 0.4 s and 0.8 s horizons—that define what it means for a model to succeed on FRED.","core_discovery":"FRED is presented as a new reference benchmark for drone perception: over 7 hours per modality of synchronized HD RGB and event streams from five drone models, with dense per-frame annotations, evaluation protocols, and canonical and challenging splits that deliberately separate easy images from domain-shifted hard cases. The central empirical claim is that event-based detectors substantially outperform RGB-only detectors on this data, with the best event model reaching 87.68 mAP50 versus 35.24 for the same YOLO architecture on RGB in the canonical split, and with a smaller gap persisting on the challenging split. The dataset is further claimed to be the first to offer event-based drone tracking and forecasting benchmarks, and the presented baselines indicate that event input alone, or combined with RGB, improves forecasting accuracy over box-only or RGB-only inputs. The authors state that fusing the modalities reduces the performance drop under distribution shift, showing the two streams capture complementary information.","pith_inferences":["If the event-vs-RGB gap generalizes beyond FRED, low-power drone detection systems may be able to deploy event-only sensors for many airspace-monitoring applications, using RGB only when texture or color matter.","The forecasting benchmarks could be extended to longer horizons and to predicting 3D position if depth or GPS telemetry were added, turning the data into a platform for interaction-aware trajectory prediction.","A direct check of the synchronization assumption—recording a blinking LED and measuring phase drift over time—would strengthen the dataset's claims and is a concrete experiment the authors do not report.","The fusion results suggest room for better fusion strategies; since event-only YOLO beats fused ER-DETR on the canonical split, a stronger event-RGB fusion that outperforms both is an open, testable target."],"forward_implications":["Event-only drone detectors are a stronger starting point than RGB-only detectors for high-speed and low-light drone perception, at least on this data.","Combining event and RGB streams helps maintain detection accuracy when the visual domain shifts (e.g., day-to-night, rain, insects), narrowing the performance drop seen by single-modality models.","The first standardized event-based tracking and forecasting benchmarks mean future work can compare drone trackers and predictors on a common, reproducible protocol.","The challenging split provides a quantitative way to test robustness to distracting events and adverse conditions, not just average accuracy on easy frames.","With over 7 hours per modality and five drone models, the dataset supports training of data-hungry multimodal detectors and forecasting models."],"supporting_citations":[{"why":"Surveys event-based vision and supplies the rationale for event cameras' high temporal resolution and dynamic range.","marker":"[4]"},{"why":"Presents NeRDD, the closest prior multimodal RGB-event drone dataset, and the ER-DETR fusion detector used as a baseline.","marker":"[12]"},{"why":"Presents F-UAV-D, the earlier 30-minute RGB-event drone dataset with synchronized camera pair, which this work scales up.","marker":"[14]"},{"why":"Provides the fixed-interval detection evaluation protocol and mAP methodology adopted for the detection benchmark.","marker":"[17]"},{"why":"Introduces YOLO v11, the detector behind the headline event-vs-RGB mAP comparison.","marker":"[8]"},{"why":"ByteTrack is the tracker used for the tracking-by-detection baselines.","marker":"[29]"},{"why":"Defines the CLEAR MOT metrics (MOTA, IDF1, ID switches) used to evaluate tracking.","marker":"[3]"}],"fun_headline_variants":["Event cameras beat RGB by 2.5x on drone detection","FRED: first RGB-event dataset for drone tracking and forecasting","Drone detection: 87.7 mAP with events vs 35.2 with RGB","Event cameras outperform RGB on new drone dataset","7-hour drone dataset reveals event camera edge in detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that aligning the RGB and event camera start times once, together with a fixed 30 FPS RGB clock, keeps the two modalities temporally synchronized to pixel-and-frame accuracy across the entire 7-hour corpus, without measurable drift or jitter.","fun_headline_variants_meta":{"raw":{"variants":["Event cameras beat RGB by 2.5x on drone detection","FRED: first RGB-event dataset for drone tracking and forecasting","Drone detection: 87.7 mAP with events vs 35.2 with RGB","Event cameras outperform RGB on new drone dataset","7-hour drone dataset reveals event camera edge in detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001767,"raw_usage":{"total_tokens":6933,"prompt_tokens":871,"completion_tokens":6062,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":5974}},"tokens_in":487,"tokens_out":6062,"duration_ms":47142,"temperature":1.0,"reasoning_tokens":5974,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:22:58.233652+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record a known periodic pattern, such as an LED blinking at a measured frequency, with both cameras at the start and end of each recording session; if the phase offset between the event stream's timestamps and the RGB frame times drifts or jitters by more than a frame over time, the cross-modal annotations become invalid for that segment.","supporting_citations":[{"cited_title":"Davison, Jörg Conradt, Kostas Daniilidis, and Davide Scaramuzza","cited_arxiv_id":null,"evidence_quote":"Surveys event-based vision and supplies the rationale for event cameras' high temporal resolution and dynamic range."},{"cited_title":"Neuromorphic drone detection: an event-rgb multimodal approach","cited_arxiv_id":null,"evidence_quote":"Presents NeRDD, the closest prior multimodal RGB-event drone dataset, and the ER-DETR fusion detector used as a baseline."},{"cited_title":"Towards real-time fast unmanned aerial vehicle detection using dynamic vision sensors","cited_arxiv_id":null,"evidence_quote":"Presents F-UAV-D, the earlier 30-minute RGB-event drone dataset with synchronized camera pair, which this work scales up."},{"cited_title":"Bytetrack: Multi-object tracking by associating every detection box","cited_arxiv_id":null,"evidence_quote":"ByteTrack is the tracker used for the tracking-by-detection baselines."}],"review_version":1}