{"id":"df0428f6-86b3-482a-b911-dc8f6e08e714","arxiv_id":"2504.17371","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DSC3D is a new open 3D trajectory dataset from drone footage with over 175,000 annotated 6DoF trajectories across five locations and 14 traffic participant classes.","lead":"This paper introduces DSC3D, a new drone-captured dataset of over 175,000 three-dimensional traffic trajectories across five European and US locations. It is among the first drone datasets to provide full 6-degree-of-freedom bounding boxes and 14 object classes, aimed at improving autonomous driving research.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 4.8 cm accuracy claim is not independently established: the detection network is finetuned on the very dataset it evaluates, so the reported median error may be optimistically biased.","rationale":"The reader identified the same load-bearing weakness: the accuracy claim rests on an evaluation that may be circular because the detection model was trained on the same dataset images used for validation. My independent review of the manuscript confirms this and adds two supporting details: the finetuning statement in Section III-F and the fact that the validation is performed on detections rather than on the final smoothed trajectories. Because the paper's central value proposition is the dataset's high accuracy and 3D completeness, and because the accuracy figure is the primary quantitative evidence for that proposition, the concern is material. However, the dataset itself, with its scale, diversity, public release, and downstream applications, is a real contribution that survives even if the accuracy claim is weakened. The reader's CONDITIONAL verdict is therefore appropriate: the dataset should be accepted with the condition that the accuracy evaluation be redone on a held-out set, or that the claim be reworded to reflect the lack of independent validation.","tokens_in":12234,"tokens_out":2665,"duration_ms":29708,"concrete_test":"Re-run the accuracy evaluation on a strictly disjoint split: first identify every track whose images were used in the finetuning stage described in Section III-F, remove all of those tracks from the evaluation set, and recompute the median positional error on the remaining tracks. If the recomputed error is materially above 4.8 cm, or if the authors cannot construct such a split because all annotated tracks were used for finetuning, then the headline accuracy figure should be withdrawn or substantially qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central contribution is a high-quality 3D trajectory dataset, and the headline number is a median positional error of 4.8 cm. For that claim to hold, the error must be measured against an independent reference, on data not used to train or finetune the annotating model. The paper does not satisfy this. Section IV-B states: 'The detection validation was performed by comparing the positional median error between predicted detections and manually annotated detections of the same objects' and then explicitly notes that 'images of this dataset were included in the training of the detection network.' Section III-F adds that the authors 'finetune our 3D detection model with highly curated, manually-annotated images.' If the manually annotated images used for finetuning overlap with those used for validation, the evaluation is circular and the 4.8 cm figure is an upper bound on the model's fit to its own training labels, not a measure of true trajectory accuracy. The paper gives no evidence of a track-disjoint split or of any held-out set. Moreover, the manual annotations themselves are not an independent ground truth: they are derived from the same monocular images and the same reconstructed ground mesh, so systematic errors in the mesh (15 cm 3D RMSE) or camera poses (1.9 m GPS RMSE) propagate into both the predictions and the reference, making the comparison potentially self-consistent while still inaccurate in absolute terms. Finally, the validation is on detections, not on the final Kalman/RTS-smoothed trajectories, so even a correct detection error does not directly certify the trajectory error that users consume. The dataset may still be useful, but the central accuracy claim is not supported as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DSC3D, a drone-captured traffic dataset with 15 hours of footage, 177,151 unique trajectories, 14 object categories, and full 3D annotations (6DoF boxes, dimensions, orientations) across five locations in Germany and the United States. The authors describe a monocular pipeline involving geo-referenced 3D scene reconstruction, ground mesh generation, camera calibration, GroundMix-based 3D detection, detection refinement, and Kalman-filter tracking. They claim a median positional accuracy of 4.8 cm, and they demonstrate applications in motion prediction, scenario mining, and generative traffic agents. The dataset and an interactive visualization platform are publicly available.","tokens_in":12597,"tokens_out":2560,"duration_ms":28177,"significance":"If the correctness of the annotations can be established, DSC3D would be a valuable community resource: it is the first drone-based dataset to offer complete 3D trajectories with 6DoF poses and metric dimensions, and its scale and class diversity substantially exceed prior drone trajectory datasets. The public release, the interactive visualization interface, and the downstream evaluations on prediction and scenario mining are strengths. However, the headline accuracy claim of 4.8 cm is not currently supported by an independent validation protocol, and because accuracy is a central advertised property of the dataset, this issue is load-bearing for the paper's main contribution.","major_comments":[{"comment":"The accuracy validation is circular as described. Section IV-B states that the positional median error is computed by comparing predicted detections with manually annotated detections, and it explicitly notes that 'images of this dataset were included in the training of the detection network.' Section III-F adds that the detection model is finetuned on 'highly curated, manually-annotated images' from the same data. Without a track-disjoint or scene-disjoint split, and without an explicit statement that the validation annotations were never used in training or finetuning, the 4.8 cm figure measures the model's fit to its own training distribution rather than the absolute accuracy of the derived trajectories. Please provide the evaluation protocol in full: how the manual annotations were partitioned, how many objects were annotated, and how the 4.8 cm median was computed from the per-frame or per-trajectory errors.","section":"Section IV-B"},{"comment":"The manual annotations are not an independent ground truth. Both the predicted detections and the reference annotations are derived from the same monocular images and the same reconstructed ground mesh, so systematic errors in the mesh (15 cm 3D RMSE) and the camera poses (1.9 m GPS RMSE) propagate into both sides of the comparison. A self-consistent comparison can therefore report small relative errors while the absolute positions are significantly less accurate. To support the 4.8 cm claim, the authors should compare against an external absolute reference (for example RTK-GPS on a survey vehicle, LiDAR scans, or surveyed ground control points) or at least quantify the worst-case error contribution of the 15 cm mesh error and 1.9 m pose error to object positions.","section":"Section III-B, III-D, IV-B"},{"comment":"The paper's headline number is not reproducible from the reported experiments. The abstract and contributions state a median positional error of 4.8 cm, but Section IV-B only reports a 'NuScenes score of 97%' and does not define the positional error metric, the number of validation frames or objects, or a per-location breakdown. Table II reports RMSE values for GPS and reconstruction but not the 4.8 cm detection error. Please add a dedicated accuracy subsection with a precise definition of the positional error (e.g., 3D center error versus ground-center error), the evaluation split, per-location results, and the number of annotated tracks used for validation.","section":"Abstract, Section IV-B, Table II"},{"comment":"The reference for the RTS-Smoother is incorrect: the text cites '[7]' (the exiD paper) for an RTS-smoother, but the exiD paper is a dataset paper and does not introduce this technique. Please cite the original Rauch-Tung-Striebel smoother reference or a method paper that describes the smoothing step, and clarify how smoothing affects the reported accuracy figures.","section":"Section III-F"}],"minor_comments":[{"comment":"The dataset name is inconsistent: the text and Table I use 'AD4CHE' while Table III uses 'AD4ACHE'; please unify the spelling.","section":"Section IV-C, Table III"},{"comment":"Reference [36] contains a typo: 'by aearial imagery' should be 'by aerial imagery'. Please also verify the authors and venue details of this reference.","section":"References"},{"comment":"The term 'NuScenes score of 97%' is ambiguous; the nuScenes detection benchmark uses a specific metric (NDS), and it is unclear how that metric is adapted to this monocular drone setting. Please define the score and its range.","section":"Section IV-B"},{"comment":"In Table I, the symbol '?' for CitySim's video availability is undefined in the caption; please either define it or replace it with a clear yes/no/partial indicator.","section":"Tables and Figures"}],"recommendation":"major_revision","confidential_remarks":"The dataset itself appears to be a genuinely useful and timely contribution, and the infrastructure behind it (public release, visualization platform, application studies) is solid. The main risk is the accuracy claim: because the validation is trained-on-the-target-data and the manual reference shares error sources with the predictions, the 4.8 cm number cannot be taken at face value. I would advise the editor that the paper is likely salvageable with a major revision that either provides an independent accuracy evaluation or substantially softens the accuracy claim and reframes the contribution around diversity and 3D completeness rather than centimeter-level precision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe key thing to know: DSC3D is a real, substantial new resource — the first drone-based trajectory dataset I know of that offers full 3D 6DoF bounding boxes with dimensions and 14 classes across five location types. The scale (175k trajectories, 15 hours) and the public release with an online viewer make it immediately useful for motion prediction, planning, and simulation research. That part is genuinely new and worth engaging with.\n\nWhat the paper does well: the pipeline is clearly described (SfM/MVS reconstruction, FlexRoad ground surface, GroundMix monocular 3D detection, PnP-based frame calibration, Kalman+RTS smoothing), and the authors are transparent about relying on their own prior components. The application vignettes — DeepUrban benchmark, traffic-rule compliance study, scenario mining distributions, generative agents — show the dataset is already being used productively. The comparison Table I is honest about the field.\n\nThe soft spots are real and need to be named. The headline \"median positional error of 4.8 cm\" is not independently established. The paper states that images from the dataset were included in the training of the detection network, and that validation compares predicted detections to manual annotations of the same objects. The manual annotations are also derived from the same monocular footage and the same reconstructed mesh, so systematic mesh/pose errors (15 cm 3D RMSE, 1.9 m GPS RMSE) can be shared between prediction and reference. On top of that, the validation is on raw detections, not the final smoothed trajectories users will download. So the 4.8 cm figure is a model-fit number, not an external accuracy certification. That said, the dataset's value does not entirely rest on that number; the relative geometry and diversity are likely sufficient for many behavioral modeling tasks. But the accuracy claim needs to be re-run on a held-out set with independent reference (e.g., GCPs) before it should be cited as ground truth.\n\nMinor: the abstract calls the data \"occlusion-free,\" while the related-work section admits building occlusions and low-height limitations. That's an overstatement, but not load-bearing.\n\nBottom line: the dataset contribution holds up; the validation section does not. If you work on trajectory prediction or simulation, this is worth citing and using after checking the data quality yourself. I would send it to a serious referee, with the clear demand that the accuracy evaluation be redone non-circularly before the paper is accepted as a benchmark reference.","headline":"Genuinely new 3D drone trajectory dataset, but the 4.8 cm accuracy claim is circular until validated on a held-out set.","tokens_in":13120,"tokens_out":3213,"would_cite":true,"duration_ms":28813,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"First public drone dataset offers full-3D traffic trajectories for 14 classes.","keywords":["3D trajectory dataset","drone data collection","autonomous driving","6DoF bounding boxes","monocular detection","traffic participants","scenario mining","motion prediction"],"falsifier":"Re-capture a subset of the five scenes with surveyed ground-control points and high-precision references such as RTK-GPS on instrumented vehicles, then compare the released trajectories against those independent positions; if the median error substantially exceeds 4.8 cm, the accuracy claim collapses.","tokens_in":12002,"feed_emoji":"🚁","tokens_out":7073,"duration_ms":61363,"temperature":0.7,"pith_summary":"The paper introduces DSC3D, a drone-captured dataset of traffic movement with complete 3D bounding-box annotations for 14 categories across five locations in Germany and the United States. It is presented as the first publicly available dataset of its kind, with more than 175,000 trajectories, about 15 hours of footage, and a claimed median positional error of 4.8 centimeters. The need arises because existing drone datasets provide only 2D or 2.5D tracks, while vehicle-mounted sensors suffer from occlusion and limited range; 3D trajectories would let planners, predictors, and simulators reason about true spatial relations. The paper demonstrates that adding scenarios extracted from the dataset to an established training set improves motion-prediction error by roughly 44%, and that the data support scenario mining and generative traffic agents.","feed_headline":"First drone dataset logs 175,000 traffic tracks in full 3D","feed_subtitle":"Complete 6DoF boxes for 14 classes across five locations aim to improve prediction and simulation.","key_machinery":"The load-bearing mechanism is a monocular ground-aware 3D tracking pipeline: the scene is first reconstructed with structure-from-motion and multi-view stereo into a geo-referenced 3D mesh, and a NURBS ground surface is fitted to the road; each frame's camera pose is obtained by feature-matching the frame to the reconstructed scene and solving PnP within RANSAC, then smoothed with a Kalman filter. Monocular detections (2D box, class, dimensions, orientation, depth) are refined by intersecting the camera ray through the detected ground center with the ground mesh, and by re-aligning the object's orientation to the ground normal. This converts per-frame pixel detections into metric 3D boxes that remain consistent across frames, after a Kalman filter and RTS smoother produce the final trajectories.","core_discovery":"DSC3D is the first publicly available drone-based trajectory dataset in which every tracked object carries a complete 6-degree-of-freedom 3D bounding box with metric dimensions, in geo-referenced coordinates, for 14 classes. The dataset covers a parking lot, a pedestrian-dense inner-city street, an unsignalized steep intersection, a federal highway, and a suburban intersection, yielding 177,151 unique trajectories over about 15 hours. The paper claims that its monocular tracking pipeline, which refines single-camera detections against a reconstructed ground mesh, achieves a median positional error of 4.8 cm when compared with manual annotations, and a 97% NuScenes-style score evaluated in 3D space. It further argues that this level of accuracy and 3D completeness is what makes the data useful for motion prediction, planning, scenario mining, and reactive traffic simulation.","pith_inferences":["The 4.8 cm median error is an internal estimate: it is computed against manual annotations that the detector itself was trained on, so users should treat it as a claim to verify independently rather than an established ground-truth accuracy.","If the accuracy claim survives independent checks, the pipeline's recipe—monocular detection plus reconstructed ground mesh—could be applied to convert existing drone video archives into 3D trajectory data at scale, not just for the five locations shown.","The inclusion of slopes up to 20% offers a natural testbed for whether current flat-ground prediction models fail on non-planar scenes, a question the paper does not directly pursue.","Public release with an interactive visualizer invites community scrutiny of annotation quality, which could either confirm or undermine the claimed accuracy more effectively than the paper's own validation."],"forward_implications":["Motion-prediction and planning models can be trained directly on metric 3D trajectories with true object sizes and orientations, enabling reasoning on non-planar surfaces like slopes and multi-level interactions.","The dataset's 14-class taxonomy and its long trajectories (up to 984 s) support scenario mining for safety-critical events, including parking maneuvers and pedestrian-heavy urban interactions.","Adding extracted scenarios from DSC3D to an existing driving dataset improves average displacement error by 44.1% and final displacement error by 44.3%, indicating that drone 3D data can improve generalization of current predictors.","The geo-referenced HD maps and 3D mesh make the dataset directly usable for simulation, digital twins, and closed-loop generative traffic agents.","Occlusion-free drone perspectives allow nearly every vehicle in a scene to serve as a potential ego vehicle, expanding the volume of usable training scenarios per hour of recording."],"supporting_citations":[{"why":"Provides the GroundMix monocular detector that produces the initial 3D bounding-box predictions refined by the pipeline.","marker":"[35]"},{"why":"Supplies FlexRoad, which generates the smooth ground mesh used to correct detection depth and orientation.","marker":"[29]"},{"why":"LoFTR feature matching registers recording frames to the reconstructed 3D scene for camera calibration.","marker":"[32]"},{"why":"LightGlue is the alternative feature matcher used for the same registration step.","marker":"[33]"},{"why":"Supplies the RTS smoother that enhances trajectory smoothness after Kalman filtering.","marker":"[7]"},{"why":"NuScenes supplies the evaluation metric and the training set that, augmented with scenarios from this dataset, shows utility for motion prediction.","marker":"[10]"},{"why":"The DeepUrban benchmark built from this dataset is used to quantify improvements in prediction and planning.","marker":"[36]"},{"why":"An external comparison of driving behaviors across datasets supports the claim that the data reflect realistic, rule-compliant driving.","marker":"[37]"}],"fun_headline_variants":["Drone dataset logs 177k full 3D traffic tracks","First drone 6DoF traffic dataset: 5 scenes, 14 classes","Occlusion-free 3D tracks: 175k trajectories from drones","DeepScenario: 177k drone-captured 6DoF trajectories"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported centimeter-level accuracy assumes the monocular detector and tracker produce trajectories reliable enough to serve as ground truth, even though the detector was trained on images from this same dataset and the error is measured against manual annotations of those same recordings.","fun_headline_variants_meta":{"raw":{"variants":["Drone dataset logs 177k full 3D traffic tracks","First drone 6DoF traffic dataset: 5 scenes, 14 classes","Occlusion-free 3D tracks: 175k trajectories from drones","DeepScenario: 177k drone-captured 6DoF trajectories"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000228,"raw_usage":{"total_tokens":1500,"prompt_tokens":992,"completion_tokens":508,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":426}},"tokens_in":608,"tokens_out":508,"duration_ms":5338,"temperature":1.0,"reasoning_tokens":426,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:41:09.588455+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-capture a subset of the five scenes with surveyed ground-control points and high-precision references such as RTK-GPS on instrumented vehicles, then compare the released trajectories against those independent positions; if the median error substantially exceeds 4.8 cm, the accuracy claim collapses.","supporting_citations":[{"cited_title":"CARLA Drone: monocular 3d object detection from a different perspective,","cited_arxiv_id":null,"evidence_quote":"Provides the GroundMix monocular detector that produces the initial 3D bounding-box predictions refined by the pipeline."},{"cited_title":"Shape your ground: Refining road surfaces beyond planar representations,","cited_arxiv_id":null,"evidence_quote":"Supplies FlexRoad, which generates the smooth ground mesh used to correct detection depth and orientation."},{"cited_title":"Lightglue: Local feature matching at light speed,","cited_arxiv_id":null,"evidence_quote":"LightGlue is the alternative feature matcher used for the same registration step."},{"cited_title":"The exid dataset: A real-world trajectory dataset of highly interactive highway scenarios in germany,","cited_arxiv_id":null,"evidence_quote":"Supplies the RTS smoother that enhances trajectory smoothness after Kalman filtering."},{"cited_title":"nuscenes: A multimodal dataset for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"NuScenes supplies the evaluation metric and the training set that, augmented with scenarios from this dataset, shows utility for motion prediction."},{"cited_title":"DeepUrban: Interaction-aware trajectory prediction and planning for automated driving by aearial imagery,","cited_arxiv_id":null,"evidence_quote":"The DeepUrban benchmark built from this dataset is used to quantify improvements in prediction and planning."},{"cited_title":"Traffic and safety rule compliance of humans in diverse driving situations,","cited_arxiv_id":null,"evidence_quote":"An external comparison of driving behaviors across datasets supports the claim that the data reflect realistic, rule-compliant driving."}],"review_version":1}