{"id":"70d80267-d2a2-47db-8ce5-46307b154a32","arxiv_id":"2506.17346","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A task-centric, five-layer data quality framework for autonomous driving, illustrated by BCS-based redundancy pruning on nuScenes that maintains or slightly improves YOLOv8 object detection.","lead":"Autonomous driving research usually focuses on models, not on whether the sensor data feeding them is good. This paper proposes a five-layer framework for judging data quality per driving task and shows, on a small dataset, that removing redundant camera and LiDAR data can keep object detection accuracy about the same or slightly better.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BCS pruning confounds redundancy removal with label-quality filtering; mAP gains don't establish the redundancy claim without random/clean-only baselines.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing concern: the BCS pruning rule conflates redundancy removal with annotation-quality filtering. The framework itself is conceptual, so the case study is the only empirical evidence for the central claim. If the mAP gains come from deleting truncated boundary boxes rather than from removing redundant information, the abstract's assertion that \"partially removing redundancy on multisource image data could improve YOLOv8 object detection task performance\" is unsupported. The paper does not provide random-pruning or clean-only baselines, error bars, repeated runs, or a clear train/test split, and the threshold is swept and selected post hoc. These issues are addressable with additional experiments, so the paper should not be rejected outright, but the central claim should not be accepted without the proposed control. The reader's CONDITIONAL verdict remains appropriate.","tokens_in":17900,"tokens_out":3055,"duration_ms":32188,"concrete_test":"Re-run the Section III-D-2 experiment with three arms at identical removal rates on a fixed train/val split with at least five seeds: (1) BCS-overlap pruning as in the paper; (2) random pruning of the same number of boxes; (3) clean-only pruning that removes the lowest-BCS boxes regardless of overlap membership. If arms (2) or (3) reproduce the mAP50 gains of arm (1) within error bars, the redundancy-specific claim is falsified; if arm (1) significantly outperforms both controls, the redundancy interpretation survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical support for the central claim rests on the BCS pruning rule in Section III-D-2(d) (Eqs. 1–2). Within each overlapping-FoV group, the rule discards the lower-BCS box whenever max BCS − min BCS > τ_BCS. Since BCS is the fraction of the full bounding box visible after clipping to image boundaries, lower-BCS boxes are precisely the ones truncated at image edges. The manipulation therefore removes low-quality annotations as well as, or instead of, redundant information. The paper's own Discussion (Section III-E) describes the effect as \"removing lower-quality duplicates,\" conceding the confound. No comparison is made against random pruning at matched removal rates, nor against cleaning-only pruning that removes low-BCS boxes outside overlapping regions. Consequently, the observed mAP50 improvement in Fig. 8 could be entirely a label-quality effect: the model is trained on fewer truncated, ambiguous boxes. The central claim that a less redundant dataset can match or surpass the full dataset requires that redundancy, not annotation quality, is the manipulated variable. That load-bearing premise is not yet secured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a five-layer task-centric data quality (DQ) framework for autonomous driving, called the Vase Framework, whose layers are Data, Data Quality, Task, Application, and Goal. The framework is intended to map DQ dimensions and metrics to task requirements and performance objectives. As a case study, the authors examine redundancy in nuScenes-mini camera images: they identify overlapping fields of view among six cameras, define a Bounding Box Completeness Score (BCS), prune lower-BCS boxes in overlapping detections, and train YOLOv8 on the pruned datasets. They also analyze image-LiDAR redundancy, report that close-range boxes are more cross-modally redundant, and prune near-range LiDAR boxes. The central empirical claim is that partially removing redundancy from multisource image data can improve or maintain YOLOv8 detection performance, so that a less redundant training dataset can match or surpass the full dataset.","tokens_in":18124,"tokens_out":3251,"duration_ms":35428,"significance":"If the central claim were established, the paper would support task-centric data curation for AV perception: redundancy evaluation could guide pruning decisions that reduce training cost without sacrificing accuracy. The paper's strengths are its clear conceptual framework, the use of a public dataset, qualitative illustrations of overlapping fields of view, and an anonymized public code link. The framework and the open research questions are useful structuring contributions. However, the empirical evidence for the load-bearing causal claim is currently weak: the pruning rule conflates redundancy with annotation completeness, the experiments lack controlled baselines and statistical confidence, and the LiDAR pruning threshold is derived post hoc from the same data. These issues need to be addressed before the claimed empirical conclusion can be accepted.","major_comments":[{"comment":"The BCS-based pruning rule conflates redundancy removal with label-quality filtering. Because BCS is the fraction of the full bounding box visible after clipping to image boundaries, the rule in Eq. (2) preferentially removes boxes truncated at image edges, i.e., lower-quality annotations, not necessarily redundant information. The observed mAP50 improvement in Fig. 8 could therefore be entirely a label-cleaning effect: the model is trained on fewer ambiguous clipped boxes. The Discussion in Section III-E concedes this when it describes the mechanism as \"removing lower-quality duplicates.\" To support the claim that redundancy is the manipulated variable, the authors must compare against random pruning at matched removal rates and against a cleaning-only baseline that removes low-BCS boxes outside overlapping regions. Without these baselines, the abstract's claim that \"partially removing redundancy ... could improve YOLOv8 object detection task performance\" is not established.","section":"Section III-D-2(d), Eqs. (1)-(2), and Section III-D-4, Fig. 8"},{"comment":"The empirical support for the central claim is a single-run mAP50 experiment on nuScenes-mini (10 scenes, 404 frames, 1,401 images), with no error bars, no repeated seeds, and no explicit train/test split description. The pruning thresholds tau_BCS are swept from 0.0 to 1.0 and the mAP50 values are compared on the same dataset from which the thresholds were selected, introducing selection bias and making it impossible to know whether the reported gains are robust. The conclusion in Section III-D-4 that \"a less redundant training dataset can achieve or even surpass the performance level of using the full training dataset\" requires repeated trials, confidence intervals, and a held-out evaluation protocol. As reported, the conclusion is not statistically supported.","section":"Section III-D-1, Section III-D-4, and Fig. 8"},{"comment":"The LiDAR pruning rule is derived post hoc from the same data it is evaluated on. The t-test showing high cross-modal redundancy for close objects (p-value=1.17e-76) is used to justify removing all boxes within distance Tdist, but Tdist is selected on the basis of the observed statistics rather than an a priori hypothesis. Furthermore, the paper does not report a quantitative detection metric for the pruned LiDAR experiments: Fig. 11 appears to show removed boxes and a qualitative statement that detections are \"unaffected,\" but no mAP, recall, or precision numbers are provided for the pruned setting. The redundancy ratio RR in Eq. (3) is computed as agreement between LiDAR-only detections and fusion detections, which is a detection-agreement measure rather than a direct data-redundancy measure and is not validated against ground truth. The claim that close-range LiDAR data is redundant and can be pruned without harming task performance is therefore not yet demonstrated.","section":"Section III-D-3, Eqs. (3)-(5), and Section III-D-4, Figs. 9-11"}],"minor_comments":[{"comment":"The abstract contains a grammatical issue: \"The proposed framework aims to mapping DQ\" should read \"aims to map DQ.\" The paper also uses inconsistent spacing in \"A Vs\" throughout.","section":"Abstract and Section I"},{"comment":"The paper says the full nuScenes dataset has 1,000 scenes but the case study uses nuScenes-mini with 10 scenes; the conclusion should explicitly state that the empirical results are a pilot study and should not be generalized, especially in the abstract and contribution list.","section":"Section III-A and Section V"},{"comment":"The description of the threshold sweep says thresholds are set from 0.0 to 1.0 with a 0.2 interval, but it is not stated whether each threshold value is evaluated once or multiple times, nor how the different camera pairs' pruned datasets are combined for training. Clarifying the training and evaluation protocol and showing per-run variability would help.","section":"Section III-D-4, Fig. 8"}],"recommendation":"major_revision","confidential_remarks":"The conceptual framework is a reasonable contribution, but the empirical validation section currently does not support the paper's central causal claim. The missing random-pruning and cleaning-only baselines, the absence of error bars or repeated trials, and the post-hoc LiDAR threshold selection are fixable with additional experiments, so I do not recommend rejection. However, the paper needs substantial revision before it can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The five-layer Vase Framework is a reasonable conceptual map for thinking about AV data quality, and the open questions in Section IV are sensible. But the case study's central empirical claim doesn't survive scrutiny: the BCS pruning rule removes truncated bounding boxes, which is label-quality filtering, not redundancy removal.\n\nWhat's actually new: the framework (data, DQ, task, application, goal) is a clean way to organize task-centric data quality, and the paper is honest that this is preliminary work, shipping code and data. The specific experiment—pruning overlapping multi-camera boxes from nuScenes-mini and retraining YOLOv8—is not in the cited prior work.\n\nWhere it gets soft. The central claim that a less redundant training dataset can match or surpass the full dataset is not supported. Equations (1)–(2) define BCS as the fraction of a box visible after cropping to image boundaries, so the pruning rule preferentially deletes boxes truncated at image edges. Those are exactly the low-quality labels. The Discussion even says the effect is \"removing lower-quality duplicates,\" which concedes the confound. Without a random-pruning baseline at matched removal rates, or a clean-only baseline removing incomplete boxes outside overlapping regions, the mAP50 gains in Fig. 8 could be entirely a label-quality effect. The LiDAR experiment is weaker: Tdist is swept and chosen post hoc on the same data, and the only quantitative support is a t-test showing close-range objects have high redundancy, not that pruning them preserves performance. On top of that, the image experiment is a single run with no error bars, no repeated seeds, and no clear train/test split, on nuScenes-mini (10 scenes). The citation pattern is fine; they draw on IoT architecture and prior DQ surveys, and relevant work is cited.\n\nNone of these flaws are fatal to the paper's spirit. The framework is useful, and the empirical issues are fixable with controlled experiments. But the abstract's phrasing—\"proves that partially removing redundancy... could improve\"—overstates what the evidence shows.\n\nWho it's for: researchers working on data-centric AI for AV, especially those thinking about data pruning or quality-aware data selection. A serious referee should engage; if the authors add random and clean-only baselines, multiple seeds, and proper splits, the paper could be solid. As it stands, the empirical claim is conditional. Send it to review but expect major revision; desk reject would be too harsh given the framework's value.","headline":"The Vase Framework is a useful conceptual map for AV data quality, but the case study's central empirical claim is confounded: the BCS rule removes truncated boxes, which is label-quality filtering, not pure redundancy removal.","tokens_in":18617,"tokens_out":2243,"would_cite":false,"duration_ms":23627,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Removing redundant camera views can match or beat full AV training data","keywords":["autonomous driving","data quality","task-centric framework","redundancy removal","object detection","multimodal data","nuScenes","YOLOv8"],"falsifier":"Train YOLOv8 on three equal-size subsets of the overlapping camera data: BCS-pruned, randomly pruned, and pruned only of boxes clipped at image boundaries. If random or boundary-only pruning matches the BCS-pruned mAP50, then the completeness score is not acting as a redundancy remover.","tokens_in":17683,"feed_emoji":"🚗","tokens_out":5050,"duration_ms":47187,"temperature":0.7,"pith_summary":"The paper argues that autonomous driving systems should manage data quality task by task, rather than only improving models, and proposes a five-layer 'Vase' framework that ties raw sensor data, quality dimensions, tasks, applications, and performance goals together. To show the framework works, the authors use redundancy as a test case: they identify overlapping fields of view among nuScenes cameras, score each duplicated bounding box for completeness, and remove the less complete duplicates. Their central empirical claim is that a less redundant training set can match or even beat the full dataset for YOLOv8 object detection, measured by mAP50. They further show that image-LiDAR redundancy concentrates near the ego vehicle, and that removing close-range LiDAR points barely hurts detection. A sympathetic reader would take away that per-task data-quality decisions are a viable lever for cheaper, faster AV perception.","feed_headline":"Pruning redundant camera data lifts AV detection accuracy","feed_subtitle":"On nuScenes, YOLOv8 trained on a reduced-redundancy dataset matches or exceeds full-data mAP50.","key_machinery":"The central object is the five-layer Vase Framework, which connects a data layer (multisource, multimodal sensor input), a data quality layer (dimensions such as completeness, consistency, and redundancy), a task layer (here, object detection), an application layer, and a goal layer (accuracy, latency, efficiency). The argument's load is carried by two pruning mechanisms. For camera redundancy, the Bounding Box Completeness Score compares the visible clipped area of a duplicated box with its full area; when the gap between the most and least complete boxes in a redundant group exceeds a threshold $\\tau_{BCS}$, the less complete box is discarded. For LiDAR redundancy, a redundancy ratio counts how many fused 3D detections also appear in LiDAR-only detection, and pruning removes all boxes whose centroid lies within a distance $T_{dist}$. These rules turn the abstract notion of redundancy into an executable data-selection policy, and the remaining loops of the framework feed detection metrics back to adjust quality thresholds.","core_discovery":"On its own terms, the paper's discovery is that data quality, measured per task, behaves like a first-class engineering quantity in autonomous driving pipelines. Concretely, the study claims that partially removing redundancy from multi-view camera images improves rather than degrades YOLOv8 object detection: for the six overlapping camera pairs in nuScenes, training on datasets with redundancy removed by a Bounding Box Completeness Score threshold reaches the same or higher mAP50 than training on the full data. For multimodal data, the paper claims that high cross-modal redundancy between image and LiDAR is concentrated at close range, with a reported T-test p-value of $1.17\\times10^{-76}$, and that removing close-range LiDAR boxes through a distance threshold preserves detection outcomes while reducing processed points. Together these results are offered as evidence that a task-centric data quality framework can guide redundancy removal without sacrificing task performance.","pith_inferences":["My inference: the reported gains may be driven more by label quality than by redundancy, because the BCS rule preferentially removes boxes clipped at image boundaries; a comparison against random pruning or cleaning only clipped boxes would isolate the cause.","The distance-based LiDAR pruning suggests a testable extension: replace the fixed $T_{dist}$ with a learned or per-scene threshold that adapts to object density, and check whether the redundancy hotspots move.","Cross-modal redundancy concentrated near the ego vehicle implies that sensor-fusion weighting could be spatially adaptive, for example by downweighting close-range LiDAR where cameras already see well.","The same BCS machinery could be applied to temporal redundancy across consecutive frames, where object persistence creates similar duplicate-information patterns."],"forward_implications":["Per-task data pruning becomes a safe default: for object detection on overlapping camera views, removing redundant instances need not cost accuracy and can improve mAP50.","Storage and compute budgets can shrink: smaller training sets and fewer LiDAR points to process yield efficiency gains while keeping detection performance stable.","The BCS rule preserves the approximate distribution of instances across camera pairs, which suggests that redundancy removal can keep inter-sensor representation balanced.","Close-range LiDAR returns are a redundancy hotspot in image-LiDAR fusion, and distance-based pruning targets exactly that hotspot.","The framework's feedback loop gives a principled way to tune quality thresholds: relax redundancy tolerance if task metrics drop, tighten it when efficiency matters."],"supporting_citations":[{"why":"Supplies the nuScenes dataset, whose six-camera 360-degree setup and image-LiDAR annotations define the case study.","marker":"[21]"},{"why":"Supplies YOLOv8, the object detection model whose mAP50 and recall measure the effects of pruning.","marker":"[27]"},{"why":"Prior evidence that removing a large share of redundant point clouds causes only a subtle performance drop, motivating the redundancy hypothesis.","marker":"[62]"},{"why":"Defines redundancy as similarity between point-cloud pairs under ego-motion, giving the paper a geometric redundancy notion to build on.","marker":"[73]"},{"why":"Frames redundancy as a content data quality issue and distinguishes element versus data redundancy, informing the paper's redundancy metrics.","marker":"[76]"},{"why":"Provides the YOLO-LiDAR fusion model used to generate baseline 3D detections for the multimodal redundancy analysis.","marker":"[83]"}],"fun_headline_variants":["Removing camera redundancy boosts AV detection","Less camera data, better YOLOv8: redundancy removal wins","Cut redundant camera views, keep YOLOv8 accuracy","Data pruning beats full datasets for AV object detection","Redundancy removal improves YOLOv8 on nuScenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the mAP50 changes come from removing redundancy itself, since the pruning rules also delete lower-quality (clipped or close-range) boxes; if the gains are actually a label-quality effect, the redundancy conclusion weakens.","fun_headline_variants_meta":{"raw":{"variants":["Removing camera redundancy boosts AV detection","Less camera data, better YOLOv8: redundancy removal wins","Cut redundant camera views, keep YOLOv8 accuracy","Data pruning beats full datasets for AV object detection","Redundancy removal improves YOLOv8 on nuScenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000533,"raw_usage":{"total_tokens":2590,"prompt_tokens":995,"completion_tokens":1595,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":1516}},"tokens_in":611,"tokens_out":1595,"duration_ms":11088,"temperature":1.0,"reasoning_tokens":1516,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:25:41.808987+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train YOLOv8 on three equal-size subsets of the overlapping camera data: BCS-pruned, randomly pruned, and pruned only of boxes clipped at image boundaries. If random or boundary-only pruning matches the BCS-pruned mAP50, then the completeness score is not acting as a redundancy remover.","supporting_citations":[{"cited_title":"A comprehensive review of yolo architectures in computer vision: From yolov1 to yolov8 and yolo-nas,","cited_arxiv_id":null,"evidence_quote":"Supplies YOLOv8, the object detection model whose mAP50 and recall measure the effects of pruning."},{"cited_title":"Ada3d : Exploiting the spatial redundancy with adaptive inference for efficient 3d object detection,","cited_arxiv_id":null,"evidence_quote":"Prior evidence that removing a large share of redundant point clouds causes only a subtle performance drop, motivating the redundancy hypothesis."},{"cited_title":"Active learning with data augmentation under small vs large dataset regimes for semantic-kitti dataset,","cited_arxiv_id":null,"evidence_quote":"Defines redundancy as similarity between point-cloud pairs under ego-motion, giving the paper a geometric redundancy notion to build on."},{"cited_title":"Quality assessment of image dataset for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Frames redundancy as a content data quality issue and distinguishes element versus data redundancy, informing the paper's redundancy metrics."},{"cited_title":"Timkie/yolo-lidar-fusion: Lidar-camera fusion for 3d object detection in autonomous driving systems,","cited_arxiv_id":null,"evidence_quote":"Provides the YOLO-LiDAR fusion model used to generate baseline 3D detections for the multimodal redundancy analysis."}],"review_version":2}