{"id":"7f55ddb6-b316-4b65-a6d0-88b6a996c7bb","arxiv_id":"2501.02822","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RDD4D, an RTMDet detector with a new Attention4D module, reports improved road damage detection on a new DRDD dataset and on the CrackTinyNet benchmark.","lead":"This paper introduces a new road damage dataset, DRDD, with 1,500 images of five damage types, and an object detection model called RDD4D that adds an attention module to the RTMDet detector. The model reports higher average precision than several baselines on DRDD and on the external CrackTinyNet benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported overall AP 0.446 equals the mean of the four class APs in Table II; the Pothole class is omitted from the DRDD evaluation, so the headline 'overall AP' and the model's claimed superiority are not established for the full five-class dataset.","rationale":"The reader's CONDITIONAL verdict is appropriate, but my most load-bearing concern differs. The arithmetic identity — Table IV's Ours AP = 0.446 equals the mean of the four per-class APs in Table II — demonstrates that the reported 'overall AP' covers only four of the five dataset classes. This is not a training-protocol ambiguity; it is an internal inconsistency that can be checked directly from the paper. If the Pothole class was excluded, the headline number is not a COCO mAP over the dataset as described, and the claimed superiority over RTMDet may not hold once the omitted class is included. I still recommend CONDITIONAL rather than REJECT because the model and dataset are released and the evaluation can be re-run; but the current manuscript does not support the broad claim as written. The reviewer should require the authors to report the Pothole AP and the five-class mAP, and to state the evaluation protocol explicitly, before the performance claims are relied upon.","tokens_in":19844,"tokens_out":7222,"duration_ms":67990,"concrete_test":"Download the released code/weights from the GitHub repository, run inference on the DRDD test split, and compute the COCO AP for the Pothole class at IoU 0.50:0.95. Then recompute the overall mAP as the mean over all five classes (Alligator, Block, Longitudinal, Transversal, Pothole). If the five-class mAP differs materially from 0.446, or if including Pothole changes the ranking against RTMDet/YOLOv7, the reported 'overall AP' is invalid. As a secondary check, inspect the evaluation script to confirm whether Pothole was excluded from the metric computation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section III.E defines DRDD with five damage types, including Potholes, and Figure 5 shows Pothole annotation counts. Yet Tables II and III report per-class results only for Alligator, Block, Longitudinal, and Transversal. The Ours row in Table IV lists overall AP = 0.446, which is exactly (0.145 + 0.900 + 0.387 + 0.352) / 4 = 0.446, the mean of the four listed classes. The Pothole class therefore appears to be silently excluded from the headline metric. If Pothole AP is below the other classes (plausible, given its distinct appearance and the class imbalance visible in Figure 5), the true five-class mAP would be lower; for example, with AP_Pothole = 0, mAP becomes 0.357, below YOLOv7's 0.255 but still above some, and the ranking over all five classes is unknown. Since the dataset contribution is five damage types and the abstract claims 'competitive performance with an overall AP,' the central claim is not currently supported for the full dataset. The missing Pothole row also prevents readers from verifying whether the model's advantage over RTMDet (0.178 AP) survives inclusion of the hardest class. This is an internal arithmetic inconsistency, independent of training-protocol questions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DRDD, a new road damage detection dataset of 1,500 images with five damage types, and proposes RDD4D, an RTMDet-based detector that inserts Attention4D blocks into the feature pyramid neck. The authors report that RDD4D outperforms several YOLO-family baselines on DRDD, particularly for large cracks, and achieves 0.825 mAP@50 on the external CrackTinyNet benchmark versus 0.601 for CrTNet. The paper also includes ablations on attention block placement and count.","tokens_in":20111,"tokens_out":6347,"duration_ms":54256,"significance":"If confirmed, the work contributes a useful diverse dataset and shows that a lightweight attention module can improve multiscale crack detection. The planned release of code, weights, dataset, and results is a concrete reproducibility asset, and the independent CrackTinyNet evaluation provides external grounding for the model's gains. However, the current reporting contains several inconsistencies that must be resolved before the performance claims can be accepted.","major_comments":[{"comment":"The DRDD evaluation omits the Pothole class entirely, although Section III.E defines Potholes as one of the five damage types and Figure 5 reports pothole annotation counts. The 'overall AP' of 0.446 in Table IV equals the mean of the four listed class APs in Table II (0.145+0.900+0.387+0.352)/4 = 0.446, not a five-class mAP. Consequently, the abstract's claim of 'competitive performance with an overall AP of 0.445' is not supported for the full dataset. The authors must either provide the missing Pothole AP and recompute the aggregate metrics, or explicitly restrict the claims to a four-class subset. The inconsistency is compounded by Table IV, where RTMDet is listed with AP=0.268 although the mean of its four class APs in Table II is 0.242, so the aggregate numbers do not align with the per-class table.","section":"VI.A, Tables II–IV"},{"comment":"The training hyperparameters are reported as 'weight decay at 0.9' and 'momentum at 5e-4,' which are inverted relative to conventional optimizer settings (momentum near 0.9, weight decay near 5e-4). Because no training details are given for the baselines, the reader cannot determine whether the reported gains of RDD4D over RTMDet (AP 0.446 vs 0.268) are due to the Attention4D module or to different optimizer settings. Please provide the correct hyperparameters and the training protocols for all compared detectors.","section":"V.A"},{"comment":"Table V is captioned as 'AP results for DRDD,' but its rows (D00 Longitudinal Crack, D10 Transverse Crack, D20 Alligator Crack, D40 Pothole, D43 White Line Blur, D44 Cross Walk Blur, D50 Manhole Cover) are the CrackTinyNet classes, not the five DRDD classes. The caption must be corrected, or the table moved to the CrackTinyNet subsection; as published, the mislabeled table undermines confidence in the experimental reporting.","section":"VI.B, Table V"}],"minor_comments":[{"comment":"The abstract contains the incomplete phrase 'on our proposed'; this should read 'on our proposed dataset.'","section":"Abstract"},{"comment":"The label 'Traversal' should be 'Transverse.'","section":"Figure 5"},{"comment":"The abstract reports an overall AP of 0.445, while Table IV reports 0.446; the values should be reconciled.","section":"Table IV"},{"comment":"The caption does not specify which classes are included; add a note clarifying the class set used for the comparison.","section":"Table II"},{"comment":"YOLOV6 is cited as [68] in the text of Section VI.A and in Table IV, but the bibliography lists it as [75]; please make the citation numbering consistent.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The omission of the Pothole class from the DRDD per-class tables is a serious internal-consistency issue that directly affects the headline results. I would ask the authors to supply the missing Pothole numbers and rerun the aggregate metrics before the paper can be considered further. The CrackTinyNet section, apart from the Table V caption, appears to provide independent support for the method."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, the central empirical claim—RDD4D beats RTMDet significantly on road damage detection—is probably true for the four crack types they report. Second, the 'overall AP' of 0.446 in the abstract and Table IV is computed on four classes only, even though the dataset, DRDD, is defined with five damage types including Potholes. The Pothole class is silently dropped from every DRDD evaluation table. That is a real reporting flaw, not a nitpick.\n\nWhat is genuinely new: DRDD is a new dataset, 1,500 annotated images at 1920×1440 with multiple damage types per image and realistic clutter. That fills a spot in the road-damage benchmark landscape. They also evaluate on the external CrackTinyNet benchmark and report mAP@.5 of 0.825 vs. 0.601 for CrTNet, which gives an independent check on the method. The Attention4D block is a simple combination of 1×1 conv, positional encoding, and Talking Head components inserted into the RTMDet neck, and they ablate placement and block count. The improvements are large enough to be interesting even if the architecture is not deeply novel.\n\nSoft spots, in order of severity. (1) The missing Pothole class. Tables II and III list only Alligator, Block, Longitudinal, and Transversal. The Ours AP of 0.446 equals the mean of those four numbers, so the headline metric excludes the fifth class. Since DRDD's whole point is five damage types, the paper needs to report Pothole AP and the true five-class mAP; the ranking over all five classes is unknown. (2) Section V.A says weight decay is 0.9 and momentum is 5e-4. Those are swapped relative to any normal training setup. Probably a typo, but as written it makes training non-reproducible, and no baseline training protocols are given. (3) Table V is captioned as DRDD but lists CrackTinyNet class codes D00–D50. A copy-paste error that should be fixed. (4) No error bars or multiple runs, which matters on a 1,500-image dataset. (5) The attention module is not compared against standard attention blocks like SE or CBAM, so the '4D attention' contribution is not well-situated.\n\nBottom line: this is a useful applied paper with a new dataset and an external validation, but it needs a corrected, class-complete evaluation and cleaner training details before the performance claim is trustworthy. A serious referee can extract that value. I would send it to review and ask for major revision.","headline":"New road-damage dataset and a modest attention-augmented RTMDet that beats baselines on four of its five classes; the dropped Pothole class makes the headline 'overall AP' misleading.","tokens_in":20669,"tokens_out":3692,"would_cite":false,"duration_ms":34638,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 4D attention block improves multi-scale road damage detection on a new diverse dataset.","keywords":["road damage detection","object detection","attention mechanism","multi-scale feature fusion","RTMDet","DRDD dataset","crack classification","smartphone dataset"],"falsifier":"Retrain RDD4D and the RTMDet baseline from the same codebase, with identical splits, augmentation, epochs, and standard hyperparameters (momentum 0.9, weight decay $5\\times10^{-4}$), and compare AP on DRDD and CrackTinyNet; if the gap shrinks to near zero, the gain is not caused by the Attention4D module. A second check is to evaluate on a set that contains small cracks, since DRDD has none and the paper's only per-scale gains are on large objects.","tokens_in":19654,"feed_emoji":"🛣️","tokens_out":8753,"duration_ms":77516,"temperature":0.7,"pith_summary":"This paper sets out to show that a small attention block inserted into the neck of the RTMDet object detector makes road damage detection markedly more accurate, and that a new dataset is needed to test this properly. It introduces DRDD, 1,500 GoPro images in which several damage types (alligator, block, longitudinal, and transverse cracks, plus potholes) appear together in one scene, at varying scales and under occlusions, shadows, and weather changes. On DRDD, the enhanced model reports an overall average precision of 0.446, with an AP for large cracks of 0.458 against 0.280 for the RTMDet baseline. On CrackTinyNet, it reports a mAP at IoU 0.5 of 0.825, about 0.21 above the previous best CrTNet score of 0.601. If these numbers hold, an inexpensive detector with one attention module could make automated road inspection cheaper and more reliable.","feed_headline":"Road-crack detector hits 0.825 mAP with 4D attention","feed_subtitle":"A small attention block added to RTMDet lifts detection on the new DRDD benchmark and beats CrTNet on CrackTinyNet.","key_machinery":"The load-bearing component is the Attention4D block, a lightweight module applied to feature maps in the neck. Each block passes a feature map through a $1\\times1$ convolution and batch normalization, splits it into Query, transposed Key, and Value branches, multiplies Query and Key, adds a positional encoding and a 'Talking Head' input, normalizes with softmax, multiplies by the Value branch with a second Talking Head input, and finishes with another $1\\times1$ convolution and batch normalization. Two blocks in the top-down path form the chosen configuration, which runs at 26.8 FPS with 38.5 million parameters and yields the reported mAP of 0.446 on DRDD. The ablation tables support the placement choice: top-down placement beats bottom-up placement (0.446 versus 0.412 mAP), and two blocks beat one block, while adding more blocks costs speed.","core_discovery":"The central claim is that Attention4D blocks let a one-stage detector refine features across scales by combining local and global context, and that this is what drives the reported improvements. The authors place two Attention4D blocks in the top-down path of the RTMDet feature pyramid neck, and their scale-specific numbers show the largest gains on large damage instances: on DRDD, AP for large objects is 0.458 versus 0.280 for RTMDet, and overall AP rises from 0.268 to 0.446. On CrackTinyNet, the model reaches 0.825 mAP at IoU 0.5 versus 0.601 for CrTNet, with recall of 0.98, and the authors attribute the gain to the attention module's handling of both fine details and high-level semantic information. The paper also argues DRDD fills a gap in existing road damage datasets, which it says mostly contain single damage types per image, whereas its images average about three damage instances per scene.","pith_inferences":["Editorial inference: the claimed multi-scale benefit is only tested on two scales, because DRDD has no small objects and the average precision for medium objects is 0.113 versus 0.123 for RTMDet; the attention module's advantage may be specific to large instances.","Editorial inference: the reported hyperparameters look swapped (weight decay 0.9, momentum $5\\times10^{-4}$), so as written the training configuration is hard to reproduce, and the 0.178 AP gain over RTMDet should be re-checked under a matched protocol before it is attributed to the module.","Editorial inference: the CrackTinyNet comparison mixes a change in architecture with a change in dataset and label-assignment protocol, so the 0.21 mAP jump may not isolate the attention module's contribution.","Editorial inference: a direct test would ablate the positional encoding and Talking Head components individually; the paper only ablates block placement and count."],"forward_implications":["Road agencies could run this model on smartphone or vehicle footage and get a reliable first pass at locating large cracks, the damage class that dominates DRDD.","Because the chosen configuration runs at 26.8 FPS with 38.5 million parameters, it is fast enough for near-real-time inspection on an edge device.","If the CrackTinyNet result transfers, the detector would find almost all damage instances (recall 0.98) while keeping precision at 0.81, which in practice means fewer missed repairs and fewer false alarms.","The scale-specific results imply that future detectors for this task should report AP separately for large, medium, and small damage, since one number can hide where the gain actually is."],"supporting_citations":[{"why":"Supplies the RTMDet baseline detector and the dynamic soft label assignment strategy that RDD4D builds on.","marker":"[66]"},{"why":"Provides the CrackTinyNet dataset and the CrTNet baseline whose 0.601 mAP the paper's model surpasses with 0.825.","marker":"[76]"},{"why":"Maeda et al.'s smartphone-based RDD-2018 dataset is the original collection and annotation paradigm that DRDD extends.","marker":"[37]"},{"why":"RDD2020 is the multi-country dataset the paper contrasts with DRDD, which it says contains mostly single-damage images.","marker":"[43]"},{"why":"CSPNeXt is the backbone used inside the RTMDet baseline, so it is part of the architecture the Attention4D blocks modify.","marker":"[56]"}],"fun_headline_variants":["Attention4D lifts road crack AP to 0.446","New DRDD dataset and 4D attention boost damage detection","RDD4D: 4D attention for multi-scale road damage spotting","4D attention nearly doubles large-crack detection AP","DRDD + Attention4D: better road damage detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim collapses if the comparison detectors were not trained under the same, correctly reported protocol, since the paper's own hyperparameters look swapped (weight decay 0.9 and momentum $5\\times10^{-4}$) and training details for the baselines are not given.","fun_headline_variants_meta":{"raw":{"variants":["Attention4D lifts road crack AP to 0.446","New DRDD dataset and 4D attention boost damage detection","RDD4D: 4D attention for multi-scale road damage spotting","4D attention nearly doubles large-crack detection AP","DRDD + Attention4D: better road damage detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000385,"raw_usage":{"total_tokens":2072,"prompt_tokens":1016,"completion_tokens":1056,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":970}},"tokens_in":632,"tokens_out":1056,"duration_ms":10860,"temperature":1.0,"reasoning_tokens":970,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:03:21.858637+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain RDD4D and the RTMDet baseline from the same codebase, with identical splits, augmentation, epochs, and standard hyperparameters (momentum 0.9, weight decay $5\\times10^{-4}$), and compare AP on DRDD and CrackTinyNet; if the gap shrinks to near zero, the gain is not caused by the Attention4D module. A second check is to evaluate on a set that contains small cracks, since DRDD has none and the paper's only per-scale gains are on large objects.","supporting_citations":[{"cited_title":"Cracktinynet: A novel deep learning model specifically designed for superior performance in tiny road surface crack detection,","cited_arxiv_id":null,"evidence_quote":"Provides the CrackTinyNet dataset and the CrTNet baseline whose 0.601 mAP the paper's model surpasses with 0.825."},{"cited_title":"Road damage detection and classification using deep neural networks with smartphone images,","cited_arxiv_id":null,"evidence_quote":"Maeda et al.'s smartphone-based RDD-2018 dataset is the original collection and annotation paradigm that DRDD extends."},{"cited_title":"Beyond bounding- box: Convex-hull feature adaptation for oriented and densely packed object detection,","cited_arxiv_id":null,"evidence_quote":"CSPNeXt is the backbone used inside the RTMDet baseline, so it is part of the architecture the Attention4D blocks modify."}],"review_version":1}