{"id":"acb4eb71-b33c-4bde-8649-cc8496d08c2f","arxiv_id":"2507.18958","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A new benchmark dataset of panoramic dental X-rays for apical periodontitis, with a DETR-based detector that claims state-of-the-art results and improved dentist performance when used as an aid.","lead":"This paper introduces PerioXrays, a new set of 3,673 panoramic dental X-rays with 5,662 annotated apical periodontitis lesions, plus a detector called PerioDet that adds attention and label-calibration modules to a DETR-style architecture. It reports state-of-the-art detection scores and a small study in which dentists performed better with the tool, though the evaluation has methodological gaps.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim rests on test-set hyperparameter tuning and single-run AP; without a validation split or error bars, the reported 3.2-point gain over Salience-DETR is not yet established.","rationale":"The reader's weakest assumption focuses on PerioXrays ground-truth correctness, which is a legitimate concern because all AP and human-collaboration numbers are measured against those annotations. However, the paper gives no inter-annotator agreement or adjudication details, so label noise is a plausible risk but not directly evidenced. My stress test identifies a more immediately load-bearing methodological gap: the absence of a validation split combined with hyperparameter selection reported in Section 4.3. Since Section 4.1 mentions only train and test splits, the parameter analysis in Fig. 4 appears to have been conducted on the test set, directly compromising the AP comparisons in Table 1. The lack of error bars or repeated runs further means the reported gaps could be within noise. This does not invalidate the dataset contribution or the human-collaboration idea, but it makes the numerical superiority claim unverified as reported. The reader's verdict of CONDITIONAL is therefore appropriate, and my concern reinforces the need for a corrected evaluation protocol rather than changing the verdict. I credit the public code repository and the patient-level split, but those do not address the test-set tuning issue. The concrete test of a validation-based tuning protocol with repeated seeds would settle whether the SOTA claim survives.","tokens_in":8192,"tokens_out":3749,"duration_ms":41836,"concrete_test":"Using the released code and dataset, construct a proper validation split (e.g., hold out 10% of training patients) or perform 5-fold cross-validation on the training split. Tune λ, α0, γ, and all baseline-specific hyperparameters on the validation folds only. Then retrain the selected configuration on train+validation and evaluate once on the held-out 673-image test set, repeating with 3–5 random seeds and reporting mean ± std for AP, AP50, AP75, and APS. If PerioDet's mean AP no longer exceeds Salience-DETR and Co-DINO by more than 2 points with non-overlapping error bars, the SOTA claim in Table 1 does not survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claimed superiority of PerioDet is not supported by a sound evaluation protocol. Section 4.1 defines only a training set of 3,000 images and a test set of 673 images, with no validation split reported. Section 4.3 then selects the critical hyperparameters λ, α0, and γ by maximizing AP on 'the PerioXrays dataset' (Fig. 4), which appears to be the same test set used for the final numbers in Tables 1 and 2. Selecting hyperparameters on the test set and then reporting those AP values as PerioDet's performance systematically overstates the method relative to the baselines, whose hyperparameters are not given the same test-set optimization. Additionally, every number in Tables 1 and 2 is a single run with no error bars, no seed information, and no repeated trials. Because AP differences of 1–3 points can easily arise from run-to-run variance or from test-set selection, the central claim that PerioDet beats Salience-DETR, Co-DINO, and CFINet is not yet quantitatively reliable. This concern is more immediately decisive than annotation noise: even if the PerioXrays labels are perfectly correct, the reported comparison as designed cannot distinguish genuine method quality from overfitting to the test set.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PerioXrays, a panoramic radiograph benchmark containing 3,673 images and 5,662 annotated instances of apical periodontitis, and proposes PerioDet, a detection pipeline with Background-Denoising Attention (BDA) and IoU-Dynamic Calibration (IDC). The method is evaluated against CNN- and transformer-based detectors, reporting AP 53.5, AP50 84.2, AP75 55.6, and APS 42.3, which the authors claim surpasses all compared baselines. A human-computer collaborative experiment with six dentists reports that PerioDet assistance raises precision from 73.1% to 92.5% and recall from 74.3% to 96.1%. The authors position the dataset as the first large-scale benchmark for automated apical periodontitis detection and the method as a clinically applicable auxiliary diagnostic tool.","tokens_in":8484,"tokens_out":4049,"duration_ms":41448,"significance":"If the evaluation is made sound, this would be a valuable contribution: the PerioXrays benchmark addresses a genuine gap in dental computer-aided diagnosis, the proposed BDA and IDC modules are simple and mechanistically plausible, and the human-computer experiment targets a clinically meaningful outcome. The authors also ship a public project page, which supports reproducibility if training code and trained models are released. However, the current experimental protocol does not yet establish the headline claims: the state-of-the-art comparison is compromised by test-set hyperparameter selection and single-run reporting, the human study is confounded by order and learning effects, and the annotation quality is not quantitatively documented. These are addressable issues, so the contribution is potentially strong but needs substantial verification.","major_comments":[{"comment":"Section 4.1 defines only a training set of 3,000 images and a test set of 673 images, with no validation split. Section 4.3 and Fig. 4 then select the critical hyperparameters λ, α0, and γ by maximizing AP on 'the PerioXrays dataset', which, given the split in Section 4.1, can only be the training or test set. If it is the test set, the final AP numbers in Tables 1 and 2 are the result of test-set tuning, which systematically favors PerioDet over baselines whose hyperparameters are not optimized on the test set. If it is the training set, the reported FIGure 4 values are not an unbiased estimate. The authors must introduce a proper validation split, select hyperparameters on it, and report test-set metrics only once; alternatively, use nested cross-validation.","section":"4.1, 4.3, Fig. 4"},{"comment":"Every AP number in Tables 1 and 2 appears to come from a single training run, with no standard deviation, number of seeds, or significance testing. The claimed advantage over the strongest baseline, Salience-DETR, is 3.2 AP points, which is within the range of run-to-run variation commonly observed for detection transformers. The authors should report mean and standard deviation over at least three to five seeds and, where appropriate, a paired significance test to support the state-of-the-art claim.","section":"Tables 1 and 2"},{"comment":"The human-computer collaborative experiment has the same six dentists judge the same 100 images in two consecutive sessions, once without and once with PerioDet. With no washout period, no randomization of image order, and no cross-over design, the reported 19.4-point precision gain and 21.8-point recall gain may be attributable to learning, memory of previous responses, or fatigue effects rather than to PerioDet assistance. The experiment should use a controlled design, such as split-image or alternating-order conditions with a washout interval, or a cross-over protocol with distinct image subsets balanced across dentists, and should report per-group (junior, mediate, senior) results.","section":"4.4, Table 3"},{"comment":"Section 2.2 states that each bounding box was reviewed by four experienced professional dentists, but no inter-annotator agreement, adjudication procedure, or validation against an external reference standard is reported. Because every AP number in Tables 1 and 2 and every precision/recall figure in Table 3 is measured against these boxes, the authors should report inter-annotator agreement (e.g., Fleiss' kappa on image-level lesion presence and mean pairwise IoU on box-level agreement) and describe how disagreements were resolved. Without this, the reliability of the ground truth, and therefore of all downstream comparisons, is not established.","section":"2.2"}],"minor_comments":[{"comment":"The captions and labels in Fig. 2 appear garbled or duplicated, with repeated counts and a stray 'Apical AP>0.5% 99' line; these should be cleaned and the plots should have clear legends.","section":"Fig. 2"},{"comment":"The term As, described as 'the minimal area definition', is never formally defined or given a value for the PerioXrays dataset; please specify its units, how it is set, and whether it is normalized by image resolution.","section":"Eq. 6"},{"comment":"The axis labels and tick values in Fig. 4 are nearly illegible; add readable axes, legends, and a note stating which data split the reported AP values are computed on.","section":"Fig. 4"},{"comment":"When reporting differences such as '5.8% improvement' and '3.2% improvement', the authors should use 'percentage points' rather than '%' to avoid ambiguity about relative versus absolute change.","section":"4.2"},{"comment":"The training details for the baselines are underspecified: the text says all baseline models use a ResNet-50 backbone, but does not state whether the same schedule, augmentation, and hyperparameter settings were used across all methods, which is needed for a fair comparison.","section":"4.1"}],"recommendation":"major_revision","confidential_remarks":"The dataset release is the strongest contribution of this paper, but the evaluation protocol is the main obstacle to acceptance. The issues identified—test-set hyperparameter selection, single-run AP, a confounded human study, and missing annotation-agreement metrics—are all fixable within the scope of a revision, so I recommend major revision rather than rejection. Please ask the editor to encourage the authors to make the training code and annotations publicly available, as the benchmark would be more useful with a documented evaluation protocol and reproducibility details."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the dataset is the paper. PerioXrays appears to be the first large annotated benchmark for apical periodontitis on panoramic radiographs, 3,673 images / 5,662 boxes with multi-stage dentist review. That is a real contribution, worth a serious referee and likely worth citing if the release matches the description.\n\nThe PerioDet method itself is an assembly of known pieces — channel attention, similarity maps, dynamic IoU thresholds, label assignment — and doesn't need to be more than that to be credible. The ablation shows both modules add something on this data.\n\nThe soft spot is exactly where the stress-test note lands: the SOTA claim is not yet established. Section 4.1 defines only train and test splits; there is no validation split. Section 4.3 tunes λ, α0, and γ on the PerioXrays dataset, which appears to be the same test set that produced Tables 1 and 2. That is test-set selection. On top of that, all AP numbers are single runs without seeds or error bars. A 3.2-point gain over Salience-DETR can easily be explained by selection or run-to-run variance. This is a load-bearing flaw for the 'superiority' claim, not a minor quibble.\n\nSecond soft spot is the annotation ground truth: four dentists reviewed each box but no inter-annotator agreement, no adjudication protocol, and no external validation. Given the whole benchmark's utility rests on label quality, that needs to be reported.\n\nThe human-computer experiment is suggestive but only loosely controlled: same six dentists on the same 100 images in consecutive sessions, so learning/order effects are plausible. The reported precision/recall jump is large, but I'd treat it as preliminary.\n\nCitation pattern is fine. The self-cited OralXrays-9, CariesXrays, and Salience-DETR are contextual, not load-bearing.\n\nWho should read this: anyone working on dental radiograph CAD, and detection researchers looking for a new small-object benchmark. If the authors add a validation split, repeated runs with variance, and label-agreement metrics, the central claim could become solid. As is, the paper is a useful dataset contribution wrapped in an overstated method claim.\n\nI would send it to peer review — the dataset deserves referee time — but the reviewers should insist on a clean evaluation protocol.","headline":"The dataset is a genuine contribution; the SOTA claim is not yet supported because hyperparameters were tuned on the test set and AP numbers are single runs without error bars.","tokens_in":9005,"tokens_out":1486,"would_cite":true,"duration_ms":12852,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PerioDet combines a new 3,673-image benchmark with noise- and small-lesion-aware modules to reach 53.5 AP.","keywords":["apical periodontitis","panoramic radiograph","PerioXrays","PerioDet","object detection","small object detection","computer-aided diagnosis","label assignment"],"falsifier":"Take the 673 test images, have an independent panel of dentists re-annotate apical periodontitis lesions from the same clinical criteria, and compare their boxes to the released ones by bounding-box agreement (for example, an IoU >= 0.5 match rate or Cohen's kappa on lesion presence per tooth). If agreement is low, the 53.5 AP and the 92.5% precision figure would need to be recomputed on the new labels; if agreement is high, the numbers stand.","tokens_in":8028,"feed_emoji":"🦷","tokens_out":7497,"duration_ms":68044,"temperature":0.7,"pith_summary":"This paper tries to establish that automated detection of apical periodontitis in panoramic dental X-rays is now feasible at a clinically useful level, and that the obstacle was missing data rather than model capacity. To that end it releases PerioXrays, 3,673 panoramic radiographs with 5,662 lesion boxes, split at the patient level into 3,000 training and 673 test images. On this benchmark, the proposed PerioDet detector, built on Background-Denoising Attention (BDA) and IoU-Dynamic Calibration (IDC), reports 53.5 AP, 84.2 AP50, and 55.6 AP75, outperforming twelve cited CNN and Transformer baselines, with the largest margin on small lesions. In a separate study, six dentists improved from 73.1% to 92.5% precision and from 74.3% to 96.1% recall when using PerioDet as an aid. A sympathetic reader would take the paper's aim to be demonstrating that this benchmark plus these two mechanisms makes computer-aided apical periodontitis diagnosis practical.","feed_headline":"Benchmark and detector push apical periodontitis detection to 53.5 AP","feed_subtitle":"A 3,673-image dental X-ray benchmark plus a noise- and small-lesion-aware detector lifts dentists' recall to 96 percent.","key_machinery":"The load-bearing mechanisms are the Background-Denoising Attention (BDA) and IoU-Dynamic Calibration (IDC) modules. BDA takes each FPN feature level $P_i$, computes a channel importance vector $z_i = \\mathrm{Sigmoid}(\\mathrm{Conv}_{1\\times1}(P_i))$, and builds a similarity map $S_i$ between the refined feature map and a scene embedding taken from the deepest level $C_5$; the output $P_i^{BD} = (1+z_i)\\cdot P_i \\odot S_i$ suppresses background while keeping target channels. IDC replaces fixed positive-sample IoU rules with an area-dependent threshold $T_{rp} = \\max(0.25, 0.2 + 0.15(\\sqrt{wh}/A_s)^\\lambda)$ and a dynamic label assignment that scores anchors by $DIoU = \\alpha AIoU + (1-\\alpha)RIoU - (1-\\alpha)|AIoU-RIoU|^\\gamma$, gradually shifting weight to the regression box during training. The threshold's dependence on object area is what gives small lesions enough positive anchors, and the BDA gate is what keeps panoramic radiograph artifacts from suppressing true detections.","core_discovery":"The central claim is that PerioXrays is the first large-scale benchmark for apical periodontitis detection and that PerioDet beats current detectors on it by explicit design for the two failure modes typical of periapical lesions: background clutter and small object size. Quantitatively, the paper reports 53.5 AP on the test split, a 3.2 point gain over the best Transformer baseline (Salience-DETR at 50.3 AP) and a 5.8 point gain over the best CNN baseline (CFINet at 47.7 AP); small-lesion AP rises to 42.3, 5.1 points above the best baseline. The ablation table attributes 1.8 AP to BDA alone, 2.4 AP to IDC alone, and 3.6 AP to both together relative to the Co-DINO base. The human-computer experiment claims that with the tool, professional dentists' precision rises from 73.1% to 92.5% and recall from 74.3% to 96.1%, which the paper reads as evidence of clinical applicability as an auxiliary diagnostic aid.","pith_inferences":["My inference: the same area-adaptive IoU threshold idea could transfer to other small-object medical tasks, such as microcalcification detection in mammograms, but PerioDet's gains would need to be re-estimated because that domain's noise profile differs.","My inference: since the human study used 100 images and six dentists, the 19-22 point clinician gains are a proof of concept; a multi-centre reader study with more images and varied expertise would show whether the benefit persists in routine workflow.","My inference: the paper does not report per-tooth or per-root localization accuracy; testing whether false positives sit on healthy roots near true lesions would tell whether the remaining errors are anatomical confusion or genuine misses."],"forward_implications":["PerioXrays gives the community a fixed, patient-level split and a standardized evaluation protocol, so future apical periodontitis detectors can be compared on equal footing.","The 5.1-point small-object gain (APS 42.3 versus 37.0) implies the IDC label-assignment trick is doing real work for tiny lesions, the very cases dentists miss under fatigue.","Under the same ResNet-50 backbone and training schedule, the gap between PerioDet and each baseline is attributable to the BDA and IDC modules plus their synergy, since only the modules differ.","The dentist-collaboration result implies that CAD assistance, rather than full automation, is the immediate clinical use: it raises both precision and recall without requiring the dentist to accept or reject model output blindly."],"supporting_citations":[{"why":"Supplies the clinical diagnostic criteria that the annotation guidelines are based on.","marker":"[1]"},{"why":"Supplies radiological criteria for identifying apical periodontitis in the annotation process.","marker":"[13]"},{"why":"Supplies clinical evaluation standards for periodontal disease used in the annotation guidelines.","marker":"[2]"},{"why":"Motivates the IoU-Dynamic Calibration design by showing traditional IoU thresholds are sensitive to positional deviations in small objects.","marker":"[7]"},{"why":"Defines the Feature Pyramid Network that BDA modifies and that the PerioDet architecture builds on.","marker":"[15]"},{"why":"Defines the AP, AP50, and AP75 evaluation metrics used in all comparisons and ablations.","marker":"[16]"},{"why":"The strongest Transformer baseline PerioDet is compared against and exceeds by 3.2 AP.","marker":"[12]"},{"why":"Co-DINO serves as both a strong Transformer baseline and the base model for the ablation study.","marker":"[30]"},{"why":"CFINet is the best CNN baseline, beaten by 5.8 AP, and represents the small-object-detection approach this work extends.","marker":"[26]"},{"why":"Faster R-CNN is the classic CNN baseline included in the comparison table.","marker":"[21]"}],"fun_headline_variants":["PerioDet lifts apical periodontitis detection to 53.5 AP","First large-scale benchmark for apical periodontitis detection","Benchmark with 3,673 X-rays targets small periapical lesions","First apical periodontitis benchmark: 3,673 images, 5,662 lesions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the PerioX-rays annotations being correct: each box was reviewed by four dentists, but the paper reports no inter-annotator agreement, no adjudication rule, and no independent reference standard, so if the labels are noisy or inconsistent the reported AP gains and dentist-improvement numbers would be measured against unreliable ground truth.","fun_headline_variants_meta":{"raw":{"variants":["PerioDet lifts apical periodontitis detection to 53.5 AP","First large-scale benchmark for apical periodontitis detection","Benchmark with 3,673 X-rays targets small periapical lesions","First apical periodontitis benchmark: 3,673 images, 5,662 lesions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000909,"raw_usage":{"total_tokens":3935,"prompt_tokens":1000,"completion_tokens":2935,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":2854}},"tokens_in":616,"tokens_out":2935,"duration_ms":22230,"temperature":1.0,"reasoning_tokens":2854,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:04:05.678091+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the 673 test images, have an independent panel of dentists re-annotate apical periodontitis lesions from the same clinical criteria, and compare their boxes to the released ones by bounding-box agreement (for example, an IoU >= 0.5 match rate or Cohen's kappa on lesion presence per tooth). If agreement is low, the 53.5 AP and the 92.5% precision figure would need to be recomputed on the new labels; if agreement is high, the numbers stand.","supporting_citations":[{"cited_title":"Endodontic topics8(1), 36–54 (2004)","cited_arxiv_id":null,"evidence_quote":"Supplies the clinical diagnostic criteria that the annotation guidelines are based on."},{"cited_title":"Endodon- tic topics1(1), 3–25 (2002)","cited_arxiv_id":null,"evidence_quote":"Supplies radiological criteria for identifying apical periodontitis in the annotation process."},{"cited_title":"Periodontology 2000 7(1), 39–53 (1995)","cited_arxiv_id":null,"evidence_quote":"Supplies clinical evaluation standards for periodontal disease used in the annotation guidelines."},{"cited_title":"Pattern Recognition137, 109256 (2023)","cited_arxiv_id":null,"evidence_quote":"Motivates the IoU-Dynamic Calibration design by showing traditional IoU thresholds are sensitive to positional deviations in small objects."},{"cited_title":"In: CVPR","cited_arxiv_id":null,"evidence_quote":"The strongest Transformer baseline PerioDet is compared against and exceeds by 3.2 AP."},{"cited_title":"In: ICCV","cited_arxiv_id":null,"evidence_quote":"Co-DINO serves as both a strong Transformer baseline and the base model for the ablation study."},{"cited_title":"In: ICCV","cited_arxiv_id":null,"evidence_quote":"CFINet is the best CNN baseline, beaten by 5.8 AP, and represents the small-object-detection approach this work extends."},{"cited_title":"In: NeurIPS","cited_arxiv_id":null,"evidence_quote":"Faster R-CNN is the classic CNN baseline included in the comparison table."}],"review_version":2}