{"id":"c5af1ce4-a77b-48b5-b9c8-55993cfaace5","arxiv_id":"2505.15637","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"OMNI is a new multi-view RGB dental image dataset with dentist labels for ten malocclusion-related categories, benchmarked with six object detectors.","lead":"The authors introduce OMNI, a dataset of 4,166 RGB dental photographs from 384 patients across five views, annotated by dentists with ten classes of malocclusion-related labels, and benchmark six deep learning models for tooth detection and classification. The paper positions OMNI as the first public multi-view dataset for automated malocclusion diagnosis, with best detection reaching a 66.4 mAP@0.5.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No patient-disjoint split is described: with ~10.9 images per participant across five views and follow-up visits, the Table III benchmark may reflect same-patient memorization rather than generalization to new patients.","rationale":"The reader identified annotation accuracy and inter-annotator agreement as the weakest assumption. That is a legitimate concern, but I see a more fundamental and more easily testable problem in the experimental protocol: the absence of any evidence that the train/test split is participant-disjoint. Annotation noise affects label quality, whereas split leakage directly undermines the interpretation of every number in Table III as a measure of diagnostic generalization. The paper's own statistics make this concern concrete: 4,166 images from 384 participants with multiple views and follow-up visits means each participant contributes roughly 11 images on average, so an image-level random split is very likely to place the same participant in both training and test. The benchmark's purpose is to show that deep learning models can diagnose malocclusion issues on new patients, and that claim cannot be evaluated if the test set contains images of training participants. This is a standard requirement in medical imaging and should be stated and verified explicitly. I therefore keep the verdict conditional, as the reader did, but for a different condition: the authors must demonstrate and document a patient-disjoint split, not merely add kappa statistics. The annotation-quality issue remains important but is secondary to the generalization validity of the benchmark.","tokens_in":16334,"tokens_out":3649,"duration_ms":36043,"concrete_test":"Obtain or request the participant identifiers for all 4,166 images from the OMNI repository and compute the overlap of participant IDs across the training, validation, and test splits. If any participant appears in more than one split, rerun the six baselines under a strict participant-disjoint split (all views and visits of a participant assigned to exactly one split) and compare the resulting mAP values to Table III. If mAP drops substantially, the reported benchmark overstates generalization; even if it does not drop, the paper should explicitly document the patient-disjoint split and release the corresponding metadata.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that OMNI enables automated malocclusion diagnosis rests on the benchmark results in Table III being meaningful for unseen patients. Section III-A1 states that data were gathered from 384 patients 'during the first visit and two follow-up visits,' and Section III-A2 says five views are acquired per patient, yielding 4166/384 ≈ 10.85 images per participant. However, Section III-A3 describes the train/validation/test split only by image counts (2481/857/828) and never states that all images of a given participant are confined to a single split. If the split is performed at the image level, the same patient's teeth, braces, attachments, lighting, and gingival appearance can appear in both training and test sets. Models could then exploit patient-specific appearance cues rather than learning malocclusion diagnosis, inflating mAP values. This risk is especially acute for the orthodontic-appliance classes (OB, FOD, CFOA, IOA), which are stable per patient over time. The dataset's stated purpose as a benchmark for automated diagnosis requires patient-independent evaluation, so the absence of any patient-level split statement is a load-bearing gap.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OMNI, a dataset of 4,166 RGB intraoral photographs from 384 patients across five views (frontal, left, right, maxillary, mandibular), annotated with ten categories related to malocclusion. The authors benchmark six object-detection baselines (Faster R-CNN, Mask R-CNN, EfficientDet, DETR, Deformable DETR, and a proposed GraphTeethNet) and report mAP metrics. They claim that OMNI is the first publicly available multi-view dental image dataset for malocclusion diagnosis and that the benchmark shows deep learning can address this task.","tokens_in":16469,"tokens_out":4323,"duration_ms":44512,"significance":"If the dataset is reliable and the split is sound, OMNI would be a useful community resource: it is larger than most existing malocclusion-related dental image collections, avoids X-ray acquisition, includes multiple views, and the authors publicly release dataset and code. The paper also provides a reasonable first baseline comparison and a new graph-based detector, GraphTeethNet, whose multi-dimensional edge features are ablated. The main strengths are the dataset release, the breadth of baseline models, and the ablation studies. However, the benchmark's validity depends on two currently unverified conditions: that no participant appears in both training and test partitions, and that the annotation unit (per-image vs. per-tooth) matches the object-detection evaluation. The absence of label-agreement metrics and single-run results further limits the quantitative claims.","major_comments":[{"comment":"The train/val/test split is described only by image counts (2,481/857/828) and no participant-disjoint restriction is stated. Since Section III-A1 says the 4,166 images come from 384 participants across first and follow-up visits, an image-level random split can place different views of the same participant in both training and test sets. Patient-specific appearance cues (teeth shape, gingiva, orthodontic appliances) could then be memorized rather than diagnostic generalization being measured, which is especially likely for patient-stable classes such as OB, FOD, and IOA. The authors should either confirm that all images of each participant are confined to one split or rerun the benchmark under a participant-disjoint split and report both sets of results.","section":"Section III-A3"},{"comment":"The annotation unit is ambiguous. Section III-B2 states that every tooth is localized with a bounding box, while Section III-B3 says malocclusion issues are 'labelled for each image' and Table I reports numbers of images per class. The benchmark, however, evaluates object-detection mAP, which requires per-instance labels. If image-level labels are attached to all teeth boxes in an image, the AP values in Table III do not have the standard detection interpretation and may be inflated. The authors must specify whether each box carries its own class label or whether the image label is copied to all boxes, and provide per-unit label statistics.","section":"Section III-B and Table I"},{"comment":"The class taxonomy mixes diagnoses and treatment artifacts in a way that needs justification: HT and TE are not malocclusion issues, while IOA, OB, CFOA, and FOD are orthodontic appliances or devices. In addition, the definitions of OB and FOD overlap, and CFOA overlaps with fixed appliances. If these are intended as separate detection categories, the paper should explain how overlapping labels were resolved during annotation; otherwise ambiguous ground truth can bias the per-category AP reported in Table III.","section":"Section III-B3"},{"comment":"No inter-annotator agreement, adjudication counts, or label-quality metric is reported, and all results come from a single training run. This is not merely a formal issue: for category TM, Table III reports AP values ranging from 8.07 (EfficientDet) to 51.81 (GraphTeethNet) despite only 147 TM images, and no confidence interval would allow a reader to judge whether any difference between models is meaningful. The authors should add agreement statistics on a labeled subset and report mean and standard deviation over multiple seeds or bootstrap confidence intervals before claiming that the models 'can accurately recognize various malocclusion issues.'","section":"Section III-B4 and Table III"}],"minor_comments":[{"comment":"The column header 'Epoches' is misspelled, and the parenthesized epoch numbers should be explained in the table caption.","section":"Table III"},{"comment":"The caption describes four acquisition panels, but the text says the dataset contains five views; please clarify whether the lateral photographs combine left and right views and make the correspondence between figure panels and the five views explicit.","section":"Fig. 1"},{"comment":"There are several grammatical errors, e.g., 'This papers conducted,' 'There are already exists numerous dental image datasets,' and 'prevent the development'; a careful language edit is needed.","section":"Section I"},{"comment":"The dataset release page is a GitHub repository, but the paper does not state a license, data-use agreement, or plan for long-term archival; adding a persistent DOI or institutional repository link would strengthen the 'publicly available' claim.","section":"Section III-A1"},{"comment":"The Tbox parameter is introduced as 'restricted to 50' but no sensitivity analysis or justification for this value is given; please note whether the reported GraphTeethNet results are stable to this choice.","section":"Section IV-C"},{"comment":"The comparison across baselines uses different training epochs (50 or 300) and different frameworks (MMDetection vs. PyTorch); while this is acceptable for a baseline benchmark, the paper should state explicitly that wall-clock or convergence criteria differ, so readers do not interpret the rankings as equal-budget comparisons.","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The core dataset-resource claim is promising and likely within the journal's scope, but the benchmark's credibility hinges on the patient-disjoint split issue and the annotation-unit ambiguity. If the current split is image-level, the reported mAP numbers may substantially overstate diagnostic generalization, and that would affect the main contribution. I would encourage the editor to request a revision that clarifies or redoes the split, reports agreement statistics, and adds uncertainty quantification; these are fixable with the existing data and code."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"OMNI is genuinely new: the first public multi-view RGB dental dataset aimed at malocclusion, and the gap in the literature is real. The collection protocol is thoughtful, the ethics and consent are addressed, and the multi-step dentist review gives some confidence in label quality. The six-detector benchmark is standard and the numbers are internally consistent. GraphTeethNet is honestly framed as an adaptation of the authors' prior GRATIS work, so no hidden novelty claim.\n\nThe soft spots are real, and the biggest one is the split. With 384 patients and 4166 images (about 10.9 images per patient, across five views and follow-up visits), an image-level split almost guarantees that the same patient appears in both training and test. The paper only reports image counts per split, not patient-level separation. For stable per-patient classes like orthodontic braces and fixed devices, models can memorize patient-specific appearance and inflate mAP. That is load-bearing: a benchmark is supposed to measure generalization to unseen patients. This is fixable by reporting a patient-disjoint split and confirming the current numbers hold under it.\n\nThe label-quality concerns are also fair: no inter-annotator agreement, no error bars, and the taxonomy mixes tooth-level findings (tooth torsion, misalignment) with patient-level skeletal conditions (mandibular retrusion) and treatment artifacts (braces, attachments, fixed devices). That makes the mAP numbers hard to interpret clinically, and the per-image vs per-tooth unit of annotation is ambiguous. The paper does state the limitation of class imbalance, which is honest.\n\nNone of this kills the dataset's value. If the release happens as promised, this is a useful resource for orthodontic screening research, especially in clinic settings where X-ray is not available. The paper deserves a serious referee, but with major revision expected: patient-level split, kappa statistics, error bars, and a clearer class ontology. I would want to see the revised version before trusting the benchmark numbers.","headline":"A genuinely novel dental dataset with a real gap-filling claim, but the benchmark numbers need a patient-disjoint split and label-quality metrics before they can be trusted.","tokens_in":17151,"tokens_out":2871,"would_cite":true,"duration_ms":25014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces OMNI, the first publicly available multi-view RGB dental image dataset for malocclusion diagnosis, and benchmarks six deep-learning detectors on it.","keywords":["malocclusion diagnosis","dental image dataset","deep learning benchmark","object detection","multi-view oral imaging","teeth localization","graph neural network","RGB dental photography"],"falsifier":"Have an independent panel of orthodontists re-annotate a random subset of the OMNI images and compare their labels with the released ones; if per-category agreement is low, or if labels diverge from X-ray or dental records, the benchmark numbers reflect annotation noise rather than diagnostic skill.","tokens_in":16049,"feed_emoji":"🦷","tokens_out":7075,"duration_ms":55659,"temperature":0.7,"pith_summary":"The paper's central claim is that the field lacked a large, accurately labelled, publicly available image dataset for malocclusion, and that the new OMNI dataset fills that gap. It reports 4,166 natural RGB photographs of the oral cavity from 384 patients, covering five standard views and annotated by dentists for ten malocclusion-related categories, including healthy teeth. The paper benchmarks six deep-learning detectors on this data and shows that they can localize teeth and recognise several malocclusion issues with useful accuracy. A sympathetic reader would care because this makes automated malocclusion screening research possible without X-ray imaging and gives future work a common evaluation set.","feed_headline":"New open dental dataset targets malocclusion without X-rays","feed_subtitle":"OMNI's 4,166 RGB images and dentist labels let malocclusion screening research proceed without X-rays.","key_machinery":"The load-bearing object is the OMNI dataset itself: a standardised five-view RGB imaging protocol with explicit rules for centering the dental midline and arch, followed by a two-stage annotation process (tooth bounding boxes, then categorical malocclusion labels) reviewed by senior dentists. The supporting pieces of machinery are the six detection baselines, of which GraphTeethNet is the paper's own addition. GraphTeethNet builds a tooth-relation graph in which each directed edge is a multi-dimensional feature learned by cross-attention between tooth proposals and global image features; the ablation shows that including these edge features raises mAP from 35.69 to 37.20 and mAP@0.5 from 62.09 to 63.89.","core_discovery":"OMNI is put forward as the first publicly available multi-view dental image dataset built for automated malocclusion diagnosis. It contains 4,166 RGB photographs taken with a standard camera from frontal, left, right, maxillary, and mandibular views, collected from 384 patients; each image has teeth localised by bounding boxes and is labelled for ten categories: healthy teeth, tooth torsion, deep overjet, invisible orthodontic attachment, tooth emergence, cast fixed orthodontic appliances, tooth misalignment, mandibular retrusion, orthodontic brace, and fixed orthodontic device. The paper argues that its multi-view RGB format avoids the radiation exposure and operational complexity of X-ray datasets, and that its size, patient diversity, and dentist-reviewed annotations support deep learning training. It validates the dataset by training three CNN detectors, two Transformer detectors, and one graph-based detector, with the graph-based GraphTeethNet adding learned multi-dimensional edge features between teeth and the best overall mAP@0.5 of 66.39 reported for Deformable DETR.","pith_inferences":["Because the images are ordinary dental-chair photographs, a primary-care or public-health screening tool for malocclusion could be built from this benchmark without X-ray equipment; the paper itself does not build such a tool.","If label quality is later verified against orthodontic records, the same five-view protocol could be extended to other oral conditions such as caries and gum inflammation.","The dataset was collected over first and follow-up visits, so a natural next step is to use the tooth-relation graph to predict treatment progression across time, which the current single-image benchmark does not exploit.","The largest performance differences between models are concentrated in rare categories, so class-balanced training and better label strategies may matter more than architecture choice for practical accuracy."],"forward_implications":["Automated malocclusion diagnosis can be studied and deployed from ordinary RGB photography, avoiding radiation exposure in screening pipelines.","The six published baselines give the community a common yardstick: future detectors on OMNI can be compared directly against these numbers.","The five-view acquisition protocol can serve as a template for collecting dental image data in clinical settings.","Graph-based modelling of inter-tooth relationships is shown to be a workable direction for oral image diagnosis, not just for teeth segmentation.","The reported category imbalance marks rare-condition recall as a concrete bottleneck for practical use."],"supporting_citations":[{"why":"an existing small dental X-ray dataset whose limitations the OMNI dataset is meant to address","marker":"[23]"},{"why":"a prior small panoramic radiograph dataset that defines the scarcity gap in dental image data","marker":"[24]"},{"why":"a large panoramic radiograph dataset with multi-level labels, the closest existing benchmark before OMNI","marker":"[25]"},{"why":"a large but only roughly labelled caries dataset that motivates the OMNI emphasis on dentist-reviewed annotations","marker":"[26]"},{"why":"the orthodontics reference from which the ten malocclusion category definitions are taken","marker":"[41]"},{"why":"the CNN detection architecture that underlies the Faster R-CNN baseline and the GraphTeethNet base","marker":"[42]"},{"why":"the Transformer detection method whose global-context modelling is benchmarked on OMNI","marker":"[38]"},{"why":"the Deformable DETR baseline that achieves the best reported mAP@0.5 on the dataset","marker":"[45]"},{"why":"the source of the multi-dimensional edge feature idea used by GraphTeethNet","marker":"[46]"}],"fun_headline_variants":["OMNI dataset brings malocclusion AI without X-rays","X-ray-free malocclusion screening: OMNI dataset","OMNI: first open dataset for malocclusion with no X-rays","4,166 photos, 384 patients: OMNI enables X-ray-free malocclusion AI","Malocclusion detection without X-rays: OMNI dataset benchmark"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the dentist-produced labels are accurate and consistent enough to serve as ground truth, since no inter-annotator agreement or independent clinical validation is reported.","fun_headline_variants_meta":{"raw":{"variants":["OMNI dataset brings malocclusion AI without X-rays","X-ray-free malocclusion screening: OMNI dataset","OMNI: first open dataset for malocclusion with no X-rays","4,166 photos, 384 patients: OMNI enables X-ray-free malocclusion AI","Malocclusion detection without X-rays: OMNI dataset benchmark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000773,"raw_usage":{"total_tokens":3441,"prompt_tokens":986,"completion_tokens":2455,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":2362}},"tokens_in":602,"tokens_out":2455,"duration_ms":17016,"temperature":1.0,"reasoning_tokens":2362,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:13:45.723928+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have an independent panel of orthodontists re-annotate a random subset of the OMNI images and compare their labels with the released ones; if per-category agreement is low, or if labels diverge from X-ray or dental records, the benchmark numbers reflect annotation noise rather than diagnostic skill.","supporting_citations":[{"cited_title":"Diffusion-based hierarchical multi-label object detection to analyze panoramic dental x-rays,","cited_arxiv_id":null,"evidence_quote":"a large panoramic radiograph dataset with multi-level labels, the closest existing benchmark before OMNI"},{"cited_title":"Digital dental x-ray database for caries screening,","cited_arxiv_id":null,"evidence_quote":"an existing small dental X-ray dataset whose limitations the OMNI dataset is meant to address"},{"cited_title":"Children’s dental panoramic radiographs dataset for caries segmentation and dental disease detection,","cited_arxiv_id":null,"evidence_quote":"a prior small panoramic radiograph dataset that defines the scarcity gap in dental image data"},{"cited_title":"Multi-level uncertainty aware learning for semi-supervised dental panoramic caries segmentation,","cited_arxiv_id":null,"evidence_quote":"a large but only roughly labelled caries dataset that motivates the OMNI emphasis on dentist-reviewed annotations"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"the orthodontics reference from which the ten malocclusion category definitions are taken"}],"review_version":1}