{"id":"89b2b3ca-d8c8-41b5-b1f3-a63c477193fe","arxiv_id":"2501.09506","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A survey of COVID-19 deep learning diagnostics that includes an internally inconsistent benchmark; the stated 99.97% image accuracy is not supported by the paper's own numbers.","lead":"This paper reviews deep learning methods for COVID-19 detection from images, text, and cough sounds, and reports a small benchmark of standard models. The headline result, MobileNet at 99.97% accuracy on X-ray images, conflicts with the paper's own tables, which report about 97.9%, so the benchmark claim is not currently reliable.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's headline MobileNet image accuracy of 99.97% is contradicted by the paper's own Table 7 (97.97%) and Table 8 (97.95%), leaving the central empirical claim unsupported.","rationale":"The reader's verdict is REJECT, and my independent check supports that rejection, so I recommend UNCHANGED. The reader identified the abstract-vs-table contradiction in the strongest_claim but framed the weakest_assumption around evaluation protocol (single 80/20 split, 10 epochs, small cough test set). My load-bearing concern is narrower and more decisive: the paper's own tables contradict its headline image accuracy, so the central numerical claim is internally unsupported before any questions of protocol or leakage. The 33-sample cough test set is a genuine secondary fragility, but it is not needed to reject the paper. The text result (BiGRU 99.89%) is internally consistent with Table 7 and therefore is not the basis for the concern. I am not alleging fraud or fabrication; the discrepancy could be a typo in the abstract or a mislabeled table. Either way, the manuscript as written does not support its own headline claim, which is sufficient grounds for the reader's REJECT verdict.","tokens_in":53741,"tokens_out":2131,"duration_ms":20631,"concrete_test":"Retrieve the code from the provided GitHub repository (https://github.com/shafiq-islam-cse/Multimodal-Marvels-of-Deep-Learning-Using-Image-Speech-and-Text-Review-of-COVID-19-Detection), rerun the MobileNet image experiment on the same 4,000-image COVID-19 Radiography subset using the stated 80/20 split, 10 epochs, batch size 256, Adam lr=0.005, L2=1e-5, and dropout=0.25, and record the best validation accuracy. If the reproduced value is 97.97% (or 97.95%), the abstract's 99.97% is unsupported; if it is 99.97%, Tables 7 and 8 require correction. Additionally, check the saved training logs or confusion matrix to confirm which number matches the actual run.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim, stated in the abstract, is that 'the MobileNet model achieved the highest accuracy of 99.97% for COVID-19 image data.' However, Section 4.3 reports MobileNetV2's highest training accuracy as 0.9797, Table 7 lists MobileNet at 97.97% for image analysis, and Table 8 reports 97.95% accuracy on the 800-sample image test set. The 99.97% figure appears nowhere in the experimental tables, and no explanation or erratum is offered for the two-percentage-point gap. Because the abstract's headline number is the key quantitative takeaway of the paper, this internal inconsistency is load-bearing: either the abstract is wrong or the experimental tables are wrong, and the manuscript does not resolve which. The cough result is less central but also fragile: Table 8's MobileNet cough accuracy of 93.69% is based on only 33 test samples, so it corresponds to about 31 correct predictions, and the abstract's 93.73% differs from the test-table value. Section 5.5 acknowledges data scarcity and generalisability limitations but never flags the abstract-versus-tables discrepancy. The text-domain claim (BiGRU 99.89%) is at least internally consistent with Table 7, so the contradiction is specific to the image claim that the paper leads with.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents a systematic review of deep learning methods for COVID-19 detection across image, text, and speech modalities, together with an experimental comparison of ten deep models (although the abstract says eleven) on public datasets. The experiments train pretrained CNN and RNN-style models for 10 epochs on an 80/20 split and report accuracy, precision, recall, and F1 for each modality. The paper concludes that MobileNet is the best image and cough model (with the abstract claiming 99.97% image accuracy) and that BiGRU is the best text model, and it proposes a cloud-based multimodal framework as future work.","tokens_in":54007,"tokens_out":7945,"duration_ms":77277,"significance":"The review component is a strength: it organizes 174 studies into a taxonomy, tabulates datasets and preprocessing tools, and provides a comparative table of previous work. The authors also make code and data available, and the limitations section acknowledges data scarcity and generalizability issues. If the experimental results were reliable, the systematic side-by-side comparison of ten models across three modalities would be a useful reference. However, the experimental contribution is not currently reliable: the headline image accuracy in the abstract is contradicted by the paper's own tables, the cough test set is extremely small, and the evaluation protocol lacks confidence intervals and leakage checks. The paper is therefore more valuable as a survey than as a benchmark, and the benchmark claims need substantial revision.","major_comments":[{"comment":"The abstract's central claim that MobileNet achieved 99.97% accuracy on COVID-19 image data is not supported anywhere in the experimental section. Section 4.3 reports MobileNetV2's highest training accuracy as 0.9797, Table 7 lists MobileNet image accuracy as 97.97%, and Table 8 reports 97.95% on the 800-sample image test set; Section 5.2 repeats 97.97%. No explanation or erratum is given for the two-percentage-point gap. Because this number is the principal quantitative takeaway of the paper, the manuscript must either correct the abstract to match the tables or provide experimental evidence for 99.97%.","section":"Abstract, §4.3, Tables 7–8"},{"comment":"The cough evaluation is too fragile to support any claim of superiority. Section 4.3 says the cough dataset contains 1257 samples (166 unique), and Section 4.4 states the cough test data comprise 166 samples, yet Table 8 reports support S=33 for every cough model and Fig. 13(c) shows a 33-sample confusion matrix with 17 positive cases. MobileNet's 93.69% cough accuracy therefore rests on roughly 31 correct predictions. No confidence intervals are provided, and no analysis of per-subject splitting, label noise, or recording confounds is reported. The authors must either supply a statistically sound evaluation (e.g., repeated stratified splits with confidence intervals and per-subject separation) or remove the cough-specific ranking.","section":"§4.3, §4.4, Table 8, Fig. 13(c)"},{"comment":"The experimental protocol is not sufficient to establish a reliable model ranking. The setup uses a single 80/20 split and a maximum of 10 epochs, and it describes the procedure as 'binary cross-validation' without defining how cross-validation was performed; Table 7 reports training accuracy, while Table 8 reports test accuracy on a different number of samples. There are no repeated runs, no confidence intervals, and no checks for class imbalance or data leakage between train and test. Since the paper's central experimental contribution is the comparison of ten models, the protocol needs to be substantially strengthened before the rankings can be accepted.","section":"§4.2.1"}],"minor_comments":[{"comment":"The heading 'Rational for the Study' should read 'Rationale'; the text also contains repeated typos such as 'pre-processiong', 'precession', and 'CNN-RN' for 'CNN-RNN'.","section":"§1.1, §4.4, Table 4"},{"comment":"The abstract states that 11 deep learning models were implemented and analyzed, while Section 1.3 and Table 7 describe and list ten models; please make the count consistent.","section":"Abstract, §1.3, Table 7"},{"comment":"The citation 'peng2020empirical' is not expanded in the reference list; please provide the full bibliographic entry.","section":"§4.1, limitation 3"},{"comment":"Section 5.1 recommends ResNet as the best picture model with 98% accuracy, which conflicts with the experimental recommendation in Section 5.2 and Table 7 where MobileNet is best; please reconcile these recommendations.","section":"§5.1, §5.2"},{"comment":"Table 4 contains unclear values such as '40,00' for text data and '28,00' for cough data; please verify these numbers and format them consistently.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the abstract-versus-tables contradiction is serious enough that I would not accept the paper in its current form. If the authors can supply a corrected abstract and a statistically sound evaluation, a revised version could be reconsidered. Otherwise, the benchmark portion should be removed and the paper reframed as a review only."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper is a review plus a small benchmark, and the benchmark's headline number is internally inconsistent. The abstract says MobileNet hit 99.97% accuracy on COVID-19 image data, but Table 7 reports 97.97% and Table 8 reports 97.95%, with no erratum or explanation. That is not a minor typo: the paper leads with this number, and the experimental section never reproduces it. The cough result is also fragile—the test set is 33 samples, so the reported 93.69% accuracy is about 31 correct predictions, and the abstract's 93.73% does not match the test table.\n\nWhat the paper does well: it assembles a broad systematic review of deep learning for COVID-19 across image, text, and speech, with 174 included studies, a taxonomy, comparative tables, and a summary of data challenges. The authors also provide a code/data link and document their training setup (10 epochs, Adam, L2, dropout). That is honest, reproducible-seeming scaffolding for a routine comparison of standard pretrained models on public datasets.\n\nThe soft spots beyond the headline contradiction: no confidence intervals, no leakage analysis, no label-quality check, and the cough test set is far too small to support ranking claims. The review portion is a restatement and reorganization of known material; I do not see a new taxonomy or framework here. The 'unifying framework' claimed in the introduction is essentially a table of features. Section 5.5 lists limitations, but it never flags the abstract-versus-tables discrepancy, which is the most important limitation of all.\n\nMy read: the empirical contribution is not reliable as written, and the review, while useful as a catalog, does not change practice. The fix is straightforward—correct the abstract, run proper evaluation with multiple splits and confidence intervals, and be transparent about the tiny cough sample. If the authors do that, the benchmark could be a modest useful contribution. In its current form, I would not send it to peer review; I would desk-reject with an invitation to resubmit after correcting the numbers and tightening the evaluation.\n\nFor the reading group, not a good pick unless you want an example of how internal consistency matters.","headline":"The abstract's 99.97% image accuracy is contradicted by the paper's own tables, so the benchmark's headline result is unsupported; the review portion is a broad but routine catalog.","tokens_in":54590,"tokens_out":3260,"would_cite":false,"duration_ms":30101,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pretrained deep learning models classify COVID-19 at near-perfect accuracy across X-rays, coughs, and tweets, the paper claims.","keywords":["deep learning","COVID-19 detection","multimodal analysis","medical imaging","transfer learning","cough classification","text classification","benchmark"],"falsifier":"Re-run the released code and check whether the reported numbers reproduce: Table 7 lists MobileNet image accuracy as 97.97% and Table 8 as 97.95%, while the abstract claims 99.97%. Then repeat the comparison with repeated stratified cross-validation and an externally collected held-out set; if the accuracy gaps between MobileNet or BiGRU and the next-best models shrink or vanish, the claimed rankings do not hold.","tokens_in":53542,"feed_emoji":"🩺","tokens_out":6311,"duration_ms":58115,"temperature":0.7,"pith_summary":"This review argues that standard deep learning architectures, fine-tuned on public datasets, can classify COVID-19 from three modalities at near-perfect accuracy: chest X-rays, cough recordings, and tweets. The paper reports that MobileNet (MobileNetV2) achieves the highest accuracy on image and cough data, while BiGRU leads on text, and it positions these results as a practical guide for model choice in pandemic screening. The authors also provide a taxonomy of deep learning methods for COVID-19 analysis, a comparative table of reviewed studies, and a proposed cloud-based multimodal diagnostic framework. If the benchmark results hold, the paper shows that transfer-learned models need only a handful of epochs to match or surpass previously published COVID-19 detection accuracy.","feed_headline":"MobileNet and BiGRU sweep near-perfect COVID-19 benchmarks","feed_subtitle":"One review benchmarks 11 standard models on X-rays, coughs, and tweets; MobileNet wins image and audio, BiGRU wins text.","key_machinery":"The load-bearing machinery is a uniform experimental protocol applied to a set of standard architectures (CNN, VGG-16, ResNetV3, Inception, DenseNet, Xception, AlexNet, CNN-RNN, EfficientNet, and MobileNetV2, with recurrent variants for text) across three public COVID-19 datasets. Images are 4,000 four-class X-rays, cough data are 1,257 samples with 166 unique samples, and text are 40,000 tweets; each model was fine-tuned for ten epochs with an 80/20 train-validation split, Adam at a 0.005 learning rate, L2 regularisation, and dropout. The protocol's power is that all models see identical preprocessing and data, so their accuracies can be ranked directly; its output is the training table, the test table with support sizes, and the recommendation that MobileNet-type CNNs generalise across image and audio while BiGRU leads text.","core_discovery":"The paper's central claim is that a systematic comparison of deep learning models across three data modalities identifies clear, dataset-specific winners: MobileNetV2 reaches 97.97% training and 97.95% test accuracy on four-class chest X-ray images (while the abstract headlines 99.97%), MobileNet reaches 93.73% training and 93.69% test accuracy on cough audio (from 33 unique test samples), and BiGRU reaches 99.89% training and 99.87% test accuracy on COVID-19 tweet text. The authors present these results as evidence that transfer-learned CNN and RNN families are data-specific in strength: convolutional models excel at image and audio, recurrent models at text. The paper frames this as a practical finding for building screening tools: when resources are limited, choose MobileNet-style lightweight CNNs for imaging and cough screening and BiGRU for text mining.","pith_inferences":["The reported near-perfect accuracies may reflect dataset-specific shortcuts rather than portable diagnostic performance, because public COVID-19 X-ray and tweet collections can contain style and wording artifacts; the same models should be re-tested on externally collected, independently labelled data before any clinical use.","For the cough benchmark, the 93.69% test accuracy rests on only 33 unique test samples, so a single mislabelled or confounded recording shifts the accuracy by about three percentage points; the speech ranking should be treated as provisional until a larger held-out patient group is used.","The paper's recommended architecture-per-modality pattern (CNN-family for image and audio, RNN-family for text) is consistent with broader deep learning experience and could be tested on other respiratory diseases using the same three-modality design."],"forward_implications":["MobileNetV2 becomes the first model to try for chest X-ray and cough-based COVID-19 screening when only transfer learning and short fine-tuning are affordable.","BiGRU becomes the recommended architecture for COVID-19 text classification on tweets, including sentiment and misinformation tasks, with near-perfect reported accuracy.","A cloud-based multimodal pipeline fed by image, cough, and text streams becomes a feasible next step, since a single lightweight architecture family performs strongly on two of the three modalities.","Researchers entering the field can use the paper's taxonomy and comparative table to locate which methods have already been tried and which research gaps remain for real-world deployment."],"supporting_citations":[{"why":"Supplies the cough dataset (1,257 samples, 166 unique) that anchors the speech benchmark claiming MobileNet at 93.73% accuracy.","marker":"link 1"},{"why":"Supplies the COVID-19 tweet text dataset (40,000 samples) used to rank BiGRU first at 99.89% accuracy.","marker":"link 2"},{"why":"Supplies the four-class chest X-ray dataset (4,000 images) used for the image benchmark ranking MobileNet first.","marker":"link 3"},{"why":"Provides the ten-CNN CT-image comparison that the review extends and uses as a state-of-the-art baseline for image models.","marker":"Ardakani et al. (2020)"},{"why":"Cough classification baseline (SVM/LSTM/ResNet) whose reported accuracy the speech benchmark is compared against.","marker":"Pahar et al. (2021)"},{"why":"Deep convolutional network baseline for respiratory sound data that anchors the speech review and comparative analysis.","marker":"Lella and Pja (2022)"},{"why":"Lexicon-based deep learning text baseline at 99% accuracy that the review's BiGRU result claims to surpass.","marker":"Ahmed et al. (2023)"}],"fun_headline_variants":["MobileNet and BiGRU lead multimodal COVID-19 detection","MobileNet tops image and audio, BiGRU tops text in COVID-19 review","Multimodal DL review: MobileNet for X-rays, BiGRU for tweets","COVID-19 model showdown: MobileNet wins images, BiGRU wins text"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark conclusions assume that a single 80/20 random split of each public dataset, trained for only ten epochs, gives a fair and stable comparison of the models, and that the test labels are clean and non-overlapping with training data.","fun_headline_variants_meta":{"raw":{"variants":["MobileNet and BiGRU lead multimodal COVID-19 detection","MobileNet tops image and audio, BiGRU tops text in COVID-19 review","Multimodal DL review: MobileNet for X-rays, BiGRU for tweets","COVID-19 model showdown: MobileNet wins images, BiGRU wins text"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000543,"raw_usage":{"total_tokens":2642,"prompt_tokens":1029,"completion_tokens":1613,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":1528}},"tokens_in":645,"tokens_out":1613,"duration_ms":14207,"temperature":1.0,"reasoning_tokens":1528,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:56:31.156250+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the released code and check whether the reported numbers reproduce: Table 7 lists MobileNet image accuracy as 97.97% and Table 8 as 97.95%, while the abstract claims 99.97%. Then repeat the comparison with repeated stratified cross-validation and an externally collected held-out set; if the accuracy gaps between MobileNet or BiGRU and the next-best models shrink or vanish, the claimed rankings do not hold.","supporting_citations":[],"review_version":1}