{"id":"7da90e0e-f08f-4f02-a5e7-2efd4bbb7a95","arxiv_id":"2501.18766","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A standard GRU classifier is claimed to reach 94% accuracy on Bangla fake news detection, but the paper's data and baseline comparisons are internally inconsistent.","lead":"This paper reports a GRU-based deep learning model for detecting fake news in Bangla, claiming 94% accuracy on a dataset of 58,478 news items. The work is undermined by internal contradictions in the dataset description and an incomplete comparison with stronger models.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 94% accuracy claim is unsupported because the dataset description is internally contradictory: Section III-A says 58,478 rows, yet Section III-B reports only 3,000 + 2,000 = 5,000 instances after oversampling.","rationale":"The reader's weakest assumption is exactly the load-bearing point: every downstream number in the paper is expressed relative to a dataset that is described twice, incompatibly. Section III-A says 58,478 rows with two classes; Section III-B says after oversampling there were 3,000 real and 2,000 fake instances, i.e., 5,000 total. Since oversampling only adds duplicate minority examples, it cannot shrink a 58,478-row corpus to 5,000 rows. Therefore the reported 94% accuracy either refers to a different, smaller dataset or the oversampling/count description contains a fundamental error. I do not treat this as evidence of misconduct; a reporting error could produce the same text. But the result is the same: the central accuracy and 'large dataset' claims are not verifiable from the manuscript. The additional inconsistencies—LSTM in the introduction versus GRU elsewhere, Table II's 'second highest' wording when 94% exceeds the listed 93.32%, and [5]'s accuracy shifting from 87% in the review to 78.62% in Table II—are consistent with the same pattern of unreconciled numbers. No code, data, or confusion matrix is provided, so no independent check is possible from the preprint alone. My concrete test targets the decisive count reconciliation; until that is resolved, the REJECT verdict should stand.","tokens_in":7170,"tokens_out":4501,"duration_ms":44230,"concrete_test":"Recompute the totals in Section III-B: 3,000 real plus 2,000 fake equals 5,000, and compare this with Section III-A's 58,478 rows. If the authors' preprocessing script or data release shows that the 5,000 post-oversampling counts refer to the same corpus used for the 94% accuracy result, the contradiction is resolved; otherwise the accuracy claim is untethered from the claimed dataset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—94% accuracy with a 32-unit GRU on a 58,478-item Bangla fake-news dataset, outperforming prior work—cannot be evaluated because the dataset used for training is not coherently specified. Section III-A states the data has 58,478 rows and two labels, Fake and Real. Section III-B then says that after oversampling 'there were 3000 real and 2000 fake data' (5,000 total). Oversampling adds minority-class copies; it cannot reduce a 58,478-row two-class corpus to 5,000 rows. Thus either the row count, the class counts, or the description of the oversampling step is wrong. The performance metrics in Table I (94% accuracy, per-class precision 92%/95%, recall 93%/94%, F1 93%/94%) are presented without the corresponding confusion matrix numbers, so they cannot be checked independently. This internal inconsistency is load-bearing: if the actual training set was 5,000 items rather than 58,478, the claimed 'large dataset' contribution and the 94% figure describe a different experiment than the one reported. Secondary contradictions (LSTM in the introduction versus GRU elsewhere; Table II's 'second highest' statement despite 94% > 93.32%; and the review's [5] 87% Gaussian NB versus Table II's 78.62% LR) reinforce the same conclusion: the reported numbers do not form a reliable evidentiary chain.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a GRU-based deep learning model for Bangla fake news detection. The authors describe a pipeline of data collection, preprocessing (null-value handling, lemmatization, tokenization, padding, label encoding), and oversampling to address class imbalance, then train a sequential Keras model with an embedding layer and a 32-unit GRU. They report 94% accuracy, with per-class precision 92%/95%, recall 93%/94%, and F1 93%/94%. The paper claims that the contribution includes a large Bangla fake news dataset of 58,478 passages and that the proposed model outperforms other Bangla fake news detection models. The evaluation is compared with a small set of prior works in Table II.","tokens_in":7417,"tokens_out":2602,"duration_ms":29063,"significance":"If the reported results were internally consistent, the paper would offer two contributions: a relatively large Bangla fake news corpus and evidence that a compact GRU with oversampling reaches 94% accuracy. Neither contribution can currently be credited. The dataset size is described inconsistently in Section III-A and Section III-B, making the experimental basis unverifiable. The claimed superiority over prior work is contradicted by the paper's own literature review, which lists BERT at 95% (ref [4]) and Bi-LSTM at 95% (ref [7]). Because the central performance claim rests on contradictory numbers, the paper in its present form does not advance the state of knowledge.","major_comments":[{"comment":"The dataset size is internally contradictory. Section III-A states that the collected data has 58,478 rows, while Section III-B reports that after oversampling \"there were 3000 real and 2000 fake data,\" i.e., 5,000 instances total. Oversampling adds copies of minority-class instances; it cannot reduce a two-class corpus from 58,478 rows to 5,000 rows. This is a load-bearing inconsistency because the 94% accuracy in Table I cannot be associated with a coherent training set. The authors must reconcile these numbers or the central empirical claim is uninterpretable.","section":"Section III-A and III-B"},{"comment":"The claim that the proposed model \"has outperformed other Bangla fake news detection models\" is contradicted by the paper's own literature review. Section II reports that BERT achieves 95% accuracy in ref [4] and that Bi-LSTM achieves 95% accuracy in ref [7], both exceeding the proposed 94%. Moreover, Table II omits these two works, so the comparison is selective and does not support the stated superiority claim. The authors need to either include these baselines in Table II or substantially qualify their claim.","section":"Abstract, Section II, and Table II"},{"comment":"The text accompanying Table II says \"our model achieved the second highest accuracy,\" yet Table II lists the proposed model at 94% and the highest competitor (ref [8]) at 93.32%. If the table is correct, the proposed model is the highest, not second highest. Additionally, the literature review in Section II attributes 87% accuracy to Gaussian Naive Bayes in ref [5], while Table II lists 78.62% for that work's Logistic Regression model. These unexplained discrepancies make the comparison table an unreliable basis for evaluating the proposed model.","section":"Section IV, Table II"},{"comment":"The introduction repeatedly identifies Long Short-Term Memory (LSTM) networks as the proposed technique: for example, \"this paper presents a spearheading endeavour in leveraging profound learning strategies, particularly Long Short-Term Memory (LSTM) systems.\" However, Section III-C and the abstract describe a Gated Recurrent Unit (GRU) model. It is therefore unclear which architecture is actually being proposed and evaluated. This ambiguity must be fixed for the model description to be meaningful.","section":"Section I versus Section III-C"}],"minor_comments":[{"comment":"The abstract contains several typographical and translation artifacts (\"profound learning,\" \"Gated Repetitive Unit,\" \"ponder,\" \"comes about\", \"noteworthy precision rate\"). The paper would benefit from a thorough language edit.","section":"Abstract"},{"comment":"The sentence on label encoding is ungrammatical and unclear: \"Then Label Encoding was applied to convert the string into 'fake' and 'real' into numeric number 0 and 1 respectably.\" It should describe how two class labels are mapped to 0 and 1.","section":"Section III-B"},{"comment":"Table I reports precision, recall, F1, and accuracy but no confusion-matrix counts; Figure 8 is a visualization without numeric values. Without the underlying counts, the reported metrics cannot be independently verified.","section":"Table I and Figure 8"},{"comment":"The paper states that 100,534 unique tokens were found but that the vocabulary was limited to 10,000 words. The relationship between these numbers, and the exact tokenization method, should be clarified.","section":"Section III-B"},{"comment":"The paper does not provide a link or repository for the claimed dataset, despite stating in Section III-A that the data were collected from news portals and social media. A public dataset release is essential for reproducibility, especially since the dataset size claim is central to the paper.","section":"References"},{"comment":"Hyperparameters such as dropout, recurrent dropout, and activation functions are not reported, and Fig. 5 (the architecture diagram) is not described in enough detail to reproduce the model. Please provide a complete layer-by-layer specification.","section":"Section III-D and Fig. 5"}],"recommendation":"reject","confidential_remarks":"The manuscript has a serious evidentiary problem: there are contradictory statements about the dataset size (58,478 rows vs. 3,000 + 2,000 instances), about the model type (LSTM in the introduction vs. GRU in the methodology), and about the comparison results (\"second highest\" despite Table II showing the highest accuracy). These are not mere presentation slips; they undermine the central claim of a 94% accurate model that outperforms prior work. Even if the authors clarified the numbers, the paper would still need to include the omitted higher-accuracy baselines from its own literature review. I do not see a limited revision that would make this submission acceptable; the experimental report would need to be redone or substantially re-presented with a consistent dataset description and fair baselines. My recommendation is reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know one thing up front: the central claim in this paper—94% GRU accuracy on a 58,478-item Bangla fake-news dataset—is not supported by the manuscript itself. Section III-A says the dataset has 58,478 rows; Section III-B says that after oversampling there were 3,000 real and 2,000 fake instances, i.e., 5,000 total. Oversampling adds copies of the minority class; it cannot shrink a corpus by an order of magnitude. Those two numbers cannot both describe the same dataset, so the reported accuracy has no reliable data foundation. The introduction also says the model is LSTM, while the rest of the paper uses GRU. The comparison table omits the BERT 95% and Bi-LSTM 95% results that the paper's own literature review cites, and it lists [5] as 78.62% LR even though the review says that paper used Gaussian Naive Bayes with 87% accuracy. These are load-bearing contradictions, not cosmetic issues.\n\nTo give credit where it is due: the problem is real and important—Bangla fake-news detection is underexplored, and a larger public dataset would genuinely help. The authors describe a standard preprocessing pipeline, report precision/recall/F1, and make an attempt to compare against prior work. But the architecture is routine (a 32-unit GRU), and prior papers in their own reference list already apply GRU to Bangla fake-news detection. The only plausible new contribution would be the dataset, but that dataset is neither released nor coherently described. No code, no data, no confusion-matrix raw counts—just a figure that cannot be checked.\n\nThe stress-test note is correct. The internal inconsistencies mean the 94% figure describes an experiment we cannot reconstruct. I do not see a salvageable core here. The authors could fix the dataset description, release the data, and rerun the comparison with BERT and Bi-LSTM, but as submitted the manuscript is not ready for peer review. It would waste referee time.\n\nRecommendation: desk reject. If the authors eventually release the dataset and correct the contradictions, the work might be worth a second look as a dataset paper. Until then, the evidence chain is broken.","headline":"The paper's 94% accuracy claim collapses under its own dataset contradictions; no code or data is released, so there is nothing verifiable to referee.","tokens_in":8092,"tokens_out":1429,"would_cite":false,"duration_ms":17229,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a 32-unit GRU network detects Bangla fake news with 94% accuracy, beating the prior detectors it compares against.","keywords":["Bangla fake news detection","Gated Recurrent Unit","oversampling","text classification","imbalanced dataset","deep learning","Bangla NLP","misinformation detection"],"falsifier":"Count the rows of the raw dataset and the class distribution before and after the described oversampling, then rerun the stated GRU training with the same 80/10/10 split and report test-set metrics. If the pre-oversampling row count is not 58,478 or the post-oversampling totals are not 3,000 real and 2,000 fake, the claimed 94% accuracy does not correspond to a clearly specified experiment.","tokens_in":6909,"feed_emoji":"📰","tokens_out":7771,"duration_ms":76554,"temperature":0.7,"pith_summary":"The paper argues that a comparatively simple deep learning model, a gated recurrent unit (GRU), can detect fake news in Bangla with 94% accuracy after text preprocessing and oversampling of the minority class. The reported precision is 92% for fake news and 95% for real news, with recall of 93% and 94% and F1 scores of 93% and 94%. The authors claim this result outperforms the earlier Bangla fake news detectors in their comparison while using the largest dataset, 58,478 rows, and they position the work as evidence that lightweight recurrent models can serve under-resourced languages. The result matters because Bangla has far fewer annotated corpora and detection tools than major languages, so a strong, reproducible baseline would lower the barrier for real-world misinformation tools.","feed_headline":"GRU model claims 94% accuracy on Bangla fake news","feed_subtitle":"The result would beat earlier detectors on the largest dataset, but two dataset-size counts in the paper disagree.","key_machinery":"The load-bearing mechanism is the Gated Recurrent Unit (GRU), a recurrent network cell that carries a hidden state through a text sequence using update and reset gates, paired here with oversampling to balance the two classes. The model is a sequential network: an embedding layer of dimension 100, one 32-unit GRU layer, and a dense sigmoid layer for binary classification, trained with the Adam optimizer, binary cross-entropy loss, batch size 32, and 10 epochs with a 20% validation split. The preprocessing chain—tokenization, vocabulary truncation, padding, label encoding—converts raw Bangla headlines and content into fixed-length numeric sequences.","core_discovery":"On the paper's own account, the central discovery is that a sequential neural network with an embedding layer of dimension 100, a 32-unit GRU layer, and a sigmoid output, trained with Adam at learning rate 1e-4 for 10 epochs on an oversampled Bangla news corpus, achieves 94% accuracy with average precision 93%, average recall 94%, and average F1 93% in distinguishing fake from real news. The authors further claim that this result surpasses earlier Bangla fake news detectors in the literature, and that their dataset of 58,478 headline-plus-content rows is the largest used in such comparisons. The proposed pipeline is lemmatization, unwanted-character removal, tokenization, vocabulary truncation to 10,000 words, padding to length 100, label encoding, and oversampling to correct class imbalance.","pith_inferences":["An implication the paper leaves implicit is that a 32-unit GRU at 10 epochs is enough to reach 94% on Bangla, which would make the marginal value of large pretrained language models uncertain for this task if the result holds.","The paper does not address test-set leakage from oversampling before splitting; a fold-internal oversampling experiment would settle whether the reported accuracy is optimistic.","The unresolved 58,478-versus-5,000 dataset-count discrepancy means the result cannot currently be reproduced; resolving the count, or rerunning on a public Bangla benchmark, is the direct test of the contribution."],"forward_implications":["If the 94% accuracy holds, a lightweight GRU with oversampling is a viable alternative to heavier transformer models for Bangla fake news detection.","The reported balanced precision and recall (93% and 94% on average) would mean the model does not simply predict the majority class, a common failure on imbalanced news corpora.","The claimed 58,478-item dataset would be the largest Bangla fake news corpus among the compared works, giving future researchers a larger training resource.","The second-highest accuracy in the comparison table, behind a Multinomial Naive Bayes model trained on 1,500 items, suggests dataset size and model complexity trade off in ways that deserve direct comparison."],"supporting_citations":[{"why":"Supplies the imbalance-handling methods (SMOTE and stacked generalization) that motivate the paper's oversampling step.","marker":"[1]"},{"why":"Provides the deep hybrid learning baseline with roughly 99% overall F1 and 82% fake-news F1 that the paper aims to surpass.","marker":"[3]"},{"why":"Provides the logistic-regression baseline with 78.62% accuracy reported in the paper's comparison table.","marker":"[5]"},{"why":"Provides the earlier GRU baseline with 77% accuracy on a 57,000-item Bangla corpus, the most direct model comparison.","marker":"[7]"},{"why":"Provides the multinomial naive Bayes baseline with 93.32% accuracy, the strongest prior accuracy in the paper's comparison table.","marker":"[8]"}],"fun_headline_variants":["Bangla fake news detection hits 94% with GRU","GRU beats other Bangla fake news detectors","94% accuracy on 58K Bangla news posts","Deep learning tackles Bangla fake news","New Bangla fake news dataset and GRU model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The account of the dataset is internally consistent: the paper first says the dataset has 58,478 rows, then reports 3,000 real and 2,000 fake instances after oversampling, a total of 5,000; if either count is wrong, the 94% accuracy cannot be tied to any reproducible dataset.","fun_headline_variants_meta":{"raw":{"variants":["Bangla fake news detection hits 94% with GRU","GRU beats other Bangla fake news detectors","94% accuracy on 58K Bangla news posts","Deep learning tackles Bangla fake news","New Bangla fake news dataset and GRU model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1262,"prompt_tokens":912,"completion_tokens":350,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":275}},"tokens_in":528,"tokens_out":350,"duration_ms":4108,"temperature":1.0,"reasoning_tokens":275,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T22:31:31.226802+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Count the rows of the raw dataset and the class distribution before and after the described oversampling, then rerun the stated GRU training with the same 80/10/10 split and report test-set metrics. If the pre-oversampling row count is not 58,478 or the post-oversampling totals are not 3,000 real and 2,000 fake, the claimed 94% accuracy does not correspond to a clearly specified experiment.","supporting_citations":[{"cited_title":"M., Awosaf, Z., Prottoy, M","cited_arxiv_id":null,"evidence_quote":"Supplies the imbalance-handling methods (SMOTE and stacked generalization) that motivate the paper's oversampling step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the deep hybrid learning baseline with roughly 99% overall F1 and 82% fake-news F1 that the paper aims to surpass."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the logistic-regression baseline with 78.62% accuracy reported in the paper's comparison table."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the earlier GRU baseline with 77% accuracy on a 57,000-item Bangla corpus, the most direct model comparison."},{"cited_title":"G., Hasan, M","cited_arxiv_id":null,"evidence_quote":"Provides the multinomial naive Bayes baseline with 93.32% accuracy, the strongest prior accuracy in the paper's comparison table."}],"review_version":1}