REVIEW 3 major objections 7 minor 36 references
Multiclass Sentiment Analysis for Identifying Political Viewpoints
T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Tamil political tweets hold best model to 0.28 macro F1 on a seven-way sentiment task.
desk verdict A low-resource shared-task baseline with plausible but internally inconsistent reported numbers; useful only after the tables are fixed and official scores confirmed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a seven-class supervised pipeline built on a 4,352-post Tamil training set, with a 544-post development set and a 544-post unlabeled test set. One arm encodes text with TF-IDF and trains an XGBoost classifier; the other tokenizes with BertTokenizer and fine-tunes bert-base-uncased for five epochs at batch size 32 and learning rate 1e-5. The load-bearing quantity is macro-averaged F1 over all seven classes, which is what ranks the models; because macro averaging weighs rare classes equally with common ones, the classifiers' poor handling of rare classes drives the low scores.
What would settle it
Sum the predicted counts in the published confusion matrices by class and compare them with the #Predicted column of Table 6 (305, 84, 50, 43, 15, 23, 24). If the sums do not match, or if recomputing macro F1 from a consistent matrix changes the 0.2835 and 0.2806 figures, the central result is not reproducible from the paper itself.
Extended reading notes
Core claim
The paper's central claim is that on the official test set, XGBoost with TF-IDF features reaches a macro F1 of 0.2835 and BERT reaches 0.2806, a near tie in which XGBoost edges ahead. Both models are strongly biased toward the Positive class: Positive receives by far the most correct predictions, while Opinionated, Sarcastic, Substantiated, None-of-the-above, and much of Negative are frequently absorbed into it. The authors present these numbers as demonstrating the difficulty of contextualized political sentiment and as baseline results for further work on this dataset and language.
Load-bearing premise
The load-bearing premise is that the published tables all come from one consistent evaluation run of the same test predictions; since the paper's own per-class counts do not fully agree, that premise is not fully established.
Editorial extensions
If this is right
- A future system for this task must clear roughly 0.28 macro F1 on the same seven classes to be a meaningful improvement.
- The error pattern points to the classes with the smallest training support, None-of-the-above and Substantiated, and to Positive as the dominant confusion target.
- Class-balanced training or cost-sensitive losses are the direct next step suggested by the data, not a new architecture.
- The near tie between a bag-of-words-plus-boosted-trees model and a fine-tuned transformer implies that architecture is not the main bottleneck on this dataset.
Reading between the lines
- A reader's inference from the two result tables is that the 0.0029 gap between XGBoost and BERT is smaller than the inconsistency between the reported per-class statistics, so the claimed ordering should not be used as evidence until the numbers are reconciled.
- A testable extension the paper leaves implicit is hierarchical classification: first coarse positive/negative/neutral, then fine-grained classes, which could reduce the dominant confusion and is directly checkable on the same test set.
- The low scores also suggest a data-limited regime, so data augmentation or pseudo-labeling on the unlabeled test posts might improve rare classes more than changing preprocessing.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes a multiclass sentiment analysis system for political Tamil tweets, developed for the DravidianLangTech 2025 shared task. The authors evaluate two models: an XGBoost classifier with TF-IDF features and a BERT-base-uncased transformer. On the official test set they report macro F1 scores of 0.2835 (XGBoost) and 0.2806 (BERT), concluding that the task is challenging and positioning their results as a baseline for future work.
Significance. If the reported results are accurate, the paper provides a useful, if modest, empirical baseline for political sentiment analysis in a low-resource language (Tamil). The use of standard evaluation metrics and the transparent reporting of BERT hyperparameters are strengths. However, the manuscript contains internal numerical inconsistencies that undermine confidence in the headline F1 scores, and it does not provide code, predictions, or official evaluation materials to support independent verification. The contribution is a system description for a shared task, acceptable in principle for a workshop venue only if the reporting errors are corrected and the results are made verifiable.
major comments (3)
- [Section 3.5, Tables 4 and 6] Table 6's predicted-instance counts contradict the precision and recall values in Table 4. From Table 4, the predicted count for each class is recall × support / precision, which gives approximately 23 (Opinionated), 50 (Sarcastic), 24 (Neutral), 305 (Positive), 43 (Substantiated), 84 (Negative), and 15 (None). Table 6 lists these same values but assigned to different labels (Opinionated 305, Sarcastic 84, Neutral 50, Positive 43, Substantiated 15, Negative 23, None 24). The text in Section 3.5 explicitly repeats the erroneous assignment when it states that the Opinionated label received 305 predictions. Because the macro F1 values in Table 4 are derived from the same per-class precision and recall values, this inconsistency makes the paper's internal reporting of the official test results unreliable and leaves the headline F1 scores without corroboration from the published tables. The authors must correct Table 6 and the accompanying discussion, or provide the official confusion matrices and predictions.
- [Sections 3.3 and 3.4] Section 3.3 states that TF-IDF and BertTokenizer are applied for the 'Logistic Regression and BERT models,' but Section 3.4 describes XGBoost (XGBClassifier) and BERT, and no Logistic Regression results appear anywhere in the paper. This methodological contradiction must be resolved: either the Logistic Regression experiment should be reported, or the text in Section 3.3 should be corrected to refer to XGBoost. As written, it is unclear which models were actually evaluated.
- [Reproducibility and verifiability] The manuscript provides no code, no data, no XGBoost hyperparameters, and no link to the official evaluation scores. Given the internal inconsistency between Tables 4 and 6, the authors should make their test-set predictions or confusion matrices available as supplementary material so that the claimed macro F1 scores (0.2835 and 0.2806) can be independently checked. Without such materials, the central comparative claim is not verifiable.
minor comments (7)
- [Section 3.1] The test set is described as 'unlabeled,' yet Tables 4 and 5 report support counts for each class. Please clarify that gold labels were obtained from the shared task organizers after submission, or describe how the support counts were derived.
- [Section 3.5] The text says the results of both approaches are presented in 'Tables 4 and 6,' but the BERT results appear in Table 5, not Table 6. The reference should be to Tables 4 and 5 for the two models, with Table 6 as a separate summary for XGBoost.
- [Section 3.4] There is a typo in 'IF-IDF' which should be 'TF-IDF.'
- [Abstract] The keyword line has a formatting artifact: 'K ˛eywords' displays an unusual accent; this should be cleaned up.
- [Section 4] The conclusion contains a duplicated phrase: 'into into seven multiclasses.'
- [Section 3.5] There are several typos in the figure discussion, including 'corret' instead of 'correct,' 'osition' instead of 'position,' and 'Figure 2 and 3' should be 'Figures 2 and 3.'
- [Section 1] The sentence 'the main contribution of this paper is summarized as follows' should begin with a capital letter after the preceding period.
Circularity Check
No significant circularity: the headline F1 scores are external measurements from an organizer-supplied test set; the internal table inconsistency is a correctness issue, not a circular derivation.
full rationale
The paper makes no first-principles derivation or fitted-prediction loop. Its central claim is that XGBoost and BERT achieve macro F1 scores of 0.2835 and 0.2806 on the official DravidianLangTech 2025 test set, which is an external benchmark. The models are trained on labeled data and evaluated on an unseen test set; no target quantity is defined in terms of the result and no fitted parameter is renamed as a prediction. Abundant self-citations in the literature review and methodology are not load-bearing for the test-set F1: the dataset comes from the shared task, not from the authors' own construction, and the cited prior work is background context, not a premise that forces the outcome. The inconsistency between Table 4's per-class precision/recall and Table 6's predicted counts is a reporting/correctness defect, not a form of circularity, because it does not make the F1 value an input to itself. Under the hard rules, no circular step can be exhibited with a quote, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- BERT learning rate =
1e-5
- BERT epochs =
5
- BERT batch size =
32
- XGBoost hyperparameters =
not specified
assumptions (5)
- domain assumption The DravidianLangTech 2025 dataset labels are correct and the seven-class scheme is a meaningful operationalization of political viewpoint sentiment.
- domain assumption bert-base-uncased, an English pretrained model, transfers to Tamil text without language adaptation.
- domain assumption Macro F1 is the appropriate primary metric for this seven-class task.
- domain assumption Standard preprocessing (removing punctuation, emojis, user mentions) improves or at least does not harm model performance.
- domain assumption The official test set of 544 instances is a sufficient sample to compare models.
Cite this review
Pith. "Pith review of Multiclass Sentiment Analysis for Identifying Political Viewpoints." pith.science (2026). https://pith.science/paper/ZYFWB3J7
@misc{pith2026260811049,
author = {Pith},
title = {Pith review of: Multiclass Sentiment Analysis for Identifying Political Viewpoints},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZYFWB3J7}},
note = {Machine review of arXiv:2608.11049}
}
read the original abstract
The rapid growth of social media has created vast amounts of political discourse, which provides valuable opportunities to analyze public opinions and identify different political perspectives. Sentiment Analysis (SA) is a core task in Natural Language Processing (NLP) that allows the computational study of attitudes and opinions in textual data, and has become increasingly important for understanding political discourse. In this work, we investigate multiclass sentiment analysis of political view- points on social media, that is to automatically discriminate multiple sentiment classes over political issues and figures. To solve this task we design and evaluate two machine-learning approaches based on XGBoost and BERT. We train and evaluate the models on a labeled dataset of political social media posts using standard classification metrics. The experimental results show that the XGBoost model reaches an F1-score of 0.2835 and the BERT- based model reaches an F1-score of 0.2806 on the test set. These results demonstrate the challenge of classifying complex and contextualized political discourse sentiment and provide a baseline for future research in multiclass political sentiment analysis.
Figures
Reference graph
Works this paper leans on
-
[1]
Recent Advances in Mathematical Research and Computer Science , volume=
Sentiment analysis: an approach for analysing tamil movie reviews using Tamil tweets , author=. Recent Advances in Mathematical Research and Computer Science , volume=
- [2]
-
[3]
DLRG@ DravidianLangTech-ACL2022: Abusive comment detection in tamil using multilingual transformer models , author=. Proceedings of the Second Workshop on Speech and Language Technologies for Dravidian Languages , year=
-
[4]
Social Media Hate and Offensive Speech Detection Using Machine Learning Method , author=. Proceedings of the Fourth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
-
[5]
Mucs@ dravidianlangtech2023: Malayalam fake news detection using machine learning approach , author=. Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages , pages=
-
[6]
Hope speech in social media texts using transformer , author=. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEURWS. org , year=
work page 2024
-
[7]
arXiv preprint arXiv:2410.02609 , year=
Ethio-fake: Cutting-edge approaches to combat fake news in under-resourced languages using explainable ai , author=. arXiv preprint arXiv:2410.02609 , year=
-
[8]
Evaluating the Quality of Data: Case of Sarcasm Dataset , author=
Show all 36 references
-
[9]
International Journal of Social Sciences & Educational Studies , volume=
The Effects of Web 2.0 Tools on Foundation English StudentsSuccess Rates at A Private University in Iraq , author=. International Journal of Social Sciences & Educational Studies , volume=. 2023 , publisher=
2023
-
[10]
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=
Deep reinforcement learning for information retrieval: Fundamentals and advances , author=. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=
-
[11]
Corpus Creation for Sentiment Analysis in Code-Mixed
Hegde, Asha and Anusha, Mudoor Devadas and Coelho, Sharal and Shashirekha, Hosahalli Lakshmaiah and Chakravarthi, Bharathi Raja , booktitle =. Corpus Creation for Sentiment Analysis in Code-Mixed. 2022 , address =
2022
-
[12]
Journal of Computer Science Research , volume=
Natural Language Processing and Its Challenges on Omotic Language Group of Ethiopia , author=. Journal of Computer Science Research , volume=
-
[13]
Transformer-Based Hate Speech Detection for Multi-Class and Multi-Label Classification , author=
-
[14]
Artificial Intelligence Review , volume=
State of the art: a review of sentiment analysis based on sequential transfer learning , author=. Artificial Intelligence Review , volume=. 2023 , publisher=
2023
-
[15]
Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages , pages=
Habesha@ DravidianLangTech: Abusive Comment Detection using Deep Learning Approach , author=. Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages , pages=
-
[16]
Multilingual Hope Speech Detection using Machine Learning , author=
-
[17]
Knowledge-Based Systems , volume=
Multitasking of sentiment detection and emotion recognition in code-mixed Hinglish data , author=. Knowledge-Based Systems , volume=. 2023 , publisher=
2023
-
[18]
Advances in Signal Processing, Embedded Systems and IoT: Proceedings of Seventh ICMEET-2022 , pages=
Multimodal Approach for Code-Mixed Speech Sentiment Classification , author=. Advances in Signal Processing, Embedded Systems and IoT: Proceedings of Seventh ICMEET-2022 , pages=. 2023 , publisher=
2022
-
[19]
vol , volume=
Development of Longest-Match Based Stemmer for Texts of Wolaita Language , author=. vol , volume=
-
[20]
American Journal of Software Engineering and Applications , volume=
Object Oriented Software Development for Artificial Intelligence , author=. American Journal of Software Engineering and Applications , volume=
-
[21]
arXiv preprint arXiv:2305.00090 , year=
NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis , author=. arXiv preprint arXiv:2305.00090 , year=
2023 arXiv
-
[22]
arXiv preprint arXiv:2211.14459 , year=
Transformer-based model for word level language identification in code-mixed kannada-english texts , author=. arXiv preprint arXiv:2211.14459 , year=
-
[23]
Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages , pages=
Csscutn@ dravidianlangtech: abusive comments detection in tamil and telugu , author=. Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages , pages=
-
[24]
Journal of Intelligent & Fuzzy Systems , number=
Detecting multilingual hate speech targeting immigrants and women on Twitter , author=. Journal of Intelligent & Fuzzy Systems , number=
-
[25]
Journal of Intelligent & Fuzzy Systems , number=
Odio-BERT: Evaluating domain task impact in hate speech detection , author=. Journal of Intelligent & Fuzzy Systems , number=
-
[26]
Proceedings of the Fourth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
Habesha@ DravidianLangTech 2024: Detecting Fake News Detection in Dravidian Languages using Deep Learning , author=. Proceedings of the Fourth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
2024
-
[27]
Procedia Computer Science , volume=
Semantic-Driven Topic Modeling Using Transformer-Based Embeddings and Clustering Algorithms , author=. Procedia Computer Science , volume=. 2024 , publisher=
2024
-
[28]
Knowledge-Based Systems , pages=
Evaluating the effectiveness of XAI techniques for encoder-based language models , author=. Knowledge-Based Systems , pages=. 2025 , publisher=
2025
-
[29]
International Conference on Advances of Science and Technology , pages=
Bilingual Word-Level Language Identification for Omotic Languages , author=. International Conference on Advances of Science and Technology , pages=. 2023 , organization=
2023
-
[30]
Diagnostics , volume=
Melanoma detection using XGB classifier combined with feature extraction and K-means SMOTE techniques , author=. Diagnostics , volume=. 2022 , publisher=
2022
-
[31]
Procedia Computer Science , volume=
Lexicon-based Language Relatedness Analysis , author=. Procedia Computer Science , volume=. 2024 , publisher=
2024
-
[32]
Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
LexiLogic@ DravidianLangTech 2025: Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments and Sentiment Analysis in Tamil and Tulu , author=. Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
2025
-
[33]
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages=
Amado at semeval-2025 task 11: Multi-label emotion detection in amharic and english data , author=. Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025) , pages=
2025
-
[34]
PeerJ Computer Science , pages=
Evaluating the capability of base and large-scale language models for multilingual sarcasm detection , author=. PeerJ Computer Science , pages=. 2026 , publisher=
2026
-
[35]
Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
Girma@ dravidianlangtech 2025: Detecting ai generated product reviews , author=. Proceedings of the Fifth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages , pages=
2025
-
[36]
Mexican International Conference on Artificial Intelligence , pages=
Pragmatic Generalization in LLMs: Insights from Fine-Tuning and Evaluating on Multilingual Sarcasm , author=. Mexican International Conference on Artificial Intelligence , pages=. 2025 , organization=
2025
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.