Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Linguistic Analysis of Sinhala YouTube Comments on Sinhala Music Videos: A Dataset Study

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that YouTube comments on Sinhala songs form a representative subset of general Sinhala, and it provides a 63,471-comment dataset and a 964-word stop-word list to support that claim.

desk verdict A genuine new Sinhala YouTube comment corpus, but the representativeness claim is unsupported and the transliteration step needs validation; worth peer review with major revisions. read the letter →

arxiv 2501.18633 v1 pith:J4264DJ7 submitted 2025-01-28 cs.CL

W. M. Yomal De Mel , Nisansa de Silva This is my paper · ORCID
classification cs.CL
keywords SinhalaNLPYouTubecommentsmusicemotionrecognitionlow-resourcelanguagestopwordscorpuscreationsentimentanalysislinguistic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that comments on Sinhala music videos are not an isolated slang pool but a representative slice of the Sinhala language, and it backs that claim with a public dataset of 63,471 cleaned Sinhala comments drawn from 20 songs. It also derives 964 frequency-based stop words for this comment domain, 182 of which match English stop words after translation. If the claim holds, the dataset and stop-word list become reusable building blocks for sentiment analysis, music emotion recognition, and other Sinhala NLP tasks that currently lack resources.

What carries the argument

The load-bearing method is a Z-score filter applied to word frequencies: after computing the mean and standard deviation of comment-word frequencies, words with a standardized frequency above 3.0 are flagged as frequent, producing the 964-word stop-word list. Representativeness is argued through a one-hot presence vector for each word across the Wikipedia, newspaper, and government corpora, a Word2Vec embedding of the combined general-domain corpus, and a t-SNE visualization showing that words shared across sources cluster together. The 36% overlap statistic is the paper's main quantitative evidence that the comment domain reflects general Sinhala.

What would settle it

Manual inspection of a random sample of the 28,043 transliterated comments would settle it: if a large fraction of them are garbled or actually English words rendered in Sinhala script, the dataset and the 36% overlap claim would need to be re-evaluated. A cheap first check is whether the 182 'true stop words' still correspond to genuine Sinhala function words when back-transliterated.

Watch

Extended reading notes

Core claim

The central discovery is a curated corpus: from 93,116 raw comments on 27 videos, the authors kept 63,471 Sinhala comments after removing English-only text, spam, and non-Sinhala characters, and after transliterating 28,043 Roman-script comments into Sinhala script. A frequency analysis found 54,834 unique Sinhala words and 328,922 distinct word pairs. The authors then built a general-domain Sinhala word set from Wikipedia articles, newspaper articles, and government documents, and found that 20,015 words (about 36% of the comment-domain unique words) appear in both domains. They interpret this overlap as evidence that the YouTube comment domain is a representative subset of the general Sinhala language domain.

Load-bearing premise

The whole dataset assumes that the automatic transliteration of 28,043 Roman-script comments into Sinhala script produces linguistically valid Sinhala text; if those conversions are noisy or mistranslate English phrases into Sinhala-looking words, every frequency count and the representativeness comparison are contaminated.

Editorial extensions

If this is right

  • A ready-made 63,471-comment Sinhala corpus can serve as training or evaluation data for Sinhala sentiment analysis, especially for music-related emotion classification.
  • The 964 stop-word list gives Sinhala NLP pipelines a domain-specific filter, which should reduce noise in downstream frequency and embedding analyses.
  • The 36% lexical overlap suggests that models trained on this comment corpus are not locked into slang; they may transfer reasonably to other written Sinhala domains.
  • The identified high-frequency words ('ලස්සනයි', 'සුපිරි') and word pairs ('හරිම ලස්සනයි') offer a starting lexicon for detecting praise and emotional engagement in music discourse.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper only uses a 36% overlap to claim representativeness, a stricter statistical test (for example, comparing frequency distributions of common words with a correlation measure) would be a natural next step; such a test is not performed in the paper.
  • The transliteration noise concern also applies to the stop-word list: if mis-transliterated English function words entered the list, then the 182-word match with English stop words may partly be an artifact of transliteration rather than true linguistic overlap.
  • A direct extension would be to run the same preprocessing on comments for other South Asian languages to see whether the 36% overlap figure is characteristic of code-mixed music comment sections or unique to Sinhala.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a new dataset of 63,471 Sinhala YouTube comments collected from 27 videos of 20 Sinhala songs, obtained after a pipeline of filtering, language detection, and transliteration of Roman-script comments via the Google Transliterator API. The authors algorithmically derive a list of 964 frequent Sinhala words claimed as stop-words using a z-score threshold of 3.0, and compare the YouTube comment vocabulary with a general-domain Sinhala corpus built from Wikipedia, newspapers, and government documents. On the basis of a 36% type overlap between the YouTube and general-domain unique word sets, the paper concludes that the YouTube comment domain is a representative subset of the general Sinhala language domain. The paper also reports word and word-pair frequency statistics and discusses implications for MIR and MER.

Significance. If validated, the dataset and the derived stop-word list would be reusable resources for Sinhala NLP, a language with few public corpora. The manuscript's strengths include the new data collection, the public GitHub repository, the algorithmic derivation of the stop-word list, and the attempted comparison with external lists (NLTK and Lakmal et al.). However, the central representativeness claim is not established by the evidence provided, and the unvalidated transliteration step poses a data-quality risk. The dataset itself, with clearer documentation and validated preprocessing, could still be a useful contribution, but the current paper overstates its conclusions.

major comments (4)
  1. [Section IV-B, Figure 6] The conclusion that the YouTube comment domain 'serves as a representative subset of the general Sinhala language domain' is based solely on the observation that 36% of YouTube unique words also appear in the general-domain corpus. Type overlap is a coverage statistic, not a representativeness criterion. The paper's own frequency analysis in Figure 6 shows systematic divergence: words such as 'වෙග්' and 'සුපිරි' are far more frequent in YouTube comments, while 21 words including 'ඇති', 'සහ', and 'කර' are significantly more frequent in the general domain. A token-level frequency correlation, a divergence measure, or a matched-sample design would be needed to support the representativeness claim. As written, the conclusion does not follow from the presented evidence.
  2. [Section III, transliteration step] The pipeline converts 28,043 Roman-script comments into Sinhala script using the Google Transliterator API, but the paper provides no manual validation or error-rate estimate for these transliterations. If English-only comments or mixed-script comments were transliterated into Sinhala-looking text with incorrect or nonce forms, all downstream frequency counts, stop-word statistics, and domain comparisons would be contaminated. The authors should report a human evaluation of a random sample of transliterated comments, or otherwise demonstrate that the transliterated text is linguistically valid Sinhala.
  3. [Section IV-A, stop-word derivation] The z>3.0 threshold identifies 964 'frequently occurring words' that are then called stop-words. However, the paper's own Table II shows that the most frequent words include content words such as 'ලස්සනයි' (beautiful) and 'සුපිරි' (super), which are sentiment-bearing adjectives. Only 182 of the 964 words match English NLTK stop-words after translation, and only 53 match the Lakmal et al. Sinhala stop-word list. The paper does not demonstrate that the remaining words are function words or otherwise semantically neutral. A stop-word list that retains strong sentiment markers is problematic for the sentiment-analysis applications the paper motivates. The authors should evaluate the list against a function-word criterion or show the impact of removing these words on a downstream task.
  4. [Section III, numeric pipeline] The dataset construction counts are ambiguous. The paper states that 35,428 comments with Sinhala characters were identified and 30,716 comments had a combination of non-English and English characters, totaling 66,144; it then reports a segregation into 83 comments containing non-English characters and 30,633 comments composed solely of English characters, but the final dataset of 63,471 is 35,428 + 28,043, leaving the fate of the 83 comments and the 2,590 unsuccessful transliterations unexplained. The paper should clarify which rows were excluded at each step and why, as this affects the reproducibility and interpretation of all subsequent statistics.
minor comments (5)
  1. [Throughout] The manuscript contains numerous typographical errors (e.g., 'efficient', 'feild', 'difficult', 'intial', 'contibution') and grammatical issues (e.g., the abstract's 'confirming latter as good representation of general domain'). A thorough proofread is needed.
  2. [Figures 5a-5h] The caption for Figure 5 lists subfigures (a) through (g) but the final panel is labeled (h) 'All Combinations'; the text also refers to 'Figure 5' without a panel letter when discussing the combined plot. Please make the cross-references and panel labels consistent.
  3. [Section IV-B, t-SNE paragraph] The description of the one-hot encoding and t-SNE visualization is not connected to the representativeness claim or to any quantitative result; either explain how the t-SNE plots support the analysis or remove this passage to avoid confusion.
  4. [Table I] The table lists song titles and comment counts, but the paper does not describe the criteria used to select the 27 videos and 20 songs, or how the 'strict linguistic reliability' mentioned in the Introduction was operationalized. A short paragraph on selection criteria would improve reproducibility.
  5. [Section IV, first paragraph] The sentence 'It was noted that a higher number of comments consist of two unique Sinhala words' is unclear: does 'two unique Sinhala words' mean comments with exactly two word tokens, or two distinct word types? Please clarify the intended statistic.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dataset, stop-word list, and domain comparison are derived from independent pipeline steps; the weak representativeness inference is a validity issue, not a circular one.

full rationale

No step in the paper reduces by construction to its own inputs. The stop-word list is derived from the newly collected YouTube comment corpus using an explicit z-score threshold (Eq. 1), and it is checked against external NLTK and Lakmal et al. lists; the citation to Wijeratne and de Silva [5] is a methodological acknowledgment, not an unverified load-bearing premise. The general-domain corpus is assembled from Wikipedia articles, newspaper articles, and prior Sinhala resources (including some by the same group), but it is independent of the YouTube comment dataset, so the 36% type-overlap comparison is not circular. The representativeness conclusion drawn from type overlap is logically weak, and the transliteration quality is a data-validity concern, but neither makes the derivation equivalent to its inputs. No fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported from self-citations as a substitute for evidence.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central resource relies on a standard corpus-building pipeline. Two hand-set z-score thresholds (same value 3.0) are the only numeric free parameters; key domain assumptions are the unvalidated transliterator, the representativeness of the self-assembled general corpus, and the NLTK filter. No invented entities are introduced.

free parameters (1)
  • Z-score threshold for stop-word selection and domain comparison = 3.0
    Words with z > 3.0 are designated frequently occurring and treated as stop-words; the same threshold defines the four quadrants in the domain comparison. No sensitivity analysis is given, and the threshold materially determines the 964-word list (Section IV-A and IV-B).
assumptions (4)
  • domain assumption Word frequency distributions can be treated with a z-score threshold as if approximately normal, so z > 3 identifies outliers.
    Natural-language frequency distributions are Zipfian and heavily skewed; the paper applies a normal-distribution-based threshold without testing this assumption (Section IV-A).
  • domain assumption The Google Transliterator API converts Roman-script Sinhala and mixed comments into faithful Sinhala script.
    The final corpus depends on 28,043 transliterated comments; no accuracy rate is reported (Section III).
  • domain assumption The general Sinhala corpus assembled from 9 Wikipedia pages, 82 newspaper articles, and prior research resources is representative of general Sinhala.
    The representativeness conclusion compares against this corpus; it is small and ad hoc, with no sample design (Section IV-B).
  • domain assumption The NLTK English word list is an adequate tool for identifying English-only comments.
    It may misclassify Singlish comments or mixed-language comments, affecting the filtering counts (Section III).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Linguistic Analysis of Sinhala YouTube Comments on Sinhala Music Videos: A Dataset Study." pith.science (2026). https://pith.science/paper/J4264DJ7

@misc{pith2026250118633,
  author       = {Pith},
  title        = {Pith review of: Linguistic Analysis of Sinhala YouTube Comments on Sinhala Music Videos: A Dataset Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J4264DJ7}},
  note         = {Machine review of arXiv:2501.18633}
}
read the original abstract

This research investigates the area of Music Information Retrieval (MIR) and Music Emotion Recognition (MER) in relation to Sinhala songs, an underexplored field in music studies. The purpose of this study is to analyze the behavior of Sinhala comments on YouTube Sinhala song videos using social media comments as primary data sources. These included comments from 27 YouTube videos containing 20 different Sinhala songs, which were carefully selected so that strict linguistic reliability would be maintained and relevancy ensured. This process led to a total of 93,116 comments being gathered upon which the dataset was refined further by advanced filtering methods and transliteration mechanisms resulting into 63,471 Sinhala comments. Additionally, 964 stop-words specific for the Sinhala language were algorithmically derived out of which 182 matched exactly with English stop-words from NLTK corpus once translated. Also, comparisons were made between general domain corpora in Sinhala against the YouTube Comment Corpus in Sinhala confirming latter as good representation of general domain. The meticulously curated data set as well as the derived stop-words form important resources for future research in the fields of MIR and MER, since they could be used and demonstrate that there are possibilities with computational techniques to solve complex musical experiences across varied cultural traditions

Figures

Figures reproduced from arXiv: 2501.18633 by the authors.

Figure 1
Figure 1. Stages of the Data Preparation Process Word2Vec developed by Mikolov et al. [18], represents a breakthrough in neural word embedding models. It is trained using a straightforward feed-forward neural network, which has made it highly influential in the domain of NLP due to its simplicity and effectiveness. Word2Vec excels in constructing dense, low-dimensional vector representations of words, which has been crucial i… view at source ↗
Figure 2
Figure 2. Data Analysis Process [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Distribution of Comment Length distribution of linguistic elements, revealing nuanced pat￾terns within the dataset. Notably, an analysis of word pairs underscores the richness of linguistic expression encapsu￾lated within the Sinhala language, with an extensive col￾lection of unique word pairings comprising the discourse. This analysis revealed a vast array of 328,922 distinct word pairs, indicative of the diverse l… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Distribution of Z Value of Word Frequency [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Word Distribution among Corpora, where the set of words in each of the corpora are denoted as: [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Scatter Plot of Z-Scores legend for easy interpretation. Words appearing in multi￾ple sources are more interspersed, indicating that shared context across sources leads to more similar embeddings. This intermixing highlights the semantic consistency of such words acros…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SalAngaBhava: A Sinhala Market Dataset for Aspect-based Sentiment Analysis

    cs.CL 2026-07 conditional novelty 6.0 of 10

    SalAngaBhava is a new, publicly released Sinhala e-commerce review dataset with 1,858 reviews manually annotated at the aspect-sentiment quadruple level for ABSA research.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    Survey on publicly available sinhala nat- ural language processing tools and research,

    N. de Silva, “Survey on publicly available sinhala nat- ural language processing tools and research,” arXiv preprint arXiv:1906.02358, 2019

    arXiv 1906

  2. [2]

    Implementing a corpus for sinhala lan- guage,

    D. Upeksha, C. Wijayarathna, M. Siriwardena, L. Lasandun, C. Wimalasuriya, N. De Silva, and G. Dias, “Implementing a corpus for sinhala lan- guage,” in Symposium on Language Technology for South Asia, vol. 2015, 2015, p. 3

    work page 2015

  3. [3]

    Sinhala text classification: observations from the perspective of a resource poor language,

    N. de Silva, “Sinhala text classification: observations from the perspective of a resource poor language,” ResearchGate, 2015

    work page 2015

  4. [4]

    The flores evaluation datasets for low-resource ma- chine translation: Nepali-english and sinhala-english,

    F. Guzmán, P.-J. Chen, M. Ott, J. Pino, G. Lam- ple, P. Koehn, V. Chaudhary, and M. Ranzato, “The flores evaluation datasets for low-resource ma- chine translation: Nepali-english and sinhala-english,” arXiv preprint arXiv:1902.01382 , 2019

    arXiv 1902

  5. [5]

    Sinhala language corpora and stopwords from a decade of sri lankan facebook,

    Y. Wijeratne and N. de Silva, “Sinhala language corpora and stopwords from a decade of sri lankan facebook,” arXiv preprint arXiv:2007.07884 , 2020

    arXiv 2007

  6. [6]

    Bertifying sinhala–a comprehensive analysis of pre-trained language models for sinhala text classification,

    V. Dhananjaya, P. Demotte, S. Ranathunga, and S. Jayasena, “Bertifying sinhala–a comprehensive analysis of pre-trained language models for sinhala text classification,” arXiv preprint arXiv:2208.07864 , 2022

    arXiv 2022

  7. [7]

    NSINA: A News Corpus for Sinhala

    H. Hettiarachchi, D. Premasiri, L. Uyangodage, and T. Ranasinghe, “Nsina: A news corpus for sinhala,” arXiv preprint arXiv:2403.16571 , 2024

    work page Pith review arXiv 2024

  8. [8]

    Content-based music information retrieval: Current directions and future 10 challenges,

    M. A. Casey, R. Veltkamp, M. Goto, M. Leman, C. Rhodes, and M. Slaney, “Content-based music information retrieval: Current directions and future 10 challenges,” Proceedings of the IEEE , vol. 96, no. 4, pp. 668–696, 2008

    work page 2008

Show all 27 references
  1. [9]

    Mul- timodal music information processing and retrieval: Survey and future challenges,

    F. Simonetta, S. Ntalampiras, and F. A vanzini, “Mul- timodal music information processing and retrieval: Survey and future challenges,” in 2019 international workshop on multilayer music representation and pro- cessing (MMRP). IEEE, 2019, pp. 10–18

    2019

  2. [10]

    Music emo- tion recognition using convolutional long short term memory deep neural networks,

    S. Hizlisoy, S. Yildirim, and Z. Tufekci, “Music emo- tion recognition using convolutional long short term memory deep neural networks,” Engineering Science and Technology, an International Journal , vol. 24, no. 3, pp. 760–767, 2021

    2021

  3. [11]

    Music emotion recognition: A state of the art review,

    Y. E. Kim, E. M. Schmidt, R. Migneco, B. G. Morton, P. Richardson, J. Scott, J. A. Speck, and D. Turn- bull, “Music emotion recognition: A state of the art review,” in Proc. ismir, vol. 86, 2010, pp. 937–952

    2010

  4. [12]

    Machine recognition of music emotion: A review,

    Y.-H. Yang and H. H. Chen, “Machine recognition of music emotion: A review,” ACM Transactions on Intelligent Systems and Technology (TIST) , vol. 3, no. 3, pp. 1–30, 2012

    2012

  5. [13]

    Investigating emotion with music: neu- roscientific approaches,

    S. Koelsch, “Investigating emotion with music: neu- roscientific approaches,” Annals of the New York Academy of Sciences , vol. 1060, no. 1, pp. 412–418, 2005

    2005

  6. [14]

    Emotion regulation through listening to music in everyday situations,

    M. V. Thoma, S. Ryf, C. Mohiyeddini, U. Ehlert, and U. M. Nater, “Emotion regulation through listening to music in everyday situations,” Cognition & emo- tion, vol. 26, no. 3, pp. 550–560, 2012

    2012

  7. [15]

    Approaches, applications, and challenges in physiological emotion recognition—a tutorial overview,

    Y. S. Can, B. Mahesh, and E. André, “Approaches, applications, and challenges in physiological emotion recognition—a tutorial overview,” Proceedings of the IEEE, 2023

    2023

  8. [16]

    Recurrent convolutional neural networks for text classification,

    S. Lai, L. Xu, K. Liu, and J. Zhao, “Recurrent convolutional neural networks for text classification,” in Proceedings of the AAAI conference on artificial intelligence, vol. 29, no. 1, 2015

    2015

  9. [17]

    Refining word embeddings using intensity scores for senti- ment analysis,

    L.-C. Yu, J. Wang, K. R. Lai, and X. Zhang, “Refining word embeddings using intensity scores for senti- ment analysis,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 26, no. 3, pp. 671–681, 2017

    2017

  10. [18]

    Efficient estimation of word representations in vector space,

    T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781 , 2013

    2013 arXiv

  11. [19]

    Distributed representations of words and phrases and their compositionality,

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Advances in neural information processing systems , vol. 26, 2013

    2013

  12. [20]

    Evaluating large-language models for dimensional music emotion prediction from social media discourse,

    P. Donnelly and A. Beery, “Evaluating large-language models for dimensional music emotion prediction from social media discourse,” in Proceedings of the 5th International Conference on Natural Language and Speech Processing (ICNLSP 2022), 2022, pp. 242–250

    2022

  13. [21]

    A music video recommender system based on emotion classification on user comments,

    P. Sarakit, T. Theeramunkong et al. , “A music video recommender system based on emotion classification on user comments,” Doctoral dissertation, 2015

    2015

  14. [22]

    Transformer-based approach towards music emotion recognition from lyrics,

    Y. Agrawal, R. G. R. Shanker, and V. Alluri, “Transformer-based approach towards music emotion recognition from lyrics,” in European conference on information retrieval. Springer, 2021, pp. 167–175

    2021

  15. [23]

    Text-based sentiment analysis and music emotion recognition,

    E. Çano, “Text-based sentiment analysis and music emotion recognition,” arXiv preprint arXiv:1810.03031, 2018

    2018 arXiv

  16. [24]

    Word embedding evaluation for sinhala,

    D. Lakmal, S. Ranathunga, S. Peramuna, and I. Herath, “Word embedding evaluation for sinhala,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, 2020, pp. 1874–1881

    2020

  17. [25]

    A corpus and machine learning models for fake news classification in sinhala,

    V. Jayawickrama, A. Ranasinghe, D. C. Attanayake, and Y. Wijeratne, “A corpus and machine learning models for fake news classification in sinhala,” 2021

    2021

  18. [26]

    Sinmin-sinhala corpus project

    C. Wijayarathna and D. Upeksha, “Sinmin-sinhala corpus project. ”

  19. [27]

    Automatic creation of a sentence aligned sinhala- tamil parallel corpus,

    R. Pathirennehelage, N. Ihalapathirana, A. Mo- hamed, M. Ranathunga, S. Jayasena, S. Dias et al. , “Automatic creation of a sentence aligned sinhala- tamil parallel corpus,” in Proceedings of the 6th Work- shop on South and Southeast Asian Natural Language Processing (W Assanl...

    2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.