{"id":"ccdb2d27-e933-4039-a8d1-41e912e63d93","arxiv_id":"2501.18633","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A new dataset of 63,471 Sinhala YouTube comments on music videos and 964 derived stop-words is presented and compared against general Sinhala corpora.","lead":"This paper builds and analyzes a dataset of 63,471 Sinhala YouTube comments on Sinhala music videos, plus a list of 964 common Sinhala words. It aims to help future research on music emotion recognition and natural language processing for Sinhala, a language with few digital resources.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Representativeness claim does not follow from 36% type overlap; the paper's own frequency comparisons undercut it.","rationale":"The reader's weakest_assumption is the unvalidated transliteration step, which is a valid data-quality concern. However, the most load-bearing weakness in the central claim is the unsupported representativeness inference. The 36% unique-word type overlap is not evidence that the YouTube comment domain is representative of general Sinhala; it only shows partial lexical coverage. The paper's own Z-score analysis in Section IV-B and Figure 6 demonstrates substantial frequency discrepancies for common words, undermining the representativeness claim regardless of transliteration quality. This concern is independent and more fundamental than transliteration noise: even a perfectly transliterated dataset would not justify the conclusion. The appropriate resolution is a major revision that either removes the representativeness claim or provides proper token-level frequency comparison and a defensible statistical definition of representativeness. The dataset and stop-word list may remain useful resources, so a conditional accept pending revision is appropriate, matching the reader's verdict but for a different primary reason.","tokens_in":11046,"tokens_out":4322,"duration_ms":36436,"concrete_test":"Recompute the Section IV-B comparison at the token level: measure the share of YouTube comment word tokens that appear in the general-domain corpus, and compute Spearman correlation of log-transformed frequencies for the 20,015 overlapping words across the two corpora. If token coverage falls below 80% or the frequency correlation is weak (e.g., rho < 0.5), the claim that YouTube comments are a representative subset of general Sinhala does not hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section IV-B concludes that the YouTube comment domain 'serves as a representative subset of the general Sinhala language domain' solely from the observation that 36% of YouTube's 54,834 unique words also appear in the general-domain corpus. This is a logical non sequitur: type overlap is a coverage statistic, not a representativeness criterion. A sample can share one-third of the vocabulary yet differ systematically in usage; the paper's own Figure 6 shows words like 'වෙග්' and 'සුපිරි' are far more frequent in YouTube comments while 'ඇති' and 'සහ' dominate the general domain, directly contradicting representativeness. No token-level analysis or frequency correlation is provided. The reader's focus on unvalidated transliteration is a real data-quality issue, and noisy transliteration could inflate the overlap if English tokens become Sinhala-script nonce forms, but even with perfect transliteration the representativeness conclusion would not be established. The inference weakness itself is more fundamental and load-bearing than the transliteration concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a new dataset of 63,471 Sinhala YouTube comments collected from 27 videos of 20 Sinhala songs, obtained after a pipeline of filtering, language detection, and transliteration of Roman-script comments via the Google Transliterator API. The authors algorithmically derive a list of 964 frequent Sinhala words claimed as stop-words using a z-score threshold of 3.0, and compare the YouTube comment vocabulary with a general-domain Sinhala corpus built from Wikipedia, newspapers, and government documents. On the basis of a 36% type overlap between the YouTube and general-domain unique word sets, the paper concludes that the YouTube comment domain is a representative subset of the general Sinhala language domain. The paper also reports word and word-pair frequency statistics and discusses implications for MIR and MER.","tokens_in":11121,"tokens_out":3020,"duration_ms":26437,"significance":"If validated, the dataset and the derived stop-word list would be reusable resources for Sinhala NLP, a language with few public corpora. The manuscript's strengths include the new data collection, the public GitHub repository, the algorithmic derivation of the stop-word list, and the attempted comparison with external lists (NLTK and Lakmal et al.). However, the central representativeness claim is not established by the evidence provided, and the unvalidated transliteration step poses a data-quality risk. The dataset itself, with clearer documentation and validated preprocessing, could still be a useful contribution, but the current paper overstates its conclusions.","major_comments":[{"comment":"The conclusion that the YouTube comment domain 'serves as a representative subset of the general Sinhala language domain' is based solely on the observation that 36% of YouTube unique words also appear in the general-domain corpus. Type overlap is a coverage statistic, not a representativeness criterion. The paper's own frequency analysis in Figure 6 shows systematic divergence: words such as 'වෙග්' and 'සුපිරි' are far more frequent in YouTube comments, while 21 words including 'ඇති', 'සහ', and 'කර' are significantly more frequent in the general domain. A token-level frequency correlation, a divergence measure, or a matched-sample design would be needed to support the representativeness claim. As written, the conclusion does not follow from the presented evidence.","section":"Section IV-B, Figure 6"},{"comment":"The pipeline converts 28,043 Roman-script comments into Sinhala script using the Google Transliterator API, but the paper provides no manual validation or error-rate estimate for these transliterations. If English-only comments or mixed-script comments were transliterated into Sinhala-looking text with incorrect or nonce forms, all downstream frequency counts, stop-word statistics, and domain comparisons would be contaminated. The authors should report a human evaluation of a random sample of transliterated comments, or otherwise demonstrate that the transliterated text is linguistically valid Sinhala.","section":"Section III, transliteration step"},{"comment":"The z>3.0 threshold identifies 964 'frequently occurring words' that are then called stop-words. However, the paper's own Table II shows that the most frequent words include content words such as 'ලස්සනයි' (beautiful) and 'සුපිරි' (super), which are sentiment-bearing adjectives. Only 182 of the 964 words match English NLTK stop-words after translation, and only 53 match the Lakmal et al. Sinhala stop-word list. The paper does not demonstrate that the remaining words are function words or otherwise semantically neutral. A stop-word list that retains strong sentiment markers is problematic for the sentiment-analysis applications the paper motivates. The authors should evaluate the list against a function-word criterion or show the impact of removing these words on a downstream task.","section":"Section IV-A, stop-word derivation"},{"comment":"The dataset construction counts are ambiguous. The paper states that 35,428 comments with Sinhala characters were identified and 30,716 comments had a combination of non-English and English characters, totaling 66,144; it then reports a segregation into 83 comments containing non-English characters and 30,633 comments composed solely of English characters, but the final dataset of 63,471 is 35,428 + 28,043, leaving the fate of the 83 comments and the 2,590 unsuccessful transliterations unexplained. The paper should clarify which rows were excluded at each step and why, as this affects the reproducibility and interpretation of all subsequent statistics.","section":"Section III, numeric pipeline"}],"minor_comments":[{"comment":"The manuscript contains numerous typographical errors (e.g., 'efficient', 'feild', 'difficult', 'intial', 'contibution') and grammatical issues (e.g., the abstract's 'confirming latter as good representation of general domain'). A thorough proofread is needed.","section":"Throughout"},{"comment":"The caption for Figure 5 lists subfigures (a) through (g) but the final panel is labeled (h) 'All Combinations'; the text also refers to 'Figure 5' without a panel letter when discussing the combined plot. Please make the cross-references and panel labels consistent.","section":"Figures 5a-5h"},{"comment":"The description of the one-hot encoding and t-SNE visualization is not connected to the representativeness claim or to any quantitative result; either explain how the t-SNE plots support the analysis or remove this passage to avoid confusion.","section":"Section IV-B, t-SNE paragraph"},{"comment":"The table lists song titles and comment counts, but the paper does not describe the criteria used to select the 27 videos and 20 songs, or how the 'strict linguistic reliability' mentioned in the Introduction was operationalized. A short paragraph on selection criteria would improve reproducibility.","section":"Table I"},{"comment":"The sentence 'It was noted that a higher number of comments consist of two unique Sinhala words' is unclear: does 'two unique Sinhala words' mean comments with exactly two word tokens, or two distinct word types? Please clarify the intended statistic.","section":"Section IV, first paragraph"}],"recommendation":"major_revision","confidential_remarks":"The central representativeness claim is not supported by the evidence, and the transliteration validation gap is a genuine data-quality concern, but both are fixable within the manuscript's scope. The dataset itself appears to be a potentially useful resource for Sinhala NLP. The paper would be strengthened by reframing the contribution as a dataset and linguistic analysis rather than claiming representative subset status without token-level evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The dataset is real and new: 63,471 Sinhala comments from 27 YouTube videos on 20 songs, with a public GitHub link, and a 964-word frequency list. That is a genuine resource for Sinhala NLP and music emotion work, and the preprocessing arithmetic from 93,116 raw comments to the final count is internally consistent. The paper also gives useful transparency about its pipeline. Credit where it is due: this is the first Sinhala YouTube music-comment corpus I know of.\n\nThe soft spots are in the analysis, not the data collection. The central claim that the YouTube comment domain is \"a representative subset of the general Sinhala language domain\" is not supported by the 36% unique-word type overlap. Type overlap is a coverage statistic, not a representativeness criterion. The paper's own Figure 6 and the lists of words that are significantly more frequent in each domain directly undercut the claim: වෙග්, සුපිරි, ලස්සනයි dominate YouTube comments while ඇති, සහ, කර dominate the general domain. That is a systematic difference in usage, and it is exactly what representativeness should not look like.\n\nThe transliteration concern raised in the review is real but secondary. The Google Transliterator API converted 28,043 Roman-script comments into Sinhala script, and there is no manual validation or error-rate estimate. If English comments were transliterated into Sinhala-script nonce forms, the unique-word counts and downstream statistics are contaminated. This is fixable with a sample check.\n\nThe stop-word list is also mislabeled. The z>3 threshold captures frequent words, not function words; ලස්සනයි (beautiful) and සුපිරි (super) are content words carrying sentiment, and only 182 of 964 match English NLTK stop-words, 53 match the Lakmal Sinhala list. Calling the list \"stop-words\" is misleading. It is a domain-specific high-frequency word list, which can still be useful.\n\nThe general-domain corpus is small and ad hoc: 9 Wikipedia articles, 82 newspaper articles, plus some earlier datasets. That is not a solid baseline for a representativeness claim. The paper should either drop the claim or replace it with a token-level frequency correlation using a larger, balanced general corpus.\n\nOverall: the resource is worth having, but the analysis overreaches. A serious referee should ask for major revision: validate transliteration, re-label the stop-word list, and re-baseline the representativeness argument. With those changes this becomes a solid dataset paper. I would accept it for peer review, and I would bring it to a reading group as an example of how a good resource can be paired with an unsupported inference.","headline":"A genuine new Sinhala YouTube comment corpus, but the representativeness claim is unsupported and the transliteration step needs validation; worth peer review with major revisions.","tokens_in":11767,"tokens_out":3151,"would_cite":false,"duration_ms":28491,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that YouTube comments on Sinhala songs form a representative subset of general Sinhala, and it provides a 63,471-comment dataset and a 964-word stop-word list to support that claim.","keywords":["Sinhala NLP","YouTube comments","music emotion recognition","low-resource language","stop words","corpus creation","sentiment analysis","linguistic analysis"],"falsifier":"Manual inspection of a random sample of the 28,043 transliterated comments would settle it: if a large fraction of them are garbled or actually English words rendered in Sinhala script, the dataset and the 36% overlap claim would need to be re-evaluated. A cheap first check is whether the 182 'true stop words' still correspond to genuine Sinhala function words when back-transliterated.","tokens_in":10724,"feed_emoji":"🎵","tokens_out":4468,"duration_ms":35973,"temperature":0.7,"pith_summary":"This paper claims that comments on Sinhala music videos are not an isolated slang pool but a representative slice of the Sinhala language, and it backs that claim with a public dataset of 63,471 cleaned Sinhala comments drawn from 20 songs. It also derives 964 frequency-based stop words for this comment domain, 182 of which match English stop words after translation. If the claim holds, the dataset and stop-word list become reusable building blocks for sentiment analysis, music emotion recognition, and other Sinhala NLP tasks that currently lack resources.","feed_headline":"Sinhala YouTube comments share 36% of vocabulary with general Sinhala","feed_subtitle":"A new 63,471-comment corpus and 964 stop words give Sinhala music emotion research a reusable foundation.","key_machinery":"The load-bearing method is a Z-score filter applied to word frequencies: after computing the mean and standard deviation of comment-word frequencies, words with a standardized frequency above 3.0 are flagged as frequent, producing the 964-word stop-word list. Representativeness is argued through a one-hot presence vector for each word across the Wikipedia, newspaper, and government corpora, a Word2Vec embedding of the combined general-domain corpus, and a t-SNE visualization showing that words shared across sources cluster together. The 36% overlap statistic is the paper's main quantitative evidence that the comment domain reflects general Sinhala.","core_discovery":"The central discovery is a curated corpus: from 93,116 raw comments on 27 videos, the authors kept 63,471 Sinhala comments after removing English-only text, spam, and non-Sinhala characters, and after transliterating 28,043 Roman-script comments into Sinhala script. A frequency analysis found 54,834 unique Sinhala words and 328,922 distinct word pairs. The authors then built a general-domain Sinhala word set from Wikipedia articles, newspaper articles, and government documents, and found that 20,015 words (about 36% of the comment-domain unique words) appear in both domains. They interpret this overlap as evidence that the YouTube comment domain is a representative subset of the general Sinhala language domain.","pith_inferences":["Because the paper only uses a 36% overlap to claim representativeness, a stricter statistical test (for example, comparing frequency distributions of common words with a correlation measure) would be a natural next step; such a test is not performed in the paper.","The transliteration noise concern also applies to the stop-word list: if mis-transliterated English function words entered the list, then the 182-word match with English stop words may partly be an artifact of transliteration rather than true linguistic overlap.","A direct extension would be to run the same preprocessing on comments for other South Asian languages to see whether the 36% overlap figure is characteristic of code-mixed music comment sections or unique to Sinhala."],"forward_implications":["A ready-made 63,471-comment Sinhala corpus can serve as training or evaluation data for Sinhala sentiment analysis, especially for music-related emotion classification.","The 964 stop-word list gives Sinhala NLP pipelines a domain-specific filter, which should reduce noise in downstream frequency and embedding analyses.","The 36% lexical overlap suggests that models trained on this comment corpus are not locked into slang; they may transfer reasonably to other written Sinhala domains.","The identified high-frequency words ('ලස්සනයි', 'සුපිරි') and word pairs ('හරිම ලස්සනයි') offer a starting lexicon for detecting praise and emotional engagement in music discourse."],"supporting_citations":[{"why":"Supplies the Z-score stop-word derivation method and the prior Sinhala Facebook corpus that this approach extends.","marker":"[5]"},{"why":"Provides the survey and the freely available Sinhala corpora that form the general-domain comparison set.","marker":"[1]"},{"why":"Contributes a Sinhala news extract used as part of the general-domain corpus.","marker":"[3]"},{"why":"Offers the reference Sinhala stop-word list against which 53 of the 964 words were checked.","marker":"[24]"},{"why":"Provides a recent large Sinhala news corpus that frames the scale of available Sinhala resources.","marker":"[7]"}],"fun_headline_variants":["Sinhala YouTube comments share 36% of words with general Sinhala","63K Sinhala comments: a new corpus for music emotion AI","Mining Sinhala song comments: a 63K-dataset for emotion research","Sinhala song comments: 36% vocabulary overlap with general language"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole dataset assumes that the automatic transliteration of 28,043 Roman-script comments into Sinhala script produces linguistically valid Sinhala text; if those conversions are noisy or mistranslate English phrases into Sinhala-looking words, every frequency count and the representativeness comparison are contaminated.","fun_headline_variants_meta":{"raw":{"variants":["Sinhala YouTube comments share 36% of words with general Sinhala","63K Sinhala comments: a new corpus for music emotion AI","Mining Sinhala song comments: a 63K-dataset for emotion research","Sinhala song comments: 36% vocabulary overlap with general language"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000907,"raw_usage":{"total_tokens":3903,"prompt_tokens":951,"completion_tokens":2952,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":2868}},"tokens_in":567,"tokens_out":2952,"duration_ms":20310,"temperature":1.0,"reasoning_tokens":2868,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T10:26:58.184022+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Manual inspection of a random sample of the 28,043 transliterated comments would settle it: if a large fraction of them are garbled or actually English words rendered in Sinhala script, the dataset and the 36% overlap claim would need to be re-evaluated. A cheap first check is whether the 182 'true stop words' still correspond to genuine Sinhala function words when back-transliterated.","supporting_citations":[{"cited_title":"Sinhala text classification: observations from the perspective of a resource poor language,","cited_arxiv_id":null,"evidence_quote":"Contributes a Sinhala news extract used as part of the general-domain corpus."},{"cited_title":"Word embedding evaluation for sinhala,","cited_arxiv_id":null,"evidence_quote":"Offers the reference Sinhala stop-word list against which 53 of the 964 words were checked."},{"cited_title":"NSINA: A News Corpus for Sinhala","cited_arxiv_id":"2403.16571","evidence_quote":"Provides a recent large Sinhala news corpus that frames the scale of available Sinhala resources."}],"review_version":1}