Pith. sign in

REVIEW 4 major objections 4 minor 18 references

A Breadth-First Catalog of Text Processing, Speech Processing and Multimodal Research in South Asian Languages

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read A broad LLM-assisted survey maps 2022–2024 South Asian text, speech, and multimodal research and ranks low-resource languages by activity.

desk verdict A useful, honest breadth-first survey of South Asian language work, but the headline counts rest on an unvalidated classifier and one clear citation error, so treat the numbers as approximate and send it to review with a request for reproducibility. read the letter →

arxiv 2501.00029 v1 pith:SATFN2H7 submitted 2024-12-20 cs.CL cs.IRcs.LG

classification cs.CLcs.IRcs.LG
keywords SouthAsianlanguageslow-resourceNLPspeechprocessingmultimodalmodelsLLM-basedliteraturesurveyNepalilanguageAssameseTelugu
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a broad survey of research published between January 2022 and October 2024 on text processing, speech processing, and multimodal models in South Asian languages. Its central claim is that the recent literature can be systematically counted and grouped into themes using an LLM-driven pipeline, and that doing so reveals where research effort is concentrated. The paper reports 369 relevant papers on language models, 84 on multimodal models, and 52 on speech processing, plus a spotlight on 21 low-resource languages in which Nepali, Assamese, and Telugu each have more than 50 papers since 2022. The value of the survey, if its counts are right, is a breadth-first map that helps new researchers see which tasks, languages, and benchmarks are active and which remain almost untouched.

What carries the argument

The load-bearing mechanism is the paper's multi-stage curation pipeline: Google Scholar queries generated from a base prompt with GPT-4o, metadata compiled through Publish or Perish; a hand-labeled sample of 20 titles used as in-context examples for GPT-4o relevance classification of all titles; irrelevant or pre-2022 papers excluded; the O1 model groups the relevant papers into themes, after BERTopic produced less coherent clusters; and for the 21-language spotlight, literal string matching is used instead of the classifier because the classifier's precision (0.33) and recall (0.7) were too low. This pipeline is what turns a large, noisy literature into the paper's tables, counts, and trend claims.

What would settle it

Manually label a random sample of the papers the pipeline classified as relevant and irrelevant (for example, 200 from the 369 'relevant' language-model papers and 200 from the excluded set), then compare the labels with the classifier's decisions; if precision or recall on that sample is far below what the survey's counts assume, the field-level counts and the spotlight orderings would need to be revised. A simpler check: rerun the same search queries for the 21 spotlight languages using only exact language-name strings and see whether the counts reproduce.

Watch

Extended reading notes

Core claim

The paper claims to provide a systematic, breadth-first catalog of recent South Asian language technology research, covering text-based language models, multimodal models, and speech processing separately, and then focusing on 21 low-resource languages that are often overshadowed in multilingual work. It identifies major thematic clusters—machine translation and cross-lingual transfer, hate speech and sentiment, bias and fairness, LLM adaptation, datasets and benchmarks, image captioning, multimodal machine translation and hate speech, code-mixed speech, automatic speech recognition, and low-resource speech—and reports paper counts for each field. The spotlight section asserts that Nepali (63 papers), Assamese (58), and Telugu (57) were the most studied low-resource languages, while Kannada shows the most advanced speech-related work, and that no papers were found for Dari, Rohingya, Kurukh, and Santali. The accompanying trends and challenges sections argue that the field is growing quickly but that publication venues are fragmented, benchmarks are under-collected in multilingual LLM evaluations, code-mixing comparisons across languages are missing, and LLM-based meta-analysis still needs better precision and coherence.

Load-bearing premise

The survey's counts and trends rest on the assumption that a GPT-4o relevance classifier trained on only 20 hand-labeled titles decides correctly which papers belong in the survey; the paper itself reports precision of 0.33 and recall of 0.7 for the spotlight classifier and gives no accuracy measures for the main field-level classifications.

Editorial extensions

If this is right

  • If the counts are right, the field's center of gravity is text-based language modeling: 369 relevant papers, versus 84 multimodal and 52 speech-processing papers.
  • Researchers looking for low-resource languages with enough literature to build on should start with Nepali, Assamese, and Telugu (more than 50 papers each), followed by Kannada (49), which the survey identifies as the most speech-advanced.
  • Dari, Rohingya, Kurukh, and Santali appear to have no dedicated papers in this window, signaling near-empty research space.
  • Aggregating the scattered South Asian benchmarks into a single hosted benchmark would help because, per the paper, many are left out of multilingual LLM evaluations.
  • LLM-based survey methods need better precision and recall and more coherent topics before their counts can be treated as stable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: because the spotlight counts rely on literal string matching and the paper found the classifier imprecise, the reported per-language counts are best read as lower bounds; work indexed under alternative names (the paper itself tried 'Rakhine' for Rohingya) could be missed.
  • My inference: the sharp gap between text papers (369) and speech or multimodal papers (84 and 52) may partly reflect how easily each type of work is discovered by title-based queries, so the true gap could be smaller than the counts suggest.
  • My inference: a direct replication using abstracts instead of titles, or using a larger hand-labeled sample, would test whether the relevance classifier's low precision changes the field-level trends; the paper's own limitation section invites this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents a breadth-first survey of NLP research in South Asian languages published between January 2022 and October 2024, covering text processing, multimodal models, and speech processing. The methodology uses Google Scholar searches, a GPT-4o-based relevance classifier trained on 20 hand-labeled titles, and an O1-based topic grouping, followed by a low-resource language spotlight on 21 languages using literal string matching. The paper reports overall relevant-paper counts (369 text, 84 multimodal, 52 speech) in Table 1 and per-language counts in Section 7 (e.g., Nepali 63, Assamese 58, Telugu 57), and identifies research themes, trends, challenges, and future directions.

Significance. If the quantitative findings were reliable, this survey would be a useful entry point for NLP researchers working on South Asian languages, particularly the low-resource spotlight that covers many under-served languages. The paper is transparent about its methodology and includes a Limitations section that acknowledges low precision/recall and topic cohesion issues, which is commendable. Its breadth-first perspective complements depth-first reviews such as Sankalp et al. (2024). However, the central counts and trends rest on an LLM relevance classifier whose accuracy is not reported for the main corpus, and the only reported metrics (precision 0.33, recall 0.7, in Section 7) are explicitly described as significantly low by the authors themselves, leading them to abandon the classifier for the spotlight section. The paper also contains a clear mis-citation in Section 5.2. These issues undermine confidence in the paper's main quantitative claims as they currently stand, although the survey's structure and curated bibliography retain value if the counts can be validated or appropriately caveated.

major comments (4)
  1. [Section 2 / Table 1 / Section 7] The main counts in Table 1 (369/84/52 relevant papers) and the theme-level trends in Sections 3–5 are produced by a GPT-4o relevance classifier, but no precision/recall or any other accuracy metric is reported for that classifier on the main corpus. The only numerical evaluation in the paper is in Section 7, where the spotlight relevance classifier is reported to have precision 0.33 and recall 0.7, a level the authors themselves call 'significantly low' and which motivated them to switch to literal string matching. Because the same classification approach underlies Table 1, the central quantitative claims are not established. The authors should provide per-category validation on a held-out labeled set (or at least a random sample) for the text, multimodal, and speech classifiers, or re-frame the counts as unverified with appropriate caveats and error bounds.
  2. [Section 5.2 / References] The paper cites Spiesberger et al. (2023) as an example of 'speech-to-speech translation for English → {Hindi, Telugu, Gujarati, Marathi, Punjabi}' in Section 5.2. The cited reference, 'Abusive speech detection in indic languages using acoustic features' (INTERSPEECH 2023), is a paper about abusive speech detection, not speech-to-speech translation. This is a load-bearing error because it inflates the speech-to-speech translation theme and indicates that the relevance/topic classification pipeline can mis-assign papers to incorrect themes. The citation should be corrected, and the other papers listed in that theme should be manually re-verified.
  3. [Section 2] The survey methodology is not reproducible. The Google Scholar queries, the 20 hand-labeled titles, and the per-paper relevance predictions from GPT-4o are not released, so the counts in Table 1 and Section 7 cannot be independently checked. The authors should provide these artifacts as supplementary material or a public repository. At a minimum, they should report the exact queries used for each language and category and the labeled examples used for in-context learning.
  4. [Section 7] The language-specific counts in Section 7 (e.g., 'Nepali, Assamese, and Telugu were the most popular languages, with more than 50 papers each since 2022') are based on literal string matching rather than the LLM classifier, which is an improvement, but the string-matching step is not validated against an independent search or manual review. Given that some language names (e.g., 'Burmese', 'Sindhi') can appear in non-linguistic or unrelated contexts, a manual verification of a sample, or a comparison with an alternative search strategy, is needed to support the 'most popular languages' claim with any confidence.
minor comments (4)
  1. [References] The reference list contains a duplicate entry for Majhi and Saha (2024): entries '2024a' and '2024b' have identical titles ('An automatic speech recognition system in odia language using attention mechanism and data augmentation') and appear to be the same paper. Please merge them and cite consistently, as both Section 6 and Section 7.7 cite these entries.
  2. [References / Section 4.3] The reference 'Singh et al., 2024b' (MIMIC: Misogyny identification in multimodal internet content) is listed twice in the reference list, once as 2024a and once as 2024b, with identical titles. Additionally, Section 4.3 cites this same work as both 'multimodal misogyny identification' and 'multimodal hate speech detection'; if the paper is specifically about misogyny, the hate-speech attribution should be corrected or clarified.
  3. [Section 7.17] The heading 'Chattisgarhi, Dhivehi, Sairaki, Rajasthani, and Balochi' contains two spelling errors: 'Chattisgarhi' should be 'Chhattisgarhi' and 'Sairaki' should be 'Saraiki', matching the spellings used in the abstract and elsewhere in the paper.
  4. [Section 7.7] There is a typo in Section 7.7: 'We alsio found work' should read 'We also found work'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey's claims are empirical aggregations of external literature, not derivations from its own assumptions.

full rationale

The paper is a literature survey. Its main quantitative outputs—the relevant-paper counts in Table 1 and the language-specific counts in Section 7—are summaries of papers found through Google Scholar searches and filtered by an LLM-based relevance classifier or, for Section 7, literal string matching. These are empirical claims about an external literature, not results derived from the paper's own definitions or fitted parameters. The classifier is trained on 20 hand-labeled titles, but the counts are not predictions of those labels; they are inferences about unseen papers, and the paper openly reports the spotlight classifier's low precision/recall (0.33 precision, 0.7 recall) and concedes in the Limitations that 'low precision/recall persist.' This is a validity and robustness concern, not circularity: no equation or construction forces the reported counts to equal the inputs. The paper's statements that 'Nepali, Assamese, and Telugu were the most popular languages' and the per-language counts in Section 7 are based on string matching, and while their accuracy is unvalidated, nothing in the paper defines those counts into existence. Self-citations are not load-bearing: the cited prior work by the author's organization or coauthors is not used to justify the survey's central classifications or counts. No step in the derivation chain reduces to its own input by construction, so the correct finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper's central claims (per-language and per-category paper counts, trend summaries) rest on the LLM classification pipeline. None of the pipeline components is validated with reported metrics for the main corpus, so the counts are load-bearing assumptions rather than measured facts.

assumptions (3)
  • domain assumption Google Scholar, queried with LLM-generated prompts, surfaces the relevant literature.
    Section 2 states the initial discovery relied solely on Google Scholar and Publish or Perish. Papers not indexed or not matched by the queries are silently excluded.
  • ad hoc to paper GPT-4o, using 20 hand-labeled titles as in-context examples, classifies relevance accurately enough for counting.
    Section 2: relevance prediction is based on only 20 examples; no precision/recall is reported for the main corpus, and the spotlight classifier had precision 0.33 (Section 7).
  • domain assumption O1-generated topic clusters are coherent enough to define the paper's themes.
    Section 2: O1 grouping is used to structure Sections 3-5, with only a qualitative comparison to BERTopic and no quantitative validation of cluster coherence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Breadth-First Catalog of Text Processing, Speech Processing and Multimodal Research in South Asian Languages." pith.science (2026). https://pith.science/paper/SATFN2H7

@misc{pith2026250100029,
  author       = {Pith},
  title        = {Pith review of: A Breadth-First Catalog of Text Processing, Speech Processing and Multimodal Research in South Asian Languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SATFN2H7}},
  note         = {Machine review of arXiv:2501.00029}
}
read the original abstract

We review the recent literature (January 2022- October 2024) in South Asian languages on text-based language processing, multimodal models, and speech processing, and provide a spotlight analysis focused on 21 low-resource South Asian languages, namely Saraiki, Assamese, Balochi, Bhojpuri, Bodo, Burmese, Chhattisgarhi, Dhivehi, Gujarati, Kannada, Kashmiri, Konkani, Khasi, Malayalam, Meitei, Nepali, Odia, Pashto, Rajasthani, Sindhi, and Telugu. We identify trends, challenges, and future research directions, using a step-wise approach that incorporates relevance classification and clustering based on large language models (LLMs). Our goal is to provide a breadth-first overview of the recent developments in South Asian language technologies to NLP researchers interested in working with South Asian languages.

Figures

Figures reproduced from arXiv: 2501.00029 by the authors.

Figure 1
Figure 1. Flowchart for the Research Paper Classifica [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 16 canonical work pages

  1. [2]

    In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1–7

    Dialect adaptation and data augmentation for low-resource asr: Taltech systems for the madasr 2023 challenge. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1–7. MB Alvi, NA Mahoto, MSA Reshan, and M Unar. 2023. Count me too: Sentiment analysis of roman sindhi script. SAGE . . .. HM Anagha, K Sairam, and J Mahesh. 2023....

  2. [4]

    Efficient assamese word recognition for so- cietal empowerment: A comparative feature-based analysis. IEEE . . .. R Budhathoki and S Timilsina. 2023. Image captioning in nepali using cnn and transformer decoder. Journal of Engineering and Sciences. SS Burramsetty and NP Gonugunta. 2022. Event ex- traction from telugu-english code mixed social media text. ...

  3. [5]

    A Comprehensive Study of the Current State-of-the-Art in Nepali Automatic Speech Recognition Systems

    Relevance of language-specific training on image caption synthesis for low resource assamese language. In 2023 International Conference on Asian Language Processing (IALP), pages 13–18. CoRover.ai. 2024. BharatGPT: Bridging the Multilin- gual Gap. https://www.corover.ai/. T Dalai, TK Mishra, and PK Sa. 2023. Part-of-speech tagging of odia language using s...

  4. [8]

    Pashto handwritten invariant character tra- jectory prediction using a customized deep learning technique. Sensors. Alvi Khan, Fida Kamal, Mohammad Abrar Chowdhury, Tasnim Ahmed, Md Tahmid Rahman Laskar, and Sabbir Ahmed. 2023. BanglaCHQ-summ: An ab- stractive summarization dataset for medical queries in Bangla conversational speech. In Proceedings of the...

  5. [9]

    Evaluating Telugu Proficiency in Large Language Models_ A Comparative Analysis of ChatGPT and Gemini

    Evaluating the diversity, equity, and inclu- sion of NLP technology: A case study for Indian languages. In Findings of the Association for Compu- tational Linguistics: EACL 2023, pages 1763–1777, Dubrovnik, Croatia. Association for Computational Linguistics. Vanlalmuansangi Khenglawt, Sahinur Rahman Laskar, Riyanka Manna, Partha Pakray, and Ajoy Kumar Kha...

  6. [10]

    Multilingual Tourist Assistance using ChatGPT: Comparing Capabilities in Hindi, Telugu, and Kannada

    Building a llama2-finetuned llm for odia lan- guage utilizing domain knowledge instruction set. . . . Conference on AI-ML . . .. S Kolar and R Kumar. 2023. Multilingual tourist assistance using chatgpt: Comparing capabilities in hindi, telugu, and kannada. arXiv preprint arXiv:2307.15376. N Koppula, J Pradeep Kumar, and K Srinivas Rao. 2022. Word sense di...

  7. [11]

    Cross-Linguistic Offensive Language Detection: BERT-Based Analysis of Bengali, Assamese, & Bodo Conversational Hateful Content from Social Media

    Data and knowledge-driven approaches for multilingual training to improve the performance of speech recognition systems of indian languages. MK Majhi and SK Saha. 2024a. An automatic speech recognition system in odia language using attention mechanism and data augmentation. International Journal of Speech Technology. MK Majhi and SK Saha. 2024b. An automa...

  8. [12]

    AxomiyaBERTa: A Phonologically-aware Transformer Model for Assamese

    Development of a dataset and a deep learn- ing baseline named entity recognizer for three low resource languages: Bhojpuri, maithili, and magahi. ACM Transactions on . . .. Rajesh Kumar Mundotiya, Swasti Mishra, and Anil Ku- mar Singh. 2022. Hierarchical self attention based sequential labelling model for bhojpuri, maithili and magahi languages. Journal o...

Show all 18 references
  1. [13]

    Data Engineering and

    Grapheme to phoneme conversion for malay- alam speech using encoder-decoder architecture. . . . Data Engineering and . . .. S Pudasaini, S Ghimire, P Ale, and A Shakya. 2024. Ap- plication of nepali large language models to improve sentiment analysis. Proceedings of the . . .....

  2. [15]

    Procedia Com- puter

    Phonogram-based automatic typo correction in malayalam social media comments. Procedia Com- puter . . .. F Rauf, R Irfan, L Mushtaq, and M Ashraf. 2022. Fake news detection in urdu using deep learning. VFAST Transactions on Software Engineering. P Ravva. 2023. Systems and reso...

  3. [16]

    Acoustics

    Visual speech recognition for kannada lan- guage using vgg16 convolutional neural network. Acoustics. F Rynjah, B Syiem, and LJ Singh. 2022. Investigating khasi speech recognition systems using a recurrent neural network-based language model. Int J Eng Trends Technol. Jayanta ...

  4. [17]

    FIRE (Working

    Sinhala and gujarati hate speech detection. FIRE (Working . . .. Arghyadeep Sen, Shantipriya Parida, Ketan Kotwal, Subhadarshi Panda, Ondˇrej Bojar, and Satya Ranjan Dash. 2022. Bengali visual genome: A multimodal dataset for machine translation and image captioning. In Intell...

  5. [18]

    Intelligent systems and

    Speech intelligibility quality in telugu speech patterns using a wavelet-based hybrid threshold trans- form method. Intelligent systems and . . .. Tushar Verma, Atul Shree, and Ashutosh Modi. 2023. Asr for low resource and multilingual noisy code- mixed speech. Interspeech. V ...

  6. [2020]

    ICT . . .. Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Ma- halakshmi J, Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan...

  7. [2021]

    Rest of World

    ViTA: Visual-linguistic translation by aligning object tags. In Proceedings of the 8th Workshop on Asian Translation (WAT2021) , pages 166–173, Online. Association for Computational Linguistics. Smita Gupta and Atreyo Banerjee. 2024. Ai, justice, and the ecosystem approach – n...

  8. [2022]

    Code-mixed Kannada . . .. PP Bansod. 2023. Hate speech detection in hindi. San Jose State University. F Barakzai, S Bhatti, and S Saddar. 2022. Sentiment analysis of sindhi news articles using deep learning. 2022 IEEE 17th International . . .. H Baruah, SR Singh, and P Sarmah....

  9. [2023]

    In 2023 14th International Conference on Computing Communica- tion and Networking Technologies (ICCCNT), pages 1–6

    Nlp based model to convert english speech to gujarati text for deaf and dumb people. In 2023 14th International Conference on Computing Communica- tion and Networking Technologies (ICCCNT), pages 1–6. A M Abirami, Wei Qi Leong, Hamsawardhini Rengara- jan, D Anitha, R Suganya, ...

  10. [2024]

    International Journal on Cyber- netics & Informatics (IJCI), 13(2)

    Direct punjabi to english speech translation using discrete units. International Journal on Cyber- netics & Informatics (IJCI), 13(2). R Kevat and S Degadwala. 2024. Developing gujarati article summarization utilizing improved page-rank system. International Journal of Scienti...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.