Pith. sign in

REVIEW 4 major objections 8 minor 1 cited by

Romanized to Native Malayalam Script Transliteration Using an Encoder-Decoder Framework

T0 review · 4 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read A Bi-LSTM encoder-decoder trained on 4.3 million romanized-Malayalam word pairs converts standard typing to native script at 7.4% character error, and degrades sharply to 22.7% when vowel signs are missing.

desk verdict A modest but honest shared-task system description; the architecture is textbook, the numbers are plausible, but the missing baseline makes the headline CER hard to interpret. read the letter →

arxiv 2412.09957 v1 pith:77Q5UJIO submitted 2024-12-13 cs.CL

classification cs.CL
keywords Malayalamtransliterationromanizedtonativescriptencoder-decoderBi-LSTMattentionmechanismcharactererrorratevowelomissionIndic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a character-level sequence-to-sequence model, an attention-based bidirectional LSTM encoder-decoder, can convert romanized Malayalam into native Malayalam script well enough for everyday standard typing. Trained on 4.3 million curated word pairs from two public Indic transliteration corpora, it reports a 7.4% character error rate on the general test set. The same model reaches only 22.7% character error on adhoc typing patterns in which vowel signs are mostly omitted, so the claim is carefully bounded to standard typing. A sympathetic reader should care because romanized input is how most Malayalam speakers type on phones and computers, while formal documents need native script; an automatic converter would remove a manual retyping burden. The paper's design choice is word-level transliteration with sentence reconstruction, and the authors state the model has no language model to exploit word context.

What carries the argument

The load-bearing mechanism is the attention-based bidirectional LSTM encoder-decoder. The encoder reads up to 57 romanized characters as 64-dimensional embeddings and processes them with a Bi-LSTM whose hidden states are concatenated into 256 dimensions and compressed to 128 per timestep; a context vector is built from these states. The decoder repeats that context vector for every output timestep, runs it through an LSTM, and an attention layer lets each decoding step focus on the relevant encoder positions. The concatenated LSTM and attention representations pass through a time-distributed dense layer to produce probabilities over 76 Malayalam characters, including vowels, vowel signs, consonants, anuswaram, visargam, virama, and chillu. The companion mechanism is the sentence pipeline: strip non-alphabetic characters, transliterate each word independently, then reinsert punctuation and numbers; this pipeline is what makes the reported CER a word-level claim rather than a sentence-level one.

What would settle it

Take the adhoc test sentences, restore the word boundaries that the preprocessing assumes, and feed the model each sentence with spaces inserted according to the ground truth. If CER stays near 22.7% even though the segmentation is now correct, the bottleneck is the missing-vowel character mapping; if CER drops sharply, the reported error is partly an artifact of the word-extraction pipeline. A second check is to rerun training after adding vowel-dropping paraphrases to the corpus and see whether Test Set-2 CER moves materially.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that an attention-enabled Bi-LSTM encoder-decoder, trained on a merged corpus of 4.344 million word-level transliteration pairs, learns the romanized-to-Malayalam character mapping well for standard typing patterns, achieving 7.4% CER on the general test set. On the adhoc set with frequent vowel omissions, CER rises to 22.7%, with WER at 66.9% and BLEU at 7.5%. The authors interpret the high CER on adhoc input as a consequence of training data that covers standard patterns but misses irregular vowel-dropping styles. They also argue that WER and BLEU misrepresent quality because a few character errors can make whole words count as wrong, and they identify phonetically similar Malayalam characters that share a romanized form as a recurring error source. The system is word-level: sentences are split into words, transliterated independently, and reassembled; the authors note this omits cross-word dependencies and limits sentence-level quality.

Load-bearing premise

The whole approach assumes word-by-word transliteration is enough, so sentence context never needs to influence how a word is converted; the paper itself notes the model has no language model to capture word dependencies.

Editorial extensions

If this is right

  • If the reported 7.4% CER is representative, a standard-pattern romanized Malayalam sentence will need roughly one character correction per fourteen characters typed, making the model usable as a real-time transliteration aid.
  • The 22.7% CER on vowel-dropping input means the model is not reliable for the informal SMS-style typing where vowels are skipped; users would see about one in four characters wrong.
  • Because WER on Test Set-1 is 34.5%, far above the CER, even good transliterations will show many whole-word mismatches under word-based scoring, so CER is the metric that reflects the character-level nature of the task.
  • Adding adhoc vowel-dropping patterns to the training data is the paper's stated next step and should directly reduce the Test Set-2 error if the architecture is sound.
  • The word-level design implies that punctuation and numbers are preserved exactly, but any ambiguity resolvable only through neighboring words is invisible to the model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A fair extension would be to rerun the same architecture on sentences rather than isolated words, or to add a language-model rescoring step; the paper's own limitation section suggests this could close much of the 22.7% gap, but that is an inference, not a demonstrated result.
  • Real-world romanized Malayalam often lacks consistent spaces and vowel signs at the same time; if so, the reported Test Set-2 CER may still overstate usability, because the word-extraction preprocessing assumes detectable word boundaries.
  • The error analysis hints that confusable character pairs sharing one romanized spelling are a main error source; a testable fix is to augment the training data with minimal pairs that force the model to use surrounding characters.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper describes a reverse transliteration system that converts romanized Malayalam into native Malayalam script using an attention-based Bi-LSTM encoder-decoder architecture. The model is trained on 4.3 million word-level transliteration pairs from the publicly available Dakshina and Aksharantar datasets. The authors evaluate on the IndoNLP-2025 shared task test sets, reporting a character error rate (CER) of 7.4% on general typing patterns (Test Set-1) and 22.7% on adhoc typing patterns with missing vowel indicators (Test Set-2), with correspondingly higher word error rates and lower BLEU scores. The paper discusses the limitations of the word-level pipeline and the absence of a language model, and it releases the training code under an MIT license.

Significance. The reported performance is plausible but currently uninterpretable without comparison to existing systems. The main strength is the reproducible setup: public training data, a shared-task test set, and released code. If the 7.4% CER holds up against baselines, the model would represent a functional word-level transliterator for standard typing patterns; however, the paper does not demonstrate that this outperforms or even matches the transformer-based IndicXlit, which is trained on the same corpus. The sentence-level reconstruction approach and the acknowledged absence of a language model limit the generality of the claims, especially for adhoc patterns. Overall, the work is a credible system description but does not yet establish a scientific contribution beyond its own reported numbers.

major comments (4)
  1. [Section 4, Table 2] The claimed performance is not situated relative to any baseline. The related work discusses IndicXlit (Madhani et al., 2023), an encoder-decoder transformer trained on Aksharantar, but no comparison is provided, nor is any rule-based or statistical baseline reported. Without a baseline, the 7.4% CER on Test Set-1 is uninterpretable; it could be strong or weak relative to the state of the art. The authors should at minimum evaluate IndicXlit on the same test sets and report pairwise comparisons, ideally with statistical significance tests across multiple runs or bootstrapped confidence intervals.
  2. [Section 3.2 and Section 4] The evaluation methodology is underspecified. The paper does not report the number of sentences in each test set, how CER/WER/BLEU are computed (e.g., tokenization at character/word level, which tools are used), or the variance of the metrics. No error bars or confidence intervals are given, and no multiple-seed training is described. In addition, key training hyperparameters such as batch size, number of epochs, optimizer, learning rate, regularization, and number of LSTM layers are omitted; the code link is not enough if the paper's central numbers are to be independently checked. These details are necessary to assess the reliability of the headline results.
  3. [Section 3 and Limitations] The sentence-level evaluation is based on a word-level model with naive reconstruction: sentences are split into words, transliterated individually, and reassembled. As the authors acknowledge in Limitations, the model lacks a language model, and word boundaries are unclear in adhoc patterns with missing vowels. The paper does not analyze how often this reconstruction fails or whether sentence context would resolve the observed errors. Since the shared task test sets are sentence-level, the reported CER/WER/BLEU reflect this pipeline, not just the transliteration model. This design choice should be characterized quantitatively, for example by comparing against a context-aware baseline.
  4. [Section 5 (Discussion)] The error analysis is incomplete and the interpretation of WER/BLEU is not supported. The sentence 'Specifically, the model frequently misclassifies character pairs such as' is cut off, so no actual error types are identified. The claim that the gap between CER (7.4%) and WER (34.5%) on Test Set-1 is due to WER/BLEU penalizing minor variations is not demonstrated; with an average of roughly 4–5 character errors per wrong word, the errors may be systematic (e.g., missing vowel signs or chillu/virama mishandling). The authors should provide a breakdown of error types and examples, and quantify how many errors are single-character versus multi-character.
minor comments (8)
  1. [Section 3.1] The section heading contains a typo: 'Datsets' should be 'Datasets'.
  2. [Section 2] The sentence 'Aksharamukha script converter is is another rule-based systems' contains a duplicated 'is' and a subject-verb agreement error; it should be 'Aksharamukha is another rule-based system'.
  3. [Table 1 caption] The word 'severly' in the caption should be 'severely'.
  4. [Section 2] The attribution of the Marian-based NMT transliteration system to Kunchukuttan et al. is unclear; please cite the specific reference and verify that the claim matches the cited work.
  5. [Section 3.2] The sentence 'The context vector extracts from this processed sequence' is grammatically awkward; rephrase to clarify what the context vector is and how it is computed.
  6. [Figures 1 and 2] The captions refer to distributions of evaluation metrics, but the figures are not included in the visible text; please ensure they have labeled axes and are properly referenced.
  7. [References] Some reference entries are incomplete (e.g., Baby et al. lacks venue details, James et al. lacks the arXiv identifier). Please complete all references.
  8. [Section 3.1] The phrase '4.100 million pairs' is ambiguous; use '4.1 million' or '4,100,000'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported transliteration results are empirical evaluations on held-out shared-task test sets, not derivations that reduce to their own inputs.

full rationale

The paper's central claim is an empirical performance report: an attention-based Bi-LSTM encoder-decoder was trained on 4.3 million publicly available Dakshina and Aksharantar transliteration pairs and evaluated on the IndoNLP-2025 shared-task test sets, yielding 7.4% CER on Test Set-1 and 22.7% CER on Test Set-2. There is no derivation chain in which an output quantity is defined in terms of the claimed result, no fitted parameter is renamed as a prediction, and no held-out number is statistically forced by the training procedure. The model outputs are generated from learned parameters, not from the test-set metrics that are later reported. The only self-citation, Baiju et al. 2024, appears in related work as a general remark about mined name-pair datasets and is not load-bearing for the main results. Citations to Dakshina, Aksharantar, Manohar et al. 2022, and the shared task organizers provide independent external support for the data, character inventory, and evaluation protocol. The missing baseline comparison against IndicXlit, the truncated error analysis, and the lack of training details are substantive completeness or verifiability concerns, but they are not circularity: failing to compare against a strong baseline does not make the reported test-set numbers equivalent to the training input. The limitation that the model lacks a language model is an honest statement about generalization, not a circular step. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

This is an empirical ML paper with no formal axioms or invented entities. The free parameters are standard architecture hyperparameters and data-dependent dimensions, all chosen without reported tuning or sensitivity analysis. The key domain assumptions are that word-level character transduction works for sentence transliteration and that the public training corpora match the test conditions.

free parameters (5)
  • embedding_dim = 64
    Model hyperparameter chosen by the authors; no tuning or ablation is reported. It affects the representation capacity of the encoder.
  • LSTM_units = 256
    Hidden size of the Bi-LSTM encoder. The paper states this value but does not justify it or explore alternatives.
  • decoder_dense_dim = 128
    Dense layer size after the encoder, noted as 128-dimensional, chosen without reported experimentation.
  • max_input_seq_len = 57
    Maximum input sequence length identified from training data. This truncation threshold affects long words and is a data-dependent choice.
  • output_vocab_size = 76
    Number of target Malayalam characters, derived from the training data. This is a data-dependent dimension rather than a tuned parameter.
assumptions (3)
  • domain assumption Character-level sequence-to-sequence mapping is sufficient for Malayalam transliteration.
    The entire method treats transliteration as a word-level character transduction, explicitly ignoring sentence context (Section 3).
  • domain assumption Dakshina and Aksharantar corpora represent the test distributions.
    Training data are public datasets; the paper does not analyze distribution shift to the shared task test sets, especially the adhoc set with missing vowels.
  • domain assumption The shared task test sets are independent of the training data.
    No deduplication or overlap analysis is reported. If the test sets derive from the same data pools (e.g., Aksharantar), scores could be inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Romanized to Native Malayalam Script Transliteration Using an Encoder-Decoder Framework." pith.science (2026). https://pith.science/paper/77Q5UJIO

@misc{pith2026241209957,
  author       = {Pith},
  title        = {Pith review of: Romanized to Native Malayalam Script Transliteration Using an Encoder-Decoder Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/77Q5UJIO}},
  note         = {Machine review of arXiv:2412.09957}
}
read the original abstract

In this work, we present the development of a reverse transliteration model to convert romanized Malayalam to native script using an encoder-decoder framework built with attention-based bidirectional Long Short Term Memory (Bi-LSTM) architecture. To train the model, we have used curated and combined collection of 4.3 million transliteration pairs derived from publicly available Indic language translitertion datasets, Dakshina and Aksharantar. We evaluated the model on two different test dataset provided by IndoNLP-2025-Shared-Task that contain, (1) General typing patterns and (2) Adhoc typing patterns, respectively. On the Test Set-1, we obtained a character error rate (CER) of 7.4%. However upon Test Set-2, with adhoc typing patterns, where most vowel indicators are missing, our model gave a CER of 22.7%.

Figures

Figures reproduced from arXiv: 2412.09957 by the authors.

Figure 1
Figure 1. The distribution of WER, CER and BLEU over the Test Set-1. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The distribution of WER, CER and BLEU over the Test Set-2. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. IndoNLP 2025: Shared Task on Real-Time Reverse Transliteration for Romanized Indo-Aryan languages

    cs.CL 2025-01 conditional novelty 4.0 of 10

    A shared task benchmark for Romanized to native script transliteration across Sinhala, Hindi, and Malayalam, with the organizers' team winning Sinhala.

Reference graph

Works this paper leans on

20 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Arun Baby, Anju Leela Thomas, NL Nishanthi, TTS Consortium, et al. 2016. Resources for I ndian languages. In Proceedings of Text, Speech and Dialogue. CBBLR Workshop

  2. [2]

    Bajiyo Baiju, Kavya Manohar, Leena G Pillai, and Elizabeth Sherly. 2024. https://doi.org/10.1109/RAICS61201.2024.10690040 Malayalam to English Named Entity Transliteration using Attention based BiLSTM . In 2024 IEEE Recent Advances in Intelligent Computational Systems (RAICS), pages 1--6

  3. [3]

    Jesin James, Deepa P Gopinath, et al. 2024. Advocating character error rate for multilingual asr evaluation. arXiv preprint arXiv:2410.07400

  4. [4]

    Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, Andr \'e F. T. Martins, and Alexandra Birch. 2018. https://doi.org/10.18653/v1/P18-4020 M arian: Fast neural machine translation in C ++ . In Proceedings of ACL 2018, System Demonstration...

  5. [5]

    Christo Kirov, Cibu Johny, Anna Katanova, Alexander Gutkin, and Brian Roark. 2024. https://doi.org/10.1162/coli_a_00510 Context-aware Transliteration of Romanized South Asian Languages . Computational Linguistics, pages 1--60

  6. [6]

    Anoop Kunchukuttan, Siddharth Jain, and Rahul Kejriwal. 2021. https://doi.org/10.18653/v1/2021.eacl-main.303 A large-scale evaluation of neural machine transliteration for I ndic languages . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3469--3475, Online. Association for...

  7. [7]

    Anoop Kunchukuttan, Mitesh Khapra, Gurneet Singh, and Pushpak Bhattacharyya. 2018. https://doi.org/10.1162/tacl_a_00022 Leveraging orthographic similarity for multilingual neural transliteration . Transactions of the Association for Computational Linguistics, 6:303--316

  8. [8]

    Anoop Kunchukuttan, Ratish Puduppully, and Pushpak Bhattacharyya. 2015. https://doi.org/10.3115/v1/N15-3017 Brahmi-net: A transliteration and script conversion system for languages of the I ndian subcontinent . In Proceedings of the 2015 Conference of the North A merican Chapter of the Association for Computational Linguistics: Demonstrations , pages 81--...

Show all 20 references
  1. [9]

    Yash Madhani, Sushane Parthan, Priyanka Bedekar, Gokul Nc, Ruchi Khapra, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh Khapra. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.4 Aksharantar: Open I ndic-language transliteration datasets and models for the next billion us...

  2. [10]

    Sreeja Manghat, Sreeram Manghat, and Tanja Schultz. 2020. https://doi.org/10.21437/Interspeech.2020-1936 Malayalam-English Code-Switched: Grapheme to Phoneme System . In Proc. Interspeech 2020, pages 4133--4137

  3. [11]

    Kavya Manohar, A. R. Jayan, and Rajeev Rajan. 2022. https://doi.org/10.1109/ACCESS.2022.3204403 Mlphon: A multifunctional grapheme-phoneme conversion tool using finite state transducers . IEEE Access, 10:97555--97575

  4. [12]

    Alok Parlikar, Sunayana Sitaram, Andrew Wilkinson, and Alan W Black. 2016. The F estvox I ndic frontend for grapheme to phoneme conversion. In WILDRE: Workshop on I ndian Language Data-Resources and Evaluation

  5. [13]

    Priyamvada, D

    R. Priyamvada, D. Govind, Vijay Krishna Menon, B. Premjith, and K. P. Soman. 2022. Grapheme to phoneme conversion for malayalam speech using encoder-decoder architecture. In Intelligent Data Engineering and Analytics, pages 41--49, Singapore. Springer Nature Singapore

  6. [14]

    Vinodh Rajan. 2018. https://www.aksharamukha.com Aksharamukha script converter web application

  7. [15]

    Mielke, Cibu Johny, I s in Demir s ahin, and Keith Hall

    Brian Roark, Lawrence Wolf-Sonkin, Christo Kirov, Sabrina J. Mielke, Cibu Johny, I s in Demir s ahin, and Keith Hall. 2020. https://www.aclweb.org/anthology/2020.lrec-1.294 Processing South Asian languages written in the Latin script: the Dakshina dataset . In Proceedings of T...

  8. [16]

    Santhosh Thottingal. 2018. https://libindic.org/Transliteration Libindic soundex and transliteration module

  9. [17]

    Santhosh Thottingal. 2023. https://huggingface.co/datasets/santhosh/english-malayalam-names Malayalam-English Name Pair Dataset

  10. [18]

    ISO 15919:2001 Transliteration. 2001. https://www.iso.org/standard/28333.html Transliteration of Devanagari and related Indic scripts into Latin characters

  11. [19]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  12. [20]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.