Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Are Language Models Agnostic to Linguistically Grounded Perturbations? A Case Study of Indic Languages

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pre-trained multilingual models are not agnostic to linguistically grounded perturbations: replacing a single character with a phonetically or orthographically similar one flips predictions across 12 Indic languages and three tasks…

desk verdict A useful first benchmark of linguistically grounded attacks for Indic languages, but the headline 'less susceptible' claim is confounded by candidate-pool size. read the letter →

arxiv 2412.10805 v1 pith:JUUYPV6L submitted 2024-12-14 cs.CL

Poulami Ghosh , Raj Dabre , Pushpak Bhattacharyya This is my paper · ORCID
classification cs.CL
keywords adversarialattackslinguisticperturbationsphonologicalsimilarityorthographicIndiclanguagesmultilinguallanguagemodelstextclassificationrobustnesscharacter-levelattack
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Pre-trained multilingual models are not agnostic to linguistically grounded perturbations. Swapping a single character for a phonologically similar sound or a visually similar glyph flips predictions in all 12 Indic languages, across three downstream tasks and four models. Random character substitutions of the same character type cause even larger accuracy drops, so linguistic attacks appear slightly less harmful than unconstrained ones—but they remain effective, and they are far harder for a human reader to detect. Human evaluators rate the linguistically perturbed sentences as close to the original in grammar and meaning, which is what makes these edits a realistic real-world risk.

What carries the argument

The carrying mechanism is a black-box attack pipeline. It first ranks words by a masking-based importance score, $I_{w_i}$, defined in Eq. (1) as the change in predicted-label probability when the word is replaced by the [MASK] token, following the method the paper adapts from its cited baseline. It then builds, for each important word, a candidate pool of single-character edits that are linguistically similar: phonologically similar characters (homorganic consonants, which share a place of articulation and differ only in voicing or aspiration; long/short vowel pairs; sibilants) or orthographically similar characters from a curated confusion set per script. Candidate edits are filtered to stay close to the original using LaBSE cosine similarity, chrF, BERTScore, and a phonetic similarity measure, with a threshold of 0.6. The random baseline substitutes a same-type character without any linguistic constraint, which gives it a larger candidate pool; the paper explicitly attributes the random attacks' higher success rate to this unconstrained search space.

What would settle it

Run the attack pipeline with random and linguistic candidate pools matched to the same size per word; if the accuracy drops become equal, the paper's conclusion that linguistic attacks cause less harm than random attacks is an artifact of search-space size rather than a property of linguistic structure.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the answer to its title question is no: language models are not agnostic to linguistically grounded perturbations. When an important word in a sentence gets one character replaced by a phonetically similar character (a homorganic consonant, a short/long vowel counterpart, or another sibilant) or by a visually similar character from the same script, the model's prediction changes with high confidence. On IndicSentiment with IndicBERTv2, phonological attacks cut accuracy from 0.937 to 0.276 and orthographic attacks to 0.462, while random substitution of the same character type cuts it to 0.194, and the paper summarizes the pattern as linguistic attacks successfully deceiving the model while causing less harm than non-linguistic random attacks. Because the human evaluation shows linguistic edits preserve grammaticality and semantic similarity far better than random edits, the paper concludes that these constrained, linguistically plausible perturbations are an effective and more realistic threat than unconstrained random noise.

Load-bearing premise

The measurement depends on the masking-based word importance score inherited from an English-focused attack method ranking the truly decision-relevant words correctly in all 12 languages, although it was never validated per language.

Editorial extensions

If this is right

  • Human judges cannot easily spot linguistically grounded edits, so these attacks are realistic routes into deployed systems rather than artificial worst cases.
  • OCR and ASR pipelines produce exactly these phonological and orthographic confusions, meaning everyday noisy input can flip a model's prediction without any adversarial intent.
  • Robustness differs sharply by language family: Dravidian languages survive better because their words are longer and morphologically richer, while low-resource Sino-Tibetan Bodo is the most fragile, so robustness claims need to be made per language.
  • Phonological attacks are more damaging than orthographic attacks in most settings, so defenses need to address specific confusion classes (homorganic consonants and vowel-length alternations) rather than treating all single-character edits alike.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the random-versus-linguistic comparison is not pool-matched; the paper's tables show random attacks drawing on more candidates per word than phonological attacks, so the lower harm of linguistic attacks could be a search-space effect rather than linguistic structure. An equal-candidate experiment would separate the two.
  • Editorial inference: the curated visual-similarity and phonetic-similarity confusion sets could be reused as a diagnostic suite or augmentation source for OCR/ASR robustness and spelling correction, not only as an attack generator.
  • Editorial inference: if the masking-based saliency ranking is unreliable in low-resource languages, the measured accuracy drops would misstate true vulnerability; a per-language check against human-identified keywords would settle whether the pipeline transfers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies whether multilingual pre-trained language models are agnostic to linguistically grounded character-level perturbations, focusing on phonological and orthographic attacks across 12 Indic languages, three NLU tasks (IndicSentiment, IndicXParaphrase, IndicXNLI), and four models (IndicBERTv2, MuRIL, XLM-R, mBERT). The authors curate cross-script character-confusion resources, build a black-box attack pipeline based on a TextFooler-style word-importance score, and compare linguistic attacks against a random character-substitution baseline with LaBSE, chrF, and BERTScore filters. The absolute finding is that linguistic perturbations cause large accuracy drops: for example, IndicBERTv2 on IndicSentiment falls from 0.937 to about 0.28 after phonological attacks, and 0.462 after orthographic attacks in Table 1. A human evaluation on Hindi and Bengali indicates that linguistic perturbations are judged closer to the original in grammaticality and similarity than random perturbations. The abstract and Section 6.1.3 further claim that PLMs exhibit a slightly lower susceptibility to linguistic attacks than to random attacks. That relative claim is the main point of contention in this report, because the two attack families are not compared under matched search-space conditions.

Significance. If the absolute vulnerability finding stands, the paper provides a useful first systematic map of multilingual PLM robustness to linguistically natural perturbations in Indic languages, and the released character-similarity tables across nine scripts are a practical resource for future robustness work. The human evaluation is a genuine strength and supports the subtlety of the crafted perturbations. The paper does not rely on machine-checked proofs or fitted derivations; its value is empirical breadth and hand-curated linguistic resources rather than formal guarantees. The relative claim about agnosticism, however, is currently under-identified because random and linguistic attacks differ in candidate-pool size and possibly in filtering constraints. This weakens the headline interpretation and requires either additional controlled experiments or a carefully qualified reformulation.

major comments (4)
  1. [§6.1.3, Table 9] The relative claim that PLMs are 'slightly lower susceptibility to linguistic attacks' is not identified because the random and linguistic attack families are compared with unconstrained candidate-pool sizes. The paper itself states in §6.1.3 that 'Due to unconstrained search space, higher number of perturbed candidates are generated in case of random attacks compared to linguistic attacks, contributing to their higher attack success.' Table 9 (IndicSentiment, IndicBERTv2, Hindi) reports average candidates per word of 3.75 for Random versus 2.725 for Phono and 1.075 for Ortho, with query counts 50.787 versus 47.856 and 36.864 respectively; the same ordering appears across languages, models, and tasks in Tables 9–20. If a model were fully agnostic to linguistic structure, a restricted candidate set would produce exactly the observed smaller accuracy drop. To answer the title's 'agnostic' question, the comparison should match candidate-pool sizes (e.g., subsample random candidates to the size of the linguistic pool for each word) or report attack success per candidate or per query. Without this control, the evidence supports the absolute claim that linguistic attacks flip predictions, but not the relative claim that models are less susceptible to them.
  2. [§5.2, Tables 1, 9–20] All results are averages over three random trials (§5.2), yet no error bars, standard deviations, or significance tests are reported for any after-attack accuracy. For example, §6.1.2 asserts that 'phonological perturbations cause more damage than orthographic perturbations' for IndicSentiment, and §6.1.4 ranks language families by robustness; these comparisons rely on differences that may be within seed noise, and the paper provides no evidence that the differences are stable across trials. Please report per-trial values or confidence intervals for at least the headline results in Table 1, and for the language-family and cross-model claims, or explicitly state that the observed differences were not tested for significance.
  3. [§3, Eq. (1)] The word-importance saliency in Eq. (1), which masks a word and compares output probabilities, is inherited from TextFooler and is assumed to transfer to 12 Indic languages across three tasks without per-language validation. The paper does not check whether the [MASK]-based probability difference reliably identifies words whose substitution changes predictions in low-resource languages such as Bodo, or in Dravidian languages with complex morphology and longer words. If the heuristic is unreliable, the attack may select suboptimal target words, so the reported accuracy drops could misrepresent the models' true vulnerability to linguistic perturbations. The authors should validate the importance ranking against an oracle or an alternative saliency method on a subset of languages spanning at least one Indo-Aryan, one Dravidian, and one Sino-Tibetan language.
  4. [§5.4] The LaBSE cosine threshold of 0.6 is a free parameter chosen 'based on experiments with various threshold values,' but no sensitivity analysis is reported. Because this threshold filters candidate words and thus directly controls candidate-pool sizes and attack success rates, the main quantitative results, including the after-attack accuracy in Table 1, may depend on this arbitrary choice. Please provide a sensitivity analysis over thresholds (e.g., 0.5, 0.6, 0.7) for at least one model/task, and clarify whether the same threshold is applied to random attacks; if it is not, the random-versus-linguistic comparison compounds another uncontrolled difference.
minor comments (5)
  1. [Table 1 and Appendix D] Several column headers are run together without separators (e.g., 'OriginalAccuracyAfter-AttackAccuracy% PerturbedWordsQueryNumber'), and the appendix tables use 'A vg.' with a stray space in 'A vg. No. of Candidates per word'; this makes the tables unnecessarily hard to read.
  2. [§6.2] The human evaluation reports average grammaticality and similarity scores from three annotators per language but provides no inter-annotator agreement measure (e.g., Cohen's kappa or per-annotator raw scores); please add this information or state that agreement was not computed.
  3. [Appendix E] There is a typo in the sentence 'Dravidian languages are most robust to even synonym-baed substitution at the word level' — 'synonym-baed' should be 'synonym-based'.
  4. [Table 15] In the mBERT or XLM-R appendix tables (e.g., Table 15, bd/Ortho row), the Phonetic Similarity column is listed as 0 even though phonetic similarity is presumably not applicable to orthographic attacks; use a dash to match the other orthographic rows.
  5. [Contributions, Section 1] The paper says 'We release linguistic resources,' but I did not find a URL or repository link in the manuscript; please include the link in the camera-ready version so the resources are actually accessible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports empirical attack measurements with hand-curated linguistic resources; no prediction reduces to a fitted parameter or self-citation chain.

full rationale

This paper does not contain a derivation chain that reduces to its inputs. The attack pipeline follows the external TextFooler/BERT-Attack word-importance heuristic (Eq. 1), which is an established method, not a parameter fitted to the paper's own outcomes. The phonological and orthographic perturbation mappings are manually curated linguistic resources that were reviewed by linguists, and the similarity threshold of 0.6 is a free evaluation choice, not a fitted constant used to produce the central claim. The main finding—that linguistic attacks successfully deceive models but cause less harm than random attacks—is an empirical measurement on public models and public benchmarks. The paper explicitly acknowledges the larger candidate pool for random attacks as a contributing factor, which is a possible confound for the comparison but not a circular step: no equation or fitted value makes the observed accuracy drops true by construction. Self-citations (e.g., IndoWordNet, IndicNLP, IndicX-TREME) provide resources and benchmarks but are not used to justify the central claim circularly. Therefore, no significant circularity is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a standard saliency-based attack pipeline and hand-curated linguistic resources; no new mathematical entities or fitted theoretical constants are introduced. The main free parameter is the similarity threshold (0.6), and the main domain assumptions concern the transfer of TextFooler's word-importance heuristic to low-resource Indic languages.

free parameters (1)
  • Similarity threshold = 0.6
    Selected in Section 5.4 based on experiments to balance attack success rate with imperceptibility; all reported attack results use this threshold.
assumptions (4)
  • domain assumption The word-importance score in Eq. (1) correctly identifies the most impactful words for model predictions in all 12 languages.
    Inherited from TextFooler (Jin et al., 2020a) and assumed to transfer to Indic languages without validation.
  • ad hoc to paper A LaBSE cosine similarity threshold of 0.6 preserves semantic equivalence across all tested languages and scripts.
    The threshold is tuned by the authors and may not generalize across languages.
  • domain assumption The curated orthographic and phonological character mappings are the appropriate formalization of real-world linguistic errors in Indic scripts.
    Mappings are manually curated and reviewed by linguists, but no evidence is given that they cover the full distribution of naturally occurring OCR or ASR errors.
  • ad hoc to paper The random substitution baseline is a fair non-linguistic counterpart.
    The random baseline replaces letters with random same-type letters, which is not the only or most standard non-linguistic attack.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Language Models Agnostic to Linguistically Grounded Perturbations? A Case Study of Indic Languages." pith.science (2026). https://pith.science/paper/JUUYPV6L

@misc{pith2026241210805,
  author       = {Pith},
  title        = {Pith review of: Are Language Models Agnostic to Linguistically Grounded Perturbations? A Case Study of Indic Languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JUUYPV6L}},
  note         = {Machine review of arXiv:2412.10805}
}
read the original abstract

Pre-trained language models (PLMs) are known to be susceptible to perturbations to the input text, but existing works do not explicitly focus on linguistically grounded attacks, which are subtle and more prevalent in nature. In this paper, we study whether PLMs are agnostic to linguistically grounded attacks or not. To this end, we offer the first study addressing this, investigating different Indic languages and various downstream tasks. Our findings reveal that although PLMs are susceptible to linguistic perturbations, when compared to non-linguistic attacks, PLMs exhibit a slightly lower susceptibility to linguistic attacks. This highlights that even constrained attacks are effective. Moreover, we investigate the implications of these outcomes across a range of languages, encompassing diverse language families and different scripts.

Figures

Figures reproduced from arXiv: 2412.10805 by the authors.

Figure 1
Figure 1. The substitution of b (ba) with orthographi￾cally similar v (va) in the target word bkAr (bekaar) causes the model to misclassify the text with high confi￾dence. This highlights the sensitivity of language mod￾els to subtle variations in the input text, where altering a single character can lead to a significant shift in the model’s output. The input text is cropped to fit within the image. These attacks involve int… view at source ↗
Figure 2
Figure 2. Similar characters across different scripts [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The figure highlights the impact of linguistic perturbations across different languages. The x-axis lists [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Examples of generated adversarial text across different languages [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 16 canonical work pages

  1. [1]

    Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998

    arXiv 2018

  2. [2]

    Pushpak Bhattacharyya. 2010. http://www.lrec-conf.org/proceedings/lrec2010/pdf/939_Paper.pdf I ndo W ord N et . In Proceedings of the Seventh International Conference on Language Resources and Evaluation ( LREC '10) , Valletta, Malta. European Language Resources Association (ELRA)

    work page 2010

  3. [3]

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

    arXiv 2021

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  5. [5]

    Amit H Choksi and Shital P Thakkar. 2012. Recognition of similar appearing gujarati characters using fuzzy-knn algorithm. International Journal of Computer Applications, 55(6)

    work page 2012

  6. [6]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

    doi:10.18653/v1/2020.acl-main.747 2020

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

  8. [8]

    Jignesh Dholakia, Archit Yajnik, and Atul Negi. 2007. Wavelet feature based confusion character sets for gujarati script. In International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007), volume 2, pages 366--370. IEEE

    work page 2007

Show all 46 references
  1. [9]

    Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2023 a . Towards leaving no indic language behind: Building monolingual corpora, benchmark and models for indic languages. In Proceedings of the 61st A...

    2023

  2. [10]

    Khapra, Anoop Kunchukuttan, and Pratyush Kumar

    Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.693 Towards leaving no I ndic language behind: Building monolingual corpora, benchmark and models f...

    2023 doi

  3. [11]

    Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751

    2017 arXiv

  4. [12]

    Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852

    2020 arXiv

  5. [13]

    Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50--56. IEEE

    2018

  6. [14]

    Siddhant Garg and Goutham Ramakrishnan. 2020. Bae: Bert-based adversarial examples for text classification. arXiv preprint arXiv:2004.01970

    2020 arXiv

  7. [15]

    Chuan Guo, Alexandre Sablayrolles, Herv \'e J \'e gou, and Douwe Kiela. 2021. Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733

    2021 arXiv

  8. [16]

    Mahesh Jangid and Sumit Srivastava. 2016. Similar handwritten devanagari character recognition by critical region estimation. In 2016 international conference on advances in computing, communications and informatics (ICACCI), pages 1936--1939. IEEE

    2016

  9. [17]

    Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020 a . Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018--8025

    2020

  10. [18]

    Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020 b . https://doi.org/10.1609/aaai.v34i05.6311 Is bert really robust? a strong baseline for natural language attack on text classification and entailment . Proceedings of the AAAI Conference on Artificial Intellige...

    2020 doi

  11. [19]

    Simran Khanuja, Diksha Bansal, Sarvesh Mehtani, Savya Khosla, Atreyee Dey, Balaji Gopalan, Dilip Kumar Margam, Pooja Aggarwal, Rajiv Teja Nagipogu, Shachi Dave, et al. 2021. Muril: Multilingual representations for indian languages. arXiv preprint arXiv:2103.10730

    2021 arXiv

  12. [20]

    Anoop Kunchukuttan. 2020. The IndicNLP Library . https://github.com/anoopkunchukuttan/indic_nlp_library/blob/master/docs/indicnlp.pdf

    2020

  13. [21]

    Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236

    2016 arXiv

  14. [22]

    Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial intelligence safety and security, pages 99--112. Chapman and Hall/CRC

    2018

  15. [23]

    Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271

    2018 arXiv

  16. [24]

    Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020 a . Bert-attack: Adversarial attack against bert using bert. arXiv preprint arXiv:2004.09984

    2020 arXiv

  17. [25]

    Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020 b . https://doi.org/10.18653/v1/2020.emnlp-main.500 BERT - ATTACK : Adversarial attack against BERT using BERT . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (E...

    2020 doi

  18. [26]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  19. [27]

    Ninareh Mehrabi, Ahmad Beirami, Fred Morstatter, and Aram Galstyan. 2022. Robust conversational agents against imperceptible toxicity triggers. arXiv preprint arXiv:2205.02392

    2022 arXiv

  20. [28]

    John X Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020. Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp. arXiv preprint arXiv:2005.05909

    2020 arXiv

  21. [29]

    Vishal A Naik and Apurva A Desai. 2017. Online handwritten gujarati character recognition using svm, mlp, and k-nn. In 2017 8th international conference on computing, communication and networking technologies (icccnt), pages 1--6. IEEE

    2017

  22. [30]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    2015 doi

  23. [31]

    Danish Pruthi, Bhuwan Dhingra, and Zachary C Lipton. 2019 a . Combating adversarial misspellings with robust word recognition. arXiv preprint arXiv:1905.11268

    2019 arXiv

  24. [32]

    Danish Pruthi, Bhuwan Dhingra, and Zachary C. Lipton. 2019 b . https://doi.org/10.18653/v1/P19-1561 Combating adversarial misspellings with robust word recognition . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5582--5591, F...

    2019 doi

  25. [33]

    Bishwajit Purkaystha, Tapos Datta, and Md Saiful Islam. 2017. Bengali handwritten character recognition using deep convolutional neural network. In 2017 20th International conference of computer and information technology (ICCIT), pages 1--5. IEEE

    2017

  26. [34]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  27. [35]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485--5551

    2020

  28. [36]

    Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085--1097

    2019

  29. [37]

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (volume 1: long papers), pages 856--865

    2018

  30. [38]

    Olarik Surinta, Mahir F Karaaba, Lambert RB Schomaker, and Marco A Wiering. 2015. Recognition of handwritten characters using local gradient feature descriptors. Engineering Applications of Artificial Intelligence, 45:405--414

    2015

  31. [39]

    Tetsushi Wakabayashi, Umapada Pal, Fumitaka Kimura, and Yasuji Miyake. 2009. F-ratio based weighted feature extraction for similar shape character recognition. In 2009 10th International Conference on Document Analysis and Recognition, pages 196--200. IEEE

    2009

  32. [40]

    Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. arXiv preprint arXiv:1908.07125

    2019 arXiv

  33. [41]

    Xiaosen Wang, Hao Jin, and Kun He. 2019. Natural language adversarial attack and defense in word level

    2019

  34. [42]

    Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2019. Word-level textual adversarial attacking as combinatorial optimization. arXiv preprint arXiv:1910.12196

    2019 arXiv

  35. [43]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675

    2019 arXiv

  36. [44]

    Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  37. [45]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.