Pith. sign in

REVIEW 2 cited by

L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languages

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2211.11418 v4 pith:CS4T64MJ submitted 2022-11-21 cs.CL cs.LG

Raviraj Joshi This is my paper · ORCID
classification cs.CLcs.LG
keywords hindimodelsbertlanguagesmarathimodelmonolingualdevanagari
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The monolingual Hindi BERT models currently available on the model hub do not perform better than the multi-lingual models on downstream tasks. We present L3Cube-HindBERT, a Hindi BERT model pre-trained on Hindi monolingual corpus. Further, since Indic languages, Hindi and Marathi share the Devanagari script, we train a single model for both languages. We release DevBERT, a Devanagari BERT model trained on both Marathi and Hindi monolingual datasets. We evaluate these models on downstream Hindi and Marathi text classification and named entity recognition tasks. The HindBERT and DevBERT-based models show significant improvements over multi-lingual MuRIL, IndicBERT, and XLM-R. Based on these observations we also release monolingual BERT models for other Indic languages Kannada, Telugu, Malayalam, Tamil, Gujarati, Assamese, Odia, Bengali, and Punjabi. These models are shared at https://huggingface.co/l3cube-pune .

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HomoEnsNER: Does Language Alignment Outperform Architectural Complexity in Gujarati Named Entity Recognition?

    cs.CL 2026-08 conditional novelty 5.0 of 10

    A majority-vote ensemble of five independently fine-tuned GujaratiBERT models achieved the best entity-level F1 (0.8442) on Gujarati Naamapadam NER, ahead of the single-model baseline (0.8347) and all heterogeneous al...

  2. SenWiCh: Sense-Annotation of Low-Resource Languages for WiC using Hybrid Methods

    cs.CL 2025-05 conditional novelty 5.0 of 10

    The authors release sense-annotated WSD/WiC datasets for ten low-resource languages and report that English-based zero-shot transfer often beats small in-language fine-tuning, while mixed training usually helps.

Pith tools