Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation

Rohit Mishra; Sumit Singh; Uma Shanker Tiwary

REVIEW 4 major objections 5 minor 41 references

Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Appending top-10 retrieved Wikipedia paragraphs to Hindi NER inputs lifts XLM-R macro F1 from 49.6 to 71.5, with gains concentrated on short low-context sentences.

desk verdict Useful comparative numbers for Hindi NER with retrieval augmentation, but the central causal claim is underdetermined by a missing control and inconsistent reporting. read the letter →

arxiv 2507.16002 v1 pith:A4TLHUGY submitted 2025-07-21 cs.CL cs.AI

Sumit Singh , Rohit Mishra , Uma Shanker Tiwary This is my paper

classification cs.CLcs.AI

keywords namedentityrecognitionHindiNERretrievalaugmentationlow-contextentitiesWikipediaknowledgebaseXLM-RMuRILlargelanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that retrieval augmentation—appending the top-10 Wikipedia paragraphs matched to an input sentence—improves Hindi named entity recognition, especially for short low-context sentences where the model has little to work with. The evidence is a comparison across fine-tuned encoders (MuRIL and XLM-R), a fine-tuned generative model (Llama2-7B), and few-shot LLMs (Llama2-70B, Llama3-70B, GPT-3.5-turbo). The largest effect is on XLM-R, whose macro F1 rises from 0.4955 to 0.7150 on the Hindi MultiCoNER-1 test set; MuRIL and GPT-3.5-turbo improve modestly, while Llama models do not benefit. If the claim holds, low-resource NER pipelines can gain substantially from a Wikipedia retrieval step before fine-tuning, without changing the model architecture.

What carries the argument

The load-bearing mechanism is retrieval augmentation (RA), a pipeline that indexes a Hindi Wikipedia dump with a search engine, queries each input sentence, retrieves the top-10 documents, concatenates matching paragraphs after the sentence with a special separator token, and labels the augmented tokens with the extra class B-X. At test time the pipeline is iterative: first sentence retrieval only, then entity retrieval using labels predicted in the previous round, repeated until improvement saturates. The comparison between with-RA and without-RA conditions is the argument's experimental core, and the length-wise F1 tables show where the mechanism acts.

What would settle it

Fine-tune XLM-R under the paper's exact settings but replace the retrieved paragraphs with randomly sampled Hindi Wikipedia paragraphs of the same length; if macro F1 stays near 71.5 rather than falling back toward 49.6, the gain is from extra tokens rather than retrieval relevance.

Watch

Extended reading notes

Core claim

The paper's central discovery is that retrieval augmentation—concatenating the top-10 Wikipedia paragraphs matched to an input sentence, separated by the model's special token, and labelling every augmented word with the extra class B-X—improves Hindi NER for models that do not already encode enough Hindi world knowledge. On the Hindi MultiCoNER-1 test set, XLM-R's macro F1 rises from 49.55 to 71.50, MuRIL's from 69.08 to 70.40, and GPT-3.5-turbo's from 20 to 33; fine-tuned Llama2-7B stays at 37 in both conditions, and Llama2-70B and Llama3-70B do not adopt the retrieved context. Length-wise analysis shows the gains appear mainly for sentences shorter than 15 words, the low-context regime the paper targets. The authors interpret this as RA helping a model extract relevant feature vectors when its own knowledge of the example is weak.

Load-bearing premise

The paper assumes the F1 gain is caused by the relevance of the retrieved Wikipedia knowledge, because its without-RA baseline simply omits the appended text; if any added text of similar length produces the same gain, the retrieval-specific claim fails.

Editorial extensions

If this is right

If RA works as reported, adding a Wikipedia retrieval step to XLM-R-style encoders is a cheap way to gain about 22 macro-F1 points on low-context Hindi NER, with no architectural change beyond inserting retrieved text and an extra label.
MuRIL's small gain suggests that models already strong on the target language may not need RA; RA is most valuable when the model lacks the relevant knowledge.
For few-shot prompting, GPT-3.5-turbo's improvement from 20 to 33 with RA indicates that retrieval can help closed LLMs, while Llama2-70B and Llama3-70B need either longer context windows or cleaner output formats to benefit.
Fine-tuning Llama2-7B fixes its output formatting but does not reach encoder accuracy (37 versus 71.5 and 70.4), so for low-resource NER with a good dataset, fine-tuned encoders remain the better resource trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported comparison does not include a random-context control, so part of the RA gain could come from giving the model more tokens rather than from retrieval relevance; testing XLM-R with generic or cross-language Wikipedia paragraphs would isolate the mechanism.
If retrieval relevance is the true driver, the gain should concentrate on entities that appear rarely in pretraining; a targeted error analysis on long-tail entities would test this prediction directly.
For Llama models, the bottleneck may be context-window fit rather than inability to use knowledge; extracting only entity-relevant sentences rather than ten full paragraphs could make RA effective for them.
The iterative entity-retrieval loop is a natural source of error propagation; a version that filters low-confidence predicted entities before the next retrieval round could push the 71.5 ceiling higher.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth reading for its numbers, not for its causal claim. The reported gain for XLM-R on Hindi MultiCoNER-1, from 49.55 to 71.50 macro F1, is large and, if it reproduces, practically interesting. The combination of retrieval augmentation with MuRIL, fine-tuned Llama2-7B, and the few-shot comparison of four generative models on this dataset is genuinely new, and the length-wise analysis showing the biggest gains on short sentences is a useful diagnostic. The authors also deserve credit for reporting negative results: RA hurt Llama2-70B and Llama3-70B and did nothing for fine-tuned Llama2-7B, and they explicitly note that MuRIL without RA is already comparable to XLM-R with RA.

That said, the central comparison is confounded. The with-RA condition appends up to ten retrieved Wikipedia paragraphs and an extra B-X label class; the without-RA condition appends nothing. There is no random-context or generic-text control, so the XLM-R gain could come from the extra input length or the added label rather than from Wikipedia knowledge. The stress-test note is right that this is a correctness risk in the interpretation, not a quibble. The paper's own length-wise analysis makes the alternative explanation plausible, since the largest gains occur exactly where the augmented input changes the most.

The reporting also has soft spots. The final numbers for XLM-R and MuRIL come from different test-time retrieval protocols, with the MuRIL result using entities predicted by the without-RA model after test-set inspection. There are no error bars, no seeds repeated, and no code released. Internally, Table 7 shows XLM-R reaching 0.7151 at iteration two, while Table 8 reports 0.6701 with "previous best model predicted entities" — the paper doesn't reconcile these. And Table 6 omits the 86.23 state-of-the-art from the very paper they build on, which makes the XLM-R result look more competitive than it is.

My verdict: the empirical results are probably real in the narrow sense that these models produced these scores under this pipeline, but the paper's general claim that "RA significantly improves performance" is not supported by the experiment as run. The fix is straightforward but not cosmetic: add a matched random-context control, fix the evaluation protocol so it is not selected after peeking at test results, report variance over multiple seeds, release code, and reconcile the tables. This is a paper that deserves a serious referee — there is a cheap, useful technique here if the main result holds up under those controls — but it should not be accepted as-is. If I were the editor I would send it to review and tell the authors exactly what the referee will demand.

Referee Report

4 major / 5 minor

Summary. The paper presents a comparative empirical study of retrieval augmentation (RA) for Hindi named entity recognition on the MultiCoNER-1 dataset. The authors fine-tune XLM-R, MuRIL, and Llama-2-7B with and without RA, and evaluate few-shot prompting with Llama-2-70B, Llama-3-70B, and GPT-3.5-turbo with and without RA. The headline results are that macro F1 for XLM-R rises from 0.4955 to 0.7150, for MuRIL from 0.6908 to 0.7040, and for GPT-3.5-turbo from 20.00 to 33.00, while Llama models show no benefit. The paper claims that RA significantly improves performance, especially for low-context examples.

Significance. If the RA effect is genuine, the study would provide a practical comparison of retrieval augmentation across encoder and decoder model families for a low-resource language, with a large-scale evaluation on the Hindi MultiCoNER-1 test set and length-wise analysis. The authors also evaluate several model architectures and openly describe their hyperparameter search. However, the significance is conditional on resolving a missing control condition and inconsistencies in the reported retrieval protocols; without those fixes, the central causal claim is underdetermined.

major comments (4)

[Section 6.2, Tables 7 and 8] The headline RA scores for XLM-R and MuRIL are obtained under different retrieval protocols, which is not disclosed in the abstract or Table 6. The XLM-R value 0.7150 in Table 6 matches the iterative entity retrieval in Table 7 (0.7153 after the second iteration), whereas the MuRIL value 0.7040 in Table 6 matches Table 8's "Retrieval Augmentation with previous best model predicted entities" (0.7041), not Table 7, where iterative RA gives only 0.6895, below the 0.6908 no-RA baseline. The text confirms this post-hoc selection: "For MuRIL, the fourth iteration's result with RA is less than the result without RA. Hence, we used entities predicted by the model without RA." This inconsistent protocol selection undercuts the uniform claim that RA improves MuRIL and makes the two model comparisons non-comparable.
[Section 4.1.1 and Section 6.2] The without-RA condition appends no text, while the with-RA condition concatenates up to ten retrieved Wikipedia paragraphs and assigns every augmented token a new B-X label (as described in Section 4.1.1 and the classification layer in Section 4.1.3). Consequently, the observed F1 gains, such as XLM-R rising from 0.4955 to 0.7150 and MuRIL from 0.6908 to 0.7040, conflate the semantic relevance of retrieved knowledge with the mere presence of additional context tokens and an extra label class. A matched control that appends random or generic paragraphs of comparable length is required to attribute the improvement to retrieval rather than to added input length. Without such a control, the abstract's claim that "RA significantly improves performance" is not justified.
[Section 4.1.1] The iterative entity retrieval procedure is stopped based on test-set performance: the paper states "Iterations are performed based on improvement," and Section 6.2 reports test F1 after each iteration and notes that "improvement was saturated after the third iteration." Selecting the number of iterations from test-set F1 is a form of test-set model selection that can inflate the reported improvement. The authors should either pre-specify the number of iterations or choose the stopping rule on the validation set.
[Tables 6-10] All F1 scores are reported as point estimates from apparently single runs, with no standard deviations, confidence intervals, or significance tests. This is especially problematic for the MuRIL comparison (0.6908 vs 0.7040, a 1.3-point difference) and the abstract's use of "significantly." At minimum, multiple seeds and variance measures are needed to establish that the reported gains are not within run-to-run noise.

minor comments (5)

[Introduction and Section 6.2] The definition of "low-context" is inconsistent: the introduction defines low-context as examples of length less than 5, citing [10], while Section 6.2 and Figures 10-12 analyze examples with lengths less than 15. The term should be defined operationally once and used consistently.
[Contributions and Table 6] The reported GPT-3.5-turbo improvement is inconsistent: the contributions list states macro F1 increased from 0.17 to 0.31, while Table 6 reports 20.00 to 33.00. Please reconcile these numbers.
[Section 6.3 and Table 6] The generative models are evaluated on different test subsets: 20,000 examples for Llama-2-70B and Llama-3-70B and 1,000 examples for GPT-3.5-turbo. Cross-model statements such as "GPT3.5-turbo adopts RA" should explicitly account for these subset differences, rather than only noting them in Section 6.3.
[Section 4.1.3] The post-processing description explains label realignment for original sentence tokens but does not state whether predictions on the augmented tokens (labeled B-X) are excluded when computing test F1. This should be stated explicitly for evaluation transparency.
[Throughout] There are several typographical errors and inconsistent model names, including "lama3-70B" in the abstract, "finetunning" in Section 4.2, and Table 6 referring to "Llama-3-70B-chat-hf" while the text elsewhere says "Llama-3-70B-Instruct". Also, GLiNER is described as a generative model but is in fact a bidirectional transformer NER model; the description should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the RA comparison is an empirical measurement, not a derivation that reduces to its inputs.

full rationale

This is an empirical comparison, not a derivation. The central claim that retrieval-augmented models outperform their no-RA counterparts on Hindi MultiCoNER-1 is supported by F1 scores measured on a held-out test set under the reported augmentation protocol (Sec. 4.1.1, Tables 6-10). No equation in the paper is fitted to the reported F1 values, and no numeric parameter is estimated from the test labels and then renamed a prediction. The test-time iterative retrieval does feed the model's own previous predictions back into the retrieval step (Sec. 4.1.1, Fig. 4), but the final scores are empirical outcomes of that feedback loop, not equal to the predictions by construction; this is self-referential behavior, not circular reasoning. The paper cites the authors' own prior work [17,18] to justify model choice and [37] for the strict-F1 metric, but those citations are background pointers and are not the argument that establishes the RA gain; moreover the no-RA MuRIL baseline (0.6908) is independently reproduced in this paper, so the prior result is not load-bearing. The selection of the MuRIL RA configuration after observing that iterative retrieval saturated below the no-RA baseline (Sec. 6.2) is a test-set selection concern, but it is a validity threat, not a circular reduction of the conclusion to its inputs. The missing random-context control would test causal attribution, but absence of a control is a confound, not circularity. No specific circular step can be quoted.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central comparison rests on the relevance of Wikipedia retrieval and on several protocol choices (k=10, three iterations, which retrieval strategy to report) that affect the final F1. No new theoretical entities are introduced.

free parameters (4)

retrieval top-k = 10
Number of retrieved Wikipedia documents concatenated per example; set by hand in Section 4.1.1.
retrieval iterations = 3
Iterative entity retrieval stopped at third iteration because 'improvement was negligible' and saturated (Section 6.2); selected after observing test-set results.
max sequence lengths = 512 (encoders), 820 (Llama2-7B), 3100 (Llama2-70B), 7680 (Llama3-70B), 16385 (GPT3.5)
Context limits set per model to fit augmented text (Sections 4.1.3, 4.2.6, 4.3.5).
hyperparameters (learning rate, batch size, epochs, seed) = LR 5e-6, batch 64, 20 epochs, seed 10/5 for encoders; LR 5e-5, batch 8, 2 epochs for Llama2-7B
Chosen by validation performance in Sections 4.1.4 and 4.2.6.

assumptions (4)

domain assumption Retrieved Wikipedia paragraphs supply relevant context that helps disambiguate low-context Hindi entities.
Section 4.1.1 assumes retrieved context is informative; no control condition with random or generic context.
domain assumption The MultiCoNER-1 Hindi dataset labels are correct and its test set is a valid benchmark.
Section 3 uses the dataset as ground truth without additional validation.
domain assumption Pretrained models (MuRIL, XLM-R, Llama, GPT) are appropriate and their tokenizers handle Hindi adequately.
Sections 4.1.2 and 4.3.1 select models based on prior reports rather than new evidence.
ad hoc to paper Sentences with length under 15 tokens are the relevant 'low-context' regime.
The paper defines low context loosely in Section 3 and then analyzes lengths under 15 in Figures 10-12.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation." pith.science (2026). https://pith.science/paper/A4TLHUGY

@misc{pith2026250716002,
  author       = {Pith},
  title        = {Pith review of: Enhancing Hindi NER in Low Context: A Comparative study of Transformer-based models with vs. without Retrieval Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A4TLHUGY}},
  note         = {Machine review of arXiv:2507.16002}
}

read the original abstract

One major challenge in natural language processing is named entity recognition (NER), which identifies and categorises named entities in textual input. In order to improve NER, this study investigates a Hindi NER technique that makes use of Hindi-specific pretrained encoders (MuRIL and XLM-R) and Generative Models ( Llama-2-7B-chat-hf (Llama2-7B), Llama-2-70B-chat-hf (Llama2-70B), Llama-3-70B-Instruct (Llama3-70B) and GPT3.5-turbo), and augments the data with retrieved data from external relevant contexts, notably from Wikipedia. We have fine-tuned MuRIL, XLM-R and Llama2-7B with and without RA. However, Llama2-70B, lama3-70B and GPT3.5-turbo are utilised for few-shot NER generation. Our investigation shows that the mentioned language models (LMs) with Retrieval Augmentation (RA) outperform baseline methods that don't incorporate RA in most cases. The macro F1 scores for MuRIL and XLM-R are 0.69 and 0.495, respectively, without RA and increase to 0.70 and 0.71, respectively, in the presence of RA. Fine-tuned Llama2-7B outperforms Llama2-7B by a significant margin. On the other hand the generative models which are not fine-tuned also perform better with augmented data. GPT3.5-turbo adopted RA well; however, Llama2-70B and llama3-70B did not adopt RA with our retrieval context. The findings show that RA significantly improves performance, especially for low-context data. This study adds significant knowledge about how best to use data augmentation methods and pretrained models to enhance NER performance, particularly in languages with limited resources.

Figures

Figures reproduced from arXiv: 2507.16002 by the authors.

**Figure 1.** Demonstration of our Model for an example of test data. In implementing RA, we constructed a knowledge base (KB) using the Wiki dump and utilized similar contexts to augment the examples, as described in [11]. The process of RA is described in Section 4.1.1. Our contributions: 1. Fine-tuned Llama2-7B model for NER task using Hindi data with and without retrieval augmentation. Results indicate that Llama2-7B produce… view at source ↗

**Figure 2.** Length-wise distribution of the dataset. Length of examples 1 2 3 4 5 6 7 8 Number of examples 142 12470 26159 24721 20727 14807 9926 6326 Length of examples 9 10 11 12 13 14 15 16 Number of examples 4011 2863 2089 1752 1678 1427 1237 1209 Length of examples 17 18 19 20 21 22 23 24 Number of examples 1125 990 959 894 806 705 656 636 Length of examples 25 26 27 28 29 30 31 32 Number of examples 503 468 464 358 319 25… view at source ↗

**Figure 3.** Generalized Transformer-based NER Model Architecture 4.1.1 Retrieval Augmentation Retrieve augmented context allows the inclusion of external contextual information to assist in disambiguating intricate named entities. Getting in more information through augmentation leads to a greater range of data that is used for getting an accurate understanding of context. For given input “ िवकी बमर्ी सािहत्य ” augmented text … view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Figure shows an overview of the process of the training and testing phase. The training phase has labelled examples; therefore, training has been done with the augmented training data ( with sentence retrieval and title retrieval). The testing phase initially has examp…

**Figure 5.** Figure 5: Some words are split into sub-word tokens during tokenization with MuRIL tokenizer. Labels are aligned accordingly. 4.1.2 Selection of Language Model |(LMs) MuRIL and XLM-R both provide prominent results for the NER task for Multiconer-1 [17] and Hiner [24] dataset. Ba…

**Figure 6.** Figure 6: Architecture of LLM for NER prediction with and without RA 4.2.4 General Architecture The overall architecture of RAFT is shown in [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: General architecture of RAG-based few-shot prompting using the chain of thoughts strategy. 4.3.1 Selection of Large Language Models (LLMs) Large Language Models (LLMs) have demonstrated significant potential as highly capable AI assistants. They excel in complex reason…

**Figure 8.** Figure 8: The results of the best models utilizing RA, XLM-R and MuRIL, are displayed in the Confusion Matrix [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

**Figure 9.** Figure 9: Figure shows the comparison between results with and without RA for language models. 6 Results and Discussion 6.1 Results with Llama2-7B-4bit-fine-tune Comparison between llama2-7B-chat-hf and fine-tuned model Llama2-7B-4bit-fine-tune (with and without RA) After fine-t…

**Figure 12.** Figure 12: Comparing length-wise F1 scores with and without RA using GPT3.5 Comparison of Llama2-7B-4bit-fine-tune with other Fine-tuned Models (Muril and XLM-R) with RA Fine-tuned transformer-encoder-based models like Muril and XLM-R produce better results compared to fine-tu…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 37 canonical work pages

[1]

& De Meulder, F

Tjong Kim Sang, E. & De Meulder, F. Introduction to the CoNLL-2003 Shared Task: Language- Independent Named Entity Recognition. Proceedings Of The Seventh Conference On Natural Language Learning At HLT-NAACL 2003 . pp. 142-147 (2003), https://aclanthology.org/W03-0419

work page 2003
[2]

Opdahl, A

Al-Moslmi, T., Gallofré Ocaña, M., L. Opdahl, A. & Veres, C. Named Entity Extraction for Knowledge Graphs: A Literature Overview. IEEE Access. 8 pp. 32862-32881 (2020)

work page 2020
[3]

& Juola, P

Al-Smadi, M., Al-Dalabih, I., Jararweh, Y. & Juola, P. Leveraging Linked Open Data to Automatically Answer Arabic Questions. IEEE Access. 7 pp. 177122-177136 (2019)

work page 2019
[4]

Guo, J., Xu, G., Cheng, X. & Li, H. Named entity recognition in query. Proceedings Of The 32nd International ACM SIGIR Conference On Research And Development In Information Retrieval . pp. 267-274 (2009)

work page 2009
[5]

& Fiza, M

Chen, C., Zhang, W., Shakeri, A. & Fiza, M. The Exploration of Knowledge-Preserving Prompts for Document Summarisation. 2023 International Joint Conference On Neural Networks (IJCNN) . pp. 1-8 (2023)

work page 2023
[6]

& Rose, R

Reddy, A. & Rose, R. Integration of Statistical Models for Dictation of Document Translations in a Machine-Aided Human Translation Task. IEEE Transactions On Audio, Speech, And Language Process- ing. 18, 2015-2027 (2010) 16 A preprint - July 21, 2025 F1 score (With KB) F1 score (Without KB) Model Length- wise Examples CW PROD PER LOC CORP GRP Macro F1 CW ...

work page 2010
[7]

& Hartley, A

Babych, B. & Hartley, A. Improving machine translation quality with automatic named entity recogni- tion. Proceedings Of The 7th International EAMT Workshop On MT And Other Language Technology Tools, Improving MT Through Other Language Technology Tools, Resource And Tools For Building MT At EACL 2003 . (2003)

work page 2003
[8]

& Croft, W

Petkova, D. & Croft, W. Proximity-based document representation for named entity retrieval. Proceed- ings Of The Sixteenth ACM Conference On Conference On Information And Knowledge Management . pp. 731-740 (2007)

work page 2007

Show all 41 references

[9]

& Smith, D

Mollá, D., Van Zaanen, M. & Smith, D. Named entity recognition for question answering. Proceedings Of The Australasian Language Technology Workshop 2006 . pp. 51-58 (2006)

2006
[10]

& Malmasi, S

Meng, T., Fang, A., Rokhlenko, O. & Malmasi, S. GEMNET: Effective gated Gazetteer representations for recognizing complex entities in low-context input. Proceedings Of The 2021 Conference Of The North American Chapter Of The Association For Computational Linguistics: Human Lan...

2021
[11]

& Jiang, Y

Wang, X., Shen, Y., Cai, J., Wang, T., Wang, X., Xie, P., Huang, F., Lu, W., Zhuang, Y., Tu, K., Lu, W. & Jiang, Y. DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition. Proceedings Of The 16th International Workshop On Semantic ...

2022
[12]

& Liu, Q

Chen, B., Ma, J., Qi, J., Guo, W., Ling, Z. & Liu, Q. USTC-NELSLIP at SemEval-2022 Task 11: Gazetteer-Adapted Integration Network for Multilingual Complex Named Entity Recognition. Proceed- ings Of The 16th International Workshop On Semantic Evaluation (SemEval-2022) . pp. 161...

2022
[13]

& Rokhlenko, O

Malmasi, S., Fang, A., Fetahu, B., Kar, S. & Rokhlenko, O. Semeval-2022 task 11: Multilingual complex named entity recognition (multiconer). Proceedings Of The 16th International Workshop On Semantic Evaluation (SemEval-2022) . pp. 1412-1437 (2022)

2022
[14]

& Polosukhin, I

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L. & Polosukhin, I. Attention Is All You Need. (2017)

2017
[15]

& Others MuRIL: Multilingual representations for indian languages

Khanuja, S., Bansal, D., Mehtani, S., Khosla, S., Dey, A., Gopalan, B., Margam, D., Aggarwal, P., Nagipogu, R., Dave, S. & Others MuRIL: Multilingual representations for indian languages. ArXiv Preprint ArXiv:2103.10730. (2021)

2021 arXiv
[16]

& Stoyanov, V

Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L. & Stoyanov, V. Unsupervised Cross-lingual Representation Learning at Scale. (2020)

2020
[17]

& Tiwary, U

Singh, S., Jawale, P. & Tiwary, U. silpa_nlp at SemEval-2022 Tasks 11: Transformer based NER models for Hindi and Bangla languages. Proceedings Of The 16th International Workshop On Semantic Evaluation (SemEval-2022) . pp. 1536-1542 (2022,7), https://aclanthology.org/2022.seme...

2022
[18]

& Tiwary, U

Singh, S. & Tiwary, U. Silp_nlp at SemEval-2023 Task 2: Cross-lingual Knowledge Transfer for Mono- lingual Learning. Proceedings Of The 17th International Workshop On Semantic Evaluation (SemEval- 2023). pp. 1183-1189 (2023,7), https://aclanthology.org/2023.semeval-1.164

2023
[19]

Huang, Z., Xu, W. & Yu, K. Bidirectional LSTM-CRF Models for Sequence Tagging. (2015)

2015
[20]

& Morwal, S

Sharma, R. & Morwal, S. & Agarwal, B. Named entity recognition using neural language model and CRF for Hindi language. Computer Speech & Language . 74 pp. 101356 (2022)

2022
[21]

& Toutanova, K

Devlin, J., Chang, M., Lee, K. & Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (2019)

2019
[22]

& Malmasi, S

Fetahu, B., Fang, A., Rokhlenko, O. & Malmasi, S. Gazetteer Enhanced Named Entity Recognition for Code-Mixed Web Queries. Proceedings Of The 44th International ACM SIGIR Conference On Research And Development In Information Retrieval . pp. 1677-1681 (2021)

2021
[23]

& Rokhlenko, O

Malmasi, S., Fang, A., Fetahu, B., Kar, S. & Rokhlenko, O. SemEval-2022 Task 11: Multilingual Com- plex Named Entity Recognition (MultiCoNER). Proceedings Of The 16th International Workshop On Semantic Evaluation (SemEval-2022) . pp. 1412-1437 (2022,7), https://aclanthology.or...

2022
[24]

& Bhattacharyya, P

Murthy, R., Bhattacharjee, P., Sharnagat, R., Khatri, J., Kanojia, D. & Bhattacharyya, P. HiNER: A Large Hindi Named Entity Recognition Dataset. (2022) 19 A preprint - July 21, 2025

2022
[25]

& Kumar Sarda, A

He, J., Uppal, A., N, M., Vignesh, S., Kumar, D. & Kumar Sarda, A. Infrrd.ai at SemEval-2022 Task 11: A system for named entity recognition using data augmentation, transformer-based sequence labeling model, and EnsembleCRF. Proceedings Of The 16th International Workshop On Se...

2022
[26]

& Kumar, P

Kakwani, D., Kunchukuttan, A., Golla, S., N.C., G., Bhattacharyya, A., Khapra, M. & Kumar, P. IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages. Findings Of The Association For Computational Linguistics: EMNLP
[28]

& Yang, D

Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M. & Yang, D. Is ChatGPT a General-Purpose Natural Language Processing Task Solver?. Proceedings Of The 2023 Conference On Empirical Methods In Natural Language Processing. pp. 1339-1384 (2023,12), https://aclanthology.org/202...

2023
[29]

& Wang, H

Xie, T., Li, Q., Zhang, J., Zhang, Y., Liu, Z. & Wang, H. Empirical Study of Zero-Shot NER with ChatGPT. Proceedings Of The 2023 Conference On Empirical Methods In Natural Language Processing . pp. 7935-7956 (2023,12), https://aclanthology.org/2023.emnlp-main.493

2023
[30]

& Shu, K

Chen, C. & Shu, K. PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners. Proceedings Of The 17th Conference Of The European Chapter Of The Association For Computational Linguistics. pp. 562-574 (2023,5), https://aclanthology.org/2023.eacl-main.41

2023
[31]

& Johnson, M

Hu, J., Ruder, S., Siddhant, A., Neubig, G., Firat, O. & Johnson, M. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. International Conference On Machine Learning. pp. 4411-4421 (2020)

2020
[32]

& Dean, J

Wu, Y., Schuster, M., Chen, Z., Le, Q., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, Ł., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., Stevens, K., Kurian, G., Patil, N., Wang, W., Young, C., Smith, J...

2016 arXiv
[33]

& Richardson, J

Kudo, T. & Richardson, J. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. (2018)

2018
[34]

& Rush, A

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T., Gugger, S., Drame, M., Lhoest, Q. & Rush, A. HuggingFace’s Transformers: S...

2020
[35]

& Amodei, D

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin,...

2020
[36]

& Scialom, T

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., H...

2023
[37]

& Tiwary, U

Singh, S. & Tiwary, U. ACRF: Aggregated Conditional Random Field for Out of Vocab (OOV) Token Representation for Hindi NER. IEEE Access. 12 pp. 22707-22717 (2024)

2024
[38]

Build gen AI models with Together AI
[39]

& Zettlemoyer, L

Dettmers, T., Pagnoni, A., Holtzman, A. & Zettlemoyer, L. QLoRA: Eﬀicient Finetuning of Quantized LLMs. (2023), https://arxiv.org/abs/2305.14314

2023 arXiv
[40]

& Charnois, T

Zaratiana, U., Tomeh, N., Holat, P. & Charnois, T. GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer. (2023), https://arxiv.org/abs/2311.08526 20 A preprint - July 21, 2025

2023 arXiv
[41]

& Nguyen, T

Lai, V., Ngo, N., Pouran Ben Veyseh, A., Man, H., Dernoncourt, F., Bui, T. & Nguyen, T. Chat- GPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning. Findings Of The Association For Computational Linguistics: EMNLP 2023 . pp. 1...

2023
[2020]

4948-4961 (2020,11), https://aclanthology.org/2020.findings-emnlp.445

pp. 4948-4961 (2020,11), https://aclanthology.org/2020.findings-emnlp.445

2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

& De Meulder, F

Tjong Kim Sang, E. & De Meulder, F. Introduction to the CoNLL-2003 Shared Task: Language- Independent Named Entity Recognition. Proceedings Of The Seventh Conference On Natural Language Learning At HLT-NAACL 2003 . pp. 142-147 (2003), https://aclanthology.org/W03-0419

work page 2003

[2] [2]

Opdahl, A

Al-Moslmi, T., Gallofré Ocaña, M., L. Opdahl, A. & Veres, C. Named Entity Extraction for Knowledge Graphs: A Literature Overview. IEEE Access. 8 pp. 32862-32881 (2020)

work page 2020

[3] [3]

& Juola, P

Al-Smadi, M., Al-Dalabih, I., Jararweh, Y. & Juola, P. Leveraging Linked Open Data to Automatically Answer Arabic Questions. IEEE Access. 7 pp. 177122-177136 (2019)

work page 2019

[4] [4]

Guo, J., Xu, G., Cheng, X. & Li, H. Named entity recognition in query. Proceedings Of The 32nd International ACM SIGIR Conference On Research And Development In Information Retrieval . pp. 267-274 (2009)

work page 2009

[5] [5]

& Fiza, M

Chen, C., Zhang, W., Shakeri, A. & Fiza, M. The Exploration of Knowledge-Preserving Prompts for Document Summarisation. 2023 International Joint Conference On Neural Networks (IJCNN) . pp. 1-8 (2023)

work page 2023

[6] [6]

& Rose, R

Reddy, A. & Rose, R. Integration of Statistical Models for Dictation of Document Translations in a Machine-Aided Human Translation Task. IEEE Transactions On Audio, Speech, And Language Process- ing. 18, 2015-2027 (2010) 16 A preprint - July 21, 2025 F1 score (With KB) F1 score (Without KB) Model Length- wise Examples CW PROD PER LOC CORP GRP Macro F1 CW ...

work page 2010

[7] [7]

& Hartley, A

Babych, B. & Hartley, A. Improving machine translation quality with automatic named entity recogni- tion. Proceedings Of The 7th International EAMT Workshop On MT And Other Language Technology Tools, Improving MT Through Other Language Technology Tools, Resource And Tools For Building MT At EACL 2003 . (2003)

work page 2003

[8] [8]

& Croft, W

Petkova, D. & Croft, W. Proximity-based document representation for named entity retrieval. Proceed- ings Of The Sixteenth ACM Conference On Conference On Information And Knowledge Management . pp. 731-740 (2007)

work page 2007