Pith. sign in

REVIEW 4 major objections 5 minor 36 references

COCO-Urdu: A Large-Scale Urdu Image-Caption Dataset with Multimodal Quality Estimation

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new 59,000-image Urdu caption dataset, COCO-Urdu, is claimed to be the largest public release of its kind, with 319,000 captions filtered by a hybrid quality pipeline.

desk verdict A genuinely large Urdu caption resource, but the quality claims rest on circular metrics and no dataset link, so treat as promising but unproven. read the letter →

arxiv 2509.09014 v1 pith:K3HN47DE submitted 2025-09-10 cs.CV cs.CL

Umair Hassan This is my paper
classification cs.CVcs.CL
keywords UrduimagecaptioningmultimodalqualityestimationmachinetranslationCLIPCOMET-KiwiBERTScorelow-resourcelanguagesMSCOCO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

COCO-Urdu aims to close a gap: Urdu is spoken by over 250 million people but has little vision-language data. The paper builds a dataset by taking 59,000 MS COCO images, translating their English captions into Urdu with a zero-shot neural translation model, and then scoring every caption with three automatic signals—reference-free translation quality, semantic consistency via back-translation, and visual grounding through a CLIP image-text similarity. Low-scoring captions (roughly 1% of the total) are rephrased iteratively, first automatically and then manually for a small subset, producing final mean component scores of 0.76, 0.97, and 0.75. If the quality bar holds, researchers gain the largest publicly available Urdu caption dataset plus a reusable recipe for building comparable datasets for other low-resource languages.

What carries the argument

The load-bearing mechanism is the hybrid multimodal quality-estimation (QE) pipeline. It produces, for every translated caption, a weighted combination of three scores: COMET-Kiwi (a reference-free neural translation-quality estimator), BERTScore computed between the original English caption and a back-translation of the Urdu caption (semantic consistency), and a CLIP-based visual-grounding score that compares the image embedding to the back-translated English caption relative to the original caption, using a harmonic-mean formulation. A caption is accepted if the ensemble score passes a threshold (0.7); otherwise it is flagged for iterative refinement, first by an open-source language model

What would settle it

Recruit native Urdu speakers to rate a random sample of, say, 500 accepted captions on semantic fidelity and visual grounding. If human ratings correlate weakly with the hybrid score, or if a substantial fraction of captions scoring above 0.7 are judged inaccurate or ungrounded, the threshold premise collapses and the dataset's quality claim is falsified.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims to introduce the largest publicly documented Urdu image–caption dataset: 59,000 images from MS COCO with 319,000 Urdu captions, translated zero-shot into Urdu and passed through a hybrid multimodal quality-estimation (QE) pipeline. The pipeline combines COMET-Kiwi for reference-free translation quality (threshold 0.7), BERTScore with back-translation for semantic consistency (threshold 0.9), and a CLIP-based relative alignment score for visual grounding (threshold 0.7). Captions scoring below the hybrid threshold (about 3,572) were automatically rewritten by a large language model, with 200 more corrected by hand. The paper reports final mean component scor

Load-bearing premise

The whole quality argument depends on the premise that COMET-Kiwi, BERTScore-back-translation, and CLIP-based similarity over an English proxy are accurate measures of Urdu caption quality, even though all three scoring models are trained predominantly on English data and the pipeline treats back-translation as lossless.

Editorial extensions

If this is right

  • Urdu vision-language models gain a training and evaluation resource that is an order of magnitude larger than existing public options.
  • The reference-free QE framework can be lifted to build similar datasets for other low-resource languages from high-resource image-caption corpora.
  • The reported threshold behavior (a ~1% flagged subset) suggests that targeted refinement, not full re-translation, is enough to push aggregate quality scores up.
  • Researchers can benchmark Urdu captioning, retrieval, and multimodal reasoning against COCO-Urdu in a standardized way, using the released pipeline to audit quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's quality claims ultimately rest on the assumption that the three English-centric metrics are valid for Urdu; a direct human study correlating ratings of Urdu captions with the hybrid score would test this, and such a study is not yet present beyond 200 hand-fixed cases.
  • Using the same family of models for translation, back-translation, and reference generation could inflate consistency scores; comparing against multiple independent reference translators or human translations would assess this.
  • A natural follow-up is to apply the pipeline to other languages and to measure whether the 0.7/0.9/0.7 thresholds generalize or need recalibration per language.
  • The dataset's licensing and derivative status are carefully stated, but downstream users must still verify Flickr image terms for each image before commercial use.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces COCO-Urdu, a large-scale Urdu image-caption dataset derived from a stratified 50% subset of MS COCO, containing 59,000 images and 319,000 Urdu captions translated with SeamlessM4T v2. A hybrid multimodal quality estimation pipeline is proposed, combining COMET-Kiwi, BERTScore with back-translation, and a CLIP-based visual grounding score; captions below threshold are refined with Qwen 14B. The paper reports mean QE scores (0.76, 0.97, 0.75) and reference-based MT scores (BLEU/SacreBLEU/CHRF), comparing favorably with smaller Urdu datasets, and describes a fault-tolerant parallel translation pipeline. The authors claim this is the largest public Urdu captioning dataset and that the QE framework provides a scalable validation method.

Significance. The dataset is a potentially valuable resource: at 59K/319K it is substantially larger than prior Urdu captioning datasets, and the stratified sampling, versioned infrastructure, and release intention are concrete engineering contributions. If the quality-control method is valid, the hybrid QE framework would be a reusable recipe for low-resource caption construction. However, the central quality claim is not yet established because the validation metrics are largely the same signals used to select and refine captions, and the only human evaluation is restricted to 200 low-scoring examples. The reported reference-based scores use another MT model as reference rather than human translations. With independent human evaluation or a calibration study, the resource could be useful; at present, the evidence supports existence and scale, not the high-fidelity claim.

major comments (4)
  1. [Table 1; §3.3.1–3.3.4] Table 1 presents mean QE scores as evidence of quality ('indicating generally high semantic fidelity'). But the thresholds in the same table (0.70/0.90/0.70) are the acceptance criteria that filter and trigger refinement, so the reported means are post-selection statistics; a mean above threshold is guaranteed by construction for the accepted set. To validate the pipeline, the paper needs to report the unfiltered score distributions, the proportion rejected, and the score changes from refinement, and ideally calibrate thresholds against human judgments. Without this, the numbers in Table 1 cannot be read as independent evidence of fidelity.
  2. [§3.3.2 and §3.3.3] The BERTScore and CLIP components both depend on back-translating the Urdu caption into English with SeamlessM4T v2, the same model that produced the Urdu from the English source. This is a self-consistency check: a systematic error that is consistently reversible (e.g., consistently swapping two object names) can yield high round-trip BERTScore and high CLIP alignment with the back-translated proxy. The CLIPScore formula in §3.3.3 is computed on the back-translated English, not on Urdu, so it cannot detect Urdu-side semantic errors that survive round-trip translation. Using a different model for back-translation, or a multilingual visual–text model that accepts Urdu, would break the circularity; at minimum this limitation should be stated and tested.
  3. [§5.1, Table 3] The reference-based evaluation uses NLLB-3B to generate references because human references are unavailable. NLLB-3B is a machine translation system; the comparison then measures similarity to one MT model's output, not to human-authored Urdu. The BLEU values are also inconsistently formatted (0.53 versus 53). The claim that this 'ensures reliable automated scoring' is unsupported. The authors should evaluate on a held-out subset with human references (or at least report agreement with human judgments) and state the limitation.
  4. [§7, §9] Human evaluation is limited to 200 low-scoring captions; no human evaluation is reported for the large body of captions that pass the thresholds. Section 7 explicitly acknowledges that 'broader human validation will be necessary before deploying the dataset in high-stakes downstream tasks,' yet the Conclusion restates the unqualified claim of 'high semantic fidelity and cross-modal consistency.' The main quality claim needs to be scoped to what the evidence supports, or supplemented with a larger human-evaluation study (e.g., a random stratified sample with adequacy/fluency ratings).
minor comments (5)
  1. [Table 3 note] The note says 'UCID' instead of 'UICD' and 'less documented' should be 'less well documented.'
  2. [Figure 2] The figure promises 'representative examples' of captions before and after refinement, but no actual Urdu sentences are shown in the text. Include the examples in the caption or text for inspection.
  3. [Reproducibility] The paper says the dataset is publicly available but provides no dataset URL or code repository. Add links for reproducibility and verification of the size/quality claims.
  4. [References] Some references appear inaccurate or incomplete: [16] is titled 'Scaling laws do not scale for low-resource languages' but is attributed to Kaplan et al., and [19] contains a placeholder URL. Please verify and correct all citations.
  5. [§3.3.4] The hybrid weights (0.4/0.4/0.2) are described as 'empirically set' but no sensitivity analysis is provided. A brief robustness check (e.g., varying weights and reporting the effect on retention and downstream scores) would strengthen the contribution.

Circularity Check

2 steps flagged · score 5.0 of 10

QE scores both select and validate the captions, and the back-translation uses the same model in both directions, so the high-fidelity claim is substantially self-referential even though the dataset size and NLLB-based BLEU benchmarks retain independent content.

  1. fitted input called prediction [Section 3.3.1 and Table 1; reiterated in Section 5.2]
    "Empirically, we set a threshold of 0.7 to flag low-scoring captions for iterative refinement. Across the dataset, translations achieve a mean COMET-Kiwi score of 0.76, indicating generally high semantic fidelity under a conservative threshold (Table 1)."

    The same COMET-Kiwi score is used as the selection/refinement filter and then reported as evidence of quality. Low-scoring captions are removed or refined so that the surviving dataset is constructed to satisfy the thresholds; the Table 1 means (COMET-Kiwi 0.76 vs 0.70, BERTScore 0.97 vs 0.90, CLIP 0.75 vs 0.70, hybrid 0.84 vs 0.70) are therefore partly forced by the filter. Reporting these means as 'generally high semantic fidelity' is a self-validation, not an independent confirmation: the evaluation metric is the same metric used to build the dataset.

  2. self definitional [Section 3.3.2 and Section 3.3.3]
    "To further ensure semantic consistency, we perform back-translation of Urdu captions using SeamlessM4T v2 [9] and compute BERTScore [37]. ... To capture cross-modal consistency, we compute a CLIP-based visual grounding score [24], leveraging back-translated captions as English proxies."

    The Urdu captions being validated were produced by SeamlessM4T v2 (Section 3.2), and the BERTScore 'semantic consistency' measure compares the original English caption with an Urdu-to-English back-translation obtained from that same SeamlessM4T v2 model. The CLIP grounding score likewise uses that same back-translated English as the text side. Thus the two largest components of the hybrid score measure round-trip self-consistency of one model, not whether the Urdu is semantically faithful: systematic errors that the same model inverts symmetrically (e.g., consistently mistranslating an entity) can pass both gates. The reported mean BERTScore of 0.97 and CLIP score of 0.75 are therefore by construction relative to the same system that generated the Urdu, and cannot independently establish f

full rationale

COCO-Urdu's existence and size claim (59K images, 319K captions) is not circular; it is a concrete construction from MS COCO via SeamlessM4T v2, stratified sampling, and refinement. The circularity is concentrated in the quality/fidelity claim. Table 1's mean QE scores are produced by the same thresholds and the same models used to select and refine the captions, so 'high semantic fidelity' is partly an artifact of the filter rather than an independent measurement. The BERTScore component is a round-trip self-consistency test through the same SeamlessM4T v2 model that generated the Urdu, and the CLIP component uses that same self-generated back-translation as the English proxy; symmetric errors are invisible to both. Section 7 itself flags the limited human evaluation (200 captions) and states that broader human validation is necessary, reinforcing that the headline quality claim lacks an external anchor. The paper does not rely on self-citations or uniqueness theorems, and the NLLB-based BLEU/SacreBLEU/chrF results provide some cross-model, non-circular signal, which keeps the paper from being wholly circular. Still, the central assertion of 'high semantic fidelity and cross-modal consistency' rests in substantial part on the selection metric and on a same-model round trip, so the overall circularity score is 5: partial circularity, with independent content in the dataset resource itself and in the reference-based benchmarks.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The ledger shows that the paper's quality-control machinery is both the filter and the evidence: six hand-set constants (thresholds, weights, the 2.5 CLIPScore factor) plus five unvalidated domain assumptions about model behavior on Urdu. No invented entities. The main external groundings are the Sechidis stratification algorithm, the NLLB-3B reference model, and 200 human-corrected captions.

free parameters (6)
  • Hybrid ensemble weights (w_COMET, w_BERT, w_CLIP) = 0.4 / 0.4 / 0.2
    Set empirically in Section 3.3.4 with no ablation or validation; they directly define the hybrid score that flags low-quality captions.
  • COMET-Kiwi acceptance threshold = 0.70
    Section 3.3.1: 'we set a threshold of 0.7 to flag low-scoring captions'. The reported mean score of 0.76 sits just above the cutoff.
  • BERTScore acceptance threshold = 0.90
    Table 1. BERTScore runs high even for weak text, so this cutoff has weak discriminative power.
  • CLIP visual grounding threshold = 0.70
    Table 1. The reported mean of 0.75 is barely above the cutoff.
  • CLIPScore scale factor and harmonic-mean form = 2.5 and H(1, s_bt/max(s_orig, eps))
    Section 3.3.3: the 2.5 factor maps typical CLIP cosines (about 0.3) to about 0.75 and is capped by min(1, ); no derivation or calibration is given.
  • Final hybrid threshold for refinement = 0.70
    Section 4: captions below 0.7 (3,572 auto-refined plus 200 manual) were rewritten; this defines the reported 1% refinement subset.
assumptions (6)
  • domain assumption COMET-Kiwi, BERTScore, and CLIP scores are valid quality indicators when applied to Urdu captions
    Invoked throughout Section 3.3; none of the three models is validated on Urdu in the paper, and CLIP cannot process Urdu text directly (Section 3.3.3 uses back-translated English proxies).
  • domain assumption Back-translation through SeamlessM4T v2 is lossless enough that BERTScore between original English and back-translated English measures translation quality
    Section 3.3.2 uses the same model for forward and backward translation; symmetric or systematic errors are invisible to the check.
  • domain assumption NLLB-3B output can serve as near-human reference translations for BLEU, SacreBLEU, and chrF
    Section 5: 'Human reference translations are unavailable at this scale, so we generated reference translations using the NLLB-3B model'.
  • domain assumption Qwen 14B refinement preserved semantic content while improving fluency
    Section 4 asserts semantic preservation; only 200 captions were human-checked, and Table 2 shows a large surface-form shift (SacreBLEU 30.82 to 75.98).
  • domain assumption MS COCO English captions are visually grounded, so the source CLIP similarity is a valid baseline for relative scoring
    Section 3.3.3 normalizes by s_orig = cos(I, Torig), assuming the original English caption's image alignment is the correct reference point.
  • standard math The Sechidis et al. iterative stratification algorithm preserves COCO's multi-label distribution
    Section 3.1 cites [30] and shows Figure 1, but the paper reports no quantitative distribution statistics to verify the claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of COCO-Urdu: A Large-Scale Urdu Image-Caption Dataset with Multimodal Quality Estimation." pith.science (2026). https://pith.science/paper/K3HN47DE

@misc{pith2026250909014,
  author       = {Pith},
  title        = {Pith review of: COCO-Urdu: A Large-Scale Urdu Image-Caption Dataset with Multimodal Quality Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3HN47DE}},
  note         = {Machine review of arXiv:2509.09014}
}
read the original abstract

Urdu, spoken by over 250 million people, remains critically under-served in multimodal and vision-language research. The absence of large-scale, high-quality datasets has limited the development of Urdu-capable systems and reinforced biases in multilingual vision-language models trained primarily on high-resource languages. To address this gap, we present COCO-Urdu, a large-scale image-caption dataset derived from MS COCO, containing 59,000 images and 319,000 Urdu captions selected through stratified sampling to preserve the original distribution. Captions were translated using SeamlessM4T v2 and validated with a hybrid multimodal quality estimation framework that integrates COMET-Kiwi for translation quality, CLIP-based similarity for visual grounding, and BERTScore with back-translation for semantic consistency; low-scoring captions were iteratively refined using open-source large language models. We further benchmark COCO-Urdu on BLEU, SacreBLEU, and chrF, reporting consistently strong results. To the best of our knowledge, COCO-Urdu is the largest publicly available Urdu captioning dataset. By releasing both the dataset and the quality estimation pipeline, we aim to reduce language bias in multimodal research and establish a foundation for inclusive vision-language systems.

Figures

Figures reproduced from arXiv: 2509.09014 by the authors.

Figure 1
Figure 1. Comparison of class distributions before and after subset selection. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Representative examples of captions before and after iterative refinement, [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Schematic of the fault-tolerant, parallel COCO-Urdu translation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 1 canonical work pages

  1. [19]

    Aligning visual and textual concepts for multimodal learning

    Shizhe Liu, Haoyang Ma, and Daniel Hsu. Aligning visual and textual concepts for multimodal learning. InNeurIPS, 2017. URL https://papers.nips. cc/paper/2017/hash/xxx-Aligning-Visual-Textual.pdf

    2017

  2. [16]

    Scaling laws do not scale for low-resource languages.arXiv preprint arXiv:2303.01234, 2023

    Jared Kaplan, Sharan Narang, Mark Chen, Tom Henighan, et al. Scaling laws do not scale for low-resource languages.arXiv preprint arXiv:2303.01234, 2023

    arXiv 2023

  3. [1]

    Generative image captioning in urdu using deep learning

    Kamran Ahmad, Bilal Raza, and Zafar Aslam. Generative image captioning in urdu using deep learning. InICDAR, 2023

    2023

  4. [2]

    A transformer-based urdu image caption generator.Journal of Computational Linguistics, 2024

    Fatima Ahmed, Salman Latif, and Waleed Arif. A transformer-based urdu image caption generator.Journal of Computational Linguistics, 2024

    2024

  5. [3]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736, 2022

    2022

  6. [4]

    Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond.Transactions of the association for computational linguistics, 7:597–610, 2019

    Mikel Artetxe and Holger Schwenk. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond.Transactions of the association for computational linguistics, 7:597–610, 2019

    2019

  7. [5]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming 12 Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfen...

    arXiv doi:10.48550/arxiv.2309.16609 2023

  8. [6]

    Multilingual vision-and-language representation learning

    Emanuele Bugliarello, Edoardo Maria Ponti, and Desmond Elliott. Multilingual vision-and-language representation learning. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1795–1810, 2022. URL https://aclanthology.org/ 2022.acl-long.125

    2022

Show all 36 references
  1. [7]

    Palm: Scaling language modeling with pathways.arXiv preprint arXiv:2204.02311, 2022

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways.arXiv preprint arXiv:2204.02311, 2022. URL https: //arxiv.org/ab...

    2022 arXiv

  2. [9]

    Seamlessm4t: Massively multilingual & multimodal machine translation

    Marta R Costa-juss `a, Chau Tran, James Cross, Marianna ˇSo´oskov´a, Shruti Bhosale, Vishrav Chaudhary, Angela Fan, Francisco Guzm ´an, et al. Seamlessm4t: Massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596, 2023

    2023 arXiv

  3. [10]

    No language left behind: Scaling human-centered machine translation.arXiv preprint arXiv:2207.04672, 2022

    Angela Fan, Shruti Bhosale, Vishrav Chaudhary, Marta R Costa-juss`a, James Cross, Francisco Guzm´an, Chien-Sheng Hsu, Gretchen Krueger, Michael Ma, Evgeny Matusov, et al. No language left behind: Scaling human-centered machine translation.arXiv preprint arXiv:2207.04672, 2022

    2022 arXiv

  4. [11]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Lillian Lee, and Shuran Shen. Clipscore: A reference-free evaluation metric for image captioning. InEMNLP, 2021

    2021

  5. [12]

    Efficient urdu caption generation using attention based lstm

    Inaam Ilahi, Hafiz Muhammad Abdullah Zia, Muhammad Ahtazaz Ahsan, Rauf Tabassam, and Armaghan Ahmed. Efficient urdu caption generation using attention based lstm. https://arxiv.org/abs/2008.01663, 2020. 13

    2008 arXiv

  6. [13]

    Openclip: An open-source reimplementation of clip.https://github.com/mlfoundations/open_clip, 2021

    Gabriel Ilharco, Mitchell Wortsman, Nicholas Carlini, Ali Farhadi, Alhussein Fawzi, and Florian Tramer. Openclip: An open-source reimplementation of clip.https://github.com/mlfoundations/open_clip, 2021

    2021

  7. [14]

    Johnson and Taghi M

    Justin M. Johnson and Taghi M. Khoshgoftaar. Survey on deep learning with class imbalance.Journal of Big Data, 6:1–54, 2019. URL https: //api.semanticscholar.org/CorpusID:102354936

    2019

  8. [15]

    The state and fate of linguistic diversity and inclusion in the nlp world

    Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. The state and fate of linguistic diversity and inclusion in the nlp world. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. URL https://aclantholog...

    2020

  9. [17]

    Bilingual–visual consistency for multimodal neural machine translation

    Chen Li, Ming Zhao, and Lin Wang. Bilingual–visual consistency for multimodal neural machine translation. InNAACL, 2024

    2024

  10. [18]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InEuropean Conference on Computer Vision (ECCV), 2014. URLhttps://cocodataset.org/

    2014

  11. [20]

    Uicd: A new dataset and approach for urdu image captioning

    Rimsha Muzaffar, Syed Yasser Arafat, Junaid Rashid, Jungeun Kim, and Usman Naseem. Uicd: A new dataset and approach for urdu image captioning. PLOS ONE, 20(6):e0320701, 2025. doi: 10.1371/journal.pone.0320701. URL https://doi.org/10.1371/journal.pone.0320701

    2025 doi

  12. [21]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 2002. URLhttps://aclanthology.org/P02-1040

    2002

  13. [22]

    chrf: character n-gram f-score for automatic mt evaluation

    Maja Popovi´c. chrf: character n-gram f-score for automatic mt evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, 2015. URLhttps://aclanthology.org/W15-3049. 14

    2015

  14. [23]

    A call for clarity in reporting bleu scores

    Matt Post. A call for clarity in reporting bleu scores. InProceedings of the Third Conference on Machine Translation: Research Papers, 2018. URL https://aclanthology.org/W18-6319

    2018

  15. [24]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamila Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InProceedings of the...

    2021 arXiv

  16. [25]

    The impact of translating resource-rich datasets to low-resource languages.ACL Findings, 2021

    Krithika Ramesh, Amanpreet Singh, and Ankit Kumar. The impact of translating resource-rich datasets to low-resource languages.ACL Findings, 2021

    2021

  17. [26]

    Comet: A neural framework for mt evaluation

    Ricardo Rei, Ana Farinha, Alon Lavie, Jo ˜ao Almeida, and Andr ´e Martins. Comet: A neural framework for mt evaluation. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, 2020. URL https://aclanthology.org/2020. e...

    2020

  18. [27]

    A dataset for movie description

    Anna Rohrbach, Zhe Qiu, Ivan Titov, and Bernt Schiele. A dataset for movie description. InCVPR, 2015. URL https://doi.org/10.1109/CVPR. 2015.7298946

    2015

  19. [28]

    Beyond english-centric multilingual nlp.arXiv preprint arXiv:2004.13958, 2020

    Sebastian Ruder. Beyond english-centric multilingual nlp.arXiv preprint arXiv:2004.13958, 2020. URL https://arxiv.org/abs/2004. 13958

    2004 arXiv

  20. [29]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Robert Kaczmarczyk Beaumont, Radu Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Richard Muller, Bernardo Zaff, Ajay Katta, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. InNeurIPS Datasets and ...

    2022

  21. [30]

    On the stratification of multi-label data

    Konstantinos Sechidis, Grigorios Tsoumakas, and Ioannis Vlahavas. On the stratification of multi-label data. InProceedings of the 2011 European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part III, ECML PKDD’11, page 145–158, Berlin, Heidelberg...

    2011

  22. [31]

    Multimodal quality estimation for machine translation

    Lucia Specia, Loic Barrault, Desmond Elliott, Stella Frank, and Khalil Sima’an. Multimodal quality estimation for machine translation. InProceedings of the 2020 Conference on Machine Translation (WMT), 2020

    2020

  23. [32]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation langua...

    2023 arXiv

  24. [33]

    Y . Wu, M. Zhang, and R. Xu. Evaluation of multilingual image captioning: How far can we get with clip?Transactions of the ACL, 2024

    2024

  25. [34]

    Cliptrans: Transferring visual knowledge with pre-trained models for multimodal machine translation

    Jian Yang, Yichao Zhou, and Hao Xu. Cliptrans: Transferring visual knowledge with pre-trained models for multimodal machine translation. In ACL, 2023

    2023

  26. [35]

    When does clip generalize better than unimodal models?arXiv preprint arXiv:2205.15237, 2022

    Mert Yuksekgonul, Haohan Wang, Rishi Bommasani Varma, and Percy Liang. When does clip generalize better than unimodal models?arXiv preprint arXiv:2205.15237, 2022

    2022 arXiv

  27. [36]

    Mobileclip: Fast image-text models for on-device multimodal learning

    Yu Zeng, Xin Li, Kai Wu, and Wei Sun. Mobileclip: Fast image-text models for on-device multimodal learning. InCVPR, 2024

    2024

  28. [37]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. InInternational Conference on Learning Representations (ICLR), 2020. URL https: //arxiv.org/abs/1904.09675. Appendix: Dataset Licensing The COCO datas...

    2020 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.