Pith. sign in

REVIEW 4 cited by

FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.06004 v1 pith:QUGJNYC4 submitted 2024-06-10 cs.CV cs.AIcs.CL

Yebin Lee , Imseong Park , Myungjoo Kang This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords evaluationfleurimagecaptioningscorecaptionsmetricsreference
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Most existing image captioning evaluation metrics focus on assigning a single numerical score to a caption by comparing it with reference captions. However, these methods do not provide an explanation for the assigned score. Moreover, reference captions are expensive to acquire. In this paper, we propose FLEUR, an explainable reference-free metric to introduce explainability into image captioning evaluation metrics. By leveraging a large multimodal model, FLEUR can evaluate the caption against the image without the need for reference captions, and provide the explanation for the assigned score. We introduce score smoothing to align as closely as possible with human judgment and to be robust to user-defined grading criteria. FLEUR achieves high correlations with human judgment across various image captioning evaluation benchmarks and reaches state-of-the-art results on Flickr8k-CF, COMPOSITE, and Pascal-50S within the domain of reference-free evaluation metrics. Our source code and results are publicly available at: https://github.com/Yebin46/FLEUR.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SPECS: Specificity-Enhanced CLIP-Score for Long Image Caption Evaluation

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A new reference-free metric, SPECS, fine-tunes LongCLIP with a specificity objective and reaches LLM-level human correlation on long captions at a fraction of the computational cost.

  2. Evaluation of Multilingual Image Captioning: How far can we get with CLIP models?

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A fine-tuned multilingual CLIP model rates image captions in ten languages with human-judgment correlation as high as English-only models on English data, using machine-translated benchmarks and native multicultural tests.

  3. A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Reference-free caption quality is scored by the downstream vision-language accuracy of a caption-conditioned reconstructed image, via a new CTTD benchmark.

  4. RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation

    cs.CL 2025-06 conditional novelty 5.0 of 10

    RIVAL iteratively re-trains a reward model adversarially against the current translator and adds a BLEU-predicting head, improving in-domain WMT and subtitle translation over SFT baselines.

Pith tools