Pith. sign in

REVIEW 3 major objections 4 minor 40 references

The Devil is in the EOS: Sequence Training for Detailed Image Captioning

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that penalizing the end-of-sequence token during sequence training is enough to make pretrained vision-language models generate longer, more detailed captions, and that this recovers latent capability rather than adding…

desk verdict EOS debiasing works in practice, but the paper does not actually test whether EOS is the critical token. read the letter →

arxiv 2507.20077 v2 pith:CRINC73Z submitted 2025-07-26 cs.CV cs.CL

Abdelrahman Mohamed , Yova Kementchedjhieva This is my paper · ORCID
classification cs.CVcs.CL
keywords EOSdebiasingsequencetrainingdetailedimagecaptioningvision-languagemodelshallucinationcaptionlengthunsupervisedadaptationend-of-sequencetoken
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a large part of the detail deficit in vision-language captioning is not missing knowledge but a training artifact: cross-entropy training on short captions biases the model toward predicting the end-of-sequence (EOS) token too early. It proposes an unsupervised remedy: during a short sequence-training phase, minimize the probability of the EOS token only, with no reward model and no detailed captions as supervision. Across three pretrained VLMs and three detailed-captioning benchmarks, this single negative signal lengthens captions, raises recall of image-relevant details, and improves retrieval, while coherence stays largely stable. The authors interpret the result as recovering capabilities already latent in the vision and language backbones, rather than instilling new ones. If right, the method offers a training-data-free first step for any task whose available examples are short and generic.

What carries the argument

The machinery is EOS debiasing as a sequence-training objective: from each model-generated caption ending in EOS, the only training signal is $\nabla L(\theta) = \nabla_\theta \log p_\theta(t_n)$, a REINFORCE-style gradient that lowers the probability of the EOS token. This suppresses an entire subspace of early-terminating sequences and reshapes the token-level probability landscape; the self-attention mechanism then redistributes probability mass toward sequences whose structure differs from early termination, so detail is integrated naturally rather than appended at the end.

What would settle it

One concrete check: take a VLM whose language backbone was not pretrained on long-form text (or whose EOS token in the VLM vocabulary is not the true sentence terminator, as in BLIP-2 OPT), apply EOS debiasing, and see whether recall and coherence rise with length; if the extra tokens are not image-grounded, the recovery-of-latent-capability explanation is wrong.

Watch

Extended reading notes

Core claim

The central discovery is that the EOS token is the handle that controls caption length: because every training caption ends in EOS, teacher-forced cross-entropy training overshoots the probability of EOS, and generation terminates prematurely. Sequence training with the gradient $\nabla L(\theta) = \nabla_\theta \log p_\theta(t_n)$ for the EOS token only, applied after standard finetuning on COCO, gradually suppresses that bias. The resulting models do not simply postpone termination; the extra detail is distributed throughout the caption and remains grounded in the image, as shown by CAPTURE gains, retrieval improvements, and training-progression plots where recall rises with length. The approach outperforms the inference-time alternative of simply blocking EOS generation, which yields incoherent concatenations of short captions.

Load-bearing premise

The load-bearing premise is that the base VLM already knows how to describe images in detail, so suppressing the probability of the EOS token will surface image-grounded detail instead of incoherent repetition; the paper also assumes, more quietly, that the token it penalizes is the model's true termination token.

Editorial extensions

If this is right

  • Any pretrained VLM can be pointed at detailed captioning without detailed data or a reward model; the paper demonstrates this on BLIP-2 with OPT, BLIP-2 with FlanT5, and PaliGemma by finetuning only the cross-modal bridge.
  • EOS debiasing beats the trivial inference-time fix of blocking EOS generation on CAPTURE and coherence for nearly every model and dataset, because gradual sequence training preserves distributional coherence.
  • The added length is informative, not padding: recall of reference objects rises with length during training, and text-to-image retrieval on Urban-1k improves sharply.
  • Hallucination grows with length, but stays in the same range as a much larger instruction-tuned model trained explicitly for detailed captioning.
  • As a direct corollary, the method transfers to any conditional task whose available training text is short and generic, provided the base model already has latent knowledge of the target output.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper recommends brief EOS debiasing before supervised finetuning; a testable extension is measuring how much less detailed caption data is needed after the debiasing stage.
  • The BLIP-2 OPT result, where the debiased 'EOS' is actually a newline token, implies practitioners should verify the identity of the termination token first; penalizing a non-termination token may be what causes the coherence drop.
  • A direct falsification of the latent-capability story would be to apply EOS debiasing to a model whose language backbone was never exposed to long-form text and show that longer output degrades into repetition.
  • The interaction between EOS debiasing and decoding is underexplored: the paper trains with contrastive decoding but generates with beam search plus repetition penalty, so the optimal inference recipe after debiasing remains an open question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes EOS debiasing, an unsupervised sequence-training method for detailed image captioning. For a VLM already finetuned on short captions (BLIP-2 OPT, BLIP-2 T5, PaliGemma on COCO), the authors continue training only the cross-modal bridge with a REINFORCE-style gradient that reduces the log-probability of the EOS token in generated sequences. They evaluate on FineCapEval, DCI, and DOCCI using CIDEr, CAPTURE, and GPT-4-based coherence, plus retrieval on Urban-1k and hallucination metrics CHAIRi and ALOHa. They report substantial length and CAPTURE gains over base models and over an inference-time EOS-blocking baseline, with increased hallucination as expected, and conclude that EOS debiasing recovers latent detailed-captioning ability without reward models or supervised detailed-caption data.

Significance. If the central claim holds, the result is significant: it offers an almost parameter-free, reward-free way to extract existing detailed-captioning ability from pretrained VLMs, closing much of the gap to a heavily instruction-tuned 3B model. The paper's strengths are its simplicity, the breadth of the evaluation (three models, three detailed-caption benchmarks, retrieval, and hallucination metrics), the qualitative evidence that added detail is integrated throughout captions rather than appended, and the explicit acknowledgment of limitations and hallucination trade-offs. The main weakness is that the paper's core causal claim, that the EOS token specifically is the devil, is not tested by any control that applies the same sequence-training recipe to a non-EOS token; without such a control, the mechanism remains unverified.

major comments (3)
  1. [§3.2, §4.4, §5.3] The central claim that EOS debiasing, rather than sequence training in general, is responsible for the gains is not supported by any control experiment. The only training comparison is against the base model and an inference-time EOS-blocking baseline ('triv.'), which shares no training component with the method and therefore does not test whether the negative signal must be applied to the EOS token. A control that applies the identical sequence-training recipe while penalizing a frequent non-EOS token (e.g., 'the' or a uniformly random token) would directly test the proposed mechanism. Without it, the observed improvements could equally be attributed to sequence training itself, to the contrastive decoding exploration described in §4.1, or to any token-level negative signal. The mechanistic account in §5.3, that the penalty suppresses 'an entire subspace of sequences that are likely to terminate early,' is a post-hoc hypothesis that this control would either support or refute.
  2. [§4.1, Table 2, Figure 2, Table 5] The checkpoint-selection and stopping procedure is underspecified and risks selection on the outcome variable. §4.1 states that training continues until generated captions reach a sequence length of 60, yet Table 2 reports final caption lengths of 21–39, not 60. Figure 2 and Table 5 evaluate four later checkpoints (steps 6000, 7500, 8400, and 9150), but the text does not state how these checkpoints were chosen or which checkpoint corresponds to the final model in Tables 2 and 3. Since the training objective directly drives up caption length, and length is itself one of the reported outcomes, stopping or selecting checkpoints based on length can inflate apparent gains. The authors should report the selection rule, results for all checkpoints, and error bars or multiple seeds for the main tables.
  3. [§5.2] The BLIP-2 OPT result is in tension with the proposed mechanism and needs explicit reconciliation. The paper reports that for BLIP-2 OPT the token being penalized (ID 50118) is the newline token '\n', not the actual OPT EOS token '</s>' (ID 2). Yet EOS debiasing still improves CAPTURE and retrieval for this model. If the method works by penalizing a newline token, then the description 'EOS debiasing' is inaccurate for this model; if it works despite penalizing a non-termination token, then the claim that the effect is specific to the true EOS token is weakened. The discussion attributes the coherence drop to this mismatch, but the success of the method under the mismatch requires more analysis, for example an ablation that penalizes the true OPT EOS token instead.
minor comments (4)
  1. [Title and §4.1] The displayed title contains an extra space in 'T raining'; please correct the typographical error.
  2. [§4.3] The GPT-4o coherence evaluation is described with a single prompt and temperature 1.0, but no information is given about the number of captions scored per model-dataset cell, the variance of scores across repeated API calls, or the agreement with human judgments. Please provide these details or cite an established protocol.
  3. [Table 2] The baseline label 'triv.' is introduced in §4.3 but is not defined in the table caption. A one-sentence definition in the caption would make the table self-contained.
  4. [§2.3 and §5.3] The related work cites Yue et al. (2024) on over-generation from weakened EOS probability, but the paper does not connect this to the observed wave-like EOS-probability dynamics in Figure 1b. A brief discussion of how the training dynamics differ from the over-generation failure mode would strengthen the analysis.

Circularity Check

1 steps flagged · score 3.0 of 10

Length gains are built into the training protocol, but the paper's substantive detail-quality claims rest on external benchmarks and are not circular.

  1. fitted input called prediction [Section 3.2 (Eq. 2) and Section 4.1]
    "Concretely, we propose to gradually reduce the probability of the EOS token through sequence training. For each generated caption in a training batch, we minimize the probability of EOS token: ∇L(θ)=∇θ log pθ(tn) ... We train until the generated captions reach a sequence length of 60, the maximum that fits on this GPU card."

    The training objective directly minimizes the probability of the EOS token, which is the termination signal of the autoregressive decoder, and training is continued until generated captions reach length 60. Longer captions are therefore the optimization target itself, not an independent empirical prediction. Reporting 'a substantial increase in the length of generated captions' as a result restates the training signal. The paper's substantive claim that the extra text contains relevant, image-grounded detail is not circular, because it is evaluated with external metrics (CAPTURE, retrieval, recall) that are not fitted by the objective.

full rationale

The paper's central mechanism—that suppressing the EOS token during sequence training makes a pretrained VLM generate longer captions—is partly true by construction. Eq. 2 defines the training loss as minimizing log p(EOS) on generated sequences, and Section 4.1 states training continues until captions reach 60 tokens; thus longer output is the fitted objective, not a discovered consequence. However, the paper does not stop at length: its headline conclusions concern relevant detail, measured on external benchmarks (CAPTURE, retrieval Recall@1, unigram recall) that are not part of the training objective and are not fitted. Those evaluations give the central claim independent empirical content. I found no self-citation load-bearing arguments, no imported uniqueness theorems, and no ansatz smuggled in via citation; the cited EOS-length results from Newman et al. and Kulikov et al. are external prior work, not self-supporting. The absence of a control that penalizes a non-EOS token is a real threat to the causal attribution 'the Devil is in the EOS,' but that is a correctness/experimental-design concern, not circularity. Overall, only the length component reduces by construction, so the appropriate circularity score is moderate rather than severe.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new entities. It rests on the assumption of latent detail-generation capability in base VLMs, on the stability of pure negative training signal, and on the EOS token being a well-defined termination control. The target length 60 and learning rate are hand-set.

free parameters (3)
  • target captions length 60 = 60 tokens
    Training is run until generated captions reach 60 tokens, the maximum fitting on an A100 80GB GPU; this directly sets the length outcome and is not derived from the task.
  • learning rate 1e-7 = 1e-7
    Chosen by hand; higher values cause degenerate behavior and lower values slow convergence (Section 4.1, Appendix A.3.2).
  • decoding hyperparameters = beam=5, repetition penalty=1.5, no-repeat-ngram=3
    Used for all generation; chosen manually and not justified against alternatives.
assumptions (3)
  • domain assumption Base VLMs already possess the capability to produce detailed captions; EOS bias is the main bottleneck.
    Section 3.3 Prerequisites: the method "relies on the strong fundamental capabilities of VLMs"; if the model lacks visual-linguistic detail knowledge, removing EOS bias would yield only degenerate text.
  • domain assumption Sequence training with only a negative EOS signal will not cause the model to collapse into repetitions or ungrounded text.
    Section 3.2 states this is "a purely empirical question"; the mechanism in Section 5.3 is a post hoc explanation, not a derivation.
  • domain assumption The EOS token used during generation is the same token that was biased during cross-entropy training and that controls termination.
    For BLIP-2 OPT the authors find the VLM's EOS token is the newline token rather than the OPT EOS token (Section 5.2), so the penalty can target the wrong token and hurt coherence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Devil is in the EOS: Sequence Training for Detailed Image Captioning." pith.science (2026). https://pith.science/paper/CRINC73Z

@misc{pith2026250720077,
  author       = {Pith},
  title        = {Pith review of: The Devil is in the EOS: Sequence Training for Detailed Image Captioning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CRINC73Z}},
  note         = {Machine review of arXiv:2507.20077}
}
read the original abstract

Despite significant advances in vision-language models (VLMs), image captioning often suffers from a lack of detail, with base models producing short, generic captions. This limitation persists even though VLMs are equipped with strong vision and language backbones. While supervised data and complex reward functions have been proposed to improve detailed image captioning, we identify a simpler underlying issue: a bias towards the end-of-sequence (EOS) token, which is introduced during cross-entropy training. We propose an unsupervised method to debias the model's tendency to predict the EOS token prematurely. By reducing this bias, we encourage the generation of longer, more detailed captions without the need for intricate reward functions or supervision. Our approach is straightforward, effective, and easily applicable to any pretrained model. We demonstrate its effectiveness through experiments with three VLMs and on three detailed captioning benchmarks. Our results show a substantial increase in caption length and relevant details, albeit with an expected increase in the rate of hallucinations.

Figures

Figures reproduced from arXiv: 2507.20077 by the authors.

Figure 1
Figure 1. Training progression of BLIP-2 T5 as measured on COCO validation set. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Comparison across different checkpoints against TinyLLaVA on COCO validation set. in Figure 1a). This wave-like pattern cannot be trivially explained and likely has to do with the complex dynamics of sequence training [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 13 canonical work pages

  1. [1]

    Paligemma: A Versatile 3b VLM for Transfer

    Lucas Beyer, Andreas Steiner, Andr \'e Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A Versatile 3b VLM for Transfer . arXiv preprint arXiv:2407.07726, 2024

    arXiv 2024

  2. [2]

    ShareGPT4V: Improving Large Multi-Modal Models with Better Captions , 2023

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. ShareGPT4V: Improving Large Multi-Modal Models with Better Captions , 2023. URL https://arxiv.org/abs/2311.12793

    arXiv 2023

  3. [3]

    Fine-grained Image Captioning with CLIP Reward

    Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, Franck Dernoncourt, Trung Bui, and Mohit Bansal. Fine-grained Image Captioning with CLIP Reward . arXiv preprint arXiv:2205.13115, 2022

    arXiv 2022

  4. [4]

    The Algebraic Theory of Context-Free Languages

    Noam Chomsky and Marcel-Paul Sch \"u tzenberger. The Algebraic Theory of Context-Free Languages . In Studies in Logic and the Foundations of Mathematics, pp.\ 118--161. Elsevier, 1959. doi:10.1016/S0049-237X(08)72023-8

    doi:10.1016/s0049-237x(08)72023-8 1959

  5. [5]

    Benchmarking and Improving Detail Image Caption

    Hongyuan Dong, Jiawen Li, Bohong Wu, Jiacong Wang, Yuan Zhang, and Haoyuan Guo. Benchmarking and Improving Detail Image Caption . arXiv preprint arXiv:2405.19092, 2024

    arXiv 2024

  6. [6]

    CapWAP: Captioning with a Purpose

    Adam Fisch, Kenton Lee, Ming-Wei Chang, Jonathan H Clark, and Regina Barzilay. Capwap: Captioning with a Purpose . arXiv preprint arXiv:2011.04264, 2020

    work page Pith review arXiv 2011

  7. [7]

    No Detail Left Behind: Revisiting Self-Retrieval for Fine-Grained Image Captioning

    Manu Gaur, Makarand Tapaswi, et al. No Detail Left Behind: Revisiting Self-Retrieval for Fine-Grained Image Captioning . arXiv preprint arXiv:2409.03025, 2024

    work page Pith review arXiv 2024

  8. [8]

    From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment

    Yusuke Hirota, Ryo Hachiuma, Chao-Han Huck Yang, and Yuta Nakashima. From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment . In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 17807--17816, Miami, Florida, USA,...

    doi:10.18653/v1/2024.emnlp-main.986 2024

Show all 40 references
  1. [9]

    The Curious Case of Neural Text Degeneration

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The Curious Case of Neural Text Degeneration . arXiv preprint arXiv:1904.09751, 2019

    1904 arXiv

  2. [10]

    Characterizing and Addressing the Issue of Oversmoothing in Neural Autoregressive Sequence Modeling

    Ilia Kulikov, Maksim Eremeev, and Kyunghyun Cho. Characterizing and Addressing the Issue of Oversmoothing in Neural Autoregressive Sequence Modeling . arXiv preprint arXiv:2112.08914, 2021

    2021 arXiv

  3. [11]

    BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

    Junnan Li, Dongxu Li, Silvio Savarese, and Li Fei-Fei. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models . arXiv preprint arXiv:2301.12597, 2023

    2023 arXiv

  4. [12]

    Microsoft COCO: Common Objects in Context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. Microsoft COCO: Common Objects in Context . In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proce...

    2014

  5. [13]

    Visual Instruction Tuning

    Haotian Liu, Chunyuan Ye, Qiyuan Shi, and Weizhu Chen. Visual Instruction Tuning . arXiv preprint arXiv:2304.08485, 2023

    2023 arXiv

  6. [14]

    Discriminability Objective for Training Descriptive Captions

    Ruotian Luo, Brian Price, Scott Cohen, and Gregory Shakhnarovich. Discriminability Objective for Training Descriptive Captions . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 6964--6974, 2018

    2018

  7. [15]

    The EOS Decision and Length Extrapolation

    Benjamin Newman, John Hewitt, Percy Liang, and Christopher D Manning. The EOS Decision and Length Extrapolation . arXiv preprint arXiv:2010.07174, 2020

    2010 arXiv

  8. [16]

    DOCCI: Descriptions of Connected and Contrasting Images

    Yasumasa Onoe, Sunayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset, Garrett Tanzer, Su Wang, and Jason Baldridge. DOCCI: Descriptions of Connected and Contrasting Images . In Proceedings of the European Conferen...

    2024 doi

  9. [17]

    GPT-4V Technical Report , 2023

    OpenAI. GPT-4V Technical Report , 2023. URL https://openai.com/contributions/gpt-4v/. Accessed: 2025-02-04

    2023

  10. [18]

    ALOHa: A New Measure for Hallucination in Captioning Models

    Suzanne Petryk, David M Chan, Anish Kachinthaya, Haodi Zou, John Canny, Joseph E Gonzalez, and Trevor Darrell. ALOHa: A New Measure for Hallucination in Captioning Models . arXiv preprint arXiv:2404.02904, 2024

    2024 arXiv

  11. [19]

    Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions , 2024

    Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, and Tong Zhang. Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions , 2024. URL https://arxiv.org/abs/2406.07502

    2024 arXiv

  12. [20]

    Sequence Level Training with Recurrent Neural Networks , 2016

    Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence Level Training with Recurrent Neural Networks , 2016. URL https://arxiv.org/abs/1511.06732

    2016 arXiv

  13. [21]

    Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel

    Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self-Critical Sequence Training for Image Captioning . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 7008--7024, 2017

    2017

  14. [22]

    Object Hallucination in Image Captioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object Hallucination in Image Captioning . arXiv preprint arXiv:1809.02156, 2018

    2018 arXiv

  15. [23]

    CVQA: Culturally-Diverse Multilingual Visual Question Answering Benchmark , 2024

    David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Rada Mihalcea, Thamar Solorio, and Alham Fikri Aji. CVQA: Culturally-Diverse Multilingual Visual Question Answering Benchmark , 2024. URL https://arxiv.org/abs/2406.05967

    2024 arXiv

  16. [24]

    From Pixels to Prose: A Large Dataset of Dense Image Captions , 2024

    Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, and Tom Goldstein. From Pixels to Prose: A Large Dataset of Dense Image Captions , 2024. URL https://arxiv.org/abs/2406.10328

    2024 arXiv

  17. [25]

    Contrastive Search is What You Need for Neural Text Generation

    Yixuan Su and Nigel Collier. Contrastive Search is What You Need for Neural Text Generation . arXiv preprint arXiv:2210.14140, 2022

    2022 arXiv

  18. [26]

    A Picture is Worth More than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions

    Jack Urbanek, Florian Bordes, Pietro Astolfi, Mary Williamson, Vasu Sharma, and Adriana Romero-Soriano. A Picture is Worth More than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  19. [27]

    Lawrence Zitnick, and Devi Parikh

    Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. CIDEr: Consensus-based Image Description Evaluation , 2015. URL https://arxiv.org/abs/1411.5726

    2015 arXiv

  20. [28]

    Show and Tell: A Neural Image Caption Generator

    Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and Tell: A Neural Image Caption Generator . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 3156--3164, 2015

    2015

  21. [29]

    Williams and David Zipser

    Ronald J. Williams and David Zipser. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks . In Neural Computation, volume 1, pp.\ 270--280. MIT Press, 1989. doi:10.1162/neco.1989.1.2.270

    1989 doi

  22. [30]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 arXiv

  23. [31]

    Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation , 2023

    Zihao Yue, Anwen Hu, Liang Zhang, and Qin Jin. Learning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation , 2023. URL https://arxiv.org/abs/2306.13460

    2023 arXiv

  24. [32]

    Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective

    Zihao Yue, Liang Zhang, and Qin Jin. Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective . arXiv preprint arXiv:2402.14545, 2024

    2024 arXiv

  25. [33]

    Long-CLIP: Unlocking the Long-Text Capability of CLIP

    Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-CLIP: Unlocking the Long-Text Capability of CLIP . In European Conference on Computer Vision, pp.\ 310--325. Springer, 2024

    2024

  26. [34]

    Judging LLM-as-a-Judge with MT-Bench and ChatBot Arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging LLM-as-a-Judge with MT-Bench and ChatBot Arena . Advances in Neural Information Processing Systems, 36: 0 46595--46623, 2023

    2023

  27. [35]

    Tinyllava: A Framework of Small-Scale Large Multimodal Models

    Baichuan Zhou, Ying Hu, Xi Weng, Junlong Jia, Jie Luo, Xien Liu, Ji Wu, and Lei Huang. Tinyllava: A Framework of Small-Scale Large Multimodal Models . arXiv preprint arXiv:2402.14289, 2024 a

    2024 arXiv

  28. [36]

    Analyzing and Mitigating Object Hallucination in Large Vision-Language Models , 2024 b

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and Mitigating Object Hallucination in Large Vision-Language Models , 2024 b . URL https://arxiv.org/abs/2310.00754

    2024 arXiv

  29. [37]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  30. [38]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  31. [39]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  32. [40]

    \\ DCI & 7,805 & 10 & 45 & Images from SA-1B with detailed captions of up to 77 tokens, summarized with an LLM from hyper-detailed captions of 1k tokens

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.