Pith. sign in

REVIEW 4 major objections 6 minor 40 references

VaccineRAG: Boosting Multimodal Large Language Models' Immunity to Harmful RAG Samples

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VaccineRAG claims that training multimodal LLMs to analyze each retrieved sample before answering makes them resistant to harmful RAG context, with a segment-level GRPO variant doing the heavy lifting.

desk verdict A useful CoT/RAG training recipe with a real evaluator-bias risk; the dataset and Partial-GRPO are worth engaging, but the evidence needs error bars and independent scoring. read the letter →

arxiv 2509.04502 v1 pith:LBP7OHRX submitted 2025-09-02 cs.CL cs.AI

Qixin Sun , Ziqin Wang , Hengyuan Zhao , Yilin Li , Kaiyou Song , Linjiang Huang , Xiaolin Hu , Qingpei Guo
show 1 more author
Si Liu
This is my paper
classification cs.CLcs.AI
keywords Retrieval-AugmentedGenerationMultimodalLargeLanguageModelsChain-of-ThoughtReasoningPreferenceOptimizationGRPORobustnessVisualQuestionAnsweringVaccineRAG
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VaccineRAG attacks an underappreciated weak point in retrieval-augmented generation: even when the retriever works well enough to find useful references, the generator often absorbs irrelevant or misleading ones. The paper introduces a 10k-example multimodal dataset in which each retrieved image or passage is paired with a structured chain of thought—a summary, a helpfulness judgment, a conclusion that cites only helpful references, and then a final answer—so a model can be explicitly taught which evidence to trust. To make that long chain of thought learnable, the authors modify GRPO so that each segment of the output gets its own reward and gradient signal, calling the result Partial-GRPO. Experiments across Qwen2-VL, Qwen2.5-VL, and InternVL3 show higher mean accuracy and lower accuracy degradation than a strong robustness baseline and than vanilla GRPO when harmful retrieved samples are added. If correct, RAG reliability can be improved by training the generator to discriminate evidence rather than by endlessly improving the retriever.

What carries the argument

The two load-bearing components are the VaccineRAG dataset and Partial-GRPO. VaccineRAG provides dense token-level supervision: each of the roughly five retrieved references per question is paired with a GPT-4o-generated summary, a helpfulness analysis verified against the original WebQA label, a citation-aware conclusion, and the final answer. Partial-GRPO splits a sampled completion into helpfulness-analysis, conclusion, and final-answer segments; each segment has its own reward (helpfulness reward, conclusion reward, format reward), and each reward is normalized across the sampled group and applied through a partial importance-sampling ratio so that gradients affect only the segment the r

What would settle it

Take the same training pipeline and deliberately flip the helpfulness labels on 20% of the training samples, then run the polluted-generation evaluation: if accuracy degradation remains close to the normal VaccineRAG result, the method does not depend on annotation accuracy and the claim survives; if degradation jumps toward zero-shot levels, annotation quality is load-bearing and the immunity claim needs qualification.

Watch

Extended reading notes

Core claim

The paper's central claim is that a multimodal RAG system's vulnerability to harmful retrieved samples is not only a retriever problem; it is a generator problem that can be trained away with dense reasoning supervision. VaccineRAG converts sparse retrieval labels into a chain of thought: for each retrieved image or passage the model must produce a summary, decide whether the reference is helpful, write a conclusion that cites only helpful references, and only then give the final answer. Training on this dataset with a short SFT warm-up followed by Partial-GRPO yields higher mean accuracy and lower accuracy degradation than SURf and vanilla GRPO on polluted generation, and in TopK generation

Load-bearing premise

The central claim assumes the GPT-4o-generated summaries, helpfulness judgments, and conclusions—manually checked only when they disagree with the original labels—are correct enough to serve as a teaching signal; if those annotations are noisy or systematically biased, the model may imitate the annotator rather than learn genuine sample discrimination.

Editorial extensions

If this is right

  • If correct, multimodal RAG systems can tolerate imperfect retrievers: a trained model keeps most of its accuracy even when harmful samples are injected into the context.
  • Robustness no longer depends on the retriever staying accurate after deployment, since the generator itself learns to discriminate helpful from unhelpful evidence.
  • The structured chain-of-thought output gives an interpretable audit trail showing which retrieved samples the model accepted, which it rejected, and why.
  • The ablation results imply all three reward functions—format, helpfulness, and conclusion—are necessary; removing any one measurably hurts accuracy or output stability.
  • The method transfers across three different 7B-8B vision-language models, suggesting the recipe is not tied to one architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: evaluating the trained models on a different multimodal RAG benchmark or on adversarially constructed distractors would test whether the discrimination skill transfers beyond WebQA's distribution.
  • Beyond the paper: if the GPT-4o-generated chain-of-thought is replaced with a cheaper or weaker annotator, the drop in performance would reveal how much of the gain is annotation-driven rather than method-driven.
  • Beyond the paper: Partial-GRPO's segment-level credit assignment should generalize to other long structured generation tasks such as multi-step tool use or long-form QA, though the paper only tests it on RAG.
  • Beyond the paper: a deployed system could tune the retriever for speed and recall rather than precision, letting the trained generator serve as the final filter against noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces VaccineRAG, a Chain-of-Thought (CoT) annotated multimodal RAG dataset constructed from WebQA, together with Partial-GRPO, a variant of GRPO that applies segment-level rewards and partial importance-sampling ratios to different parts of a completion. The authors train Qwen2-VL-7B, Qwen2.5-VL-7B, and InternVL3-8B with SFT followed by Partial-GRPO, and report higher mean accuracy (MA) and lower accuracy degradation rate (ADR) than zero-shot, SURf, SURf+CoT, and vanilla GRPO under polluted retrieval settings, plus gains in Top-K retrieval. The central claim is that the method makes MLLMs robust to harmful retrieved samples while preserving accuracy.

Significance. If the results hold, VaccineRAG would be a useful dataset for studying retrieval robustness in multimodal RAG, and the idea of decomposing rewards by CoT segment is interesting. The three-reward design (format, helpfulness, conclusion) is intuitive, and the authors provide an ablation showing that each reward contributes. The paper also makes a methodological proposal for multi-component GRPO that could be relevant beyond RAG. However, the current evidence is weakened by an unvalidated GPT-4o evaluation, an inconsistent ADR pattern across models, and an incomplete specification of the Partial-GRPO objective. With these issues addressed, the contribution could be significant.

major comments (4)
  1. [§5.2, Eq. (3)] The 'partial importance sampling ratio' is defined as the arithmetic mean of per-token likelihood ratios. For a sequence-level policy ratio, the correct importance weight is the product over tokens of π_θ/π_θ_old. An average of ratios is not a likelihood ratio, so Eq. (4) and Eq. (10) are not unbiased surrogates for the GRPO objective as written. If the implementation uses a product, the equations should be corrected; if the average is deliberate, the authors need to supply a bias/variance analysis and show that the objective still optimizes the intended expected reward.
  2. [§4.1 and §6.2] GPT-4o is used both to generate the CoT annotations in the training data and to score the model answers in the main evaluation. The paper asserts that GPT-4o scoring 'aligns much more closely with human assessments' but reports no human study, agreement statistics, or alternative metrics. This introduces a confound: models trained on GPT-4o-generated CoT may imitate GPT-4o's answer style and receive systematically higher scores from the same model, independent of true robustness. Please provide a human evaluation on at least a random subset of all models, with inter-annotator agreement, or use an objective metric (e.g., exact match, token F1) as a second evaluation.
  3. [Tables 1–3] No error bars, multiple seeds, or significance tests are reported. Differences such as Qwen2-VL MA5 56.27 vs. GRPO 54.50 and ADR5 15.98 vs. 23.75 could be within run-to-run noise. Please report results for at least three independent training runs (mean±std) and use a significance test (e.g., paired bootstrap or Wilcoxon) for the main comparisons. This is essential for the central claim of robustness improvement.
  4. [Table 1, Qwen2.5-VL row] The claim in §6.4 that Partial-GRPO 'significantly improved' ADR relative to baselines is contradicted by the Qwen2.5-VL results: vanilla GRPO has ADR5=0.21 while Partial-GRPO has ADR5=10.43, and SURf+CoT has ADR5=5.83. Thus the robustness benefit is not consistent across models. If the main claim is that Partial-GRPO improves MA while preserving or improving ADR, Table 1 shows a model where ADR is much worse than vanilla GRPO. This needs to be discussed and the claim qualified.
minor comments (6)
  1. [Figure 1, template] The template lists '[reference_4]' twice; the second should presumably be '[reference_5]'.
  2. [§5.2, Eq. (5)] The format reward is defined only as 'adheres to the format'. Please specify the exact format-checking rule (e.g., presence of the begin/end markers) and how it is applied automatically.
  3. [§5.2, Eq. (8)] The conclusion reward depends on detecting whether a retrieved sample is 'cited' in the conclusion. This is not defined. Please specify the citation-detection mechanism (string matching, model-based span detection, etc.).
  4. [§4.1] The helpfulness analysis is generated by prompting GPT-4o with the ground-truth helpfulness label. This is post-hoc rationalization rather than independent annotation. Please clarify how this could bias the CoT and whether the manual verification addresses this.
  5. [§6.2] The claim that GPT-4o scores are 'consistent across multiple assessments' needs a protocol description: how many repeated evaluations, what variance, and whether the same prompt was used each time.
  6. [Abstract / §7] The code and dataset are promised to be 'publicly released soon'. For a reproducibility-oriented paper, please provide a link or state the intended release venue.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is self-contained and the central claims do not reduce to their inputs.

full rationale

VaccineRAG's claimed contribution is a CoT-annotated training dataset plus a Partial-GRPO objective, evaluated on held-out WebQA validation questions against external baselines (zero-shot, SURf, GRPO). The training signal comes from WebQA ground-truth helpfulness labels (used in reward functions Eq. 6-9) and GPT-4o-generated CoT text, while the reported accuracy is scored by GPT-4o on open-ended answers. Although the same model (GPT-4o) generates training annotations and evaluates answers, this is an evaluation-validity concern, not a circularity: the evaluation targets are WebQA ground-truth answers on a held-out set, and no fitted parameter or training quantity is used as its own prediction. The helpfulness and conclusion rewards are explicitly defined against ground-truth and internal-consistency targets, and the paper does not claim to derive those labels from the model. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation. Therefore the derivation is self-contained and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the reliability of WebQA labels, GPT-4o annotation quality, the validity of GPT-4o as an evaluator, and the design of the reward decomposition. No numeric free parameters are fitted to data in the proposed method; hyperparameters are deferred to the supplementary. No new physical entities are introduced.

assumptions (5)
  • domain assumption WebQA provides reliable ground-truth helpfulness labels for retrieved samples.
    The dataset construction (Section 4.1) adopts WebQA labels as the ground truth for the helpfulness reward and for manual verification of GPT-4o annotations.
  • domain assumption GPT-4o-generated CoT annotations (summaries, analyses, conclusions) are correct and teachable after manual verification.
    Sections 4.1 and 4.2 rely on GPT-4o annotations as training targets; the claim assumes these are accurate enough to improve sample discrimination.
  • domain assumption GPT-4o-based scoring of candidate answers is a valid proxy for correctness and aligns with human assessment.
    Section 6.2 asserts alignment with human assessment but reports no human agreement data; all accuracy numbers depend on this assumption.
  • ad hoc to paper The three reward functions (helpfulness, conclusion, format) and their token-level scopes decompose the desired reasoning behavior correctly.
    Section 5.2 defines rewards designed for this task; the claim that Partial-GRPO improves learning assumes this decomposition is the right objective.
  • standard math Standard GRPO and importance-sampling assumptions: the partial importance sampling ratio and group-relative advantage estimation provide unbiased or low-variance gradients.
    Section 5.2 uses importance sampling ratios and group normalization without new mathematical justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VaccineRAG: Boosting Multimodal Large Language Models' Immunity to Harmful RAG Samples." pith.science (2026). https://pith.science/paper/LBP7OHRX

@misc{pith2026250904502,
  author       = {Pith},
  title        = {Pith review of: VaccineRAG: Boosting Multimodal Large Language Models' Immunity to Harmful RAG Samples},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBP7OHRX}},
  note         = {Machine review of arXiv:2509.04502}
}
read the original abstract

Retrieval Augmented Generation enhances the response accuracy of Large Language Models (LLMs) by integrating retrieval and generation modules with external knowledge, demonstrating particular strength in real-time queries and Visual Question Answering tasks. However, the effectiveness of RAG is frequently hindered by the precision of the retriever: many retrieved samples fed into the generation phase are irrelevant or misleading, posing a critical bottleneck to LLMs' performance. To address this challenge, we introduce VaccineRAG, a novel Chain-of-Thought-based retrieval-augmented generation dataset. On one hand, VaccineRAG employs a benchmark to evaluate models using data with varying positive/negative sample ratios, systematically exposing inherent weaknesses in current LLMs. On the other hand, it enhances models' sample-discrimination capabilities by prompting LLMs to generate explicit Chain-of-Thought (CoT) analysis for each sample before producing final answers. Furthermore, to enhance the model's ability to learn long-sequence complex CoT content, we propose Partial-GRPO. By modeling the outputs of LLMs as multiple components rather than a single whole, our model can make more informed preference selections for complex sequences, thereby enhancing its capacity to learn complex CoT. Comprehensive evaluations and ablation studies on VaccineRAG validate the effectiveness of the proposed scheme. The code and dataset will be publicly released soon.

Figures

Figures reproduced from arXiv: 2509.04502 by the authors.

Figure 1
Figure 1. Top: Our multimodal RAG dataset VaccineRAG annotated with CoT. The reasoning process encompasses summa [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Partial GRPO calculates the objective function by multiplying the rewards applied to different parts of the completion [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 22 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and Hajishirzi, H. 2024. Self- RAG : Learning to Retrieve, Generate, and Critique through Self-Reflection. In The Twelfth International Conference on Learning Representations

    2024

  4. [4]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923

    arXiv 2025

  5. [5]

    Caffagni, D.; Cocchi, F.; Moratelli, N.; Sarto, S.; Cornia, M.; Baraldi, L.; and Cucchiara, R. 2024. Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 1818--1826

    work page 2024

  6. [6]

    Chang, Y.; Narang, M.; Suzuki, H.; Cao, G.; Gao, J.; and Bisk, Y. 2022. Webqa: Multihop and multimodal qa. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16495--16504

    work page 2022

  7. [7]

    Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z. 2024 a . M 3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics: ACL 2024, 2318--2335. Bangkok, Thailand: Associatio...

    work page 2024

  8. [8]

    Chen, T.; Wang, H.; Chen, S.; Yu, W.; Ma, K.; Zhao, X.; Zhang, H.; and Yu, D. 2024 b . Dense X Retrieval: What Retrieval Granularity Should We Use? In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 15159--15177. Miami, Florida, USA: Association for Computational Li...

    work page 2024

Show all 40 references
  1. [9]

    Chen, Z.; Wang, Z.; Wang, Z.; Liu, H.; Yin, Z.; Liu, S.; Sheng, L.; Ouyang, W.; Qiao, Y.; and Shao, J. 2023. Octavius: Mitigating task interference in mllms via lora-moe. arXiv preprint arXiv:2311.02684

    2023 arXiv

  2. [10]

    Chen, Z.; Wu, J.; Wang, W.; Su, W.; Chen, G.; Xing, S.; Zhong, M.; Zhang, Q.; Zhu, X.; Lu, L.; et al. 2024 c . Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  3. [11]

    Chen, Z.; Xu, C.; Qi, Y.; and Guo, J. 2024 d . MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training. CoRR, abs/2407.21439

    2024 arXiv

  4. [13]

    Gao, Y.; Xiong, Y.; Gao, X.; Jia, K.; Pan, J.; Bi, Y.; Dai, Y.; Sun, J.; Wang, M.; and Wang, H. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997

    2024 arXiv

  5. [14]

    He, X.; Tian, Y.; Sun, Y.; Chawla, N.; Laurent, T.; LeCun, Y.; Bresson, X.; and Hooi, B. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems, 37: 132876--132907

    2024

  6. [15]

    Houlsby, N.; Giurgiu, A.; Jastrzebski, S.; Morrone, B.; De Laroussilhe, Q.; Gesmundo, A.; Attariyan, M.; and Gelly, S. 2019. Parameter-efficient transfer learning for NLP. In International conference on machine learning, 2790--2799. PMLR

    2019

  7. [16]

    J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. Lo RA : Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

    2022

  8. [17]

    Hu, W.; Gu, J.-C.; Dou, Z.-Y.; Fayyaz, M.; Lu, P.; Chang, K.-W.; and Peng, N. 2025. MRAG -Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models. In The Thirteenth International Conference on Learning Representations

    2025

  9. [18]

    Jiang, Z.; Ma, X.; and Chen, W. 2024. Longrag: Enhancing retrieval-augmented generation with long-context llms. arXiv preprint arXiv:2406.15319

    2024 arXiv

  10. [19]

    Karimi Mahabadi, R.; Henderson, J.; and Ruder, S. 2021. Compacter: Efficient low-rank hypercomplex adapter layers. Advances in Neural Information Processing Systems, 34: 1022--1035

    2021

  11. [20]

    Lan, T.; Cai, D.; Wang, Y.; Huang, H.; and Mao, X.-L. 2023. Copy is All You Need. In The Eleventh International Conference on Learning Representations

    2023

  12. [21]

    u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt\

    Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; K\" u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt\" a schel, T.; Riedel, S.; and Kiela, D. 2020 a . Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Larochelle, H.; Ranzato, M.; Hadsell, R...

    2020

  13. [22]

    u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \

    Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; K \"u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \"a schel, T.; et al. 2020 b . Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33: 9459--9474

    2020

  14. [23]

    L.; and Liang, P

    Li, X. L.; and Liang, P. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190

    2021 arXiv

  15. [24]

    Lin, S.-C.; Lee, C.; Shoeybi, M.; Lin, J.; Catanzaro, B.; and Ping, W. 2025. MM - EMBED : UNIVERSAL MULTIMODAL RETRIEVAL WITH MULTIMODAL LLMS . In The Thirteenth International Conference on Learning Representations

    2025

  16. [25]

    Liu, X.; Ji, K.; Fu, Y.; Tam, W.; Du, Z.; Yang, Z.; and Tang, J. 2022. P -Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Compu...

    2022

  17. [26]

    Lu, P.; Mishra, S.; Xia, T.; Qiu, L.; Chang, K.-W.; Zhu, S.-C.; Tafjord, O.; Clark, P.; and Kalyan, A. 2022. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., ed...

    2022

  18. [27]

    Lu, P.; Qiu, L.; Yu, W.; Welleck, S.; and Chang, K.-W. 2023. A Survey of Deep Learning for Mathematical Reasoning. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2023

  19. [28]

    Qiao, S.; Ou, Y.; Zhang, N.; Chen, X.; Yao, Y.; Deng, S.; Tan, C.; Huang, F.; and Chen, H. 2023. Reasoning with Language Model Prompting: A Survey. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computationa...

    2023

  20. [29]

    Ram, O.; Levine, Y.; Dalmedigos, I.; Muhlgay, D.; Shashua, A.; Leyton-Brown, K.; and Shoham, Y. 2023. In-Context Retrieval-Augmented Language Models. Transactions of the Association for Computational Linguistics, 11: 1316--1331

    2023

  21. [30]

    Sarto, S.; Cornia, M.; Baraldi, L.; Nicolosi, A.; and Cucchiara, R. 2024. Towards Retrieval-Augmented Architectures for Image Captioning. ACM Trans. Multimedia Comput. Commun. Appl., 20(8)

    2024

  22. [31]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  23. [32]

    T.; Nayeem, M

    Shohan, F. T.; Nayeem, M. T.; Islam, S.; Akash, A. U.; and Joty, S. 2024. XL - H ead T ags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association f...

    2024

  24. [33]

    Sun, J.; Zhang, J.; Zhou, Y.; Su, Z.; Qu, X.; and Cheng, Y. 2024. SUR f: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural...

    2024

  25. [34]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Fan, Y.; Dang, K.; Du, M.; Ren, X.; Men, R.; Liu, D.; Zhou, C.; Zhou, J.; and Lin, J. 2024. Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. arXiv...

    2024 arXiv

  26. [35]

    V.; and Zhou, D

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; ichter, b.; Xia, F.; Chi, E.; Le, Q. V.; and Zhou, D. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in Neural...

    2022

  27. [36]

    Xu, G.; Jin, P.; Li, H.; Song, Y.; Sun, L.; and Yuan, L. 2024. LLaVA-CoT: Let Vision Language Models Reason Step-by-Step. arXiv:2411.10440

    2024 arXiv

  28. [37]

    Yan, Y.; and Xie, W. 2024. E cho S ight: Advancing Visual-Language Models with W iki Knowledge. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 1538--1551. Miami, Florida, USA: Association for Computa...

    2024

  29. [38]

    Yu, Z.; He, L.; Wu, Z.; Dai, X.; and Chen, J. 2023. Towards Better Chain-of-Thought Prompting Strategies: A Survey. arXiv:2310.04959

    2023 arXiv

  30. [39]

    Zhang, Z.; Zhang, A.; Li, M.; Zhao, H.; Karypis, G.; and Smola, A. 2023. Multimodal Chain-of-Thought Reasoning in Language Models. arXiv preprint arXiv:2302.00923

    2023 arXiv

  31. [40]

    Zhao, X.; Zhang, Y.; Zhang, W.; and Wu, X.-M. 2024. U ni F ashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generation. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language...

    2024

  32. [41]

    J.; Lian, D.; and Xiong, Y

    Zhou, J.; Liu, Z.; Liu, Z.; Xiao, S.; Wang, Y.; Zhao, B.; Zhang, C. J.; Lian, D.; and Xiong, Y. 2024. MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval. arXiv preprint arXiv:2412.14475

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.