Pith. sign in

REVIEW 3 major objections 7 minor 7 cited by

VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training a vision-language model with retrieval-aware reinforcement learning and a crop-and-zoom action space raises visual RAG accuracy by roughly 20 to 30 points on three benchmarks.

desk verdict Plausible RL recipe for visual RAG with a clean crop-and-zoom action space, but the 20-30% numbers are not yet supported because the evaluation judge is undisclosed and the training reward is a Qwen model. read the letter →

arxiv 2505.22019 v2 pith:V7CVRBQN submitted 2025-05-28 cs.CL cs.AIcs.CV

Qiuchen Wang , Ruixue Ding , Yu Zeng , Zehui Chen , Lin Chen , Shihang Wang , Pengjun Xie , Fei Huang
show 1 more author
Feng Zhao
This is my paper
classification cs.CLcs.AIcs.CV
keywords retrieval-augmentedgenerationvision-languagemodelsreinforcementlearningvisualperceptionactionsGRPOvisuallyrichdocumentsmulti-turnreasoningcrop-and-zoom
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VRAG-RL is a reinforcement-learning recipe for retrieval-augmented question answering over images, slides, charts, tables, and dense document layouts. The paper argues that prior multimodal RAG systems fail in two ways: they drop images into the context without any visual-specific perception, and they cannot phrase search queries well enough to retrieve the right images. To fix this, the paper trains a 7B and a 3B vision-language model to emit search queries and also to select, crop, and zoom into regions of retrieved images, then optimizes the whole trajectory with GRPO, a reinforcement-learning update that compares a group of sampled trajectories. On SlideVQA, ViDoSeek, and MMLongBench, the trained models improve overall accuracy from 34.7 to 57.1 with the 7B model and from 21.3 to 53.5 with the 3B model over the strongest RL-based RAG baseline. If the result holds, visual RAG should be trained as an interactive agent rather than built as a fixed pipeline.

What carries the argument

The load-bearing machinery is the visual perception action space combined with the RAG-specific reward. The action space treats select, crop, and zoom as a first-class tool: the model writes a bounding box in <region> tokens, the system maps those normalized coordinates back onto the raw image, crops the region, and re-encodes it, so an information-dense area receives many more vision tokens than it would inside a downscaled full-page image. The reward is $r = \alpha r_{\mathrm{Ret}} + \beta r_{\mathrm{Ans}} + \gamma r_{\mathrm{Pat}}$: $r_{\mathrm{Ret}}$ scores how early and completely the trajectory retrieves golden relevant images using NDCG, $r_{\mathrm{Ans}}$ uses the paper's specified 7B instruction-tuned judge to decide whether the final answer is correct, and $r_{\mathrm{Pat}}$ checks adherence to the predefined search, crop, and answer format. GRPO then optimizes the policy over group-sampled multi-turn trajectories, with the model-based reward intended to give a smoother learning signal than exact match for long answers.

What would settle it

Re-score VRAG-RL's outputs and the Search-R1-VL baseline on SlideVQA or ViDoSeek with human annotations, or with a held-out judge that was not used in training, and compare the accuracy gap; if the gap collapses or the judge disagrees with humans on a large fraction of VRAG-RL's True answers, the central claim is about judge optimization rather than understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that reinforcement learning can teach a vision-language model to perform retrieval, visual inspection, and reasoning as one learned policy. Its VRAG-RL framework defines a Thought-Action-Observation loop in which the model can search, emit a bounding-box crop action using <region> tokens, receive the re-encoded crop as a new observation, and finally answer inside <answer> tags. Training first runs supervised fine-tuning on multi-expert trajectories, then applies GRPO with the composite reward $r_{\phi} = \alpha r_{\mathrm{Ret}} + \beta r_{\mathrm{Ans}} + \gamma r_{\mathrm{Pat}}$, where $r_{\mathrm{Ret}}$ is a modified Normalized Discounted Cumulative Gain that compares the order of retrieved images against golden relevant images, $r_{\mathrm{Ans}}$ is a binary model-based judge of the final answer against a reference answer, and $r_{\mathrm{Pat}}$ rewards following the action format. The paper reports that this beats vanilla RAG, ReAct, and Search-R1-style baselines on SlideVQA, ViDoSeek, and MMLongBench, with ablations attributing part of the gain to the visual-perception action space and part to the RAG-specific reward.

Load-bearing premise

The reported gains assume the model-based binary judge scores answers as a human would; if the trained model has simply learned to satisfy that judge, the improvement is not real understanding.

Editorial extensions

If this is right

  • If the framework's results hold, RL-tuned vision-language models could replace fixed OCR-based pipelines for slide, chart, table, and layout questions, since OCR-based RAG scores far lower throughout the reported comparisons.
  • Retrieval quality becomes a trainable behavior: the model learns to issue queries that surface relevant images early and to stop searching once the context is sufficient, so answer correctness and search efficiency improve together.
  • The crop-and-zoom action space makes visual perception a first-class action, and the reported gains concentrate in exactly the categories where low resolution hurts: layout, chart, and figure reasoning on MMLongBench.
  • Because the model-based reward replaces exact-match scoring, the training signal accepts paraphrase-correct answers, which the paper argues gives more stable and more general RL for long-form visual question answering.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The retrieval reward depends on golden relevant images at training time; in open-domain deployment no such list exists, so a practical system would need to learn relevance from weak labels or self-generated feedback, a transfer the paper does not test.
  • The binary answer judge is itself an instruction-tuned language model, so a natural next measurement is to compare VRAG-RL's answers against human labels and a second judge; without that, part of the reported gap could reflect alignment to the judge's notion of correctness rather than task understanding.
  • The coarse-to-fine crop action resembles how a human reads a dense document, so the same action space could transfer to other visual-agent tasks such as web navigation, screenshot-based UI grounding, or high-resolution image search; the paper evaluates documents only.
  • The data-scaling recipe uses larger models to produce trajectories and grounding coordinates for smaller models, so the reported 3B and 7B gains are partly a distillation effect; an open-source-only version would test how much teacher strength matters.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The manuscript proposes VRAG-RL, a reinforcement learning framework for retrieval-augmented generation with vision-language models. A VLM policy is trained with GRPO to interact with a search engine in a Thought-Action-Observation loop, with an action space that includes a cropping and zooming operation on already retrieved images. The reward is a weighted combination of a rule-based pattern reward, an NDCG-style retrieval efficiency reward computed against golden relevant images, and a model-based answer-correctness reward. The authors first collect expert trajectories with a large VLM and a grounding model, perform SFT, and then apply GRPO. Experiments on SlideVQA, ViDoSeek, and MMLongBench report large gains over Vanilla RAG, ReAct, and Search-R1 baselines, with the abstract claiming about 20% improvement for Qwen2.5-VL-7B and 30% for Qwen2.5-VL-3B. The central contribution is the combination of visual perception actions with retrieval-aware RL rewards for multimodal RAG.

Significance. If the reported gains are real, the paper would be a useful demonstration that RL can train VLMs to perform iterative visual retrieval and coarse-to-fine perception in an agentic RAG setting, and the released code would support follow-up work. The paper also has strengths: the action space is concrete, the reward decomposition is explicit, and the training pipeline (SFT then GRPO) is described in enough detail to be partially reproducible. However, the headline results depend on an unvalidated model-based binary evaluator whose identity is not disclosed. Because every number in Tables 1 and 2 flows through that evaluator, and because the retrieval reward uses golden relevance sets that are unavailable at deployment, the evidence for the paper's central claim is not yet sufficient. The stress-test concern about evaluator independence lands: Appendix A names Qwen2.5-7B-Instruct as the training reward model, while Section 3.1 only says that the evaluation metric is a binary model-based judge, without naming the judge or reporting agreement with human labels.

major comments (3)
  1. [§3.1, Appendix A, Table 1] The evaluation metric is a binary model-based judge, but the paper never identifies the judge, reports its agreement with human judgments, or compares it with an alternative metric. Appendix A identifies Qwen2.5-7B-Instruct as the model used for the training reward, and the prompt in Figure 12 is the only model-based evaluation protocol described. If the same model, or a model from the same family, is used for evaluation, the 20–30% improvements in Tables 1 and 2 may partly reflect the policy learning to satisfy that judge rather than improved visual RAG. Please disclose the evaluation judge, report inter-annotator agreement or correlation with human labels, and include at least one judge-independent accuracy measure.
  2. [§2.3, Eqs. (7)–(9), §3.2] The retrieval efficiency reward is computed against Drel, a collection of golden relevant images, and is used as an NDCG-style reward during RL. Section 1 and Section 3.2 claim the framework aligns with real-world applications, but golden relevance labels are not available when the system is deployed. The paper does not state which datasets provide Drel for training, how the reward behaves when Drel is incomplete or noisy, or how retrieval quality is evaluated at test time without oracle labels. This is load-bearing for the real-world alignment claim and should be addressed with a discussion of train/test relevance annotation and at least one experiment showing behavior when Drel is unavailable or approximated.
  3. [Tables 1–3, Figures 3–6] All reported numerical claims are point estimates from single evaluation runs, with no standard deviations, multiple seeds, or significance tests. This is especially important because the evaluation metric itself is model-based and potentially sensitive to the judge, and because the paper acknowledges that RL training with VLMs can be unstable. Please provide variance estimates or significance tests for at least the main results and the ablation in Table 2; without these, the magnitude of the claimed improvements cannot be assessed.
minor comments (7)
  1. [§2.3, Eq. (8)] In the IDCG definition, n is not defined, and the denominator log2(i+1) is used for both relevant and irrelevant positions; the expression should be rewritten so the reader can see exactly what ideal ranking is assumed.
  2. [Table 2] The rows of Table 2 are not labeled, so it is unclear which configuration corresponds to each accuracy value; please add explicit row labels for the four ablation settings.
  3. [§2.2] The sentence about coordinate normalization to [0, δ] does not say which models use this convention or what δ is; please add a citation and clarify the coordinate mapping for the action extraction.
  4. [§3.1 vs Appendix G] Section 3.1 says full parameter fine-tuning is used, but Table 4 reports that the vision tower and multimodal projector are frozen; please reconcile these statements.
  5. [Figure 3] The caption 'Inference w/ Noise Upper Bound' is not explained, and the plot appears to show a single curve with no indication of how the number of reference images was varied; please clarify the protocol.
  6. [§3.2] The description of Search-R1-VL says it is implemented on the authors' framework with the same reward and post-processing, so the comparison with 'strong baselines' should state more explicitly which components are shared and which differ.
  7. [Throughout] There are several typos and inconsistencies, including 'cold strat' in Section 3.3, 'understaning' in Section 3.3, and the reference formatting of 'Vidorag'; please proofread.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported 20–30% accuracy gains are measured by an unnamed binary model-based judge whose description matches the binary model-based reward used to train the policy; without disclosure or human validation, the headline result may simply reflect optimization of that judge.

  1. fitted input called prediction [Section 2.3 (Eq. 11); Section 3.1; Appendix A]
    "Regarding outcome reward, ... we adopt a model-based reward: rAns ∼πRM(·|Q,A golden,A pred) ... The model-based evaluation metric is binary 0 or 1, indicating the accuracy of the model’s responses. ... we utilize Qwen2.5-7B-Instruct Yang et al. (2024) as our reward model ... outputs a binary value (0 or 1) to represent the accuracy of the answer."

    The paper never names the evaluation judge or validates it against human labels, while describing the metric with the same binary query+reference+answer correctness judgment used as the training reward rAns. If the same Qwen2.5-7B-Instruct judge is used at inference, as the text does not exclude, then the 'accuracy' reported in Table 1 is exactly the objective the policy was optimized to maximize. The 20–30% gains then measure the policy's fit to the judge, not an independent visual-RAG capability; the central claim reduces, by construction, to 'the policy scores well on πRM', and all headline numbers inherit this reduction.

full rationale

VRAG-RL's framework components are not internally circular: the crop-and-zoom action space, the NDCG retrieval reward constructed from golden Drel, the pattern reward, and the GRPO objective are defined independently of the reported improvements. Auxiliary evaluations (retrieval Recall@1, finish/invalid action rates) are rule-based and objective. The serious risk is the main accuracy metric: Section 3.1 states only 'The model-based evaluation metric is binary 0 or 1', while Section 2.3 Eq. (11) and Appendix A define the training reward as the binary correctness output of Qwen2.5-7B-Instruct. The evaluator is never named nor validated against human judgments, so the paper does not exclude the case where the test-time judge is the same model used as the RL reward. In that case Table 1's accuracy is rAns itself, and the 20%/30% gains are a measure of reward optimization, not independent evidence of better retrieval or visual reasoning. The Section 3.3 assertion that the model-based reward is 'more stable' and 'as demonstrated in Appendix A' lacks any comparative experiment in Appendix A; this is an additional missing-proof issue, though not itself a circularity. The ViDoSeek benchmark comes from the same group, but using one's own benchmark is not circular. Overall, partial circularity of the central empirical claim: the headline numbers reduce to the judge unless the authors disclose a distinct, human-validated evaluator.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. It relies on three domain assumptions: the availability of golden relevance labels for the retrieval reward, the validity of a model-based answer judge, and correct coordinate mapping for cropping. The reward weights and the interaction limit are free parameters that influence the central results.

free parameters (2)
  • Reward weights alpha, beta, gamma = Not reported; gamma = 0 or 0.1, alpha and beta unspecified
    Equation 12 defines the final reward as a weighted sum with alpha+beta+gamma=1, but only gamma is assigned a value; alpha and beta are not reported or swept, so the final reward may be hand-tuned to the benchmarks.
  • Maximum interaction iterations T = Not reported
    Algorithm 1 runs until T interactions, but T is not specified, and it directly controls trajectory length and thus the opportunities for retrieval and cropping.
assumptions (3)
  • domain assumption A golden set of relevant images Drel is available for every training query.
    Equations 7-9 compute the retrieval efficiency reward using Drel, a predefined relevance set. In real-world RAG deployments such labels are absent, so the trained retrieval behavior may not transfer outside benchmark datasets.
  • domain assumption The model-based answer judge returns a valid binary correctness signal.
    Equation 11 and Appendix A use Qwen2.5-7B-Instruct to judge whether a generated answer is True or False. No agreement with human judgments or other evaluators is reported, so the judge's validity is assumed.
  • domain assumption The coordinate mapping from encoded context tokens to original image pixels is accurate.
    Equation 2 maps crop coordinates from the vision encoder's token space back to the raw image dimensions. If this mapping is inaccurate, the cropped observation will be wrong, breaking the visual perception action space.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning." pith.science (2026). https://pith.science/paper/V7CVRBQN

@misc{pith2026250522019,
  author       = {Pith},
  title        = {Pith review of: VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V7CVRBQN}},
  note         = {Machine review of arXiv:2505.22019}
}
read the original abstract

Effectively retrieving, reasoning and understanding visually rich information remains a challenge for RAG methods. Traditional text-based methods cannot handle visual-related information. On the other hand, current vision-based RAG approaches are often limited by fixed pipelines and frequently struggle to reason effectively due to the insufficient activation of the fundamental capabilities of models. As RL has been proven to be beneficial for model reasoning, we introduce VRAG-RL, a novel RL framework tailored for complex reasoning across visually rich information. With this framework, VLMs interact with search engines, autonomously sampling single-turn or multi-turn reasoning trajectories with the help of visual perception tokens and undergoing continual optimization based on these samples. Our approach highlights key limitations of RL in RAG domains: (i) Prior Multi-modal RAG approaches tend to merely incorporate images into the context, leading to insufficient reasoning token allocation and neglecting visual-specific perception; and (ii) When models interact with search engines, their queries often fail to retrieve relevant information due to the inability to articulate requirements, thereby leading to suboptimal performance. To address these challenges, we define an action space tailored for visually rich inputs, with actions including cropping and scaling, allowing the model to gather information from a coarse-to-fine perspective. Furthermore, to bridge the gap between users' original inquiries and the retriever, we employ a simple yet effective reward that integrates query rewriting and retrieval performance with a model-based reward. Our VRAG-RL optimizes VLMs for RAG tasks using specially designed RL strategies, aligning the model with real-world applications. The code is available at https://github.com/Alibaba-NLP/VRAG.

Figures

Figures reproduced from arXiv: 2505.22019 by the authors.

Figure 1
Figure 1. Overall Framework of our Reinforcement Learning Framework. (a) demonstrates the interaction process between the model and the external environment, as well as the implementation of the GRPO algorithm. (b) shows the proposed visual perception action space which allows the model to extract information from a coarse-to-fine perspective. (c) is the specially designed reward for RAG, which combines outcome and retrieval … view at source ↗
Figure 2
Figure 2. Comparison between our VRAG-RL and the traditional RAG in terms of perception methods. (a) Traditional methods lack effective perception, which easily leads to repetitive and ineffective retrieval calls and suboptimal outcomes. (b) Our VRAG-RL is efficient and accurate, enabling the model to perceive information-dense regions from a coarse-to-fine perspective. The policy model πθ interacts with the environment in th… view at source ↗
Figure 3
Figure 3. Experiments on the impact of context length on model performance. Unlike traditional RL methods that focus only on out￾put results, VRAG-RL emphasizes optimizing retrieval in RAG, as retrieval quality directly affects overall per￾formance. We designed a reward function with three components: pattern reward, retrieval efficiency reward, and model-based outcome reward, guiding the model to efficiently retrieve informa… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Retrieval perfor￾mance of our approach. Better retrieval facilitates high-quality generation. Our VRAG￾RL framework significantly enhances the retrieval efficiency, which is crucial for constructing a coherent and informative context for high-quality generation. As dem…
Figure 5
Figure 5. Figure 5: Relative perfor￾mance on MMLongBench. Visual perception action space provides a fine-grained perspective. The visual perception action space introduced in our framework further enhances understaning by allowing the model to focus on information￾dense regions of images …
Figure 6
Figure 6. Figure 6: Latency Analysis on Generation. Time efficiency. As shown in [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Case 1 for VRAG-RL. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Case 2 for VRAG-RL. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Prompt of VRAG-RL. Search-R1(-VL) Prompt. System Prompt: Answer the given question. You must conduct reasoning inside <think> and </think> first every time you get new information. After reasoning, if you find you lack some knowledge, you can call a search engine by <s…
Figure 10
Figure 10. Figure 10: Prompt of Search-R1(-VL) and ReAct RAG. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Prompt of Vanilla RAG. Reward Model Prompt. System Prompt: Character Introduction You are an expert evaluation system for a question answering chatbot. You are given the following information: - the query - a generated answer - a reference answer Your task is to evalu…
Figure 12
Figure 12. Figure 12: Prompt of Reward Model. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Video-DeepResearch trains video agents with a two-stage SFT plus GRPO pipeline to do visual web searches before text searches, and introduces a 200-question benchmark requiring both video grounding and open-web knowledge.

  2. HierDoc: Hierarchical Page-to-Region Evidence Routing for Long-Document Visual Question Answering

    cs.CV 2026-07 conditional novelty 6.0 of 10

    HierDoc trains separate page- and region-selection policies with GRPO, then answers from full selected pages plus selected region crops and text.

  3. BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

    cs.CV 2026-07 conditional novelty 6.0 of 10

    BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.

  4. POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Management

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    POINTS-Seeker-8B is an 8B multimodal model trained from scratch for agentic search that uses seeding and visual-space history folding to outperform prior models on six visual reasoning benchmarks.

  5. M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generation

    cs.IR 2025-08 conditional novelty 6.0 of 10

    M2IO-R1 uses GRPO reinforcement learning to train a 3B model that selects and places retrieved images into generated text answers, improving MRAMG quality on several benchmarks.

  6. Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback

    cs.CV 2025-07 conditional novelty 6.0 of 10

    RRVF trains an image-to-code MLLM using reinforcement learning with a render-and-compare visual feedback loop, and it outperforms supervised fine-tuning on chart and web benchmarks.

  7. Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Scene-aware multi-agent document synthesis plus error-driven hard-example expansion improves compact Qwen3-VL models on constrained and open-category KIE, topping reported on-device baselines.

Reference graph

Works this paper leans on

54 extracted references · 8 canonical work pages · cited by 7 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  3. [3]

    Benchmarking large language models in retrieval-augmented generation

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.\ 17754--17762, 2024 a

    work page 2024

  4. [4]

    R1-v: Reinforcing super generalization ability in vision-language models with less than \ 3

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than \ 3. https://github.com/Deep-Agent/R1-V, 2025 a . Accessed: 2025-02-02

    work page 2025

  5. [5]

    Research: Learning to reason with search for llms via reinforcement learning

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z Pan, Wen Zhang, Huajun Chen, Fan Yang, et al. Research: Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470, 2025 b

    arXiv 2025

  6. [6]

    Mindsearch: Mimicking human minds elicits deep ai searcher

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Jiangning Liu, Wenwei Zhang, Kai Chen, and Feng Zhao. Mindsearch: Mimicking human minds elicits deep ai searcher. arXiv preprint arXiv:2407.20183, 2024 b

    arXiv 2024

  7. [7]

    Agent-flan: Designing data and methods of effective agent tuning for large language models

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-flan: Designing data and methods of effective agent tuning for large language models. arXiv preprint arXiv:2403.12881, 2024 c

    arXiv 2024

  8. [8]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024 d

    arXiv 2024

Show all 54 references
  1. [9]

    M3docrag: Multi-modal retrieval is what you need for multi-page multi-document understanding

    Jaemin Cho, Debanjan Mahata, Ozan Irsoy, Yujie He, and Mohit Bansal. M3docrag: Multi-modal retrieval is what you need for multi-page multi-document understanding. arXiv preprint arXiv:2411.04952, 2024

    2024 arXiv

  2. [10]

    Pp-ocr: A practical ultra lightweight ocr system, 2020

    Yuning Du, Chenxia Li, Ruoyu Guo, Xiaoting Yin, Weiwei Liu, Jun Zhou, Yifan Bai, Zilin Yu, Yehua Yang, Qingqing Dang, and Haoshuang Wang. Pp-ocr: A practical ultra lightweight ocr system, 2020. URL https://arxiv.org/abs/2009.09941

    2020 arXiv

  3. [11]

    Colpali: Efficient document retrieval with vision language models

    Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, C \'e line Hudelot, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models. In The Thirteenth International Conference on Learning Representations, 2024

    2024

  4. [12]

    Retrieval-augmented generation for large language models: A survey

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2: 0 1, 2023

    2023 arXiv

  5. [13]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  6. [14]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  7. [15]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  8. [16]

    Mmsearch: Benchmarking the potential of large models as multi-modal search engines

    Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, et al. Mmsearch: Benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409.12959, 2024

    2024 arXiv

  9. [17]

    Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning

    Pengcheng Jiang, Jiacheng Lin, Lang Cao, Runchu Tian, SeongKu Kang, Zifeng Wang, Jimeng Sun, and Jiawei Han. Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning. arXiv preprint arXiv: 2503.00223, 2025. URL https://arx...

    2025 arXiv

  10. [18]

    Long-context llms meet rag: Overcoming challenges for long inputs in rag

    Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik. Long-context llms meet rag: Overcoming challenges for long inputs in rag. arXiv preprint arXiv:2410.05983, 2024

    2024 arXiv

  11. [19]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    2025 arXiv

  12. [20]

    Reinforcement learning: A survey

    Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4: 0 237--285, 1996

    1996

  13. [21]

    Nv-embed: Improved techniques for training llms as generalist embedding models

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428, 2024

    2024 arXiv

  14. [22]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing...

    2020

  15. [23]

    Search-o1: Agentic search-enhanced large reasoning models

    Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366, 2025

    2025 arXiv

  16. [24]

    Benchmarking multimodal retrieval augmented generation with dynamic vqa dataset and self-adaptive planning agent

    Yangning Li, Yinghui Li, Xinyu Wang, Yong Jiang, Zhen Zhang, Xinran Zheng, Hui Wang, Hai-Tao Zheng, Fei Huang, Jingren Zhou, et al. Benchmarking multimodal retrieval augmented generation with dynamic vqa dataset and self-adaptive planning agent. arXiv preprint arXiv:2411.02937, 2024

    2024 arXiv

  17. [25]

    Towards general text embeddings with multi-stage contrastive learning

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023

    2023 arXiv

  18. [26]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 26296--26306, 2024

    2024

  19. [27]

    LlamaIndex , 11 2022

    Jerry Liu. LlamaIndex , 11 2022. URL https://github.com/jerryjliu/llama_index

    2022

  20. [28]

    Visual-rft: Visual reinforcement fine-tuning

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  21. [29]

    Mmlongbench-doc: Benchmarking long-context document understanding with visualizations

    Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, et al. Mmlongbench-doc: Benchmarking long-context document understanding with visualizations. arXiv preprint arXiv:2407.01523, 2024

    2024 arXiv

  22. [30]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  23. [31]

    Simpo: Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37: 0 124198--124235, 2024

    2024

  24. [32]

    Nv-retriever: Improving text embedding models with effective hard-negative mining

    Gabriel de Souza P Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge. Nv-retriever: Improving text embedding models with effective hard-negative mining. arXiv preprint arXiv:2407.15831, 2024

    2024 arXiv

  25. [33]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/, 2024

    2024

  26. [34]

    Introducing gemini 2.0: our new ai model for the agentic era, 2024

    Sundar Pichai, D Hassabis, and K Kavukcuoglu. Introducing gemini 2.0: our new ai model for the agentic era, 2024

    2024

  27. [35]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36: 0 53728--53741, 2023

    2023

  28. [36]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  29. [37]

    Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems, ...

    2024

  30. [38]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  31. [39]

    Reinforcement learning

    Richard S Sutton, Andrew G Barto, et al. Reinforcement learning. Journal of Cognitive Neuroscience, 11 0 (1): 0 126--134, 1999

    1999

  32. [40]

    Slidevqa: A dataset for document visual question answering on multiple images

    Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. Slidevqa: A dataset for document visual question answering on multiple images. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.\ 13636--13645, 2023

    2023

  33. [41]

    Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents

    Qiuchen Wang, Ruixue Ding, Zehui Chen, Weiqi Wu, Shihang Wang, Pengjun Xie, and Feng Zhao. Vidorag: Visual document retrieval-augmented generation via dynamic iterative reasoning agents. arXiv preprint arXiv:2502.18017, 2025 a

    2025 arXiv

  34. [42]

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning, 2025 b

    Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding self-evolution in ll...

    2025 arXiv

  35. [43]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8: 0 229--256, 1992

    1992

  36. [44]

    Webwalker: Benchmarking llms in web traversal

    Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, et al. Webwalker: Benchmarking llms in web traversal. arXiv preprint arXiv:2501.07572, 2025 a

    2025 arXiv

  37. [45]

    Unfolding the headline: Iterative self-questioning for news retrieval and timeline summarization

    Weiqi Wu, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, and Hai Zhao. Unfolding the headline: Iterative self-questioning for news retrieval and timeline summarization. arXiv preprint arXiv:2501.00888, 2025 b

    2025 arXiv

  38. [46]

    Rule: Reliable multimodal rag for factuality in medical vision language models

    Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. Rule: Reliable multimodal rag for factuality in medical vision language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 1081--1093, 2024

    2024

  39. [47]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  40. [48]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  41. [49]

    Perception-r1: Pioneering perception policy with reinforcement learning

    En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, et al. Perception-r1: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954, 2025 a

    2025 arXiv

  42. [50]

    Introducing visual perception token into multimodal large language model

    Runpeng Yu, Xinyin Ma, and Xinchao Wang. Introducing visual perception token into multimodal large language model. arXiv preprint arXiv:2502.17425, 2025 b

    2025 arXiv

  43. [51]

    Visrag: Vision-based retrieval-augmented generation on multi-modality documents

    Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, et al. Visrag: Vision-based retrieval-augmented generation on multi-modality documents. arXiv preprint arXiv:2410.10594, 2024

    arXiv

  44. [52]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024 arXiv

  45. [53]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  46. [54]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.