Pith. sign in

REVIEW 4 major objections 6 minor 17 references

Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Retrieval-centered chain-of-thought, generated from what the retriever actually confuses, is what lets a unified multimodal retriever correct its own mistakes and outperform query-only reasoning baselines on MMEB-V2.

desk verdict A genuinely new embedder-adviser loop for retrieval-conditioned CoT, with careful engineering and released code, but the key ablation is confounded and some headline numbers contradict the table. read the letter →

arxiv 2608.06060 v1 pith:3JN3WURI submitted 2026-08-06 cs.CV

Zelong Sun , Jun Wang , Kaicheng Yang , Tiancheng Gu , Ziyong Feng , Zhiwu Lu This is my paper
classification cs.CV
keywords multimodalretrievalretrieval-centricchain-of-thoughthardnegativesfeedbackembedder-adviserframeworkGRPOrerank-or-retrieveLVLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that effective retrieval reasoning must be grounded in retrieval feedback, not in the query alone. Existing CoT-based retrievers explain what the query describes but not what the embedder misunderstands; UniME-R1 instead has an adviser examine the initial top-k candidates and generate a Retrieval-Centric Chain-of-Thought that names the confused cues and rewrites the query to correct the retrieval direction. If the target is already in the top-k, the system reranks; otherwise it re-retrieves over the full corpus with the refined query, reusing the candidate index. The paper reports that this conditioning improves unified multimodal retrieval on MMEB-V2 (69.9 overall at 2B scale, 70.3 at 4B) and on general image/video/document benchmarks, with an ablation attributing a 2.9-point gain over query-only CoT to actual retrieval feedback. The underlying principle—diagnose failures from retrieved candidates before refining the query—is the contribution the paper is trying to establish.

What carries the argument

The load-bearing object is the Retrieval-Centric Chain-of-Thought (RC-CoT), built from two fields: <cot_focus>, a hint that names the attributes, relations, temporal events, or document details the embedder failed to encode, and <cot_answer>, a concise rewriting of the query that carries those cues. The adviser that produces RC-CoT also emits a candidate ranking and a binary path judgement, letting the system adaptively choose reranking versus re-retrieval. A dual-mode embedder uses one representation (<dis_emb>) for initial retrieval and for encoding every candidate once, and a second representation (<gen_emb>) for the RC-CoT-augmented query, so re-retrieval searches the same reusable candidate index. Training is driven by mined hard negatives that simulate realistic retrieval failures and by GRPO with four rewards that check format validity, path correctness, candidate ordering, and whether the RC-CoT actually improves re-retrieval.

What would settle it

Run UniME-R1 on a retrieval corpus whose hard negatives are crafted by an independent process, such as adversarially paraphrased captions designed to trip the embedder, and check whether RC-CoT still improves over query-only CoT; a drop to parity would show the benefit is tied to the mining pipeline rather than to retrieval-feedback reasoning itself.

Watch

Extended reading notes

Core claim

The central claim is that retrieval reasoning should be conditioned on the retriever's own failures. UniME-R1 adds an adviser that compares the query with each of the initial top-k candidates and produces a two-part Retrieval-Centric CoT: <cot_focus> summarizes which discriminative cues the embedder confuses, and <cot_answer> converts those cues into a concise retrieval-oriented description. When a match is already in the candidate set, the adviser reranks and returns the result; when no match is present, the refined query is embedded through a generative mode and used for full-corpus re-retrieval, while candidate representations stay fixed in the discriminative mode. The paper reports consistent gains over embedding-only and reasoner-embedder baselines across MMEB-V2 and several general multimodal retrieval benchmarks. An ablation attributes a 2.9-point overall gain over query-only CoT to conditioning on actual retrieved candidates rather than random or query-derived context.

Load-bearing premise

The whole pipeline assumes that hard negatives mined by an off-the-shelf embedder and filtered by an LVLM judge resemble the confusions the target embedder will actually meet at test time; if that resemblance fails, the adviser learns to diagnose benchmark-specific mistakes rather than real retrieval failures.

Editorial extensions

If this is right

  • CoT-based retrieval methods that reason from the query alone leave a measurable gap; conditioning the rationale on the actual top-k candidates becomes a reusable design choice.
  • Because candidates are encoded once with the discriminative mode, failure-aware re-retrieval does not require rebuilding the candidate index or generating reasoning for every candidate.
  • Mined hard negatives serve three roles in one pipeline: contrastive training for the embedder, supervision for the adviser, and reward computation for reinforcement learning.
  • Adaptive routing between reranking and re-retrieval avoids unnecessary full-corpus searches when the target is already present, which keeps the added reasoning cost modest.
  • Iterative retrieval feedback yields further gains, with the largest improvements on video tasks, supporting the idea that retrieval failure diagnosis can be repeated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's oracle-routing experiment suggests the largest remaining headroom is in the binary path decision; a better judge could improve results without changing the embedder or the CoT generator, a direction the paper does not pursue.
  • The same failure-diagnosis loop could be applied to retrieval-augmented generation, where a model sees retrieved passages but currently conditions its reasoning only on the question.
  • A natural stress test would replace the LVLM judge in hard-negative mining with the embedder's own similarity scores, isolating whether a separate judge is necessary or whether mining can be made self-supervised.
  • Multi-round inference in the paper shows diminishing returns after one round, implying that a learned stopping criterion for when to stop re-retrieving could be a worthwhile extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes UniME-R1, an embedder–adviser framework for unified multimodal retrieval. A dual-mode embedder first retrieves a top-k set using a discriminative query token (<dis_emb>); a retrieval-aware adviser analyzes the retrieved candidates and emits five structured fields, including a rerank list, a path-judge token, and two RC-CoT fields (<cot_focus>, <cot_answer>). If the adviser predicts the target is in the top-k, the rerank list is used; otherwise the RC-CoT is appended to the query and re-encoding with a generative token (<gen_emb>) triggers full-corpus re-retrieval. Training uses mined hard negatives for contrastive learning, SFT on teacher-generated structured annotations, and GRPO with format, judge, rerank, and re-retrieval rewards. Experiments on MMEB-V2 and several zero-shot benchmarks report consistent gains over Reasoner–Embedder and embedding-only baselines, with final overall scores of 69.9 (2B) and 70.3 (4B).

Significance. The core idea is timely and potentially significant: conditioning retrieval-time reasoning on the embedder's own confusions rather than on the query alone is a plausible route to finer-grained multimodal retrieval, and the rerank-or-retrieve routing with a reusable candidate index is a practical efficiency win. The paper ships detailed ablations, a latency analysis, per-dataset video results, and qualitative case studies, and it states that code and models are released. The central attribution, however, currently rests on an ablation that compares a trained RC-CoT branch against an untrained query-only input type, and the reward structure is coupled to the same LVLM judge that defines the hard negatives; both issues need to be addressed before the empirical claim is fully convincing.

major comments (4)
  1. [Table 3, Eq. (9), §A.4] The central claim that retrieval feedback, not query-only CoT, drives the improvement is confounded in Table 3. Equation (9) defines L_gen using the teacher-generated RC-CoT r, and Eq. (17) plus §A.4 restrict L_gen to examples in D_cot, i.e., examples with valid RC-CoT. Therefore the <gen_emb> branch was trained only on RC-CoT inputs; query-only CoT is an out-of-distribution input for that branch. The 2.9-point gap between Query-only CoT (65.6) and RC-CoT (68.5) therefore compares a seen input type against an unseen one, rather than comparing equally trained systems. A control embedder trained on query-only CoT (or on both input types) is required to attribute the gain to retrieval feedback; the Random-candidate CoT row (66.5), which is also out-of-distribution, suggests that input distribution alone can account for part of the difference.
  2. [§3.5, Eqs. (11)–(14); §A.1] The training and evaluation loop is partially coupled to the LVLM judge used to define hard negatives. In §A.1, hard negatives are selected using a(q,c) = logit(yes) - logit(no) from an LVLM judge, and the same judge scores a_i are used as the relevance ground truth for the rerank reward R_rank in Eq. (13); R_cot in Eq. (14) is evaluated against the hard-negative pool derived from that judge. As a result, improved MMEB-V2 scores may partly reflect alignment with this judge's relevance judgments rather than general retrieval quality. I ask for an independent control: recompute the main results with an alternative judge (or human labels) for reward construction, and report sensitivity to the mining threshold tau_h and the number m of mined negatives.
  3. [Eq. (2), Eq. (12), Figure 10] There is a concrete inconsistency in the label coding for <rerank_judge>. Equation (2) defines g in {0,1} for whether the target appears in the top-k set, and Eq. (12) defines the judge reward as 1[hat(g)=g]. However, the inference prompt in Figure 10 instructs the model to output -1 when no candidate truly matches. If the model emits -1 for target-out instances, the judge reward in Eq. (12) is always zero for those instances, and the SFT teacher annotations and GRPO rewards are not defined on the same label set. Please unify the coding across Eq. (2), Eq. (12), the teacher annotations, and the prompt.
  4. [§5.1, Table 3 caption] Table 3 reports an RC-CoT overall score of 68.5, which is lower than the final UniME-R1 overall of 69.9 in Table 1, yet §5.1 describes Table 3 as showing the effect of retrieval feedback. The main text says these numbers come from an early stage with a fixed adviser (Qwen3-VL-235B), but the table caption does not state this. Please clarify in the caption and in the text that Table 3 uses a zero-shot teacher adviser and a fixed generation budget, and explain how it relates to the final trained pipeline.
minor comments (6)
  1. [§4.4] The sentence 'the 2B model improves the overall score by 9.8 and 6.8 points' should specify that these gains are over UME-R1 (final overall 60.1) and TTE (final overall 63.1), respectively; the preceding sentence about gains of 3.1 and 1.4 points refers to the strongest baseline in each size group, and the two sets of references are easy to conflate.
  2. [§5.1] The text refers to a fixed adviser 'Qwen3-VL-235B'; this appears to be a typo, likely for Qwen3-VL-2B or Qwen3-VL-4B. Please verify the model designation.
  3. [Figures 6–10 and Appendix H] The repeated token 'passengers actively' appears in multiple figure captions and in the inference prompt (e.g., before 'Query:' in Figures 6–10). This looks like a formatting artifact and should be cleaned.
  4. [§A.1] The hard-negative similarity threshold tau_h is named but its numerical value is never reported; please give the value, or at least the range, and describe how it was chosen.
  5. [Appendix B.1, Table 9] Table 9 shows that Top-7 context reaches 70.5 overall, exceeding the final system's 69.9 in Table 1; since Table 9 uses a zero-shot teacher adviser, the caption should state that these numbers are not directly comparable to the main results.
  6. [Appendix D] The manuscript itself notes that routing-frequency changes 'measure aggregate calibration and do not establish whether the correct individual queries are routed.' This is an honest limitation; I encourage either per-query routing accuracy or a small error analysis to strengthen the routing claims.

Circularity Check

1 steps flagged · score 6.0 of 10

Table 3's claim that retrieval feedback is essential is built into gen_emb training: query-only CoT is an out-of-distribution input for an embedder trained exclusively on RC-CoT.

  1. fitted input called prediction [Section 5.1 / Table 3; Section 3.4 Eq. (9); Appendix A.4 Eq. (17)]
    "Query-only CoT obtains 65.6 overall, while conditioning on random candidates provides only a modest gain to 66.5. RC-CoT based on the actual top-k results reaches 68.5, improving over these variants by 2.9 and 2.0 points, respectively. ... The RC-CoT used by Lgen is generated by the annotation teacher. ... training <gen_emb> requires a reliable teacher-generated RC-CoT. We consequently apply the generative contrastive objective only to examples that pass the RC-CoT quality filters."

    The generative branch <gen_emb> is trained only on teacher-generated RC-CoT inputs (Eq. 9, Lgen restricted to Dcot in Eq. 17). Table 3 then compares RC-CoT against query-only CoT and random-candidate CoT using this same embedder. Query-only CoT and random-candidate CoT are input types <gen_emb> has never seen during training, while RC-CoT is exactly the training distribution, so the 2.9-point gap reflects input-distribution mismatch rather than the causal benefit of retrieval feedback. A fair control would train <gen_emb> on query-only CoT as well; without it, the attribution in the abstract (retrieval feedback, not query-only CoT, drives gains) is forced by construction.

full rationale

The paper's external benchmark results (Tables 1 and 2) are self-contained comparisons against strong baselines with a fully trained system, so the overall claim that UniME-R1 improves retrieval is not circular. The circularity burden sits on the internal attribution that retrieval feedback causes the gain: Section 3.4 defines Lgen using teacher RC-CoT, Appendix A.4 applies it only to RC-CoT-valid examples, and Section 5.1 uses this RC-CoT-only embedder to conclude that query-only CoT is worse. Because query-only CoT is an out-of-distribution input for <gen_emb>, the comparison is not between equally trained conditions. The same-group citations (e.g., Gu et al. 2026 for the mining pipeline) are engineering references rather than load-bearing uniqueness theorems and do not by themselves raise circularity. The LVLM judge is used as a training reward and hard-negative filter, but final evaluation is against ground truth, so that reuse is not circular. Score 6 reflects that the central attribution claim is partially circular, while the model's overall benchmark gains remain independent evidence.

Assumptions & free parameters 4 free parameters · 4 assumptions · 3 invented entities

The framework depends on several effectively free design choices that are not reported as fitted values: the hard-negative mining threshold, the number of mined negatives per query, the adviser context size whose value is shown to matter in Table 9, and the GRPO reward weights. The paper relies on the MMEB-V2 training distribution as a stand-in for general multimodal retrieval, and on an LVLM judge both to construct the hard-negative pool and to define the rerank reward. The invented entities, mainly the RC-CoT fields, the dual-mode embedder, and the rerank-or-retrieve adviser, do have falsifiable handles in the benchmark results but still represent inductive choices.

free parameters (4)
  • hard-negative similarity threshold tau_h = not reported in main text
    Used to filter potential false negatives in hard negative mining; its value governs which candidates count as hard, and robustness to this choice is not analyzed.
  • mined hard negatives per query m = not stated explicitly
    Controls the contrastive pool and candidate-set construction; the choice affects both embedder training and adviser supervision.
  • top-k for adviser context = 5 by default, with 7 reported in Table 9
    The number of retrieved candidates visible to the adviser is a design choice; the paper shows performance varies with it, so the default is effectively tuned on a validation signal.
  • reward weights lambda_f, lambda_j, lambda_r, lambda_c = not reported
    The relative weights of the four GRPO rewards are not stated; changing them would change the adviser's policy.
assumptions (4)
  • domain assumption MMEB-V2 training data transfers to general multimodal retrieval tasks including Flickr30K, COCO, ShareGPT4V, Urban1K, and UVRB.
    The embedder and adviser are trained only on MMEB-V2-derived data; the paper then reports zero-shot results on other benchmarks. The transfer premise is plausible but not demonstrated by an out-of-domain training analysis.
  • domain assumption The LVLM judge's soft relevance scores a(q,c) are reliable enough to serve both as hard-negative labels and as the ideal ranking for the rerank reward.
    The rerank reward uses LVLM judge scores as the target ranking, and the same judge builds the hard-negative pool, so the whole training signal assumes judge correctness.
  • domain assumption The GRPO rollouts are sufficiently informative after discarding samples where all eight rollouts agree.
    The paper assumes the remaining samples provide useful relative supervision and that eight rollouts suffice.
  • domain assumption Hard negatives mined by an off-the-shelf embedder and an LVLM judge match the confusions of the SFT embedder at test time.
    The training data construction and embedder-aligned GRPO data assume that failures induced by mined hard negatives approximate real retrieval failures.
invented entities (3)
  • RC-CoT with <cot_focus> and <cot_answer> fields independent evidence
    purpose: A structured query-side rationale that encodes the embedder's observed failure mode and the corrected retrieval direction.
    The paper provides ablations showing that retrieval feedback, the focus field, and the answer field all change retrieval scores, which is a falsifiable handle within the benchmark.
  • Dual-mode embedder with <dis_emb> and <gen_emb> query tokens independent evidence
    purpose: Separates the direct retrieval representation from the RC-CoT-refined representation while keeping a shared candidate index.
    The two modes are evaluated through the main results and the embedder ablation in Table 5, providing an empirical handle.
  • Adviser with rerank-or-retrieve routing independent evidence
    purpose: A separate model that reranks candidates or decides to trigger full-corpus re-retrieval with the generated RC-CoT.
    Table 4 and the GRPO ablations give measurable evidence for its contribution, including oracle routing as an upper bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval." pith.science (2026). https://pith.science/paper/3JN3WURI

@misc{pith2026260806060,
  author       = {Pith},
  title        = {Pith review of: Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3JN3WURI}},
  note         = {Machine review of arXiv:2608.06060}
}
read the original abstract

Unified multimodal retrieval aims to identify candidates that satisfy complex user intent expressed through heterogeneous inputs. Although Large Vision-Language Model (LVLM)-based retrievers are efficient and scalable, directly encoding raw multimodal inputs often misses fine-grained discriminative cues, leading to confusion among semantically similar candidates. Recent methods mitigate this limitation by generating Chain-of-Thought (CoT) rationales to enrich the query representation. However, such reasoning is typically derived from the query alone: it explains what the query describes, but not what the retriever misunderstands. We argue that effective retrieval reasoning should instead be conditioned on retrieval feedback. Based on this insight, we introduce UniME-R1, an embedder-adviser framework that learns to reason over initially retrieved candidates and generate Retrieval-Centric Chain-of-Thought (RC-CoT). The adviser analyzes candidates individually to identify the discriminative cues confused by the embedder. If the target appears in the initial top-k set, UniME-R1 directly reranks the candidates; otherwise, it generates RC-CoT to refine the retrieval direction and performs full-corpus re-retrieval with a dual-mode embedder. To train the framework, we mine hard negatives to simulate realistic retrieval failures, jointly optimize direct retrieval and RC-CoT-augmented retrieval, and align the adviser with retrieval outcomes through supervised learning and retrieval-oriented reinforcement learning. Extensive experiments on MMEB-V2 and a diverse set of general multimodal retrieval benchmarks demonstrate that UniME-R1 consistently improves retrieval performance over strong baselines.

Figures

Figures reproduced from arXiv: 2608.06060 by the authors.

Figure 1
Figure 1. Comparison of generic CoT and RC-CoT. Generic query-only CoT broadly enriches query semantics, whereas RC-CoT analyzes retrieved candidates to diagnose failures and correct the retrieval direction with targeted evidence. Applying such reasoning indiscriminately can also introduce redundant or noisy semantics when the initial retrieval is already reliable. We argue that effective retrieval reasoning should instead be… view at source ↗
Figure 2
Figure 2. Overview of UniME-R1. The model integrates a dual-mode embedder for direct and RC-CoT-enhanced retrieval with a retrieval-aware adviser that examines the initial top-k candidates and adaptively determines whether to perform reranking or full-corpus re-retrieval with RC-CoT. representation space and retrieves Rk (q) = TopKc∈C sθ (q, c), (1) where sθ (q, c) denotes query–candidate similarity. If the target c + is in R… view at source ↗
Figure 3
Figure 3. Training data construction for UniME-R1. Mined hard negatives are used to construct target-in and target-out candidate sets for supervised learning, and model-retrieved candidate sets aligned with the SFT embedder for GRPO. Joint Contrastive Training. For each training query qi , we construct a candidate pool Pi containing the target c + i , in-batch negatives, and m mined hard negatives. Given a query representatio… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Routing calibration before and after GRPO. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 6
Figure 6. Figure 6: Case study of adviser-generated RC-CoT. Candidate-level analysis identifies corrective [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Case study of adviser-guided reranking. Candidate-level relevance analysis separates the [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Comparison of adviser-generated RC-CoT before and after RL. RL yields a more concise [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Comparison of adviser-generated reranking before and after RL. RL improves alignment [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Comparison of adviser-generated reranking before and after RL. RL improves alignment [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 12 linked inside Pith

  1. [1]

    Moca: Modality-aware continual pre-training makes better bidirectional multimodal embeddings.arXiv preprint arXiv:2506.23115,

    Haonan Chen, Hong Liu, Yuping Luo, Liang Wang, Nan Yang, Furu Wei, and Zhicheng Dou. Moca: Modality-aware continual pre-training makes better bidirectional multimodal embeddings.arXiv preprint arXiv:2506.23115,

    arXiv

  2. [6]

    Embed-rl: Reinforcement learning for reasoning-driven multimodal embeddings.arXiv preprint arXiv:2602.13823,

    Haonan Jiang, Yuji Wang, Yongjie Zhu, Xin Lu, Wenyu Qin, Meng Wang, Pengfei Wan, and Yansong Tang. Embed-rl: Reinforcement learning for reasoning-driven multimodal embeddings.arXiv preprint arXiv:2602.13823,

    arXiv

  3. [7]

    Vlm2vec: Training vision-language models for massive multimodal embedding tasks.arXiv preprint arXiv:2410.05160,

    Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. Vlm2vec: Training vision-language models for massive multimodal embedding tasks.arXiv preprint arXiv:2410.05160,

    arXiv

  4. [8]

    Glint Lab Fanheng Kong, Jingyuan Zhang, Yahui Liu, Hongzhi Zhang, Shi Feng, Xiaocui Yang, Daling Wang, Yu Tian, Fuzheng Zhang, Guorui Zhou, et al

    24 Preprint. Glint Lab Fanheng Kong, Jingyuan Zhang, Yahui Liu, Hongzhi Zhang, Shi Feng, Xiaocui Yang, Daling Wang, Yu Tian, Fuzheng Zhang, Guorui Zhou, et al. Modality curation: Building universal embeddings for advanced multimodal information retrieval.arXiv preprint arXiv:2505.19650,

    arXiv

  5. [9]

    Ume-r1: Exploring reasoning- driven generative multimodal embeddings.arXiv preprint arXiv:2511.00405,

    Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, and Jinsong Su. Ume-r1: Exploring reasoning- driven generative multimodal embeddings.arXiv preprint arXiv:2511.00405,

    arXiv

  6. [11]

    Vlm2vec-v2: Advancing multimodal embedding for videos, images, and visual documents.arXiv preprint arXiv:2507.04590,

    Rui Meng, Ziyan Jiang, Ye Liu, Mingyi Su, Xinyi Yang, Yuepeng Fu, Can Qin, Zeyuan Chen, Ran Xu, Caiming Xiong, et al. Vlm2vec-v2: Advancing multimodal embedding for videos, images, and visual documents.arXiv preprint arXiv:2507.04590,

    arXiv

  7. [13]

    Videoclip-xl: Ad- vancing long description understanding for video clip models.arXiv preprint arXiv:2410.00741, 2024a

    Jiapeng Wang, Chengyu Wang, Kunzhe Huang, Jun Huang, and Lianwen Jin. Videoclip-xl: Ad- vancing long description understanding for video clip models.arXiv preprint arXiv:2410.00741, 2024a. Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for mu...

    arXiv

  8. [14]

    Internvideo2: Scaling foundation models for multimodal video understanding

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, et al. Internvideo2: Scaling foundation models for multimodal video understanding. InECCV, pp. 396–416, 2024b. Mingjun Xu, Jinhan Dong, Jue Hou, Zehui Wang, Sihang Li, Zhifeng Gao, Renxin Zhong, and Hengxing Cai. Mm-r5: Multimodal reasoning-en...

    arXiv

Show all 17 references
  1. [15]

    Glint Lab An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al

    25 Preprint. Glint Lab An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    arXiv

  2. [16]

    thinking with images

    Hao Yu, Zhuokai Zhao, Shen Yan, Lukasz Korycki, Jianyu Wang, Baosheng He, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, and Hanchao Yu. Cafe: Unifying representation and generation with contrastive-autoregressive finetuning. InICCV, pp. 6286–6297, 2025a. Shi Yu, Chaoyue Tang, Bokai Xu...

    2025 arXiv

  3. [17]

    Megapairs: Massive data synthesis for universal multimodal retrieval.arXiv preprint arXiv:2412.14475,

    Junjie Zhou, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, Defu Lian, and Yongping Xiong. Megapairs: Massive data synthesis for universal multimodal retrieval.arXiv preprint arXiv:2412.14475,

    arXiv

  4. [2021]

    Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models.arXiv preprint arXiv:2402.03300,

    arXiv

  5. [2022]

    Rzenembed: Towards comprehensive multimodal retrieval.arXiv preprint arXiv:2510.27350,

    Weijian Jian, Yajun Zhang, Dawei Liang, Chunyu Xie, Yixiao He, Dawei Leng, and Yuhui Yin. Rzenembed: Towards comprehensive multimodal retrieval.arXiv preprint arXiv:2510.27350,

  6. [2023]

    Rematch: Boosting representation through matching for multimodal retrieval.arXiv preprint arXiv:2511.19278, 2025a

    Qianying Liu, Xiao Liang, Zhiqiang Zhang, Zhongfei Qing, Fengfan Zhou, Yibo Chen, Xu Tang, Yao Hu, and Paul Henderson. Rematch: Boosting representation through matching for multimodal retrieval.arXiv preprint arXiv:2511.19278, 2025a. Yikun Liu, Yajie Zhang, Jiayin Cai, Xiaolon...

  7. [2024]

    Think then embed: Generative context improves multimodal embedding.arXiv preprint arXiv:2510.05014,

    Xuanming Cui, Jianpeng Cheng, Hong-you Chen, Satya Narayan Shukla, Abhijeet Awasthi, Xichen Pan, Chaitanya Ahuja, Shlok Kumar Mishra, Yonghuan Yang, Jun Xiao, et al. Think then embed: Generative context improves multimodal embedding.arXiv preprint arXiv:2510.05014,

  8. [2025]

    Colpali: Efficient document retrieval with vision language models, 2024.URL https://arxiv

    Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, Céline Hudelot, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models, 2024.URL https://arxiv. org/abs/2407.01449. Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier...

    2024 arXiv

  9. [2026]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025a

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025a. Daya Guo, Dejian Yang, Haowe...

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.