Pith. sign in

REVIEW 4 major objections 4 minor 4 cited by

Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training pipeline built entirely from unlabeled images and an AI captioner outperforms all prior zero-shot composed image retrieval baselines on FashionIQ, CIRR, and CIRCO.

desk verdict The method is a plausible and genuinely new way to build CIR training triplets from unlabeled images, but the headline gains are inflated because the only curation hyperparameters were tuned on the same test benchmarks used for the final numbers. read the letter →

arxiv 2505.19952 v1 pith:Q2YGX2JK submitted 2025-05-26 cs.CV cs.IR

Rong-Cheng Tu , Wenhao Sun , Hanzhe You , Yingjie Wang , Jiaxing Huang , Li Shen , Dacheng Tao This is my paper · ORCID
classification cs.CVcs.IR
keywords zero-shotcomposedimageretrievalmultimodalreasoningagenttripletconstructionunlabeledimagescontrastivelearningInfoNCEQ-Formerlargelanguagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Zero-shot composed image retrieval asks a model to find a target image from a reference image plus a short text describing the change. The paper's claim is that this task can be learned without any annotated triplets: sample pairs of unlabeled images that are moderately similar, ask a multimodal reasoning agent to caption both and then write the modification text, and fine-tune a vision-language model so the composed query embedding matches the target image embedding. Trained this way on 10k unlabeled ImageNet images, the model reaches 37.85 average R@10 on FashionIQ, 37.98 R@1 on CIRR, and 27.14 mAP@5 on CIRCO, ahead of the strongest zero-shot baselines in each case. If correct, this removes the need for expensive manual annotation and avoids the accumulated errors of approaches that route queries through generated target texts.

What carries the argument

The load-bearing object is the multimodal reasoning agent (MRA), a multimodal large language model that first captions each member of an image pair and then writes a command describing how the reference becomes the target. The second mechanism is moderate-similarity target selection: a BLIP-2 encoder computes token-level maximum-cosine similarities between all image pairs, and the target is drawn uniformly from ranks 51–60, balancing trivial image-to-image matching against pure text-to-image retrieval. The third is the learning objective: a Q-Former compresses the composed query and the candidate target into token embeddings, similarity is the average over query tokens of the maximum cosine similarity to target tokens, and an InfoNCE loss pulls matched pairs together. The theoretical part of the paper shows that under a bijective token-alignment assumption, this token-max loss is a lower bound of the standard InfoNCE loss, so training it implicitly optimizes the standard objective.

What would settle it

Use the exact same pipeline but replace each MRA-written modification text with a caption of an unrelated image from the same training set, then re-fine-tune and evaluate on FashionIQ; if the retrieval scores stay near 37.85 average R@10 rather than collapsing, the claimed mechanism of text-guided triplet supervision is not what drives the result.

Watch

Extended reading notes

Core claim

The central claim is that the text-anchored training used by recent zero-shot CIR methods is unnecessary: training directly on triplets of the form <reference image, modification text, target image> built from unlabeled images gives better retrieval than training on generated target texts. The empirical evidence is that MRA-CIR reaches an average R@10 of 37.85 on FashionIQ against 30.3 for the strongest prior baseline, an R@1 of 37.98 on CIRR against 28.3, and an mAP@5 of 27.14 on CIRCO against 18.57 for CIReVL and 17.67 for MCL. The paper further argues that the token-level maximum-cosine InfoNCE loss used to pair queries with targets is a lower bound of the standard InfoNCE loss, providing theoretical support for the training objective.

Load-bearing premise

The load-bearing premise is that the modification texts produced by the multimodal agent truthfully and completely describe how the reference image must change into the target image, so the synthetic triplets carry accurate supervision.

Editorial extensions

If this is right

  • Manual annotation becomes optional: any unlabeled image collection can be converted into CIR training data through pairing and captioning.
  • The intermediate-text bottleneck disappears, so errors in query-to-text and text-to-image mappings no longer accumulate before retrieval.
  • The moderate-similarity selection rule gives a concrete recipe: targets neither so close that the text is trivial nor so far that the text becomes a caption.
  • Token-level maximum-cosine InfoNCE can be used as a drop-in contrastive objective with the same convergence guarantees as standard InfoNCE.
  • Retrieval improves specifically on fine-grained attributes such as colors, patterns, and small scene changes across fashion and natural-image benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would apply the same triplet-construction recipe to other pairwise visual tasks, such as image editing direction learning, visual analogy, or change captioning, where a transformation between two examples can stand in for a label.
  • The rank window 51–60 was chosen from sensitivity curves on the three evaluation datasets; on a different image collection the optimal window could shift, so treating it as a tunable per-domain parameter would be a safer reading of the paper.
  • The method's ceiling is tied to the reasoning agent: a stronger MLLM should produce cleaner modification texts and improve retrieval, while a weaker one should degrade it, making the choice of MRA a testable lever rather than a fixed component.
  • The lower-bound argument suggests that any token-level max-similarity contrastive loss, beyond the specific Q-Former setup used here, can be justified as a principled surrogate for standard InfoNCE.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes MRA-CIR, a zero-shot composed image retrieval (CIR) framework that constructs synthetic training triplets <reference image, modification text, target image> from unlabeled images alone. Reference-target pairs are selected at a moderate similarity rank (controlled by hyper-parameters q1 and q2), and a multimodal LLM (MiniCPM-VL-2_6) generates the modification text in a two-step caption-then-diff process. A Q-Former-based BLIP-2 model is then fine-tuned with a token-level max-cosine InfoNCE loss. Experiments on FashionIQ, CIRR, and CIRCO report large gains over prior zero-shot CIR methods, and the appendix provides a theoretical lower-bound argument connecting the proposed loss to standard InfoNCE.

Significance. If the reported results are reproducible under a genuinely fixed zero-shot protocol, the paper would make a valuable contribution: it shows that direct query-to-target alignment on automatically curated triplets can replace text-anchored training, potentially reducing error propagation in ZS-CIR. The idea of selecting moderately similar image pairs before generating modification text is simple and plausible, and the ablation study in Section 4.3 supports the importance of this design. However, the central empirical claim is weakened by the way q1 and q2 are selected on the test benchmarks themselves, and the absence of variance information or code prevents an independent check of the reported margins. The theoretical analysis is also built on a very strong assumption that is close to the desired conclusion.

major comments (4)
  1. [Section 4.4 / Figure 2 / Section 4.1.2] The hyper-parameters q1 and q2, which control the similarity rank range for target selection, are chosen by sweeping the range and observing performance on FashionIQ, CIRR, and CIRCO in Figure 2, and the paper states that [51,60] 'produces consistently great retrieval results across FashionIQ, CIRR, and CIRO.' These are the same three evaluation sets reported in Tables 1-3. This constitutes post-hoc tuning on the test benchmarks, so the reported numbers are selected rather than predicted, and the 'zero-shot' claim in Section 4.1.2 is undercut. To restore the zero-shot claim, the authors should choose q1 and q2 on a separate validation set (or with a pre-registered rule) and/or report results across the full range of ranks without singling out the best one.
  2. [Section 4.1.2 / Tables 1-3] The manuscript states that each experiment is repeated three times with different random seeds, but only the means are reported and no standard deviations, confidence intervals, or per-seed numbers are given. Without information about run-to-run variance, it is impossible to assess whether the reported margins over baselines (e.g., 37.85 versus 30.3 average R@10 on FashionIQ) are statistically meaningful. The paper also does not release code or the curated triplets, which makes independent verification difficult.
  3. [Section 3.1 / Appendix A] The entire training signal comes from modification texts generated by the MLLM, and Appendix A openly admits that 'any misinterpretation of the image's semantic content by the MLLM can lead to erroneous modification text' and that such errors propagate inaccuracies through the retrieval pipeline. Since this is the sole supervision source, the paper should provide some evidence of triplet quality, for instance a small human evaluation of the generated modification texts or an automated measure of their agreement with the actual visual difference, to justify the claim that the curated triplets are 'high-quality.' Without such evidence, the strong empirical gains are hard to attribute to a reliable training signal rather than to particular MLLM outputs.
  4. [Section 3.2 / Proposition 1 / Corollary 1] The theoretical justification of the token-level max-cosine InfoNCE loss rests on Assumption 3.1 (Eq. 6), which postulates the existence of a bijective token-level alignment between the composed query and the matched target after sufficient training. This is essentially the property that the contrastive loss is intended to produce, so the argument is close to circular. Corollary 1 then depends on unverified constants p1 and p2, making the bound in Eq. (9) of limited practical content. The authors should either reframe this analysis as a heuristic motivation, or provide empirical evidence that the assumed bijective alignment actually emerges after training.
minor comments (4)
  1. [Table 3] The row for the proposed method is labeled 'MIR-CIR' in Table 3, but the method is called MRA-CIR elsewhere; please correct the label and ensure all table entries use the same name.
  2. [Section 4.2, observation (1)] The text says that on CIRCO the method outperforms CIReVL and MCL by 8.57% and 9.47% 'in R@10,' but CIRCO is evaluated with mAP@K and the cited numbers (18.57 and 17.67) are mAP@5 values from Table 3; please correct the metric name.
  3. [Section 4.4 / Figure 2] The dataset name 'CIRCO' is consistently misspelled as 'CIRO' in Section 4.4 and in Figure 2; please correct this.
  4. [Table 4] The header 'FashinIQ-Dress' contains a typo and should read 'FashionIQ-Dress'; the same typo appears in the table caption.

Circularity Check

1 steps flagged · score 5.0 of 10

Zero-shot results are partly fitted: the target-selection window q1/q2 was chosen by sweeping the same test benchmarks that produce the headline numbers.

  1. fitted input called prediction [Section 4.4 (RQ3, Figure 2) and Section 4.1.2 (Implementation Details)]
    "In particular, picking images ranked between the 51st and 60th most similar produces consistently great retrieval results across FashionIQ, CIRR, and CIRO. Therefore, we set q1 = 51 and q2 = 60 as 51 and 60 in other experiments."

    The headline zero-shot results in Tables 1-3 are obtained using q1=51 and q2=60. Section 4.4 shows that this window was selected by sweeping the similarity rank range on the exact same evaluation datasets (FashionIQ, CIRR, CIRCO) and choosing the range that 'produces consistently great retrieval results' there. Thus the reported benchmark numbers are selected on the test benchmarks rather than predicted by a fixed zero-shot pipeline: the key data-curation hyperparameter is fitted to the target quantity (retrieval accuracy on those benchmarks) and then presented as zero-shot performance. This is test-set model selection, so the central empirical claim of zero-shot generalization is partially constructed by evaluation feedback from the very benchmarks used to report the gains.

full rationale

The main circularity concern is the q1/q2 choice. The paper tunes the only significant data-curation hyperparameters on the three evaluation benchmarks and then reports results on those same benchmarks as zero-shot, which compromises the strength of the headline 'prediction.' I do not count the Appendix A limitation about MLLM misinterpretation as circularity: that is a data-quality limitation, not a derivation loop. The theoretical lower-bound claim in Proposition 1 is also self-contained: under Assumption 3.1, the inequality L <= Ls follows algebraically from replacing a sum of maxima with a sum of chosen permutation terms, and Corollary 1 bounds the gap under stated thresholds. That analysis does not assume the empirical result it purports to justify. The paper's method itself, independent of the q1/q2 leakage, is a legitimate direct-triplet construction, so the circularity is partial rather than total.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central method rests on three unproved inputs: the domain transferability of the unlabeled ImageNet subset, the faithfulness of MLLM-generated text, and a strong trained-model alignment assumption used only in the theoretical analysis. The two hyperparameters q1 and q2 are effectively fitted to the benchmark test sets.

free parameters (1)
  • q1, q2 (target similarity rank range) = q1=51, q2=60
    Chosen by sweeping rank ranges on the three evaluation benchmarks (Figure 2), then fixed for all final results. This makes part of the reported zero-shot performance a fitted quantity.
assumptions (4)
  • domain assumption Unlabeled ImageNet subset is representative of target CIR domains
    Section 4.1.2: fine-tune on 10k unlabeled images from ImageNet-1k and evaluate on FashionIQ/CIRR/CIRCO without further tuning; the transfer is assumed.
  • domain assumption MLLM-generated captions and modification texts are accurate
    Section 3.1, Eq. (2)-(3); acknowledged in Appendix A as a limitation. The entire method depends on this faithfulness.
  • ad hoc to paper Assumption 3.1: after sufficient training, a bijective token-level alignment exists
    Section 3.2, Eq. (6); used to prove that the token-level max-cosine InfoNCE loss is a lower bound of standard InfoNCE. It is a strong assumption about the trained model, introduced solely for the theoretical claim.
  • standard math ETF convergence results of mini-batch InfoNCE apply here
    Appendix E Lemma 1, citing [46,47]; used to claim neural collapse properties, not central to the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval." pith.science (2026). https://pith.science/paper/Q2YGX2JK

@misc{pith2026250519952,
  author       = {Pith},
  title        = {Pith review of: Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2YGX2JK}},
  note         = {Machine review of arXiv:2505.19952}
}
read the original abstract

Zero-Shot Composed Image Retrieval (ZS-CIR) aims to retrieve target images given a compositional query, consisting of a reference image and a modifying text-without relying on annotated training data. Existing approaches often generate a synthetic target text using large language models (LLMs) to serve as an intermediate anchor between the compositional query and the target image. Models are then trained to align the compositional query with the generated text, and separately align images with their corresponding texts using contrastive learning. However, this reliance on intermediate text introduces error propagation, as inaccuracies in query-to-text and text-to-image mappings accumulate, ultimately degrading retrieval performance. To address these problems, we propose a novel framework by employing a Multimodal Reasoning Agent (MRA) for ZS-CIR. MRA eliminates the dependence on textual intermediaries by directly constructing triplets, <reference image, modification text, target image>, using only unlabeled image data. By training on these synthetic triplets, our model learns to capture the relationships between compositional queries and candidate images directly. Extensive experiments on three standard CIR benchmarks demonstrate the effectiveness of our approach. On the FashionIQ dataset, our method improves Average R@10 by at least 7.5\% over existing baselines; on CIRR, it boosts R@1 by 9.6\%; and on CIRCO, it increases mAP@5 by 9.5\%.

Figures

Figures reproduced from arXiv: 2505.19952 by the authors.

Figure 1
Figure 1. The illustration of (a) automatic triplets generation and (b) the framework of MRA-CIR. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Sensitivity analysis on the similarity ranking range [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Caption based modification text generation Prompt template [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Image captioning Prompt template Pc. # Task Description You are an expert in image understanding and modification. Given image 1 and image 2, your task is to generate a clear and concise modification instruction that, when applied to image 1, will make it visually rese…
Figure 5
Figure 5. Figure 5: Directly modification text generation Prompt template [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: The examples of triplets curated by MRA based on unlabeled images. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Retrieved results on the FashionIQ dataset. The target image is marked with the red box. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Retrieved results on the CIRCO dataset. The target images are marked with the red box. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Retrieved results on the CIRR dataset. The target image is marked with the red box. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories

    cs.CV 2026-02 conditional novelty 6.5 of 10

    The paper reframes image retrieval as agentic exploration over personal visual histories and shows the best tested multimodal agent scores only 28.7 exact match on its new DISBench benchmark.

  2. DeliCIR: Memory-Guided Test-Time Deliberation via Multi-Agent Collaboration for Composed Image Retrieval

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Proposes PDF, a hierarchical multi-agent Perception-to-Deliberation Framework that adds experience self-evolution and test-time scaling to composed image retrieval, claiming SOTA on CIRR, CIRCO, and FashionIQ.

  3. CoVR-R:Reason-Aware Composed Video Retrieval

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Zero-shot LMM reasoning over edit after-effects (states, phases, camera, tempo) plus a new CoVR-R benchmark yields large recall gains on implicit-effect composed video retrieval without task-specific training.

  4. SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding

    cs.CV 2025-06 conditional novelty 6.0 of 10

    SPAZER, a VLM-driven agent, combines 3D rendered views with 2D camera images in a progressive pipeline to achieve state-of-the-art zero-shot 3D visual grounding.

Reference graph

Works this paper leans on

60 extracted references · 26 canonical work pages · cited by 4 Pith papers

  1. [1]

    Distribution consistency guided hashing for cross-modal retrieval

    Yuan Sun, Kaiming Liu, Yongxiang Li, Zhenwen Ren, Jian Dai, and Dezhong Peng. Distribution consistency guided hashing for cross-modal retrieval. In Jianfei Cai, Mohan S. Kankanhalli, Balakrishnan Prabhakaran, Susanne Boll, Ramanathan Subramanian, Liang Zheng, Vivek K. Singh, Pablo César, Lexing Xie, and Dong Xu, editors,Proceedings of the 32nd ACM Inter- ...

    arXiv 2024

  2. [2]

    Similarity transitivity broken-aware multi-modal hashing.IEEE Trans

    Rong-Cheng Tu, Xian-Ling Mao, Jinyu Liu, Yatai Ji, Wei Wei, and Heyan Huang. Similarity transitivity broken-aware multi-modal hashing.IEEE Trans. Knowl. Data Eng., 36(11):7003– 7014, 2024. doi: 10.1109/TKDE.2024.3396492. URL https://doi.org/10.1109/TKDE. 2024.3396492

    arXiv 2024

  3. [3]

    Data-aware proxy hashing for cross-modal retrieval

    Rong-Cheng Tu, Xian-Ling Mao, Wenjin Ji, Wei Wei, and Heyan Huang. Data-aware proxy hashing for cross-modal retrieval. In Hsin-Hsi Chen, Wei-Jou (Edward) Duh, Hen-Hsen Huang, Makoto P. Kato, Josiane Mothe, and Barbara Poblete, editors,Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 202...

    arXiv 2023

  4. [4]

    Where does the performance improvement come from?: - A reproducibility concern about image-text retrieval

    Jun Rao, Fei Wang, Liang Ding, Shuhan Qi, Yibing Zhan, Weifeng Liu, and Dacheng Tao. Where does the performance improvement come from?: - A reproducibility concern about image-text retrieval. In Enrique Amigó, Pablo Castells, Julio Gonzalo, Ben Carterette, J. Shane Culpepper, and Gabriella Kazai, editors,SIGIR ’22: The 45th International ACM SIGIR Confere...

    arXiv 2022

  5. [5]

    Com- posing text and image for image retrieval - an empirical odyssey

    Nam V o, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. Com- posing text and image for image retrieval - an empirical odyssey. InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 6439–6448. Computer Vision Foundation / IEEE, 2019. doi: 10.1109/CVPR.2019. 00660. URL...

    doi:10.1109/cvpr.2019 2019

  6. [6]

    Sim- ple but effective raw-data level multimodal fusion for composed image retrieval

    Haokun Wen, Xuemeng Song, Xiaolin Chen, Yinwei Wei, Liqiang Nie, and Tat-Seng Chua. Sim- ple but effective raw-data level multimodal fusion for composed image retrieval. In Grace Hui 10 Yang, Hongning Wang, Sam Han, Claudia Hauff, Guido Zuccon, and Yi Zhang, editors,Proceed- ings of the 47th International ACM SIGIR Conference on Research and Development i...

    work page 2024

  7. [7]

    Dual-path semantic construction network for composed query-based image retrieval

    Shenshen Li. Dual-path semantic construction network for composed query-based image retrieval. In Ioannis Kompatsiaris, Jiebo Luo, Nicu Sebe, Angela Yao, Vasileios Mazaris, Symeon Papadopoulos, Adrian Popescu, and Zi Helen Huang, editors,Proceedings of the 2023 ACM International Conference on Multimedia Retrieval, ICMR 2023, Thessaloniki, Greece, June 12-...

    arXiv 2023

  8. [8]

    Multi-modal transformer with global-local alignment for composed query image retrieval.IEEE Trans

    Yahui Xu, Yi Bin, Jiwei Wei, Yang Yang, Guoqing Wang, and Heng Tao Shen. Multi-modal transformer with global-local alignment for composed query image retrieval.IEEE Trans. Multim., 25:8346–8357, 2023. doi: 10.1109/TMM.2023.3235495. URL https://doi.org/ 10.1109/TMM.2023.3235495

    work page arXiv 2023

Show all 60 references
  1. [9]

    Sentence-level prompts benefit composed image retrieval.arXiv preprint arXiv:2310.05473, 2023

    Yang Bai, Xinxing Xu, Yong Liu, Salman Khan, Fahad Khan, Wangmeng Zuo, Rick Siow Mong Goh, and Chun-Mei Feng. Sentence-level prompts benefit composed image retrieval.arXiv preprint arXiv:2310.05473, 2023

    2023 arXiv

  2. [10]

    Newsam, and Kofi Boakye

    Yuxin Tian, Shawn D. Newsam, and Kofi Boakye. Image search with text feedback by additive attention compositional learning.CoRR, abs/2203.03809, 2022. doi: 10.48550/ARXIV .2203. 03809. URLhttps://doi.org/10.48550/arXiv.2203.03809

    2022 doi arXiv

  3. [11]

    Image retrieval on real-life images with pre-trained vision-and-language models

    Zheyuan Liu, Cristian Rodriguez Opazo, Damien Teney, and Stephen Gould. Image retrieval on real-life images with pre-trained vision-and-language models. In2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 210...

    2021

  4. [12]

    Composed image retrieval using contrastive learning and task-oriented clip-based features.ACM Trans

    Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. Composed image retrieval using contrastive learning and task-oriented clip-based features.ACM Trans. Multim. Comput. Commun. Appl., 20(3):62:1–62:24, 2024. doi: 10.1145/3617597. URL https: //doi.org/10.1...

    2024 doi

  5. [13]

    Fine- grained textual inversion network for zero-shot composed image retrieval

    Haoqiang Lin, Haokun Wen, Xuemeng Song, Meng Liu, Yupeng Hu, and Liqiang Nie. Fine- grained textual inversion network for zero-shot composed image retrieval. In Grace Hui Yang, Hongning Wang, Sam Han, Claudia Hauff, Guido Zuccon, and Yi Zhang, editors,Proceedings of the 47th I...

    2024

  6. [14]

    Pic2word: Mapping pictures to words for zero-shot composed image retrieval

    Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, and Tomas Pfister. Pic2word: Mapping pictures to words for zero-shot composed image retrieval. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, ...

    2023

  7. [15]

    MLLM-I2W: harnessing multimodal large language model for zero-shot composed image retrieval

    Tong Bao, Che Liu, Derong Xu, Zhi Zheng, and Tong Xu. MLLM-I2W: harnessing multimodal large language model for zero-shot composed image retrieval. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, Proceedings ...

    2025

  8. [16]

    Kankanhalli

    Wei Li, Hehe Fan, Yongkang Wong, Yi Yang, and Mohan S. Kankanhalli. Improving context understanding in multimodal large language models via multimodal composition learning. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21- 27, 20...

    2024

  9. [17]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervi- sion. In Marina Meila an...

    2021

  10. [18]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. BLIP-2: bootstrapping language- image pre-training with frozen image encoders and large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, edit...

    2023

  11. [19]

    Seeing what you miss: Vision-language pre-training with semantic completion learning

    Yatai Ji, Rongcheng Tu, Jie Jiang, Weijie Kong, Chengfei Cai, Wenzhe Zhao, Hongfa Wang, Yujiu Yang, and Wei Liu. Seeing what you miss: Vision-language pre-training with semantic completion learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, V...

    2023

  12. [20]

    Global and local semantic completion learning for vision-language pre-training.CoRR, abs/2306.07096, 2023

    Rong-Cheng Tu, Yatai Ji, Jie Jiang, Weijie Kong, Chengfei Cai, Wenzhe Zhao, Hongfa Wang, Yujiu Yang, and Wei Liu. Global and local semantic completion learning for vision-language pre-training.CoRR, abs/2306.07096, 2023. doi: 10.48550/ARXIV .2306.07096. URL https: //doi.org/10...

    2023 doi arXiv

  13. [21]

    Zero-shot composed text- image retrieval

    Yikun Liu, Jiangchao Yao, Ya Zhang, Yanfeng Wang, and Weidi Xie. Zero-shot composed text- image retrieval. In34th British Machine Vision Conference 2023, BMVC 2023, Aberdeen, UK, November 20-24, 2023, page 381. BMV A Press, 2023. URLhttp://proceedings.bmvc2023. org/381/

    2023

  14. [22]

    Compo- sitional image retrieval via instruction-aware contrastive learning.CoRR, abs/2412.05756, 2024

    Wenliang Zhong, Weizhi An, Feng Jiang, Hehuan Ma, Yuzhi Guo, and Junzhou Huang. Compo- sitional image retrieval via instruction-aware contrastive learning.CoRR, abs/2412.05756, 2024. doi: 10.48550/ARXIV .2412.05756. URLhttps://doi.org/10.48550/arXiv.2412.05756

    2024 doi arXiv

  15. [23]

    Llama: Open and efficient foundation language models.CoRR, abs/2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation langua...

    2023 doi arXiv

  16. [24]

    Yu, and Ming Zhang

    Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, Rongcheng Tu, Xiao Luo, Wei Ju, Zhiping Xiao, Yifan Wang, Meng Xiao, Chenwu Liu, Jingyang Yuan, Shichang Zhang, Yiqiao Jin, Fan Zhang, Xian Wu, Hanqing Zhao...

    2025 doi arXiv

  17. [25]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information...

    2023

  18. [26]

    Spagent: Adaptive task decomposition and model selection for general video generation and editing

    Rong-Cheng Tu, Wenhao Sun, Zhao Jin, Jingyi Liao, Jiaxing Huang, and Dacheng Tao. Spagent: Adaptive task decomposition and model selection for general video generation and editing. CoRR, abs/2411.18983, 2024. doi: 10.48550/ARXIV .2411.18983. URL https://doi.org/ 10.48550/arXiv...

    2024 doi arXiv

  19. [27]

    Grounding language models to images for multimodal inputs and outputs

    Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and outputs. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, 12 Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors,International Conference on Machine...

    2023

  20. [28]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, ...

    2020

  21. [29]

    Minicpm-v: A GPT-4V level MLLM on your phone.CoRR, abs/2408.01800,

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. M...

    arXiv

  22. [30]

    Representation learning with contrastive predictive coding.CoRR, abs/1807.03748, 2018

    Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.CoRR, abs/1807.03748, 2018. URL http://arxiv.org/abs/1807.03748

    2018 arXiv

  23. [31]

    URL https://doi.org/10.48550/arXiv.2408

    doi: 10.48550/ARXIV .2408.01800. URL https://doi.org/10.48550/arXiv.2408. 01800

    doi arXiv

  24. [32]

    Weighted gaussian loss based hamming hashing

    Rong-Cheng Tu, Xian-Ling Mao, Cihang Kong, Zihang Shao, Ze-Lin Li, Wei Wei, and Heyan Huang. Weighted gaussian loss based hamming hashing. In Heng Tao Shen, Yueting Zhuang, John R. Smith, Yang Yang, Pablo César, Florian Metze, and Balakrishnan Prabhakaran, editors, MM ’21: ACM...

    2021

  25. [33]

    Unsupervised hashing with semantic concept mining.Proc

    Rong-Cheng Tu, Xian-Ling Mao, Kevin Qinghong Lin, Chengfei Cai, Weize Qin, Wei Wei, Hongfa Wang, and Heyan Huang. Unsupervised hashing with semantic concept mining.Proc. ACM Manag. Data, 1(1):3:1–3:19, 2023. doi: 10.1145/3588683. URLhttps://doi.org/10. 1145/3588683

    2023 doi

  26. [34]

    Unsupervised cross-modal hashing with modality-interaction.IEEE Trans

    Rong-Cheng Tu, Jie Jiang, Qinghong Lin, Chengfei Cai, Shangxuan Tian, Hongfa Wang, and Wei Liu. Unsupervised cross-modal hashing with modality-interaction.IEEE Trans. Circuits Syst. Video Technol., 33(9):5296–5308, 2023. doi: 10.1109/TCSVT.2023.3251395. URL https://doi.org/10....

    2023

  27. [35]

    Partial-softmax loss based deep hashing

    Rong-Cheng Tu, Xian-Ling Mao, Jia-Nan Guo, Wei Wei, and Heyan Huang. Partial-softmax loss based deep hashing. In Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia, editors,WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 20...

    2021

  28. [36]

    Unsupervised cross-modal hashing via semantic text mining.IEEE Trans

    Rong-Cheng Tu, Xian-Ling Mao, Qinghong Lin, Wenjin Ji, Weize Qin, Wei Wei, and Heyan Huang. Unsupervised cross-modal hashing via semantic text mining.IEEE Trans. Multim., 25: 8946–8957, 2023. doi: 10.1109/TMM.2023.3243608. URL https://doi.org/10.1109/TMM. 2023.3243608

    2023

  29. [37]

    Deep cross-modal proxy hashing.IEEE Trans

    Rong-Cheng Tu, Xian-Ling Mao, Rongxin Tu, Bin-Bin Bian, Chengfei Cai, Hongfa Wang, Wei Wei, and Heyan Huang. Deep cross-modal proxy hashing.IEEE Trans. Knowl. Data Eng., 35 (7):6798–6810, 2023. doi: 10.1109/TKDE.2022.3187023. URL https://doi.org/10.1109/ TKDE.2022.3187023

    2023

  30. [38]

    Knowledge-enhanced dual-stream zero- shot composed image retrieval

    Yucheng Suo, Fan Ma, Linchao Zhu, and Yi Yang. Knowledge-enhanced dual-stream zero- shot composed image retrieval. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 26941–26952. IEEE,

    2024

  31. [39]

    Zero-shot composed image retrieval with textual inversion

    Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Alberto Del Bimbo. Zero-shot composed image retrieval with textual inversion. InIEEE/CVF International Conference on 13 Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 15292–15301. IEEE,

    2023

  32. [41]

    Target-guided composed image retrieval

    Haokun Wen, Xian Zhang, Xuemeng Song, Yinwei Wei, and Liqiang Nie. Target-guided composed image retrieval. InProceedings of the ACM International Conference on Multimedia, pages 915–923. ACM, 2023

    2023

  33. [43]

    Enhance composed image retrieval via multi-level collaborative localization and semantic activeness perception

    Gangjian Zhang, Shikui Wei, Huaxin Pang, Shuang Qiu, and Yao Zhao. Enhance composed image retrieval via multi-level collaborative localization and semantic activeness perception. IEEE Trans. Multim., 26:916–928, 2024. doi: 10.1109/TMM.2023.3273466. URL https: //doi.org/10.1109...

    2024

  34. [44]

    Data roaming and quality assessment for composed image retrieval

    Matan Levy, Rami Ben-Ari, Nir Darshan, and Dani Lischinski. Data roaming and quality assessment for composed image retrieval. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors,Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Si...

    2024

  35. [45]

    Internvl: Scaling up vision foundation models and aligning for generic visual- linguistic tasks.CoRR, abs/2312.14238, 2023

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual- linguistic tasks.CoRR, abs/23...

    2023 doi arXiv

  36. [46]

    Self- training boosted multi-factor matching network for composed image retrieval.IEEE Trans

    Haokun Wen, Xuemeng Song, Jianhua Yin, Jianlong Wu, Weili Guan, and Liqiang Nie. Self- training boosted multi-factor matching network for composed image retrieval.IEEE Trans. Pattern Anal. Mach. Intell., 46(5):3665–3678, 2024

    2024

  37. [47]

    Context- i2w: Mapping images to context-dependent words for accurate zero-shot composed image retrieval

    Yuanmin Tang, Jing Yu, Keke Gai, Jiamin Zhuang, Gang Xiong, Yue Hu, and Qi Wu. Context- i2w: Mapping images to context-dependent words for accurate zero-shot composed image retrieval. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors,Thirty- Eighth AAAI ...

    2024

  38. [48]

    Fashion IQ: A new dataset towards retrieving im- ages by natural language feedback

    Hui Wu, Yupeng Gao, Xiaoxiao Guo, Ziad Al-Halah, Steven Rennie, Kristen Grau- man, and Rogério Feris. Fashion IQ: A new dataset towards retrieving im- ages by natural language feedback. InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-2...

    2021

  39. [49]

    A corpus for reasoning about natural language grounded in photographs

    Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs. In Anna Korhonen, David R. Traum, and Lluís Màrquez, editors,Proceedings of the 57th Conference of the Association for Computa...

    2019 doi

  40. [50]

    Mini-batch optimization of contrastive loss.Transactions on Machine Learning Research, 2024

    Jaewoong Cho, Kartik Sreenivasan, Keon Lee, Kyunghoo Mun, Soheun Yi, Jeong-Gwan Lee, Anna Lee, Jy yong Sohn, Dimitris Papailiopoulos, and Kangwook Lee. Mini-batch optimization of contrastive loss.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://open...

    2024

  41. [51]

    Bridging mini-batch and asymptotic analysis in contrastive learning: From infoNCE to kernel-based losses

    Panagiotis Koromilas, Giorgos Bouritsas, Theodoros Giannakopoulos, Mihalis Nicolaou, and Yannis Panagakis. Bridging mini-batch and asymptotic analysis in contrastive learning: From infoNCE to kernel-based losses. InForty-first International Conference on Machine Learning,

  42. [52]

    URLhttps://openreview.net/forum?id=SvvvB5t5EW. 14

  43. [53]

    Language- only efficient training of zero-shot composed image retrieval

    Geonmo Gu, Sanghyuk Chun, Wonjae Kim, Yoohoon Kang, and Sangdoo Yun. Language- only efficient training of zero-shot composed image retrieval. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 13225–13234. IEE...

    2024

  44. [54]

    isearle: Improving textual inversion for zero-shot composed image retrieval.CoRR, abs/2405.02951, 2024

    Lorenzo Agnolucci, Alberto Baldrati, Marco Bertini, and Alberto Del Bimbo. isearle: Improving textual inversion for zero-shot composed image retrieval.CoRR, abs/2405.02951, 2024. doi: 10.48550/ARXIV .2405.02951. URLhttps://doi.org/10.48550/arXiv.2405.02951

    2024 doi arXiv

  45. [55]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in context. In David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors,Computer Vision - ECCV 2014 ...

    2014 doi

  46. [56]

    Bernstein, Alexander C

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhi- heng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge.Int. J. Comput. Vis., 115 (3):211–252, 20...

    2015 doi

  47. [57]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URLhttps://openreview.net/forum?id=Bkg6RiCqY7

    2019

  48. [60]

    Vision-by- language for training-free compositional image retrieval

    Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini, and Zeynep Akata. Vision-by- language for training-free compositional image retrieval. InThe Twelfth International Con- ference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenRe- view.net, 20...

    2024

  49. [61]

    Effective conditioned and composed image retrieval combining clip-based features

    Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. Effective conditioned and composed image retrieval combining clip-based features. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18- 24, 2022, pag...

    2022

  50. [62]

    cap1" and image 2 with the caption

    Jianfeng Lu and Stefan Steinerberger. Neural collapse under cross-entropy loss.Applied and Computational Harmonic Analysis, 59:224–241, 2022. ISSN 1063-5203. doi: https://doi.org/10. 1016/j.acha.2021.12.011. URL https://www.sciencedirect.com/science/article/pii/ S1063520321001...

    2022

  51. [2023]

    URL https://doi.org/10.1109/ICCV51070

    doi: 10.1109/ICCV51070.2023.01407. URL https://doi.org/10.1109/ICCV51070. 2023.01407

    2023

  52. [2024]

    URL https://doi.org/10.1145/3626772.3657727

    doi: 10.1145/3626772.3657727. URL https://doi.org/10.1145/3626772.3657727

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.