Pith. sign in

REVIEW 3 major objections 5 minor 63 references

Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CIR-LVLM turns a large vision-language model into a single-pass encoder for composed image retrieval, reporting state-of-the-art recall on Fashion-IQ, Shoes, and CIRR.

desk verdict A useful single-pass LVLM encoder for composed image retrieval, but the advertised claim that LVLMs surpass VLPMs is not supported — the ablations change scale, pooling, and architecture at once. read the letter →

arxiv 2412.11087 v1 pith:DYOSRTHT submitted 2024-12-15 cs.IR

Zelong Sun , Dong Jing , Guoxing Yang , Nanyi Fei , Zhiwu Lu This is my paper · ORCID
classification cs.IR
keywords composedimageretrievallargevision-languagemodeluserintentpromptpoolhybridinstructioncontrastivelearningFashion-IQCIRR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes CIR-LVLM, a framework that repurposes a large vision-language model (LVLM) as a user intent-aware encoder for composed image retrieval. The paper claims that the LVLM's reasoning and instruction-following abilities let it capture what a user wants changed across a reference image and a relative caption, which earlier vision-language pretraining approaches tend to miss. A hybrid intent instruction module supplies two levels of guidance: a task prompt that clarifies what the model should do, and an instance-specific soft prompt selected from a learnable prompt pool. On the Fashion-IQ, Shoes, and CIRR benchmarks, CIR-LVLM reports state-of-the-art recall while keeping inference fast enough for practical use. The paper also claims to be the first to show that LVLMs can surpass VLPMs in multimodal retrieval tasks that require reasoning.

What carries the argument

The machinery is the LVLM-as-encoder design plus a hybrid intent instruction module. A Connector with learnable query embeddings maps the reference image into a sentence-level prompt; a task prompt tells the model what to do differently for queries versus target images; and a prompt pool with paired image/text keys selects the top-K soft prompts per instance. Because causal attention in a decoder-only LLM only lets later tokens see the whole sequence, the model uses position-weighted mean pooling to aggregate token representations, and a batch-based contrastive loss aligns query and target embeddings.

What would settle it

A same-scale control experiment would settle the claim: run the same architecture with a comparable-size pure text encoder or VLPM of roughly 7B parameters, and also run the LVLM without the Connector and prompt pool. If recall parity persists, the reasoning claim fails; if the LVLM only wins when the full instruction module is present, the mechanism is the prompts rather than the backbone.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuning an LVLM such as Qwen-VL-Chat to act as a bi-encoder retriever, rather than as a caption generator, makes it a stronger composed-image-retrieval model than cross-attention early fusion, textual inversion, or multi-pass LLM caption revision. The Connector condenses the reference image into a sentence-level prompt, the LLM produces a pooled representation that respects both the image content and the relative caption, and retrieval is done by cosine similarity to target embeddings. The authors report state-of-the-art results, including 56.21 Rmean on Fashion-IQ, 63.51 Rmean on Shoes, and 53.64 R@1 with 83.76 R@5 on CIRR, and attribute the gains to the LVLM's intent-aware reasoning combined with the hybrid instruction prompts.

Load-bearing premise

The paper's headline conclusion rests on the assumption that the LVLM's higher recall comes from its user-intent reasoning, because the ablation compares the LVLM encoder to much smaller CLIP and BLIP-2 text encoders while changing both model size and architecture.

Editorial extensions

If this is right

  • Replacing conventional text encoders with an LVLM encoder improves composed image retrieval accuracy on both fashion-domain and open-domain benchmarks.
  • Single-pass encoding sidesteps the multi-pass decoding cost of generative LVLM retrieval, giving a per-query inference time of about 0.08 seconds compared with 1.38 seconds for a generative baseline.
  • Instance-specific soft prompts selected from a prompt pool outperform a universal soft prompt, suggesting that per-instance guidance is a measurable source of accuracy.
  • The framework transfers to different LVLM backbones, with stronger backbones yielding higher retrieval performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the LVLM's reasoning ability is the true driver, then future improvements in LVLM backbones should translate almost directly into retrieval gains without redesigning the fusion strategy.
  • The prompt pool behaves like a small learned routing memory, and the same selection mechanism could be applied to other retrieval tasks with complex, multimodal or multi-step queries.
  • A same-scale control is needed to separate reasoning ability from model scale, since the paper's main ablations compare the LVLM against much smaller CLIP and BLIP-2 text encoders while changing both architecture and parameter count.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CIR-LVLM, a framework that fine-tunes the Qwen-VL-Chat large vision-language model (LVLM) as a dual encoder for composed image retrieval (CIR). A Connector with learnable query embeddings converts the reference or target image into a sentence-level prompt, and a hybrid intent instruction module adds a task-level hard prompt and an instance-specific soft prompt selected from a learnable prompt pool using image and text keys. The model is trained with a batch-based contrastive loss and uses position-weighted mean pooling over the decoder outputs. The authors report state-of-the-art results on Fashion-IQ, Shoes, and CIRR, and claim to be the first to show that LVLMs can surpass vision-language pre-trained models (VLPMs) in multimodal retrieval tasks that require reasoning.

Significance. If the claims hold, the paper provides a practical recipe for adapting a 7B LVLM as a single-pass retriever with acceptable inference cost, and the hybrid prompt-pool module is a reasonable and interesting design. The paper is also transparent about failure cases and includes ablations over fine-tuning strategies, pooling strategies, and backbones. The key scientific claim, however, is not yet established: the evidence for LVLM superiority over VLPMs rests on comparisons that change model scale, pooling, and connector architecture at the same time. The empirical SOTA claim on CIRR is also marginal, with no error bars. The work is a solid systems/empirical contribution, but the headline conceptual claim needs either a scale-matched controlled experiment or a substantial tempering.

major comments (3)
  1. [§5 Ablation study, Table 3 rows A.1-A.2; Appendix C] The ablation that replaces the LLM with CLIP or BLIP-2 text encoders simultaneously changes the parameter count (7B decoder vs. a much smaller text encoder), the pooling strategy (position-weighted mean pooling vs. [CLS] token), the projection head, and the connector design. The Rmean gap (66.17 vs. 45.34 and 54.85) is therefore not evidence that the LVLM's reasoning or instruction-following ability is the cause; it could be due to model scale, pooling, or the extra MLP described in Appendix C. To support the conclusion in Section 5 that 'compared to regular text encoders, the fine-tuned LLM is more advantageous in discerning user intent,' a scale-matched control is required, e.g., a 7B text-only encoder or a VLPM of comparable size using the same connector, pooling, and LoRA fine-tuning protocol.
  2. [Table 2, CIRR results] On CIRR, CIR-LVLM's Avg. metric is 81.44 vs. SPRC's 81.38, a difference of 0.06 absolute points, while Recallsubset@1 is 1.53 points lower (79.12 vs. 80.65). No error bars, repeated runs, or significance testing are reported anywhere in the paper. Given the small margin, the abstract's claim of state-of-the-art performance 'across three prominent benchmarks' overstates the CIRR result. Please report variance over multiple seeds or a paired significance test, and adjust the claim to the metrics where the improvement is non-negligible.
  3. [§4 Comparison with SOTA; §5 Ablation study] The cross-method comparisons in Tables 1 and 2 vary backbone, pretraining data, training loss, and fine-tuning protocol simultaneously, so the observed gains over VLPM-based methods cannot be attributed to the LVLM's user-intent reasoning per se. For example, the discussion in Section 4 claims the gain over TG-CIR is 'attributed to' the Connector and LLM capabilities, but no experiment isolates these factors. A baseline that uses the same training pipeline (LoRA, task prompts, prompt pool, position-weighted pooling) with a competitive VLPM of comparable scale is needed to make the mechanism claim credible. Without such a control, the SOTA numbers remain an empirical observation, not evidence for the 'LVLMs surpass VLPMs' conclusion stated in Contribution (4).
minor comments (5)
  1. [Table 3 vs. Appendix Table 7] The Rmean for the 'Detailed Prompt' setting is reported as 65.50 in Table 3 B.3 but 65.55 in Appendix Table 7 C.3; please reconcile these values or explain the rounding.
  2. [Figure 2(b)] The computation of q(I) and q(T) is only fully specified in Section 3; adding a brief sentence in the figure caption or a forward reference would improve readability.
  3. [Appendix C, Table 4 caption] The caption reads 'Ablation studies on Fashion-IQ dataset with regard to different pooling strategies,' but the table compares fine-tuning strategies; please correct the caption.
  4. [Section 7 Conclusion] The conclusion states LVLMs can 'surpass the VLMs,' while the rest of the paper uses 'VLPMs'; please use consistent terminology.
  5. [Throughout] There are several typos: 'loRA' should be 'LoRA,' 'retrival' in Figure 2, and 'Tabel' in Appendix C; a careful proofread is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports empirical benchmark results against external public datasets and published baselines, with no derivation that reduces to its own inputs.

full rationale

This paper is an empirical systems paper, not a derivation. Its central claims are supported by retrieval accuracy on the public Fashion-IQ, Shoes, and CIRR benchmarks, compared with published baselines and controlled ablations. The soft-prompt selection mechanism uses the model's own learned keys and embeddings as an architectural choice, not as a fitted quantity that is later reported as a prediction. The ablation replacing Qwen-VL-Chat with CLIP or BLIP-2 text encoders changes model scale, pooling strategy, and architecture simultaneously, which weakens the causal attribution of gains to LVLM reasoning ability, but this is a validity and control concern rather than circularity: no result is defined in terms of another result, and no parameter is fitted to the test metrics. Citations, including the weighted-mean pooling reference, are standard external support and are not load-bearing in a circular way. The paper is self-contained against external benchmarks, so the appropriate circularity finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. The 'instance-specific soft prompt' is a learned parameter vector within a pool, a mechanism rather than an invented entity with independent evidence. The only adjustable components are the hyperparameters listed above, which were chosen by validation performance.

free parameters (4)
  • prompt length Lp = 5 for all datasets
    Length of each soft prompt in tokens, tuned on Fashion-IQ validation (Table 8).
  • number of selected prompts K = 8 for Fashion-IQ/Shoes, 12 for CIRR
    Number of nearest prompts retrieved from the pool; tuned per dataset.
  • prompt pool size M = 45 for Fashion-IQ/Shoes, 55 for CIRR
    Number of prompts in the shared pool; tuned per dataset and shown to affect performance (Table 8).
  • temperature lambda = 100 (Fashion-IQ), 130 (CIRR), 70 (Shoes)
    Temperature in the contrastive loss, set per dataset (Section 4).
assumptions (3)
  • domain assumption Qwen-VL-Chat has sufficient visual understanding and instruction-following ability to serve as a retrieval encoder after LoRA fine-tuning.
    The whole method rests on the pretrained LVLM's capabilities; the paper tests this by ablation (Table 3) but does not prove it generally.
  • standard math The contrastive loss with batch-based classification (Eq. 5) creates a shared embedding space where query and target embeddings align for the retrieval task.
    Standard contrastive learning assumption used throughout the CIR literature; the paper relies on it without new justification.
  • domain assumption Position-weighted mean pooling (Eq. 4) produces a representation that captures user intent from the causally masked decoder outputs.
    This pooling choice is justified by the causal mask and ablation (Table 6), but it is still a modeling assumption for the task.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval." pith.science (2026). https://pith.science/paper/DYOSRTHT

@misc{pith2026241211087,
  author       = {Pith},
  title        = {Pith review of: Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DYOSRTHT}},
  note         = {Machine review of arXiv:2412.11087}
}
read the original abstract

Composed Image Retrieval (CIR) aims to retrieve target images from candidate set using a hybrid-modality query consisting of a reference image and a relative caption that describes the user intent. Recent studies attempt to utilize Vision-Language Pre-training Models (VLPMs) with various fusion strategies for addressing the task.However, these methods typically fail to simultaneously meet two key requirements of CIR: comprehensively extracting visual information and faithfully following the user intent. In this work, we propose CIR-LVLM, a novel framework that leverages the large vision-language model (LVLM) as the powerful user intent-aware encoder to better meet these requirements. Our motivation is to explore the advanced reasoning and instruction-following capabilities of LVLM for accurately understanding and responding the user intent. Furthermore, we design a novel hybrid intent instruction module to provide explicit intent guidance at two levels: (1) The task prompt clarifies the task requirement and assists the model in discerning user intent at the task level. (2) The instance-specific soft prompt, which is adaptively selected from the learnable prompt pool, enables the model to better comprehend the user intent at the instance level compared to a universal prompt for all instances. CIR-LVLM achieves state-of-the-art performance across three prominent benchmarks with acceptable inference efficiency. We believe this study provides fundamental insights into CIR-related fields.

Figures

Figures reproduced from arXiv: 2412.11087 by the authors.

Figure 1
Figure 1. Workflows of existing CIR methods and our proposed CIR-LVLM: (a) Early-fusion, (b) Textual inversion, and (c) Our [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Illustration of the architecture overview of our proposed model. All the parameters are shared between the query [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of intent instructions for the hybrid [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Influence of (a) length of soft prompt and (b) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Attention map visualization (right side of the first [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Generated caption of the modified version of the reference image according to the relative caption. Bolded: helpful [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Attention map visualization and the sum of the attention weights over all the visual or relative caption tokens for the [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Failure retrieval examples obtained by our proposed model for the CIR task. Reasons for failed retrieval are highlighted [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Successful retrieval examples obtained by our proposed model for the CIR task. The example contains operations [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 22 canonical work pages

  1. [1]

    Agrawal, H.; Desai, K.; Wang, Y.; Chen, X.; Jain, R.; Johnson, M.; Batra, D.; Parikh, D.; Lee, S.; and Anderson, P. 2019. Nocaps: Novel object captioning at scale. In Proceedings of the IEEE/CVF international conference on computer vision, 8948--8957

    2019

  2. [2]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023 a . Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    arXiv 2023

  3. [3]

    Bai, Y.; Xu, X.; Liu, Y.; Khan, S.; Khan, F.; Zuo, W.; Goh, R. S. M.; and Feng, C.-M. 2023 b . Sentence-level prompts benefit composed image retrieval. arXiv preprint arXiv:2310.05473

    arXiv 2023

  4. [4]

    Baldrati, A.; Bertini, M.; Uricchio, T.; and Del Bimbo, A. 2022. Conditioned and Composed Image Retrieval Combining and Partially Fine-Tuning CLIP-Based Features. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4959--4968

    work page 2022

  5. [5]

    L.; Berg, A

    Berg, T. L.; Berg, A. C.; and Shih, J. 2010. Automatic attribute discovery and characterization from noisy web data. In European Conference on Computer Vision, 663--676

    work page 2010

  6. [6]

    D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877--1901

    2020

  7. [7]

    Chen, Y.; Gong, S.; and Bazzani, L. 2020. Image search with text feedback by visiolinguistic attention learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3001--3011

    work page 2020

  8. [8]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P. N.; and Hoi, S. 2024. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36

    2024

Show all 63 references
  1. [9]

    Dao, T. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691

    2023 arXiv

  2. [10]

    S.; Csurka, G.; and Larlus, D

    Delmas, G.; de Rezende, R. S.; Csurka, G.; and Larlus, D. 2022. ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity. arXiv preprint arXiv:2203.08101

    2022 arXiv

  3. [11]

    S.; Shlens, J.; Bengio, S.; Dean, J.; Ranzato, M.; and Mikolov, T

    Frome, A.; Corrado, G. S.; Shlens, J.; Bengio, S.; Dean, J.; Ranzato, M.; and Mikolov, T. 2013. Devise: A deep visual-semantic embedding model. Advances in Neural Information Processing Systems, 26: 2121--2129

    2013

  4. [12]

    H.; Chechik, G.; and Cohen-Or, D

    Gal, R.; Alaluf, Y.; Atzmon, Y.; Patashnik, O.; Bermano, A. H.; Chechik, G.; and Cohen-Or, D. 2022. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618

    2022 arXiv

  5. [13]

    Gao, D.; Jin, L.; Chen, B.; Qiu, M.; Li, P.; Wei, Y.; Hu, Y.; and Wang, H. 2020. Fashionbert: Text and image matching with adaptive loss for cross-modal retrieval. In ACM SIGIR Conference on Research and Development in Information Retrieval, 2251--2260

    2020

  6. [14]

    Goenka, S.; Zheng, Z.; Jaiswal, A.; Chada, R.; Wu, Y.; Hedau, V.; and Natarajan, P. 2022. Fashionvlp: Vision language transformer for fashion retrieval with feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14105--14115

    2022

  7. [15]

    Gordo, A.; Almaz \'a n, J.; Revaud, J.; and Larlus, D. 2016. Deep image retrieval: Learning global representations for image search. In European Conference on Computer Vision, 241--257

    2016

  8. [16]

    Guo, X.; Wu, H.; Cheng, Y.; Rennie, S.; Tesauro, G.; and Feris, R. 2018. Dialog-based interactive image retrieval. Advances in Neural Information Processing Systems, 31: 676--686

    2018

  9. [17]

    Han, X.; Zhao, W.; Ding, N.; Liu, Z.; and Sun, M. 2022. Ptr: Prompt tuning with rules for text classification. AI Open, 3: 182--192

    2022

  10. [18]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  11. [19]

    Huang, Q.; Dong, X.; Zhang, P.; Wang, B.; He, C.; Wang, J.; Lin, D.; Zhang, W.; and Yu, N. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  12. [20]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6700--6709

    2019

  13. [21]

    Jin, W.; Cheng, Y.; Shen, Y.; Chen, W.; and Ren, X. 2021. A good prompt is worth millions of parameters: Low-resource prompt-based learning for vision-language models. arXiv preprint arXiv:2110.08484

    2021 arXiv

  14. [22]

    Karthik, S.; Roth, K.; Mancini, M.; and Akata, Z. 2023. Vision-by-language for training-free compositional image retrieval. arXiv preprint arXiv:2310.09291

    2023 arXiv

  15. [23]

    Kim, J.; Yu, Y.; Kim, H.; and Kim, G. 2021. Dual compositional learning in interactive image retrieval. In AAAI Conference on Artificial Intelligence, 1771--1779

    2021

  16. [24]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  17. [25]

    Lee, S.; Kim, D.; and Han, B. 2021. Cosmo: Content-style modulation for image retrieval with text feedback. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 802--812

    2021

  18. [26]

    Levy, M.; Ben-Ari, R.; Darshan, N.; and Lischinski, D. 2023. Data Roaming and Early Fusion for Composed Image Retrieval. arXiv preprint arXiv:2303.09429

    2023 arXiv

  19. [27]

    Li, B.; Wang, R.; Wang, G.; Ge, Y.; Ge, Y.; and Shan, Y. 2023 a . Seed-bench: Benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125

    2023 arXiv

  20. [28]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 b . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 19730--19742. PMLR

    2023

  21. [29]

    Li, J.; Li, D.; Xiong, C.; and Hoi, S. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, 12888--12900. PMLR

    2022

  22. [30]

    X.; and Wen, J.-R

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, W. X.; and Wen, J.-R. 2023 c . Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355

    2023 arXiv

  23. [31]

    Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2023 a . Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744

    2023 arXiv

  24. [32]

    Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; et al. 2023 b . Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281

    2023 arXiv

  25. [33]

    Liu, Z.; Luo, P.; Qiu, S.; Wang, X.; and Tang, X. 2016. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In IEEE Conference on Computer Vision and Pattern Recognition, 1096--1104

    2016

  26. [34]

    Liu, Z.; Rodriguez-Opazo, C.; Teney, D.; and Gould, S. 2021. Image retrieval on real-life images with pre-trained vision-and-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2125--2134

    2021

  27. [35]

    Liu, Z.; Sun, W.; Teney, D.; and Gould, S. 2023 c . Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encoder. arXiv preprint arXiv:2305.16304

    2023 arXiv

  28. [36]

    Ma, X.; Wang, L.; Yang, N.; Wei, F.; and Lin, J. 2023. Fine-tuning llama for multi-stage text retrieval. arXiv preprint arXiv:2310.08319

    2023 arXiv

  29. [37]

    K.; and Chakraborty, A

    Mishra, A.; Shekhar, S.; Singh, A. K.; and Chakraborty, A. 2019. Ocr-vqa: Visual question answering by reading text in images. In 2019 international conference on document analysis and recognition (ICDAR), 947--952. IEEE

    2019

  30. [38]

    Muennighoff, N. 2022. Sgpt: Gpt sentence embeddings for semantic search. arXiv preprint arXiv:2202.08904

    2022 arXiv

  31. [39]

    Muennighoff, N.; Su, H.; Wang, L.; Yang, N.; Wei, F.; Yu, T.; Singh, A.; and Kiela, D. 2024. Generative representational instruction tuning. arXiv preprint arXiv:2402.09906

    2024 arXiv

  32. [40]

    OpenAI. 2023. ChatGPT. https://chat.openai.com/

    2023

  33. [41]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  34. [42]

    Saito, K.; Sohn, K.; Zhang, X.; Li, C.-L.; Lee, C.-Y.; Saenko, K.; and Pfister, T. 2023. Pic2word: Mapping pictures to words for zero-shot composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19305--19314

    2023

  35. [43]

    G.; Malinowski, M.; Pascanu, R.; Battaglia, P.; and Lillicrap, T

    Santoro, A.; Raposo, D.; Barrett, D. G.; Malinowski, M.; Pascanu, R.; Battaglia, P.; and Lillicrap, T. 2017. A simple neural network module for relational reasoning. Advances in Neural Information Processing Systems, 30: 4974--4983

    2017

  36. [44]

    L.; Wallace, E.; and Singh, S

    Shin, T.; Razeghi, Y.; Logan IV, R. L.; Wallace, E.; and Singh, S. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980

    2020 arXiv

  37. [45]

    Suhr, A.; Zhou, S.; Zhang, A.; Zhang, I.; Bai, H.; and Artzi, Y. 2018. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491

    2018 arXiv

  38. [46]

    Sun, S.; Ye, F.; and Gong, S. 2023. Training-free Zero-shot Composed Image Retrieval with Local Concept Reranking. arXiv preprint arXiv:2312.08924

    2023 arXiv

  39. [47]

    Tang, Y.; Yu, J.; Gai, K.; Zhuang, J.; Xiong, G.; Hu, Y.; and Wu, Q. 2024. Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 5180--5188

    2024

  40. [48]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  41. [49]

    Vo, N.; Jiang, L.; Sun, C.; Murphy, K.; Li, L.-J.; Fei-Fei, L.; and Hays, J. 2019. Composing text and image for image retrieval-an empirical odyssey. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6439--6448

    2019

  42. [50]

    Wang, J.; Zhou, Y.; Xu, G.; Shi, P.; Zhao, C.; Xu, H.; Ye, Q.; Yan, M.; Zhang, J.; Zhu, J.; et al. 2023. Evaluation and Analysis of Hallucination in Large Vision-Language Models.(Aug. arXiv preprint arxiv:2308.15126

    2023 arXiv

  43. [51]

    Wang, Z.; Zhang, Z.; Lee, C.-Y.; Zhang, H.; Sun, R.; Ren, X.; Su, G.; Perot, V.; Dy, J.; and Pfister, T. 2022. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 139--149

    2022

  44. [52]

    Wen, H.; Song, X.; Yang, X.; Zhan, Y.; and Nie, L. 2021. Comprehensive linguistic-visual composition network for image retrieval. In ACM SIGIR Conference on Research and Development in Information Retrieval, 1369--1378

    2021

  45. [53]

    Wen, H.; Zhang, X.; Song, X.; Wei, Y.; and Nie, L. 2023. Target-guided composed image retrieval. In Proceedings of the 31st ACM International Conference on Multimedia, 915--923

    2023

  46. [54]

    Wu, H.; Gao, Y.; Guo, X.; Al-Halah, Z.; Rennie, S.; Grauman, K.; and Feris, R. 2021. Fashion iq: A new dataset towards retrieving images by natural language feedback. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11307--11317

    2021

  47. [55]

    Yang, Z.; Xue, D.; Qian, S.; Dong, W.; and Xu, C. 2024. LDRE: LLM-based Divergent Reasoning and Ensemble for Zero-Shot Composed Image Retrieval. In SIGIR, 80--90

    2024

  48. [56]

    Yu, W.; Yang, Z.; Li, L.; Wang, J.; Lin, K.; Liu, Z.; Wang, X.; and Wang, L. 2023. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490

    2023 arXiv

  49. [57]

    Zang, Y.; Li, W.; Zhou, K.; Huang, C.; and Loy, C. C. 2022. Unified vision and language prompt learning. arXiv preprint arXiv:2210.07225

    2022 arXiv

  50. [58]

    Zhao, Y.; Song, Y.; and Jin, Q. 2022. Progressive Learning for Image Retrieval with Hybrid-Modality Queries. arXiv preprint arXiv:2204.11212

    2022 arXiv

  51. [59]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9): 2337--2348

    2022

  52. [60]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023 a . Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

    2023 arXiv

  53. [61]

    Zhu, H.; Wei, Y.; Zhao, Y.; Zhang, C.; and Huang, S. 2023 b . Amc: Adaptive multi-expert collaborative network for text-guided image retrieval. ACM Transactions on Multimedia Computing, Communications and Applications, 19(6): 1--22

    2023

  54. [62]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  55. [63]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.