Pith. sign in

REVIEW 4 major objections 7 minor 23 references

MCoT-RE: Multi-Faceted Chain-of-Thought and Re-Ranking for Training-Free Zero-Shot Composed Image Retrieval

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free composed image retriever beats zero-shot baselines by having a multimodal LLM write two captions for the same query — one for filtering, one for re-ranking alongside the reference image.

desk verdict A credible but overclaimed training-free CIR paper; the dual-caption idea is sound, but the SOTA claim rests on an incomplete baseline set and benchmark-tuned hyperparameters. read the letter →

arxiv 2507.12819 v1 pith:WRWMEMFM submitted 2025-07-17 cs.CV

Jeong-Woo Park , Seong-Whan Lee This is my paper · ORCID
classification cs.CV
keywords composedimageretrievalzero-shottraining-freechain-of-thoughtpromptingmultimodallargelanguagemodelre-rankingFashionIQCIRR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Composed image retrieval asks a system to find a target image given a reference image plus a text instruction such as 'make the dog sit down.' This paper argues that a training-free, zero-shot retriever can outperform prior no-training approaches by using a multimodal LLM more deliberately: instead of writing one caption, it prompts the model to reason in steps and produce two captions — one that names only the requested changes, and one that integrates those changes with the unstated visual context of the reference image. The first caption filters the gallery to top-$k$ candidates; the second, combined with the first and with the reference image itself, re-ranks those candidates. On FashionIQ and CIRR the method reports Recall improvements of up to 6.24 percentage points at Recall@10 and 8.58 percentage points at Recall@1 over the strongest training-free baselines. If the claim holds, strong composed-image retrieval is available without any task-specific training or extra annotated data.

What carries the argument

The mechanism that carries the argument is the multi-faceted chain-of-thought prompt, which forces the multimodal LLM to separate explicit modifications from implicit visual context and to verbalize both as two distinct target captions. The first caption, $C_{Modi}$, is a filter: its embedding is compared by cosine similarity to every gallery image and the top-$k$ candidates are kept. The second caption, $C_{Integ}$, joins $C_{Modi}$ and the reference image in a weighted sum of embeddings, $F_{comb} = \alpha F_{Modi} + \beta F_{Integ} + (1 - \alpha - \beta) F_{R}$, whose similarity scores re-rank only the retained candidates. With the hyperparameters fixed at $\alpha = 0.05$ and $\beta = 0.9$, and $k$ set to 150 for FashionIQ and 200 for CIRR, the pipeline turns a single multimodal-LLM call into a coarse-to-fine search that preserves both the instruction and the surrounding visual scene.

What would settle it

Re-run MCoT-RE on FashionIQ and CIRR while varying the fusion weights (for example $\alpha \in \{0, 0.05, 0.2, 0.5\}$, $\beta \in \{0.5, 0.7, 0.9\}$) and the candidate count $k$ from 50 to 300, or replace the combined embedding with each single caption; if the reported gains are not stable across a neighborhood of these settings, or if the weighted combination does not beat both single captions on the same candidates, then the re-ranking mechanism is not what carries the result.

Watch

Extended reading notes

Core claim

The central claim is that MCoT-RE establishes a new state of the art among training-free zero-shot composed image retrieval methods. The pipeline feeds the reference image and modification text together to a multimodal LLM under a four-step chain-of-thought prompt: understand the reference image, interpret the modification, reason about visual changes, and anticipate the target. The model then emits two captions: $C_{Modi}$, which describes only the elements the instruction changes, and $C_{Integ}$, which preserves background, style, and other contextual cues that should survive the edit. $C_{Modi}$ is embedded and used to select the top-$k$ gallery images; then a combined feature $F_{comb} = \alpha F_{Modi} + \beta F_{Integ} + (1 - \alpha - \beta) F_{R}$ is scored against those candidates, fusing the modification caption, the integration caption, and the reference image embedding. The paper reports that this two-stage design beats all compared training-free baselines on FashionIQ and CIRR, including larger gains on CIRR's harder Recall@1, and that ablations removing either stage or either caption hurt performance.

Load-bearing premise

The claim rests on the assumption that the pre-trained contrastive image-text embedding space is linearly composable — that adding the two caption embeddings and the reference image embedding in the proportions $\alpha = 0.05$ and $\beta = 0.9$ yields a query whose cosine similarity ranks the true target correctly — and the paper fixes these weights and the candidate count $k$ per dataset without reporting sensitivity tests, so the state-of-the-art result depends on that unverified combination.

Editorial extensions

If this is right

  • Training-free zero-shot composed image retrieval can be improved without training new model weights, because the reported gains come from how an existing multimodal LLM is prompted and how its outputs are assembled at retrieval time.
  • The two-caption decomposition gives a cheap control over the trade-off between fidelity to the instruction and preservation of context, since adjusting $\alpha$ and $\beta$ shifts the weight of each caption in the re-ranking query.
  • On CIRR, where correct images sit among visually similar distractors, the method reports its largest relative gain at Recall@1, suggesting context-preserving captions matter most when the modification text is underspecified.
  • Because both stages use frozen pre-trained encoders and a multimodal LLM, the same recipe transfers to a new gallery or domain without dataset-specific fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes $\alpha = 0.05$, $\beta = 0.9$, and per-dataset $k$ without reporting a sensitivity analysis; an independent test of whether retrieval stays high across a neighborhood of these settings would show whether the gains come from the fusion mechanism or from a finely tuned operating point.
  • The method's reliance on linear combination in embedding space invites the hypothesis that contrastive image-text embeddings support additive query composition more directly than previously assumed; this could be tested by measuring retrieval quality under interpolation of arbitrary caption and reference-image pairs.
  • The same two-caption decomposition could transfer to neighbouring tasks such as text-guided image editing evaluation or reference-based generation, where separating what changed from what stayed is the same underlying problem.
  • A natural stress test is to replace the proprietary multimodal LLM with an open-weight model and re-run the pipeline, which would reveal how much of the improvement depends on the specific model's reasoning quality rather than on the prompting structure itself.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes MCoT-RE, a training-free zero-shot composed image retrieval (CIR) framework. It uses a multimodal large language model (MLLM) guided by a multi-faceted chain-of-thought (MCoT) prompt to generate two target captions: a modification-focused caption capturing explicit changes and an integration-focused caption preserving contextual visual cues. The modification caption is used to retrieve a top-k candidate set, and the final ranking is obtained by scoring candidates against a weighted combination of the two caption embeddings and the reference image embedding (Eq. 4). Experiments on FashionIQ and CIRR with two CLIP backbones are reported, and the paper claims state-of-the-art performance among training-free methods with gains up to 6.24% (abstract) or 6.63% (Section IV.C) in Recall@10 on FashionIQ and 8.58% in Recall@1 on CIRR.

Significance. If the results hold, the framework would be a useful contribution to training-free zero-shot CIR, showing that multi-faceted caption generation plus a simple re-ranking step can outperform existing single-caption approaches. The ablation study in Table III supports the contribution of each component. However, the central SOTA claim is currently not fully substantiated: the comparison set omits several training-free methods cited in the paper, and the hyperparameters (k, alpha, beta) are selected on the evaluation benchmarks without sensitivity analysis. The paper does not provide code or the full prompt template, limiting reproducibility.

major comments (4)
  1. [Section II.B; Tables I and II] The paper claims state-of-the-art results among training-free methods, but evaluates only WeiMoCIR [8], LDRE [11], and OSrCIR [13] in Tables I and II. CoTMR [12] is explicitly discussed in Section II.B as a training-free CoT-based CIR method and is not included in the comparison; refs [7], [9], and [10] are also zero-shot/training-free methods cited in the related work but absent from the tables. The central SOTA claim cannot be sustained without comparing against all cited training-free baselines under the same CLIP backbones. Please add these comparisons or revise the claim to be specific to the tabulated baselines.
  2. [Section IV.B; Eq. (4)] The hyperparameters alpha=0.05, beta=0.9, and k (150 for FashionIQ, 200 for CIRR) are determined empirically on the evaluation benchmarks. This weakens the characterization of the method as 'training-free' and raises the possibility that the reported gains are partly due to fitting these constants to the test/validation sets. Provide a sensitivity analysis for alpha, beta, and k (e.g., a grid over reasonable values) and report how the margins in Tables I and II change across the operating range.
  3. [Abstract and Section IV.C] The maximum improvement on FashionIQ is reported as 6.24% in the abstract and 6.63% in Section IV.C, while Table I shows improvements of 6.24 points on Dress and 6.63 points on Toptee. These headline numbers are inconsistent and need to be reconciled; the abstract should report the actual maximum improvement under the stated metric.
  4. [Section III.D; Eq. (4)] The re-ranking step assumes that a weighted sum of CLIP text and image embeddings forms a valid composed-query embedding, but the paper does not analyze the sensitivity of the ranking to this linear fusion or to the specific weights. The ablation in Table III shows that re-ranking helps at the chosen operating point, but without robustness checks (e.g., varying alpha and beta, or comparing against alternative fusion strategies such as late-score averaging), the reported margins remain conditional on a single hand-picked combination.
minor comments (7)
  1. [Figure 2] The alpha, beta, and (1-alpha-beta) labels in the pipeline are not explained in the caption; refer to Eq. (4) and state the chosen values in the caption.
  2. [Figure 3] The prompt template is shown in abbreviated form with placeholders. For reproducibility, include the full MCoT prompt, including the three in-context examples, in an appendix or supplementary material.
  3. [Section IV.B] The text 'Gemini [22] 1.5' should be formatted as 'Gemini 1.5 [22]', and the specific model version and decoding settings (e.g., temperature, top-p) should be reported, as MLLM outputs can be stochastic.
  4. [Tables I and II] No variance or number of runs is reported; given the stochasticity of LLM-based caption generation, error bars or an average over multiple runs would improve the reliability of the reported numbers.
  5. [Algorithm 1 and Eq. (1)] The notation for the MLLM call is inconsistent between Algorithm 1 line 6 and Eq. (1); standardize the notation.
  6. [Table III] It would be helpful to include a row for first-stage-only retrieval (CModi with no re-ranking) to directly quantify the re-ranking gain, since the 'w/o Re-Ranking' row still uses both captions in a single combined score.
  7. [Introduction] Reference [2], an AIoT survey, appears unrelated to the CIR motivation; please verify the citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the MCoT-RE pipeline is an empirical MLLM/CLIP composition, and no reported quantity is definitionally forced by its own inputs.

full rationale

MCoT-RE does not derive its reported Recall values from its construction; it empirically evaluates an MLLM/CLIP pipeline on FashionIQ and CIRR. The two captions CModi and CInteg are generated by prompting an MLLM with the reference image and modification text (Eq. 1), then embedded and combined in Eq. (4). No equation defines the benchmark labels or target images in terms of these captions, and no parameter is fitted to a subset in order to predict a closely related quantity. The hyperparameters alpha=0.05, beta=0.9 and k are selected per dataset in Section IV.B; this may raise evaluation-contamination concerns, but it is model selection rather than a circular reduction. The self-citations in the paper ([18]-[21], [23]) are unrelated prior works and are not load-bearing for the method or the results. The omission of CoTMR [12] and other training-free baselines from Tables I-II is a legitimate comparison-coverage weakness, but it does not make the method's output equivalent to its inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three categories of assumptions: the CLIP embedding space is sufficiently aligned for text-caption matching and additive feature fusion; the Gemini MLLM produces two reliable captions under the hand-written prompt; and the hyperparameters chosen on the evaluation benchmarks are not overfitted. The first two are common domain assumptions in this literature; the third is a mild ad hoc selection that weakens the training-free framing.

free parameters (4)
  • alpha (fusion weight for modification-focused caption) = 0.05
    Set in Section IV.B and used in Eq. 4; no sensitivity analysis is reported, and it is tuned by the authors rather than derived.
  • beta (fusion weight for integration-focused caption) = 0.9
    Set in Section IV.B; combined with alpha it leaves only 0.05 weight for the reference image embedding in Eq. 4.
  • k (number of candidates kept for re-ranking) = 150 for FashionIQ, 200 for CIRR
    Empirically determined per dataset (Section IV.B); affects which candidates enter the second stage and hence the final ranking.
  • MCoT prompt template with three in-context examples = Hand-written prompt (Fig. 3)
    The prompt is part of the method and the examples are elided in Fig. 3; caption quality depends on this hand-designed text, making it a free design choice.
assumptions (4)
  • domain assumption CLIP text and image embeddings live in a shared space where cosine similarity is a valid retrieval score.
    Used in Eqs. 2 and 5 and throughout; a standard but empirical assumption.
  • domain assumption Weighted sums of CLIP embeddings remain semantically meaningful queries.
    Eq. 4 assumes Fcomb approximates the target image embedding; the paper provides no derivation or verification of additivity in CLIP space.
  • domain assumption Gemini 1.5 produces captions that faithfully describe the reference image and the requested modification.
    The entire pipeline depends on the MLLM's outputs; no hallucination checks or human verification are reported (Section IV.B).
  • ad hoc to paper Hyperparameters alpha, beta, and k chosen on the evaluation benchmarks transfer and are not overfitted.
    Section IV.B states these are empirically determined; no held-out tuning set or sensitivity study is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MCoT-RE: Multi-Faceted Chain-of-Thought and Re-Ranking for Training-Free Zero-Shot Composed Image Retrieval." pith.science (2026). https://pith.science/paper/WRWMEMFM

@misc{pith2026250712819,
  author       = {Pith},
  title        = {Pith review of: MCoT-RE: Multi-Faceted Chain-of-Thought and Re-Ranking for Training-Free Zero-Shot Composed Image Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WRWMEMFM}},
  note         = {Machine review of arXiv:2507.12819}
}
read the original abstract

Composed Image Retrieval (CIR) is the task of retrieving a target image from a gallery using a composed query consisting of a reference image and a modification text. Among various CIR approaches, training-free zero-shot methods based on pre-trained models are cost-effective but still face notable limitations. For example, sequential VLM-LLM pipelines process each modality independently, which often results in information loss and limits cross-modal interaction. In contrast, methods based on multimodal large language models (MLLMs) often focus exclusively on applying changes indicated by the text, without fully utilizing the contextual visual information from the reference image. To address these issues, we propose multi-faceted Chain-of-Thought with re-ranking (MCoT-RE), a training-free zero-shot CIR framework. MCoT-RE utilizes multi-faceted Chain-of-Thought to guide the MLLM to balance explicit modifications and contextual visual cues, generating two distinct captions: one focused on modification and the other integrating comprehensive visual-textual context. The first caption is used to filter candidate images. Subsequently, we combine these two captions and the reference image to perform multi-grained re-ranking. This two-stage approach facilitates precise retrieval by aligning with the textual modification instructions while preserving the visual context of the reference image. Through extensive experiments, MCoT-RE achieves state-of-the-art results among training-free methods, yielding improvements of up to 6.24% in Recall@10 on FashionIQ and 8.58% in Recall@1 on CIRR.

Figures

Figures reproduced from arXiv: 2507.12819 by the authors.

Figure 1
Figure 1. Comparison of existing training-free ZS-CIR method with our [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall pipeline of MCoT-RE. Our framework employs an MCoT process using an MLLM to jointly reason over the reference image and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Prompt template for generating the two target captions. The [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Successful retrieval examples with MCoT-RE from CIRR. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages

  1. [8]

    Training-free zero-shot com- posed image retrieval via weighted modality fusion and similarity,

    R.-D. Wu, Y .-Y . Lin, and H.-F. Yang, “Training-free zero-shot com- posed image retrieval via weighted modality fusion and similarity,” arXiv preprint arXiv:2409.04918 , 2024

    arXiv 2024

  2. [11]

    LDRE: LLM- based divergent reasoning and ensemble for zero-shot composed image retrieval,

    Z. Yang, D. Xue, S. Qian, W. Dong, and C. Xu, “LDRE: LLM- based divergent reasoning and ensemble for zero-shot composed image retrieval,” in Proc. Int. ACM SIGIR Conf. Res. Dev. Inf. Retr. (SIGIR) , 2024, pp. 80–90

    work page 2024

  3. [13]

    Reason-before-retrieve: One-stage reflective chain-of- thoughts for training-free zero-shot composed image retrieval,

    Y . Tang et al., “Reason-before-retrieve: One-stage reflective chain-of- thoughts for training-free zero-shot composed image retrieval,” arXiv preprint arXiv:2412.11077, 2024

    arXiv 2024

  4. [12]

    CoTMR: chain-of-thought multi-scale reasoning for training-free zero-shot composed image retrieval,

    Z. Sun, D. Jing, and Z. Lu, “CoTMR: chain-of-thought multi-scale reasoning for training-free zero-shot composed image retrieval,” arXiv preprint arXiv:2502.20826, 2025

    arXiv 2025

  5. [7]

    Vision-by- language for training-free compositional image retrieval,

    S. Karthik, K. Roth, M. Mancini, and Z. Akata, “Vision-by- language for training-free compositional image retrieval,” arXiv preprint arXiv:2310.09291, 2023

    arXiv 2023

  6. [9]

    Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxy

    Y . Li, F. Ma, and Y . Yang, “Imagine and seek: Improving com- posed image retrieval with an imagined proxy,” arXiv preprint arXiv:2411.16752, 2024

    work page Pith review arXiv 2024

  7. [10]

    MLLM-I2W: Harnessing multimodal large language model for zero-shot composed image retrieval,

    T. Bao, C. Liu, D. Xu, Z. Zheng, and T. Xu, “MLLM-I2W: Harnessing multimodal large language model for zero-shot composed image retrieval,” in Proc. of the 31st Int. Conf. on Comput. Linguistics (COLING), 2025, pp. 1839–1849

    work page 2025

  8. [1]

    Composing text and image for image retrieval-an empirical odyssey,

    N. V o et al. , “Composing text and image for image retrieval-an empirical odyssey,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2019, pp. 6439–6448

    work page 2019

Show all 23 references
  1. [2]

    Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things,

    J. Zhang and D. Tao, “Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things,” IEEE Internet Things J. , vol. 8, pp. 7789–7817, 2020

    2020

  2. [3]

    Comprehensive linguistic-visual composition network for image retrieval,

    H. Wen, X. Song, X. Yang, Y . Zhan, and L. Nie, “Comprehensive linguistic-visual composition network for image retrieval,” in Proc. 44th Int. ACM SIGIR Conf. Res. Dev. Inf. Retr. (SIGIR) , 2021, pp. 1369–1378

    2021

  3. [4]

    Pic2Word: mapping pictures to words for zero-shot composed image retrieval,

    K. Saito et al. , “Pic2Word: mapping pictures to words for zero-shot composed image retrieval,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 19 305–19 314

    2023

  4. [5]

    Zero-shot composed image retrieval with textual inversion,

    A. Baldrati, L. Agnolucci, M. Bertini, and A. Del Bimbo, “Zero-shot composed image retrieval with textual inversion,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , 2023, pp. 15 338–15 347

    2023

  5. [6]

    Learning transferable visual models from natural language supervision,

    A. Radford et al. , “Learning transferable visual models from natural language supervision,” in Int. Conf. Mach. Learn. (ICML) , 2021, pp. 8748–8763

    2021

  6. [14]

    FashionIQ: A new dataset towards retrieving images by natural language feedback,

    H. Wu et al. , “FashionIQ: A new dataset towards retrieving images by natural language feedback,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, pp. 11 307–11 317

    2021

  7. [15]

    Image retrieval on real-life images with pre-trained vision-and-language models,

    Z. Liu, C. Rodriguez-Opazo, D. Teney, and S. Gould, “Image retrieval on real-life images with pre-trained vision-and-language models,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2021, pp. 2125–2134

    2021

  8. [16]

    CompoDiff: Versatile composed image retrieval with latent diffusion,

    G. Gu et al. , “CompoDiff: Versatile composed image retrieval with latent diffusion,” Trans. Mach. Learn. Res. , 2024

    2024

  9. [17]

    Language-only training of zero-shot composed image retrieval,

    G. Gu, S. Chun, W. Kim, Y . Kang, and S. Yun, “Language-only training of zero-shot composed image retrieval,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2024, pp. 13 225– 13 234

    2024

  10. [18]

    Uncertainty-aware mesh decoder for high fidelity 3d face reconstruction,

    G.-H. Lee and S.-W. Lee, “Uncertainty-aware mesh decoder for high fidelity 3d face reconstruction,” in Proc. IEEE/CVF Con. Comput. Vis. Pattern Recognit. (CVPR), 2020, pp. 6100–6109

    2020

  11. [19]

    Automatic video parsing using shot boundary detection and camera operation analysis,

    M.-S. Lee, Y .-M. Yang, and S.-W. Lee, “Automatic video parsing using shot boundary detection and camera operation analysis,” Pattern Recognit., vol. 34, no. 3, pp. 711–719, 2001

    2001

  12. [20]

    Multilayer cluster neural network for totally unconstrained handwritten numeral recognition,

    S.-W. Lee, “Multilayer cluster neural network for totally unconstrained handwritten numeral recognition,” Neural Networks, vol. 8, no. 5, pp. 783–792, 1995

    1995

  13. [21]

    Information captur- ing camera and developmental issues,

    H. Fujisawa, H. Sako, Y . Okada, and S.-W. Lee, “Information captur- ing camera and developmental issues,” in Proc. Int. Conf. Document Anal. Recognit., 1999, pp. 205–208

    1999

  14. [22]

    Gemini: a family of highly capable multimodal models,

    G. Team et al. , “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805 , 2023

    2023 arXiv

  15. [23]

    Multiresolution recog- nition of unconstrained handwritten numerals with wavelet transform and multilayer cluster neural network,

    S.-W. Lee, C.-H. Kim, H. Ma, and Y . Y . Tang, “Multiresolution recog- nition of unconstrained handwritten numerals with wavelet transform and multilayer cluster neural network,” Pattern Recognit. , vol. 29, no. 12, pp. 1953–1961, 1996

    1953

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.