Pith. sign in

REVIEW 5 cited by

iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrieval

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.02951 v2 pith:572UXXAA submitted 2024-05-05 cs.CV cs.IR

Lorenzo Agnolucci , Alberto Baldrati , Alberto Del Bimbo , Marco Bertini This is my paper · ORCID
classification cs.CVcs.IR
keywords imagecomposeddatasetretrievalcaptionisearlelabeledreference
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Given a query consisting of a reference image and a relative caption, Composed Image Retrieval (CIR) aims to retrieve target images visually similar to the reference one while incorporating the changes specified in the relative caption. The reliance of supervised methods on labor-intensive manually labeled datasets hinders their broad applicability. In this work, we introduce a new task, Zero-Shot CIR (ZS-CIR), that addresses CIR without the need for a labeled training dataset. We propose an approach named iSEARLE (improved zero-Shot composEd imAge Retrieval with textuaL invErsion) that involves mapping the visual information of the reference image into a pseudo-word token in CLIP token embedding space and combining it with the relative caption. To foster research on ZS-CIR, we present an open-domain benchmarking dataset named CIRCO (Composed Image Retrieval on Common Objects in context), the first CIR dataset where each query is labeled with multiple ground truths and a semantic categorization. The experimental results illustrate that iSEARLE obtains state-of-the-art performance on three different CIR datasets -- FashionIQ, CIRR, and the proposed CIRCO -- and two additional evaluation settings, namely domain conversion and object composition. The dataset, the code, and the model are publicly available at https://github.com/miccunifi/SEARLE.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Simple Edits: Composed Video Retrieval with Dense Modifications

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A new benchmark with much longer, denser modification texts, plus a single-encoder fusion model, raises composed video retrieval Recall@1 by 3.4 points on its own test set.

  2. Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A zero-shot composed image retrieval model trained on synthetic triplets, generated by an MLLM from moderately similar unlabeled image pairs, beats prior methods on three benchmarks.

  3. MLLM-Guided VLM Fine-Tuning with Joint Inference for Zero-Shot Composed Image Retrieval

    cs.CV 2025-05 conditional novelty 6.0 of 10

    MVFT-JI trains a Q-Former VLM with two MLLM-generated retrieval tasks and fuses VLM and MLLM similarities at inference, achieving state-of-the-art zero-shot composed image retrieval on three benchmarks.

  4. Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion

    cs.CV 2025-02 conditional novelty 6.0 of 10

    CLIP's intra-modal embeddings are miscalibrated: converting one side of an image-image or text-text comparison into the other modality improves retrieval accuracy on 15+ datasets.

  5. Zero Shot Composed Image Retrieval

    cs.CV 2025-06 reject novelty 2.0 of 10

    Fine-tuning BLIP-2 with a Q-Former raises FashionIQ validation Recall@10 to roughly 45%, but the zero-shot framing is inaccurate and the DPO variant omits the reference image.

Pith tools