Composite Sketch+Text Queries for Retrieving Objects with Elusive Names and Complex Interactions

Anand Mishra; Dhriti Prasanna Paul; Kshitij Parikh; Manish Gupta; Prajwal Gatti

REVIEW 3 major objections 3 minor 61 references

Composite Sketch+Text Queries for Retrieving Objects with Elusive Names and Complex Interactions

T0 review · 3 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes composite sketch+text image retrieval as a distinct practical task and presents STNET, a multimodal transformer that jointly encodes a rough object sketch and a short complementary text to rank natural scene images.

desk verdict Useful new benchmark, but the core claim that the text is complementary rests on an undocumented data-construction step; acceptance depends on verifying the masking of object names. read the letter →

arxiv 2502.08438 v1 pith:KBRTUTLD submitted 2025-02-12 cs.CV cs.AIcs.CLcs.IRcs.MM

Prajwal Gatti , Kshitij Parikh , Dhriti Prasanna Paul , Manish Gupta , Anand Mishra This is my paper · ORCID

classification cs.CVcs.AIcs.CLcs.IRcs.MM

keywords compositesketch+textimageretrievalelusiveobjectnamesmultimodalquerynaturalscenecontrastivelearningsketch-guideddetectionCSTBIRdatasetSTNETmodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that users who cannot name an object but can sketch it, and who can describe interactions that are hard to draw, need a retrieval interface that combines both modalities. It introduces CSTBIR, a dataset of about 2 million composite queries paired with 108K natural scene images, built by pairing Quick, Draw! sketches with Visual Genome region descriptions. Its proposed model, STNET, jointly encodes sketch and text, uses the sketch to attend to the relevant object in the image, and is trained with contrastive, classification, object-detection, and sketch-reconstruction losses. The reported results support the central claim: on Test-1K, STNET achieves R@10 of 73.7, compared with 67.0 for a two-stage baseline that converts the sketch to an object name and 50.6 for text-only CLIP. If the results hold, the contribution is a new benchmark and a strong baseline for the composite sketch+text retrieval setting.

What carries the argument

The central mechanism is STNET, a multimodal transformer with three encoders: a CLIP text encoder, a ViT sketch encoder, and a CLIP image encoder. The sketch embedding computes dot-product attention over the image patch embeddings, producing a sketch-guided image representation that focuses on the queried object. Training combines an InfoNCE contrastive loss with three auxiliary objectives: object classification from both text and image encodings, a YOLO-style sketch-guided object detection loss over the attended image grid, and a sketch reconstruction loss that decodes image features back into the query sketch. These auxiliary objectives are what push the joint embedding to be object-centric, forcing the text and image to carry object identity, the image representation to localize the object, and the representation to retain sketchable shape information.

What would settle it

Inspect the CSTBIR text descriptions for object names or near-synonyms: if a large fraction of the held-out text sentences contains the category label or a word that identifies the category, then the benchmark does not test the elusive-name scenario and the claimed advantage over text-only retrieval could come from name leakage. A simple frequency analysis of the 258 category names against the text tokens in the test sets would settle this.

Watch

Extended reading notes

Core claim

The paper's central claim is that composite sketch+text queries form a distinct retrieval modality: the sketch supplies the object's visual identity, the text supplies complementary information such as pose, color, location, or interaction, and the two together solve searches that fail when either modality is used alone. Concretely, it claims that jointly encoding sketch and text with a multimodal transformer, trained with contrastive loss plus object classification, sketch-guided object detection, and sketch reconstruction losses, outperforms text-only CLIP, sketch-only models, TIRG, Taskformer, and a two-stage pipeline that first predicts the object name and then performs text retrieval. The paper reports R@10 of 73.7 on Test-1K and 38.7 on Test-5K, and 37.2 on an open-category test set of 70 unseen object classes. It positions CSTBIR as the first rigorous study of this setting, with approximately 2M composite queries over 108K natural scene images.

Load-bearing premise

The load-bearing premise is that the text part of each query is genuinely partial and complementary, meaning it does not reveal the object's name; the paper draws text from Visual Genome region descriptions without describing how object names are suppressed, and it acknowledges that the main dataset contains no truly difficult-to-name objects, so name leakage or a too-easy vocabulary would undercut the stated motivation.

Editorial extensions

If this is right

A user interface accepting a rough object sketch plus a short complementary text can retrieve images from large natural-scene galleries, with reported R@10 of 73.7 on Test-1K and 38.7 on the harder Test-5K.
Jointly encoding sketch and text outperforms a two-stage pipeline that first names the object from the sketch and then runs text retrieval, with the gap widening on instance-level sketches where the two-stage model loses pose and shape detail.
Adding object classification, sketch-guided detection, and sketch reconstruction losses each improves retrieval, and removing the classification loss hurts most, indicating that object-centric auxiliary objectives are useful for this task.
The model generalizes to unseen object categories: on an open-category set of 70 novel classes it reaches R@10 of 37.2 versus 29.0 for the two-stage baseline, suggesting the sketch encoder captures visual semantics beyond a fixed category list.
On the CSTBIR benchmark, sketch-only retrieval is weaker than text-only retrieval, and composite retrieval is stronger than either, which directly supports the complementarity premise of the task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, a natural next step is to isolate how much of the gain comes from the sketch-guided attention pooling versus the auxiliary losses, since the contrastive loss alone already yields 68.4 R@10 on Test-1K.
The classification loss may be quietly teaching the text encoder to recover the object name from the text, which would undermine the partial-text assumption; an audit counting how often category names or synonyms appear in the text could settle this.
The dataset could be extended toward true open-vocabulary retrieval by replacing the fixed 258-class classification head with a text-conditioned classifier, removing the closed-world limitation for novel objects.
The same benchmark could directly evaluate the sketch-guided attention as an object localizer, measuring how often the attended image regions contain the ground-truth object box, which would tell whether retrieval gains come from localization or from generic CLIP embeddings.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline: this is a genuinely new benchmark and a reasonable baseline model, but the defining data-construction step is not documented. The paper claims the text is 'complementary' and excludes the object name, yet Section 3 only says the text comes from Visual Genome region descriptions. There is no description of any masking or filtering to remove object names. Figure 2 even shows blanks where the name would appear, but the text never says such blanks were inserted. That matters because Section 4 uses a text-classification loss premised on 'the object's label is not mentioned in the text sentence.' That premise is asserted, not established. If VG descriptions retain object names, the dataset reduces to text-only retrieval in practice and the elusive-name motivation collapses. The code/dataset release could resolve this, but the paper as written does not.

What is genuinely new: the task framing (object-level sketch plus partial text over natural scenes) and the scale of the dataset. Prior work used focused-object images or scene-level sketches, and none paired Quick Draw object sketches with VG region descriptions to produce ~2M queries. The authors also include a separate Open-Category test set with 50 hard-to-name objects, which is a sensible attempt to evaluate the actual motivating scenario. STNET is not architecturally novel—it is a composition of CLIP encoders, dot-product attention, and off-the-shelf losses—but it is a reasonable strong baseline, and the ablations show each loss contributes.

Soft spots in proportion: the biggest is the complementarity gap described above. Beyond that, results lack error bars; the gains over the two-stage baseline (73.7 vs 67.0 on R@10) are plausible but could be within noise for a single seed. There is also a minor inconsistency in the ablation description: Model 1 is sketch-only, but the authors say they do not use sketch-image attention there, whereas the full model does; worth clarifying. The two-stage baselines are given an advantage by being augmented with the 70 test classes on the Open-Category set, while STNET is not retrained—that is actually a fair stress test, but it should be stated more clearly.

Overall: the paper is worth a serious referee. The dataset is a real asset, and the task is well-motivated. But reviewing must verify the text-construction pipeline before accepting the benchmark claim. Ask for the dataset curation code, a sample of masked versus original texts, and multi-seed results. Who this is for: people working on sketch-based retrieval, composed multimodal retrieval, and benchmark construction. If the complementarity issue is resolved, it likely becomes a standard testbed.

Referee Report

3 major / 3 minor

Summary. The paper introduces CSTBIR, a composite sketch+text image retrieval task and dataset built from Visual Genome region descriptions and Quick, Draw! sketches, yielding roughly 2M composite queries over 108K natural scene images. It proposes STNET, a CLIP-based multimodal encoder trained with contrastive, object classification, sketch-guided detection, and sketch-reconstruction losses. On Test-1K, Test-5K, an Open-Category set, and an instance-level sketch variant, the authors report that STNET outperforms text-only, sketch-only, and other composite-query baselines; for example, R@10 of 73.7 versus 67.0 for the two-stage baseline on Test-1K.

Significance. If the benchmark construction is valid, this is a useful new task and a strong baseline, and the paper has concrete strengths: the dataset and code are released, the ablation in Table 4 isolates the contribution of each loss, and the Open-Category test set provides a meaningful generalization probe for unseen object classes. However, the central claim that the text is 'partial/complementary' is not yet substantiated. The task definition, the text-only baseline, and the interpretation of the classification loss all depend on the text not revealing the target object's name, and the paper does not specify how this property is enforced. The significance of the reported gains therefore cannot be fully assessed until the dataset construction is documented and verified.

major comments (3)

[Section 3, Figure 2, Table 1] The defining property of the benchmark is that the text is 'partial/complementary' to the sketch, but the dataset construction never states how Visual Genome region descriptions are converted into such text. Figure 2 shows queries with blanks where the object name would appear (e.g., 'Person dressed in a suit standing beside a .'), yet Section 3 only says that images and text descriptions are 'taken from Visual Genome' and paired with Quick, Draw! sketches, with no masking, deletion, or filtering procedure described. It is also not reported how often the target object's name or a synonym remains in the text. Since Text-Only CLIP already reaches R@10 50.6 and the two-stage baseline inserts the predicted name into 'the incomplete text,' the entire elusive-name scenario rests on this unstated preprocessing. The released code and data may resolve the question, but as written the benchmark claim is unsubstantiated.
[Section 4, Object Classification (LT_CLS)] The rationale for the text classification loss is that 'the object's label is not mentioned in the text sentence,' but this property is never established in Section 3. If object names are present in the text, LT_CLS can be solved by name matching rather than contextual reasoning, and the same leakage would inflate the text-only CLIP baseline, making the reported gains of STNET harder to interpret. Please specify the exact preprocessing and report a quantitative estimate of name leakage in the released text, such as the fraction of text descriptions containing the ground-truth class name, a synonym, or a hypernym.
[Section 5, Baselines (Two-stage models)] The two-stage baselines 'insert the predicted object name into the incomplete text' before querying CLIP. Because Section 3 does not explain how the text is made incomplete or whether it contains explicit placeholders, the input to the second stage is not well defined; without a documented blanking rule, the baseline may be filling in text that already names the object, or may be unable to produce a grammatical full query. Please specify the blanking and insertion format and include examples of the intermediate full-text queries used in the experiments.

minor comments (3)

[Table 2] The table reports 'Test Instances 5000,' but the text describes Test-1K with 1K queries, Test-5K with 4K queries, and an Open-Category set with 750 queries, which sums to 5,750. Please reconcile these numbers.
[Section 5, Experiments and Results] There is an editorial typo in the Taskformer discussion: 'we finetuned the publicly available checkpoint using ouris because' should read 'using our dataset. This is because' or similar.
[Section 3, Dataset limitations] The sentence 'Although this dataset does not have difficult-to-name objects, it is a reasonable proxy' is a useful caveat, but its force depends on the text being genuinely complementary; if object names are present in the text, the proxy property is not established.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: STNET is trained and evaluated on held-out splits, and the reported benchmark numbers are independent of any fitted constants or self-citation chain.

full rationale

The paper's central empirical claim is that the CSTBIR dataset instantiates a new composite sketch+text retrieval task and that STNET outperforms strong baselines. The derivation chain is not circular in the sense relevant here: the dataset is assembled from Visual Genome region descriptions, Quick, Draw! sketches, and object-category intersections; the model is trained with contrastive, classification, detection, and reconstruction losses; and retrieval quality is measured on held-out Test-1K, Test-5K, and Open-Category galleries. None of the reported R@K or median-rank numbers is obtained by re-inserting a fitted parameter or by computing a quantity that equals its own input by construction. The auxiliary classification and detection losses use the same object labels that define the benchmark, but that is ordinary supervised learning rather than a prediction that reduces to a fit. The paper cites prior work by one of its authors (Tripathi et al. 2020/2023; Pandey et al. 2020) only as related work or architectural inspiration, not as a load-bearing uniqueness theorem, so no self-citation chain forces the result. The main weakness is a benchmark-validity concern, not circularity: the manuscript asserts that the Visual Genome text is 'partial' or 'complementary' yet does not describe how object names are removed or masked, and the text-classification loss assumes the label is absent from the sentence. An unverified or underdocumented premise is different from a derivation that is equivalent to its inputs, and no specific circular reduction can be exhibited from the text provided. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the reliability of Visual Genome object and region annotations, Quick Draw sketches as category-consistent user sketches, CLIP pretrained representations, and the recall-at-K protocol. Loss weights and detection-grid sizes are chosen hyperparameters. No new theoretical entities are introduced.

free parameters (4)

alpha, beta (sketch reconstruction loss weights) = not reported in main text
Weights for LSR = alpha*LBCE + beta*LDICE; no sensitivity analysis provided.
Detection grid S=7, B=2 = 7x7x268 output
YOLO-style head hyperparameters chosen without ablation.
Sketches per category sampled from Quick Draw = 10K
Randomly sampled 10K sketches per category for pairing; number chosen for dataset size.
Stroke thinning normalization = not specified
Applied to compare sketch pixel coverage with prior work; preprocessing choice.

assumptions (5)

domain assumption Visual Genome region descriptions and object boxes are reliable for generating query text and localization targets.
Dataset is built by pairing VG annotations with Quick Draw sketches; errors propagate.
domain assumption Quick Draw sketches are representative of user-drawn object sketches and category-consistent with VG objects.
Only 258 intersecting categories used; rough sketches from non-experts assumed to match the task.
domain assumption CLIP text and image encoders transfer to this sketch+text retrieval setting.
STNET relies on pretrained CLIP without re-training from scratch; if CLIP features do not align well with sketches, results would degrade.
domain assumption Single-ground-truth Recall@K is a valid evaluation protocol even when multiple gallery images contain the same object.
The gallery may contain several images with the query object; only the paired image is counted as correct.
domain assumption The Open-Category set (70 novel classes, 50 rare) is a fair test of generalization.
Manual curation and sketch collection for novel classes are not described; validity assumed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Composite Sketch+Text Queries for Retrieving Objects with Elusive Names and Complex Interactions." pith.science (2026). https://pith.science/paper/KBRTUTLD

@misc{pith2026250208438,
  author       = {Pith},
  title        = {Pith review of: Composite Sketch+Text Queries for Retrieving Objects with Elusive Names and Complex Interactions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KBRTUTLD}},
  note         = {Machine review of arXiv:2502.08438}
}

read the original abstract

Non-native speakers with limited vocabulary often struggle to name specific objects despite being able to visualize them, e.g., people outside Australia searching for numbats. Further, users may want to search for such elusive objects with difficult-to-sketch interactions, e.g., numbat digging in the ground. In such common but complex situations, users desire a search interface that accepts composite multimodal queries comprising hand-drawn sketches of difficult-to-name but easy-to-draw objects and text describing difficult-to-sketch but easy-to-verbalize object attributes or interaction with the scene. This novel problem statement distinctly differs from the previously well-researched TBIR (text-based image retrieval) and SBIR (sketch-based image retrieval) problems. To study this under-explored task, we curate a dataset, CSTBIR (Composite Sketch+Text Based Image Retrieval), consisting of approx. 2M queries and 108K natural scene images. Further, as a solution to this problem, we propose a pretrained multimodal transformer-based baseline, STNET (Sketch+Text Network), that uses a hand-drawn sketch to localize relevant objects in the natural scene image, and encodes the text and image to perform image retrieval. In addition to contrastive learning, we propose multiple training objectives that improve the performance of our model. Extensive experiments show that our proposed method outperforms several state-of-the-art retrieval methods for text-only, sketch-only, and composite query modalities. We make the dataset and code available at our project website.

Figures

Figures reproduced from arXiv: 2502.08438 by the authors.

**Figure 2.** Examples from our dataset – CSTBIR. It contains [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Overview of the proposed method, STNET for the CSTBIR problem. CLIP. Further, as our model utilizes the pretrained CLIP, which lacks joint modeling of text, sketch, and image modalities, we propose three additional training losses to be optimized concomitantly with the contrastive objective. Object Classification (L T CLS and L I CLS) Given that the CSTBIR problem focuses on object-specific queries, we propose sep… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Qualitative results of our STNET. We show top-5 retrieved results for the multimodal (sketch+text) queries shown in left most column. From top to bottom, the sketch are for capybara, sitar, penny-farthing, and okapi. The ground truth image is shown with a green frame. …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 55 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Baldrati, A.; Bertini, M.; Uricchio, T.; and Del Bimbo, A. 2022. Effective conditioned and composed image retrieval combining CLIP-based features. In CVPR

work page 2022
[4]

K.; Chowdhury, P

Bhunia, A. K.; Chowdhury, P. N.; Yang, Y.; Hospedales, T. M.; Xiang, T.; and Song, Y.-Z. 2021. Vectorization and rasterization: Self-supervised learning for sketch and handwriting. In CVPR

work page 2021
[5]

K.; Sain, A.; Shah, P

Bhunia, A. K.; Sain, A.; Shah, P. H.; Gupta, A.; Chowdhury, P. N.; Xiang, T.; and Song, Y.-Z. 2022. Adaptive fine-grained sketch-based image retrieval. In ECCV

work page 2022
[6]

K.; Yang, Y.; Hospedales, T

Bhunia, A. K.; Yang, Y.; Hospedales, T. M.; Xiang, T.; and Song, Y.-Z. 2020. Sketch less for more: On-the-fly fine-grained sketch-based image retrieval. In CVPR

work page 2020
[7]

Changpinyo, S.; Pont-Tuset, J.; Ferrari, V.; and Soricut, R. 2021. Telling the what while pointing to the where: Multimodal queries for image retrieval. In ICCV

work page 2021
[8]

N.; Bhunia, A

Chowdhury, P. N.; Bhunia, A. K.; Sain, A.; Koley, S.; Xiang, T.; and Song, Y.-Z. 2023 a . SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text. In CVPR

work page 2023

Show all 61 references

[9]

N.; Bhunia, A

Chowdhury, P. N.; Bhunia, A. K.; Sain, A.; Koley, S.; Xiang, T.; and Song, Y.-Z. 2023 b . What Can Human Sketches Do for Object Detection? In CVPR

2023
[10]

N.; Sain, A.; Bhunia, A

Chowdhury, P. N.; Sain, A.; Bhunia, A. K.; Xiang, T.; Gryaditskaya, Y.; and Song, Y.-Z. 2022. FS-COCO: Towards understanding of freehand sketches of common objects in context. In ECCV

2022
[11]

Collomosse, J.; Bui, T.; and Jin, H. 2019. Livesketch: Query perturbations for guided sketch-based visual search. In CVPR

2019
[12]

Dey, S.; Riba, P.; Dutta, A.; Llados, J.; and Song, Y.-Z. 2019. Doodle to search: Practical zero-shot sketch-based image retrieval. In CVPR

2019
[13]

Dodds, E.; Culpepper, J.; Herdade, S.; Zhang, Y.; and Boakye, K. 2020. Modality-agnostic attention fusion for visual search with text feedback. arXiv preprint arXiv:2007.00145

2020 arXiv
[14]

Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR

2021
[15]

Dutta, A.; and Akata, Z. 2019. Semantically tied paired cycle consistency for zero-shot sketch-based image retrieval. In CVPR

2019
[16]

Eitz, M.; Hays, J.; and Alexa, M. 2012. How do humans sketch objects? ACM Transactions on graphics (TOG)

2012
[17]

Gomez, R.; Gibert, J.; Gomez, L.; and Karatzas, D. 2020. Location sensitive image retrieval and tagging. In ECCV

2020
[18]

Guo, X.; Wu, H.; Cheng, Y.; Rennie, S.; Tesauro, G.; and Feris, R. 2018. Dialog-based interactive image retrieval. NeurIPS

2018
[19]

Ha, D.; and Eck, D. 2018. A Neural Representation of Sketch Drawings. In ICLR

2018
[20]

X.; Zhang, X.; Zhu, M.; Li, Y.; Zhao, Y.; and Davis, L

Han, X.; Wu, Z.; Huang, P. X.; Zhang, X.; Zhu, M.; Li, Y.; Zhao, Y.; and Davis, L. S. 2017. Automatic spatially-aware fashion concept discovery. In ICCV

2017
[21]

He, K.; Zhang, X.; Ren, S.; and Sun, J. 2015. Deep Residual Learning for Image Recognition. CVPR

2015
[22]

Isola, P.; Zhu, J.-Y.; Zhou, T.; and Efros, A. A. 2017. Image-to-image translation with conditional adversarial networks. In CVPR

2017
[23]

Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q.; Sung, Y.-H.; Li, Z.; and Duerig, T. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML

2021
[24]

Kim, W.; Son, B.; and Kim, I. 2021. Vilt: Vision-and-language transformer without convolution or region supervision. In ICML

2021
[25]

Kovashka, A.; Parikh, D.; and Grauman, K. 2012. Whittlesearch: Image search with relative attribute feedback. In CVPR

2012
[26]

A.; et al

Krishna, R.; Zhu, Y.; Groth, O.; Johnson, J.; Hata, K.; Kravitz, J.; Chen, S.; Kalantidis, Y.; Li, L.-J.; Shamma, D. A.; et al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJCV

2017
[27]

Lee, K.-H.; Chen, X.; Hua, G.; Hu, H.; and He, X. 2018. Stacked cross attention for image-text matching. In ECCV

2018
[28]

H.; Yatskar, M.; Yin, D.; Hsieh, C.-J.; and Chang, K.-W

Li, L. H.; Yatskar, M.; Yin, D.; Hsieh, C.-J.; and Chang, K.-W. 2020 a . What Does BERT with Vision Look At? In ACL

2020
[29]

Li, M.; Lin, Z.; Mech, R.; Yumer, E.; and Ramanan, D. 2019. Photo-sketching: Inferring contour drawings from images. In WACV

2019
[30]

Li, X.; Yin, X.; Li, C.; Zhang, P.; Hu, X.; Zhang, L.; Wang, L.; Hu, H.; Dong, L.; Wei, F.; et al. 2020 b . Oscar: Object-semantics aligned pre-training for vision-language tasks. In ECCV

2020
[31]

Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In ECCV

2014
[32]

Ling, Z.; Xing, Z.; Zhou, J.; and Zhou, X. 2022. Conditional Stroke Recovery for Fine-Grained Sketch-Based Image Retrieval. In ECCV

2022
[33]

Liu, F.; Zou, C.; Deng, X.; Zuo, R.; Lai, Y.-K.; Ma, C.; Liu, Y.-J.; and Wang, H. 2020. Scenesketcher: Fine-grained image retrieval with scene sketches. In ECCV

2020
[34]

Liu, L.; Shen, F.; Shen, Y.; Liu, X.; and Shao, L. 2017. Deep sketch hashing: Fast free-hand sketch-based image retrieval. In CVPR

2017
[35]

Nakatsuka, T.; Hamasaki, M.; and Goto, M. 2023. Content-Based Music-Image Retrieval Using Self-and Cross-Modal Feature Embedding Memory. In WACV

2023
[36]

Oord, A. v. d.; Li, Y.; and Vinyals, O. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

2018 arXiv
[37]

K.; Mittal, A.; and Murthy, H

Pandey, A.; Mishra, A.; Verma, V. K.; Mittal, A.; and Murthy, H. 2020. Stacked adversarial network for zero-shot sketch based image retrieval. In WACV

2020
[38]

M.; Xiang, T.; and Song, Y.-Z

Pang, K.; Li, K.; Yang, Y.; Zhang, H.; Hospedales, T. M.; Xiang, T.; and Song, Y.-Z. 2019. Generalising fine-grained sketch-based image retrieval. In CVPR

2019
[39]

Pang, K.; Song, Y.-Z.; Xiang, T.; and Hospedales, T. M. 2017. Cross-domain Generative Learning for Fine-Grained Sketch-Based Image Retrieval. In BMVC

2017
[40]

Pont-Tuset, J.; Uijlings, J.; Changpinyo, S.; Soricut, R.; and Ferrari, V. 2020. Connecting vision and language with localized narratives. In ECCV

2020
[41]

W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML

2021
[42]

Redmon, J.; Divvala, S.; Girshick, R.; and Farhadi, A. 2016. You only look once: Unified, real-time object detection. In CVPR

2016
[43]

Ribeiro, L. S. F.; Bui, T.; Collomosse, J. P.; and Ponti, M. A. 2020. Sketchformer: Transformer-Based Representation for Sketched Structure. CVPR

2020
[44]

B.; Noy, A.; and Zelnik, L

Ridnik, T.; Baruch, E. B.; Noy, A.; and Zelnik, L. 2021. ImageNet-21K Pretraining for the Masses. In NeurIPS Track on Datasets and Benchmarks

2021
[45]

K.; Potlapalli, V.; Chowdhury, P

Sain, A.; Bhunia, A. K.; Potlapalli, V.; Chowdhury, P. N.; Xiang, T.; and Song, Y.-Z. 2022. Sketch3t: Test-time training for zero-shot sbir. In CVPR

2022
[46]

K.; Yang, Y.; Xiang, T.; and Song, Y.-Z

Sain, A.; Bhunia, A. K.; Yang, Y.; Xiang, T.; and Song, Y.-Z. 2021. Stylemeup: Towards style-agnostic sketch-based image retrieval. In CVPR

2021
[47]

Sangkloy, P.; Jitkrittum, W.; Yang, D.; and Hays, J. 2022. A Sketch is Worth a Thousand Words: Image Retrieval with Text and Sketch. In ECCV

2022
[48]

H.; Yoon, J.; Choi, S.; and Avrithis, Y

Song, C. H.; Yoon, J.; Choi, S.; and Avrithis, Y. 2023. Boosting vision transformers for image retrieval. In WACV

2023
[49]

Song, J.; Song, Y.-Z.; Xiang, T.; and Hospedales, T. M. 2017 a . Fine-Grained Image Retrieval: the Text/Sketch Input Dilemma. In BMVC

2017
[50]

Song, J.; Yu, Q.; Song, Y.-Z.; Xiang, T.; and Hospedales, T. M. 2017 b . Deep spatial-semantic attention for fine-grained sketch-based image retrieval. In ICCV

2017
[51]

H.; Li, W.; Vercauteren, T.; Ourselin, S.; and Jorge Cardoso, M

Sudre, C. H.; Li, W.; Vercauteren, T.; Ourselin, S.; and Jorge Cardoso, M. 2017. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In DLMIA/ML-CDS@MICCAI

2017
[52]

Sun, Z.; Wang, C.; Zhang, L.; and Zhang, L. 2012. Sketch2Tag: automatic hand-drawn sketch recognition. In ACM-MM

2012
[53]

Tian, Y.; Newsam, S.; and Boakye, K. 2023. Fashion Image Retrieval With Text Feedback by Additive Attention Compositional Learning. In WACV

2023
[54]

R.; Mishra, A.; and Chakraborty, A

Tripathi, A.; Dani, R. R.; Mishra, A.; and Chakraborty, A. 2020. Sketch-guided object localization in natural images. In ECCV

2020
[55]

R.; Mishra, A.; and Chakraborty, A

Tripathi, A.; Dani, R. R.; Mishra, A.; and Chakraborty, A. 2023. Multimodal query-guided object localization. Multimedia Tools and Applications

2023
[56]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. NeurIPS

2017
[57]

Vo, N.; Jiang, L.; Sun, C.; Murphy, K.; Li, L.-J.; Fei-Fei, L.; and Hays, J. 2019. Composing text and image for image retrieval-an empirical odyssey. In CVPR

2019
[58]

Young, P.; Lai, A.; Hodosh, M.; and Hockenmaier, J. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL

2014
[59]

M.; and Loy, C.-C

Yu, Q.; Liu, F.; Song, Y.-Z.; Xiang, T.; Hospedales, T. M.; and Loy, C.-C. 2016. Sketch Me That Shoe. In CVPR

2016
[60]

Zhang, Q.; Lei, Z.; Zhang, Z.; and Li, S. Z. 2020. Context-aware attention network for image-text retrieval. In CVPR

2020
[61]

Zhou, W.; Li, H.; and Tian, Q. 2017. Recent advance in content-based image retrieval: A literature survey. arXiv preprint arXiv:1706.06064

2017 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Baldrati, A.; Bertini, M.; Uricchio, T.; and Del Bimbo, A. 2022. Effective conditioned and composed image retrieval combining CLIP-based features. In CVPR

work page 2022

[4] [4]

K.; Chowdhury, P

Bhunia, A. K.; Chowdhury, P. N.; Yang, Y.; Hospedales, T. M.; Xiang, T.; and Song, Y.-Z. 2021. Vectorization and rasterization: Self-supervised learning for sketch and handwriting. In CVPR

work page 2021

[5] [5]

K.; Sain, A.; Shah, P

Bhunia, A. K.; Sain, A.; Shah, P. H.; Gupta, A.; Chowdhury, P. N.; Xiang, T.; and Song, Y.-Z. 2022. Adaptive fine-grained sketch-based image retrieval. In ECCV

work page 2022

[6] [6]

K.; Yang, Y.; Hospedales, T

Bhunia, A. K.; Yang, Y.; Hospedales, T. M.; Xiang, T.; and Song, Y.-Z. 2020. Sketch less for more: On-the-fly fine-grained sketch-based image retrieval. In CVPR

work page 2020

[7] [7]

Changpinyo, S.; Pont-Tuset, J.; Ferrari, V.; and Soricut, R. 2021. Telling the what while pointing to the where: Multimodal queries for image retrieval. In ICCV

work page 2021

[8] [8]

N.; Bhunia, A

Chowdhury, P. N.; Bhunia, A. K.; Sain, A.; Koley, S.; Xiang, T.; and Song, Y.-Z. 2023 a . SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text. In CVPR

work page 2023