Pith. sign in

REVIEW 3 major objections 4 minor 54 references

Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FemmIR claims that multimodal retrieval can be taught with edit-distance weak labels instead of similarity annotations or fine-tuning.

desk verdict The edit-distance weak supervision idea and HARG graph construction are worth attention, but the retrieval evaluation is circular and the paper's own Table II contradicts its headline claim. read the letter →

arxiv 2506.20070 v1 pith:U6ZLKQ7J submitted 2025-06-25 cs.IR cs.LGcs.MM

KMA Solaiman , Bharat Bhargava This is my paper · ORCID
classification cs.IRcs.LGcs.MM
keywords multimodalinformationretrievalweaksupervisiongrapheditdistanceattributedrelationalquerybyexamplemissingpersonhumanattributerecognitionMuQNOL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FemmIR claims that multimodal retrieval can be solved without similarity labels or fine-tuning by treating relevance as an edit cost between graphs built from extracted properties. A query example and a candidate sample are each converted into a hierarchical attributed relational graph; matching those graphs costs more for samples that differ in the properties a user cares about. Those edit costs serve as weak supervision for a graph-similarity network that learns to rank. On a new missing-person benchmark, MuQNOL, the framework retrieves images, text, and video with mean average precision comparable to an exact-join baseline and well above a fine-grained cross-modal model that was not fine-tuned. The authors claim this opens retrieval to applications with scarce annotations.

What carries the argument

The load-bearing object is the Hierarchical Attributed Relational Graph (HARG), a multi-level tree whose leaf nodes hold property values and whose internal nodes represent entities and relations. Comparisons are computed on Entity-with-Property-in-Leaf (EPL) vertices, so that the cost of matching a person includes the costs of their clothing and relation edges. The Content Edit Distance (CED) is computed by a modified Munkres bipartite assignment over these vertices with per-property replacement and insertion costs, plus an optional cumulative pass that propagates parent costs to children to respect dependencies; the result is normalized and exponentiated to give similarity labels. Those labels train SimGNN, a graph neural network with a Neural Tensor Network interaction layer, to approximate the edit distance at inference.

What would settle it

Take MuQNOL queries with gold property annotations and replace the noisy video and image property identifiers with the ground-truth attributes; if FemmIR's mAP does not rise substantially beyond 27–37%, the gap would indicate that edit-cost weak labels, rather than identifier noise, are the binding constraint. Alternatively, collect human relevance judgments on a sample of query–candidate pairs and test whether the ranking induced by CED agrees with them better than a naive matched-attribute count; if not, the edit-distance signal carries no extra information.

Watch

Extended reading notes

Core claim

The paper's central claim is that weak supervision from a purpose-built graph edit distance, Content Edit Distance (CED), is enough to train a ranked cross-modal retriever without any similarity labels or application-specific fine-tuning. For each data sample, a property extractor produces attributes that are arranged into a hierarchical relational graph; the CED measures the minimal cost of converting one sample's graph into another's, where replacement costs express user priorities such as gender mismatch being worse than shirt-color mismatch. The normalized distance is converted to a similarity score via $\mathrm{SIM} = e^{-\mathrm{nCED}}$, and this score trains SimGNN to predict similarity from graph embeddings, so at inference time only the property graphs and the learned mapping are needed. The paper argues this works because the edit cost implicitly encodes the structure and dependencies among properties, not just matched attribute counts, and demonstrates it on the MuQNOL missing-person retrieval task.

Load-bearing premise

The load-bearing premise is that relevance between samples is fully captured by the edit cost over the user-specified property graphs, so retrieval quality is capped by how well that small attribute set, such as gender, clothing colors, and height, describes what actually makes a person relevant.

Editorial extensions

If this is right

  • Retrieval systems can be built for new applications by plugging in existing property extractors and defining replacement costs, without collecting similarity annotations.
  • Performance of the retriever is bounded by the quality of the property identifiers: the paper reports that video and image identifiers with average F1 near 79.59% yield mAP scores of 27–37%, while text identifiers with higher F1 give the best mAP.
  • The edit-cost formulation lets users express which properties matter, such as gender outweighing top color, through replacement costs, keeping the system adaptable to different information needs.
  • The MuQNOL benchmark provides a testbed for weakly supervised multimodal retrieval where relevance is derived from gold property annotations rather than similarity labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same weak-supervision mechanism should transfer to other entity-centric domains, such as product search or surveillance, as long as a finite property vocabulary and consistent entity types can be fixed in advance.
  • Because CED labels are only as good as the property graphs, improving attribute extractors, especially video and image identifiers, should translate almost linearly into better retrieval mAP, a prediction that can be tested by swapping in stronger identifiers on MuQNOL.
  • The cumulative Munkres variant suggests that relation-aware edit costs matter most when properties are interdependent; ablating the cumulative pass would isolate how much of the ranking signal comes from structural dependencies rather than from simple attribute matching.
  • One implicit risk is that using edit cost over a small fixed attribute set defines relevance operationally, so samples that are relevant through unmodeled attributes would be misranked even if the framework itself works as claimed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FemmIR, a framework for multimodal retrieval by example without similarity labels. Samples are represented as hierarchical attributed relational graphs built from property identifiers; a content edit distance (CED) computed by a Munkres assignment is used as weak supervision to train SimGNN to rank query examples against candidates. The paper also introduces HART for attribute extraction from unstructured text and a benchmark, MuQNOL, built from MARS and InciText. Experiments on a missing-person use case report mAP for cross-modal retrieval and compare with EARS and FGCross-Net.

Significance. If the central claim were established, reusing pretrained property identifiers plus edit-distance weak labels would be a practical way to avoid similarity-label annotation in multimodal retrieval, and MuQNOL and HART would be useful resources. The paper is explicit about its weak-supervision assumptions and describes a concrete algorithm, which is a strength. However, the evaluation does not currently support the central claim because the relevance ground truth is generated from the same property-mismatch penalties used to define CED, and the main baseline EARS outperforms FemmIR on every reported row.

major comments (3)
  1. [Section V-A and Algorithm 1] The retrieval evaluation is circular with the weak-label generation. The MuQNOL ground truth is created by ranking samples by mismatched properties with hand-assigned penalties rcost(TOP-COLOR)=1, rcost(BOTTOM-COLOR)=2, rcost(GENDER)=3, and relevance is declared when CED < 3. CED is exactly the Munkres-optimized sum of these same property replacement/insertion costs in Algorithm 1 (lines 10-26), and the same CED scores are used as weak labels for training SimGNN. Consequently, the reported mAP mainly measures how well SimGNN reproduces the hand-defined penalty rule on a fixed attribute set, not whether edit distance tracks the missing-person relevance claimed. The paper should evaluate against an independent relevance signal (e.g., person identity labels in MARS/InciText or human relevance judgments) and should compare against a direct weighted-mismatch ranking baseline.
  2. [Section V-e, Table II] Table II contradicts the contribution claim that FemmIR "surpasses existing multimodal knowledge retrieval methods without fine-tuning." EARS, an exact inference model that requires no training, achieves higher mAP than FemmIR on every row (e.g., Text-to-Text 1.00 vs 0.52; all-modality average 0.44 vs 0.33). The abstract and Section VII describe "comparable" performance, which is accurate, but the stronger comparative claim in the introduction and contribution list is unsupported as stated. The authors should either revise the claim to match Table II or provide a fairer comparison, for example by using identical property identifiers for both systems.
  3. [Section III-A] The open-world generalization claim is constrained by assumptions (I) and (II): system-specific properties are assumed known, and entity types must be consistent through the system lifetime. These assumptions are reasonable for the missing-person use case, but they are not reflected in the abstract's "open world" framing. More importantly, because the property set is fixed and small (gender, top/bottom color, clothing, race, height), the paper does not demonstrate that edit-distance weak supervision transfers to novel attributes or modalities. A transfer experiment or an explicit limitation statement is needed to support the general claim.
minor comments (4)
  1. [Section III-B] The text uses "Munkers" for the Munkres algorithm; please correct the spelling for consistency with the rest of the paper.
  2. [Figure 4] The figure caption lists panels (a) and (c), while the body text refers only to (a) and (b); the panel labels should be made consistent.
  3. [Section V-A] MuQNOL is presented as a new benchmark, but no download URL or access mechanism is provided; a public release statement is needed for reproducibility.
  4. [Table I] The column headers "Clothes Attr-only" and "Clothes Attr-value" appear misaligned with the values in the precision/recall/F1 rows; please reformat the table.

Circularity Check

1 steps flagged · score 6.0 of 10

Evaluation relevance is defined by the same CED penalty function that produces the weak training labels, so the reported mAP cannot validate CED as an independent relevance signal.

  1. self definitional [Section V, 'Dataset Construction' and 'Retrieval Performance of FemmIR'; Algorithm 1 lines 10-26]
    "For developing the ground truth, we ranked the data samples in ascending order of the mismatched properties. ... we set the penalty for each mismatched property as follows: rcost(TOP-COLOR) = 1, rcost(BOTTOM-COLOR) = 2, and rcost(GENDER) = 3 ... We consider data samples with CED < 3 in comparison to the query object, as relevant for that query."

    Algorithm 1 constructs CED as the Munkres-optimized sum of these same property-mismatch costs: lines 10-17 add RCOST(op) for each mismatched scalar property, line 24 runs Munkres, and line 26 converts CED into the weak training label SIM = e^{-nCED}. The evaluation ground truth is therefore a thresholded version of the very function that generates the weak labels: a sample is 'relevant' exactly when its CED lies below a cutoff expressed in the hand-assigned rcost units. Since both training target and test relevance are functions of the identical CED computation, the mAP measures how well SimGNN reproduces the hand-defined penalty rule over the fixed attribute set, not whether edit distance tracks independently established missing-person relevance.

full rationale

The central retrieval claim is that CED weak supervision is sufficient to rank cross-modal relevance, but the experiments do not measure relevance independently of CED. The MuQNOL ground truth is built by ranking samples by mismatched properties with rcost(TOP-COLOR)=1, rcost(BOTTOM-COLOR)=2, rcost(GENDER)=3, and the test relevance criterion is literally 'CED < 3'; Algorithm 1 defines CED as the optimized sum of exactly those property-mismatch penalties. Thus the weak label and the evaluation label are the same construction, making the mAP a self-consistency check rather than an external validation of CED as a relevance signal. This is partial circularity rather than complete equivalence: SimGNN is a learned approximation of CED, the HARG construction and Munkres optimization add nontrivial algorithmic content, and the visual/text property identifiers are benchmarked against independent gold annotations. The HART component is also evaluated against InciText ground truth and is not implicated in this circularity. However, the paper's own observation that retrieval mAP (27-37%) tracks property-identifier F1 (79.59%) reinforces that the weak labels inherit the semantics and errors of the predefined property/penalty layer. An independent relevance signal is required before the headline claim is supported.

Assumptions & free parameters 5 free parameters · 4 assumptions · 3 invented entities

The framework depends on careful manual design of property costs, thresholds, and the choice of which properties are relevant. The paper acknowledges this by noting that retrieval performance correlates with property identifier accuracy. The graph representation and edit distance are internal constructs without independent validation.

free parameters (5)
  • Replacement costs rcost(TOP-COLOR), rcost(BOTTOM-COLOR), rcost(GENDER) = 1, 2, 3
    Penalty weights for mismatched properties in the Munkres cost matrix; chosen by hand based on domain intuition.
  • Insertion cost ICOST(op) = not specified in the paper
    Used in Algorithm 1 for missing properties; the paper does not specify how these are set.
  • OCOMP control variable = not specified
    Controls whether list comparison uses ordered or unordered comparison; mixing is not described.
  • Threshold theta_H for candidate sentence extraction = 0.5, 0.85, 0.9 depending on model
    Empirical thresholds tuned per model in Figure 2.
  • Relevance threshold CED < 3 = 3
    Used to define relevant samples for mAP evaluation; chosen so that only color mismatches are returned.
assumptions (4)
  • domain assumption Relevance is fully captured by the user-specified object properties and the edit cost between them.
    Section III-A, assumptions I and II; the retrieval problem is defined entirely through the PROP relation.
  • domain assumption Pretrained or rule-based property identifiers are available for all modalities and produce consistent labels.
    Section III-A and the evaluation; retrieval quality depends on the accuracy of the property identifiers.
  • standard math The Munkres bipartite matching gives a valid approximation of the true graph edit distance.
    Section III-B, the paper adopts the bipartite graph matching method from Riesen et al.
  • domain assumption SimGNN can learn a mapping from graph embeddings to CED scores that generalizes to unseen samples.
    Section III-C, the neural network is trained on weak labels and used at inference.
invented entities (3)
  • HARG (Hierarchical Attributed Relational Graph)
    purpose: Graph representation of data samples with properties and relations.
    A representation introduced in this paper; no falsifiable predictions outside the paper.
  • EPL vertex (Entity-with-Property-in-Leaf)
    purpose: Nodes in HARG with attached property-value leaves, used for cost computation.
    An internal construct for the edit distance calculation.
  • Content Edit Distance (CED)
    purpose: Weak supervision signal for training the retrieval model.
    A new distance metric proposed in the paper, not independently verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision." pith.science (2026). https://pith.science/paper/U6ZLKQ7J

@misc{pith2026250620070,
  author       = {Pith},
  title        = {Pith review of: Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U6ZLKQ7J}},
  note         = {Machine review of arXiv:2506.20070}
}
read the original abstract

Existing multi-media retrieval models either rely on creating a common subspace with modality-specific representation models or require schema mapping among modalities to measure similarities among multi-media data. Our goal is to avoid the annotation overhead incurred from considering retrieval as a supervised classification task and re-use the pretrained encoders in large language models and vision tasks. We propose "FemmIR", a framework to retrieve multimodal results relevant to information needs expressed with multimodal queries by example without any similarity label. Such identification is necessary for real-world applications where data annotations are scarce and satisfactory performance is required without fine-tuning with a common framework across applications. We curate a new dataset called MuQNOL for benchmarking progress on this task. Our technique is based on weak supervision introduced through edit distance between samples: graph edit distance can be modified to consider the cost of replacing a data sample in terms of its properties, and relevance can be measured through the implicit signal from the amount of edit cost among the objects. Unlike metric learning or encoding networks, FemmIR re-uses the high-level properties and maintains the property value and relationship constraints with a multi-level interaction score between data samples and the query example provided by the user. We empirically evaluate FemmIR on a missing person use case with MuQNOL. FemmIR performs comparably to similar retrieval systems in delivering on-demand retrieval results with exact and approximate similarities while using the existing property identifiers in the system.

Figures

Figures reproduced from arXiv: 2506.20070 by the authors.

Figure 1
Figure 1. HARG and Weak Label Generation; Left sided graph refers to [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Performance of Different Candidate Sentence Extraction Models based on Clothes Property Identification [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Comparison of Property Identifiers for Videos with Accuracy (acc) and F1 measure on MARS dataset (%) [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: (a) Precision-recall curves for the text as query and data [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 40 canonical work pages

  1. [1]

    End-to- end knowledge retrieval with multi-modal queries,

    M. Luo, Z. Fang, T. Gokhale, Y . Yang, and C. Baral, “End-to- end knowledge retrieval with multi-modal queries,” arXiv preprint arXiv:2306.00424, 2023

    arXiv 2023

  2. [2]

    Weakly-supervised semantic guided hashing for social image retrieval,

    Z. Li, J. Tang, L. Zhang, and J. Yang, “Weakly-supervised semantic guided hashing for social image retrieval,” International Journal of Computer Vision, vol. 128, no. 8, pp. 2265–2278, 2020

    work page 2020

  3. [3]

    Weakly supervised video moment retrieval from text queries,

    N. C. Mithun, S. Paul, and A. K. Roy-Chowdhury, “Weakly supervised video moment retrieval from text queries,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 11 592–11 601

    work page 2019

  4. [4]

    Structure label prediction us- ing similarity-based retrieval and weakly supervised label mappingstruc- ture label prediction,

    Y . Alaudah, M. Alfarraj, and G. AlRegib, “Structure label prediction us- ing similarity-based retrieval and weakly supervised label mappingstruc- ture label prediction,” Geophysics, vol. 84, no. 1, pp. V67–V79, 2019

    work page 2019

  5. [5]

    Open-learning framework for multi- modal information retrieval with weakly supervised joint embedding,

    K. Solaiman and B. Bhargava, “Open-learning framework for multi- modal information retrieval with weakly supervised joint embedding,” AAAI Spring Symposium , 2022

    work page 2022

  6. [6]

    Mars: A video benchmark for large-scale person re-identification,

    L. Zheng, Z. Bie, Y . Sun, J. Wang, C. Su, S. Wang, and Q. Tian, “Mars: A video benchmark for large-scale person re-identification,” in European Conference on Computer Vision . Springer, 2016, pp. 868–884

    work page 2016

  7. [7]

    Feature centric multi-modal information retrieval in open world environment (femmir),

    K. Solaiman and B. Bhargava, “Feature centric multi-modal information retrieval in open world environment (femmir),” TechRxiv, 2023

    work page 2023

  8. [8]

    A new approach to cross-modal multime- dia retrieval,

    N. Rasiwasia, J. Costa Pereira, E. Coviello, G. Doyle, G. R. Lanckriet, R. Levy, and N. Vasconcelos, “A new approach to cross-modal multime- dia retrieval,” in Proceedings of the 18th ACM international conference on Multimedia, 2010, pp. 251–260

    work page 2010

Show all 54 references
  1. [9]

    Multi-view canonical correlation anal- ysis,

    J. Rupnik and J. Shawe-Taylor, “Multi-view canonical correlation anal- ysis,” in Conference on Data Mining and Data Warehouses (SiKDD 2010), 2010, pp. 1–4

    2010

  2. [10]

    Generalized semi- supervised and structured subspace learning for cross-modal retrieval,

    L. Zhang, B. Ma, G. Li, Q. Huang, and Q. Tian, “Generalized semi- supervised and structured subspace learning for cross-modal retrieval,” IEEE Transactions on Multimedia , vol. 20, no. 1, pp. 128–141, 2017

    2017

  3. [11]

    On deep multi- view representation learning,

    W. Wang, R. Arora, K. Livescu, and J. Bilmes, “On deep multi- view representation learning,” in International conference on machine learning. PMLR, 2015, pp. 1083–1092

    2015

  4. [12]

    Multi-view deep network for cross-view classification,

    M. Kan, S. Shan, and X. Chen, “Multi-view deep network for cross-view classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 4847–4855

    2016

  5. [13]

    Ccl: Cross-modal correlation learning with multigrained fusion by hierarchical network,

    Y . Peng, J. Qi, X. Huang, and Y . Yuan, “Ccl: Cross-modal correlation learning with multigrained fusion by hierarchical network,” IEEE Trans- actions on Multimedia , vol. 20, no. 2, pp. 405–420, 2017

    2017

  6. [14]

    Cross-media shared representation by hierarchical learning with multiple deep networks,

    Y . Peng, X. Huang, and J. Qi, “Cross-media shared representation by hierarchical learning with multiple deep networks,” in Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence , ser. IJCAI’16. AAAI Press, 2016, p. 3846–3853

    2016

  7. [15]

    Vse++: Improv- ing visual-semantic embeddings with hard negatives,

    F. Faghri, D. J. Fleet, J. R. Kiros, and S. Fidler, “Vse++: Improv- ing visual-semantic embeddings with hard negatives,” arXiv preprint arXiv:1707.05612, 2017

    2017 arXiv

  8. [16]

    Deep adversarial metric learning for cross-modal retrieval,

    X. Xu, L. He, H. Lu, L. Gao, and Y . Ji, “Deep adversarial metric learning for cross-modal retrieval,” World Wide Web, vol. 22, no. 2, pp. 657–672, 2019

    2019

  9. [17]

    Universal weighting metric learning for cross-modal matching,

    J. Wei, X. Xu, Y . Yang, Y . Ji, Z. Wang, and H. T. Shen, “Universal weighting metric learning for cross-modal matching,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13 005–13 014

    2020

  10. [18]

    Applying machine learning and data fusion to the “missing person

    K. Solaiman, T. Sun, A. Nesen, B. Bhargava, and M. Stonebraker, “Applying machine learning and data fusion to the “missing person” problem,” Computer, vol. 55, no. 06, pp. 40–55, jun 2022

    2022

  11. [19]

    Fellbaum, WordNet: An Electronic Lexical Database

    C. Fellbaum, WordNet: An Electronic Lexical Database . Bradford Books, 1998

    1998

  12. [20]

    Surveillance video querying with a human-in-the-loop,

    M. Stonebraker, B. Bhargava, M. Cafarella, Z. Collins, J. McClellan, A. Sipser, T. Sun, A. Nesen, K. Solaiman, G. Mani, K. Kochpatcharin, P. Angin, and J. MacDonald, “Surveillance video querying with a human-in-the-loop,” in Proceedings of the Workshop on Human-In-the- Loop Da...

    2020

  13. [21]

    Bipartite graph matching for computing the edit distance of graphs,

    K. Riesen, M. Neuhaus, and H. Bunke, “Bipartite graph matching for computing the edit distance of graphs,” in International Workshop on Graph-Based Representations in Pattern Recognition . Springer, 2007, pp. 1–12

    2007

  14. [22]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” CoRR, vol. abs / 1609.02907, 2016. [Online]. Available: http://arxiv.org/abs/1609.02907

    2016 arXiv

  15. [23]

    Graph edit distance computation via graph neural networks,

    Y . Bai, H. Ding, S. Bian, T. Chen, Y . Sun, and W. Wang, “Graph edit distance computation via graph neural networks,” arXiv preprint arXiv:1808.05689, 2018

    2018 arXiv

  16. [24]

    Graph based shapes representation and recognition,

    R. J. Qureshi, J.-Y . Ramel, and H. Cardot, “Graph based shapes representation and recognition,” in International Workshop on Graph- Based Representations in Pattern Recognition . Springer, 2007, pp. 49–60

    2007

  17. [25]

    Reasoning with neural tensor networks for knowledge base completion,

    R. Socher, D. Chen, C. D. Manning, and A. Ng, “Reasoning with neural tensor networks for knowledge base completion,” in Advances in Neural Information Processing Systems 26 , C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, Eds. Curran Associates, I...

    2013

  18. [26]

    Efficient estimation of word representations in vector space,

    T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” 2013

    2013

  19. [27]

    Verb semantics and lexical selection,

    Z. Wu and M. Palmer, “Verb semantics and lexical selection,” arXiv preprint cmp-lg/9406033, 1994

    1994 arXiv

  20. [28]

    Benchmarking zero-shot text classifi- cation: Datasets, evaluation and entailment approach,

    W. Yin, J. Hay, and D. Roth, “Benchmarking zero-shot text classifi- cation: Datasets, evaluation and entailment approach,” arXiv preprint arXiv:1909.00161, 2019

    1909 arXiv

  21. [29]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. s Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” 2019

    2019

  22. [30]

    Nltk: The natural language toolkit,

    E. Loper and S. Bird, “Nltk: The natural language toolkit,” in Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics - Volume 1 , ser. ETMTNLP ’02. USA: Association for Computational Lingu...

    2002

  23. [31]

    A broad-coverage challenge corpus for sentence understanding through inference,

    A. Williams, N. Nangia, and S. Bowman, “A broad-coverage challenge corpus for sentence understanding through inference,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 ...

    2018

  24. [32]

    Video-based pedestrian attribute recognition,

    Z. Chen, A. Li, and Y . Wang, “Video-based pedestrian attribute recognition,” CoRR, vol. abs/1901.05742, 2019. [Online]. Available: http://arxiv.org/abs/1901.05742

    1901 arXiv

  25. [33]

    Recurrent convolu- tional network for video-based person re-identification,

    N. McLaughlin, J. M. Del Rincon, and P. Miller, “Recurrent convolu- tional network for video-based person re-identification,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 1325–1334

    2016

  26. [34]

    3d convolutional neural networks for human action recognition,

    S. Ji, W. Xu, M. Yang, and K. Yu, “3d convolutional neural networks for human action recognition,” IEEE transactions on pattern analysis and machine intelligence , vol. 35, no. 1, pp. 221–231, 2012

    2012

  27. [35]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  28. [36]

    A new benchmark and approach for fine-grained cross-media retrieval,

    X. He, Y . Peng, and L. Xie, “A new benchmark and approach for fine-grained cross-media retrieval,” in Proceedings of the 27th ACM international conference on multimedia , 2019, pp. 1740–1748

    2019

  29. [37]

    Scalable deep multimodal learning for cross-modal retrieval,

    P. Hu, L. Zhen, D. Peng, and P. Liu, “Scalable deep multimodal learning for cross-modal retrieval,” in Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR’19. New York, NY , USA: ACM, 2019, pp. 635–

    2019

  30. [38]

    Multi-label cross-modal retrieval,

    V . Ranjan, N. Rasiwasia, and C. Jawahar, “Multi-label cross-modal retrieval,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 4094–4102

    2015

  31. [39]

    Adversarial cross-modal retrieval,

    B. Wang, Y . Yang, X. Xu, A. Hanjalic, and H. T. Shen, “Adversarial cross-modal retrieval,” in Proceedings of the 25th ACM international conference on Multimedia , 2017, pp. 154–162

    2017

  32. [40]

    Deep cross-modal projection learning for image- text matching,

    Y . Zhang and H. Lu, “Deep cross-modal projection learning for image- text matching,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 686–701

    2018

  33. [41]

    Deep coupled metric learning for cross-modal matching,

    V . E. Liong, J. Lu, Y .-P. Tan, and J. Zhou, “Deep coupled metric learning for cross-modal matching,” IEEE Transactions on Multimedia , vol. 19, no. 6, pp. 1234–1244, 2016

    2016

  34. [42]

    Devise: A deep visual-semantic embedding model,

    A. Frome, G. Corrado, J. Shlens et al., “Devise: A deep visual-semantic embedding model,” in Advances in Neural Information Processing Systems, vol. 26, 2013

    2013

  35. [43]

    Stacked cross attention for image-text matching,

    K.-H. Lee, X. Chen, G. Hua, H. Hu, and X. He, “Stacked cross attention for image-text matching,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 201–216

    2018

  36. [44]

    Visual semantic reasoning for image-text matching,

    K. Li, Y . Zhang, K. Li, Y . Li, and Y . Fu, “Visual semantic reasoning for image-text matching,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 4654–4662

    2019

  37. [45]

    Polysemous visual-semantic embedding for cross-modal retrieval,

    Y . Song and M. Soleymani, “Polysemous visual-semantic embedding for cross-modal retrieval,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 1979–1988

    2019

  38. [46]

    Aligned attention for com- mon multimodal embeddings,

    S. Sah, S. Gopalakrishnan, and R. Ptucha, “Aligned attention for com- mon multimodal embeddings,” Journal of Electronic Imaging , vol. 29, pp. 023 013 – 023 013, 2020

    2020

  39. [47]

    Dice: Data discovery by example,

    E. K. Rezig, A. Bhandari, A. Fariha, B. Price, A. Vanterpool, V . Gadepally, and M. Stonebraker, “Dice: Data discovery by example,” Proc. VLDB Endow., vol. 14, no. 12, p. 2819–2822, jul 2021. [Online]. Available: https://doi.org/10.14778/3476311.3476353

    2021

  40. [48]

    Multimodal multimedia retrieval with vitrivr,

    R. Gasser, L. Rossetto, and H. Schuldt, “Multimodal multimedia retrieval with vitrivr,” in Proceedings of the 2019 on International Conference on Multimedia Retrieval , ser. ICMR ’19. New York, NY , USA: Association for Computing Machinery, 2019, p. 391–394. [Online]. Availabl...

    2019

  41. [49]

    Query by example for cross-lingual event retrieval,

    S. M. Sarwar and J. Allan, “Query by example for cross-lingual event retrieval,” in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, ser. SIGIR ’20. New York, NY , USA: Association for Computing Machinery, 2020, p...

    2020

  42. [50]

    Wip - skod: A framework for situational knowledge on demand,

    S. Palacios, K. Solaiman, P. Angin, A. Nesen, B. Bhargava, Z. Collins, A. Sipser, M. Stonebraker, and J. Macdonald, “Wip - skod: A framework for situational knowledge on demand,” in Heterogeneous Data Manage- ment, Polystores, and Analytics for Healthcare, V . Gadepally, T. Ma...

    2019

  43. [51]

    Multimedia search and retrieval using multimodal annotation propagation and indexing techniques,

    M. Lazaridis, A. Axenopoulos, D. Rafailidis, and P. Daras, “Multimedia search and retrieval using multimodal annotation propagation and indexing techniques,” Signal Processing: Image Communication , vol. 28, no. 4, pp. 351 – 367, 2013. [Online]. Available: http://www.sciencedi...

    2013

  44. [52]

    Combining rich features and deep learning for finding similar sentences in electronic medical records,

    Q. Chen, J. Du, S. Kim, W. J. Wilbur, and Z. Lu, “Combining rich features and deep learning for finding similar sentences in electronic medical records,” Proceedings of the BioCreative/OHNLP Challenge , pp. 5–8, 2018

    2018

  45. [53]

    Finding similar examples for aiding academic writing using sentence embeddings,

    C. L. GOH and Y . LEPAGE, “Finding similar examples for aiding academic writing using sentence embeddings,” 2020

    2020

  46. [644]

    Available: http://doi.acm.org/10.1145/3331184.3331213

    [Online]. Available: http://doi.acm.org/10.1145/3331184.3331213

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.