Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search
Reviewed by Pithpith:SHJVHXGEopen to challenge →
read the original abstract
Text-based person search aims at retrieving target person in an image gallery using a descriptive sentence of that person. It is very challenging since modal gap makes effectively extracting discriminative features more difficult. Moreover, the inter-class variance of both pedestrian images and descriptions is small. So comprehensive information is needed to align visual and textual clues across all scales. Most existing methods merely consider the local alignment between images and texts within a single scale (e.g. only global scale or only partial scale) then simply construct alignment at each scale separately. To address this problem, we propose a method that is able to adaptively align image and textual features across all scales, called NAFS (i.e.Non-local Alignment over Full-Scale representations). Firstly, a novel staircase network structure is proposed to extract full-scale image features with better locality. Secondly, a BERT with locality-constrained attention is proposed to obtain representations of descriptions at different scales. Then, instead of separately aligning features at each scale, a novel contextual non-local attention mechanism is applied to simultaneously discover latent alignments across all scales. The experimental results show that our method outperforms the state-of-the-art methods by 5.53% in terms of top-1 and 5.35% in terms of top-5 on text-based person search dataset. The code is available at https://github.com/TencentYoutuResearch/PersonReID-NAFS
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search
ROGLE automates region-level supervision via Region-to-Sentence Matching and introduces the P-VLG benchmark to improve fine-grained alignment in text-based person search over CLIP-based models.
-
InterPartAbility: Phrase-Region Grounding for Interpretable Text-to-Image Person Re-Identification
InterPartAbility performs explicit part-wise matching in text-to-image person re-identification via a patch-phrase interaction module to produce grounded explanations and achieves SOTA interpretability scores while ma...
-
InterPartAbility: Phrase-Region Grounding for Interpretable Text-to-Image Person Re-Identification
InterPartAbility adds an open-vocabulary patch-phrase interaction module and a perturbation-based interpretability protocol to TI-ReID, claiming SOTA explainability scores with competitive retrieval accuracy on three ...
-
Cross-Resolution Semantic Transfer for Robust Text-to-Image Retrieval in Low-Resolution Surveillance
CRST improves ultra-low-resolution text-to-image person retrieval by 5.7% Rank-1 and 5.3% mAP on average across three datasets while stabilizing mixed-resolution galleries.
-
ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search
ROGLE introduces automated pseudo region-sentence pairs via RSM and multi-granular learning to boost fine-grained alignment in text-based person search, plus the P-VLG benchmark with over 100k annotated regions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.