Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A plug-in uncertainty module claims state-of-the-art partially relevant video retrieval by encoding queries and videos as Gaussian distributions and gating uninformative query words.

desk verdict Solid PRVR method with real gains, but a train/test mismatch around the query support set and test-set hyperparameter tuning need to be fixed before the numbers can be trusted. read the letter →

arxiv 2509.01383 v1 pith:OJQIEN2B submitted 2025-09-01 cs.CV cs.MM

Long Zhang , Peipei Song , Jianfeng Dong , Kun Li , Xun Yang This is my paper · ORCID
classification cs.CVcs.MM
keywords partiallyrelevantvideoretrievalprobabilisticembeddingsmultivariateGaussianuncertaintymodelingconfidence-awarealignmenttext-to-videountrimmedquery-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Partially Relevant Video Retrieval (PRVR) asks a model to find untrimmed videos that only partly match a text query, with the query often ambiguous and the video full of irrelevant moments. This paper claims that existing methods fail because they treat query-video matching as a deterministic pointwise similarity, making them vulnerable to distractor videos with spurious semantic overlap. The proposed Robust Alignment Learning (RAL) instead treats both modalities as probability distributions: Gaussian mean/variance representations quantify uncertainty, sampled proxies provide multiple candidate alignments, and a learned confidence gate down-weights uninformative words. Across three PRVR baselines it reports consistent gains, a new best SumR of 198.8 on TVR (9.7 above the previous best) and 156.1 on ActivityNet, plus a transfer gain on trimmed text-to-video retrieval. The paper's own limitation section identifies named-entity mismatches, such as the character 'Beckett' in a TVR query, as a remaining failure mode.

What carries the argument

The load-bearing mechanism is a pair of modules added to the frame-level branch of an existing PRVR model. Multimodal Semantic Robust Alignment (MSRA) estimates a mean and variance for each video and query through multi-granularity aggregation, treating the representations as multivariate Gaussians; the variance quantifies aleatoric uncertainty and enables proxy-level matching through a multi-instance InfoNCE loss and a KL distribution-alignment loss. Confidence-aware Set-to-Set Alignment (CSA) predicts a word-level confidence score with a small MLP and uses it to weight the word-frame similarity matrix, replacing the usual mean pooling over words. Together they convert a one-to-one determin

What would settle it

Evaluate a trained RAL model on TVR with the query support set disabled at inference so each query is scored alone; if SumR falls back toward the 189.1 GMMFormer v2 baseline, the gain is an artifact of training-time aggregation rather than a property of the Gaussian alignment. A second check: replace the learned confidence gate with fixed equal weights and measure the drop on high-uncertainty queries.

Watch

Extended reading notes

Core claim

The paper claims that the central obstacle in PRVR is not missing multi-scale clip features but spurious semantic correlations caused by two forms of data uncertainty: query ambiguity, where a query incompletely describes the target video and contains uninformative tokens, and partial video relevance, where untrimmed videos contain many query-irrelevant segments. RAL replaces deterministic pointwise query-clip matching with probabilistic alignment: video and query features are encoded as multivariate Gaussian distributions, a query support set aggregates all queries describing the same video to enrich the text distribution, sampled proxy embeddings supply multiple alignment candidates, and a

Load-bearing premise

The text distribution is estimated from a query support set that aggregates every training query attached to a video, and the paper marks that input 'Training only' without specifying how a single query forms its distribution at test time; the reported improvements may therefore depend on seeing multiple descriptions of the same video during training.

Editorial extensions

If this is right

  • Adding RAL to MS-SL, GMMFormer, and GMMFormer v2 raises SumR on TVR by 6.7, 9.5, and 9.7 points respectively, so the mechanism transfers across clip-model architectures.
  • The margin over GMMFormer v2 widens as query uncertainty increases; at extreme uncertainty the baseline's R@1 approaches zero while RAL remains near 20, implying the benefit is concentrated in the hardest ambiguous cases.
  • When extra irrelevant segments are inserted into test videos, RAL shows the smallest SumR decline among compared methods, suggesting the distributional and confidence weighting make retrieval less sensitive to noise.
  • On trimmed text-to-video retrieval with CLIP4Clip, RAL improves R@1 by about 6.1 points (ViT-B/32) and 6.6 points (ViT-B/16), indicating the mechanism is not restricted to PRVR.
  • Ablations attribute roughly independent gains to MSRA (+5.7 SumR) and CSA (+5.4 SumR), with the full model reaching +9.7, so the two modules can be studied or reused separately.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because Figure 2 labels the query support set 'Training only' and the authors never specify how a single query forms a text distribution at inference, the reported gains may partially depend on privileged multi-query information during training; a single-query test-time evaluation would settle how much of the improvement survives deployment.
  • Beyond the paper: the confidence-gating idea is not inherently tied to words—the same learnable-weight mechanism could be applied to frames, clips, or sentence-level descriptions, and the ablations suggest it would act nearly additively with other alignment modules.
  • Beyond the paper: the paper's own failure cases point to named-entity identity as the next bottleneck; an extension that couples textual character names to visual person-specific cues, e.g., clothing or appearance, is the natural follow-up and is testable on the same TVR split.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Robust Alignment Learning (RAL) for partially relevant video retrieval (PRVR). The method has two components: (1) Multimodal Semantic Robust Alignment (MSRA), which encodes video and text as multivariate Gaussians, uses a training-time query support set to form enriched text distributions, and optimizes distribution-alignment and proxy-matching losses; (2) Confidence-aware Set-to-Set Alignment (CSA), which weights word-level similarities by learned confidence scores. RAL is plugged into MS-SL, GMMFormer, and GMMFormer-v2 and evaluated on TVR and ActivityNet, reporting a new state-of-the-art SumR of 198.8 on TVR. The paper also reports transfer to trimmed T2VR on MSR-VTT.

Significance. If the reported gains are reproducible and transfer to the actual single-query inference setting, the probabilistic-alignment view of PRVR is a useful contribution. The plug-and-play nature across several backbones is attractive, and the component ablations and robustness analyses (noise, M/V ratio, uncertainty) are informative. The paper does not provide code, and the central novelty—query support sets for uncertainty modeling—raises a train/test specification gap that currently prevents the reader from verifying whether the headline gains are real in deployment. The method itself is mostly standard probabilistic embedding plus contrastive losses, but the empirical scope is broad and the reported improvements are large if valid.

major comments (4)
  1. [Section 3.2 / Eq. (5), Figure 2, Appendix Fig. 9, Table 4] The query support set is defined only for training. Eq. (5) sets X_q = Q_s by concatenating all queries associated with a training video, and Figure 2 and Appendix Fig. 9 explicitly label this 'Training only'. At inference, a test query has no associated query set, yet the paper never states what X_q is at test time. If the model receives a single query at inference, the text-distribution input differs from the training-time distribution, and L_DA/L_PM were optimized against a representation that encodes privileged multi-query information. If the support set is discarded at test, the method is under-specified. Table 4's first row shows that training with X_q=Q lowers SumR from 198.8 to 196.9, indicating the support set contributes to the reported gain. No experiment evaluates the Q_s-trained model on single-query inference. This must be resolved before the SOTA claim can be accepted.
  2. [Appendix B.2 / Section 4.1] The hyperparameters λ3 and λ4 appear to be selected on the TVR test set. Appendix B.2 reports SumR values for sweeps over λ3 and λ4 and chooses the optimum, but it does not state that a validation split was used. Since the final TVR numbers in Table 1 are test-set numbers, selecting hyperparameters on the test set would inflate the reported SOTA. Additionally, Section 4.1 states λ3=0.001 and λ4=0.004, while Appendix B.2 concludes that the optimum is λ3=0.004 and λ4=0.001. These values conflict and must be reconciled. The authors should specify the exact split used for hyperparameter selection and report the final configuration consistently.
  3. [Table 1 / Section 4.2] All reported numbers in Table 1 appear to come from single runs with no error bars or multiple-seed statistics. The central claim is a new state of the art by a 9.7 SumR margin over GMMFormer v2. Given typical run-to-run variance in retrieval training, the paper should report mean ± std over at least three seeds, or provide a statistical significance test, for at least the proposed method and the main baseline. Without this, the magnitude of the improvement and its robustness are not established.
  4. [Section 3.3 / Eq. (12)] The confidence weights g are defined as g = MLP(Q) with no stated activation or normalization constraint. As written, the MLP output is unconstrained, so the 'confidence' weights can be negative or arbitrarily large, and the final similarity S(q,v) = Σ g_i s_i is not a normalized weighted average. This undermines the interpretation of CSA as a confidence-weighting mechanism and may affect optimization. The authors should specify the activation (e.g., sigmoid or softmax) and any normalization used to map MLP outputs to valid confidence values.
minor comments (5)
  1. [Eq. (10)] The proxy matching loss reuses the index k for both text and video proxies in a confusing way. For each text proxy, the positive set contains all K video proxies, so the summation indices should be distinct (e.g., i over text proxies and j over video proxies). Please rewrite the formula to make the intended matching explicit.
  2. [Table 5] The two RAL variants on MSR-VTT are both labeled 'ViT-B/32'; one should presumably be ViT-B/16. Please correct the typo.
  3. [Appendix A] The appendix refers to the proposed framework as 'URAL' in several places (e.g., 'proposed URAL framework'), while the main text uses 'RAL'. Please unify the terminology.
  4. [Figure 3] The description of the uncertainty-level experiment is terse. Please specify how many query sets are formed, the size of each set, and how R@1 is averaged across sets. The claim that GMMFormer v2 'collapses' with R@1 near zero on one set would be more credible with the set sizes and error bars.
  5. [Section 4.5] The sentence 'T2VR can be viewed as a simple case of PRVR, where videos are trimmed to correspond to queries' is reasonable, but the support-set issue also applies to the MSR-VTT experiments: during training multiple captions per video are available, while at test only one query is given. Please state explicitly how the support set is handled in the T2VR transfer experiments.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RAL is an empirical plug-in model whose components are trained end-to-end on retrieval losses and evaluated on external benchmarks.

full rationale

The paper's derivation chain is empirical rather than formal: Eq. (3) estimates Gaussian parameters with learned MLPs from input features; Eqs. (8) and (10) define auxiliary training losses; Eq. (12) defines the final retrieval score using MLP-predicted word confidences. None of these quantities is defined in terms of the reported retrieval metric or of a fitted prediction renamed as a result. The query support set in Eq. (5) is explicitly labeled 'Training only' in Figure 2 and Appendix Figure 9, and it does not appear in the inference-time score Eq. (12). The ablation in Table 4 (X_q=Q drops SumR from 198.8 to 196.9) shows the support set contributes to training, but it is a training-time supervisory signal, not a circular redefinition of the target metric. The underspecified inference-time handling of the support set is a genuine train/test distribution concern, but it is not a circularity because the test ranker still consumes a single query. Self-citations (e.g., Dong et al. 2022/2023, Song et al. 2025) are used as baselines or task references, not as load-bearing authority for the core claim; no uniqueness theorem from the authors is invoked. The stated limitation about named entities is an empirical failure analysis, not a circular step. Therefore, no circularity is present.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard Gaussian uncertainty assumptions and on the paper-specific use of a training-only query support set. The main free parameters are the loss coefficients and the proxy count, which are hand-chosen and partly tuned on the TVR test set.

free parameters (6)
  • λ1 (weight for Lnce) = 0.05
    Chosen by hand to balance the contrastive loss.
  • λ2 (weight for Ltrip) = 1
    Chosen by hand.
  • λ3 (weight for LDA) = 0.001 (main text) / 0.004 (appendix, claimed optimal)
    The main text and appendix report inconsistent values; both are hand-selected or tuned, and the paper states the choice was based on TVR performance.
  • λ4 (weight for LPM) = 0.004 (main text) / 0.001 (appendix, claimed optimal)
    Same inconsistency and tuning dependence as λ3.
  • τ (temperature in LPM) = not reported
    Temperature factor in Eq. (10) is never given a value in the paper.
  • K (number of proxies) = 6
    Chosen after comparing K=2,4,6 in Table 3; a hyperparameter fitted to the test set.
assumptions (4)
  • domain assumption A diagonal Gaussian distribution N(μ, σ²I) adequately quantifies the aleatoric uncertainty of video and text embeddings for PRVR
    Invoked in Eq. (4) without evidence that off-diagonal correlations or other distribution families are unnecessary.
  • ad hoc to paper Using all queries associated with a video during training (query support set) yields a text distribution that transfers to single-query inference
    Section 3.2 defines the support set as a training-time enrichment; the paper never demonstrates that the text distribution estimated from a single query at test time is consistent.
  • domain assumption Max word-frame similarity with learned confidence weights is a sufficient query-video scoring rule
    Eqs. (11)-(12) assert this scoring function; it is plausible but not proven to be optimal for PRVR.
  • standard math The reparameterization trick provides valid and unbiased gradients for the proxy matching loss
    Eq. (9) uses the standard VAE reparameterization, a well-established technique.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning." pith.science (2026). https://pith.science/paper/OJQIEN2B

@misc{pith2026250901383,
  author       = {Pith},
  title        = {Pith review of: Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OJQIEN2B}},
  note         = {Machine review of arXiv:2509.01383}
}
read the original abstract

Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos partially relevant to a given query. The core challenge lies in learning robust query-video alignment against spurious semantic correlations arising from inherent data uncertainty: 1) query ambiguity, where the query incompletely characterizes the target video and often contains uninformative tokens, and 2) partial video relevance, where abundant query-irrelevant segments introduce contextual noise in cross-modal alignment. Existing methods often focus on enhancing multi-scale clip representations and retrieving the most relevant clip. However, the inherent data uncertainty in PRVR renders them vulnerable to distractor videos with spurious similarities, leading to suboptimal performance. To fill this research gap, we propose Robust Alignment Learning (RAL) framework, which explicitly models the uncertainty in data. Key innovations include: 1) we pioneer probabilistic modeling for PRVR by encoding videos and queries as multivariate Gaussian distributions. This not only quantifies data uncertainty but also enables proxy-level matching to capture the variability in cross-modal correspondences; 2) we consider the heterogeneous informativeness of query words and introduce learnable confidence gates to dynamically weight similarity. As a plug-and-play solution, RAL can be seamlessly integrated into the existing architectures. Extensive experiments across diverse retrieval backbones demonstrate its effectiveness.

Figures

Figures reproduced from arXiv: 2509.01383 by the authors.

Figure 1
Figure 1. (a) Toy examples of spurious semantic corre [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed framework. It mainly consists of two components: (1) Multimodal Semantic [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison between our model [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Performance of different methods under dif [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: Performance on different types of queries. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Query-video cosine similarities between test [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: PRVR results on TVR: top-1 retrieved video by our method and GMMFormer v2 ( [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualization of CSA mechanism. (a) Word [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 10
Figure 10. Figure 10: The variation trend of data uncertainty and [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 9
Figure 9. Figure 9: Integration of RAL with PRVR baselines, where MSRA and CSA stand for the proposed multi￾modal semantic robust alignment and confidence-aware set-to-set alignment modules, respectively. RAL is inte￾grated into the frame-level branch, and the final retrieval score is a c…
Figure 11
Figure 11. Figure 11: The impact of the loss coefficients, λ3 and λ4, of distribution alignment loss LDA and proxy matching loss LPM. as training progresses, and the model’s retrieval performance improves. This indicates that mitigat￾ing uncertainty is crucial for improving retrieval accur…
Figure 12
Figure 12. Figure 12: More visualization results on TVR and ActivityNet. Top-1 retrieved videos from our method and [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Failure case on TVR. red boxes indicate the top-1 retrieved video by our method and GMMFormer v2 (Wang et al., 2024d). Green box indicates the ground truth video. for large-scale applications. C More Visualization Results C.1 Qualitative Retrieval Results [PITH_FULL_…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation

    cs.IR 2026-07 conditional novelty 5.5 of 10

    Disentangling LLM hidden states into attribute-aligned geo and semantic slots before dual-stream residual quantization cuts SID collisions and improves local-life recommendation AUC.

Reference graph

Works this paper leans on

53 extracted references · 45 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Simion-Vlad Bogolin, Ioana Croitoru, Hailin Jin, Yang Liu, and Samuel Albanie. 2022. Cross modal retrieval with querybank normalisation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5194--5205

    work page 2022

  4. [4]

    Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299--6308

    2017

  5. [5]

    Zhiguo Chen, Xun Jiang, Xing Xu, Zuo Cao, Yijun Mo, and Heng Tao Shen. 2023. Joint searching and grounding: Multi-granularity video content retrieval. In Proceedings of the 31st ACM International Conference on Multimedia , pages 975--983

    work page 2023

  6. [6]

    Cheol-Ho Cho, WonJun Moon, Woojin Jun, MinSeok Jung, and Jae-Pil Heo. 2025. Ambiguity-restrained text-video representation learning for partially relevant video retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 2500--2508

    work page 2025

  7. [7]

    Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus. 2021. Probabilistic embeddings for cross-modal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8415--8424

    work page 2021

  8. [8]

    Jianfeng Dong, Xianke Chen, Minsong Zhang, Xun Yang, Shujie Chen, Xirong Li, and Xun Wang. 2022. Partially relevant video retrieval. In Proceedings of the 30th ACM International Conference on Multimedia, pages 246--257

    work page 2022

Show all 53 references
  1. [9]

    Jianfeng Dong, Xirong Li, Chaoxi Xu, Xun Yang, Gang Yang, Xun Wang, and Meng Wang. 2021. Dual encoding for video retrieval by text. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4065--4080

    2021

  2. [10]

    Jianfeng Dong, Minsong Zhang, Zheng Zhang, Xianke Chen, Daizong Liu, Xiaoye Qu, Xun Wang, and Baolong Liu. 2023. Dual learning with dynamic knowledge distillation for partially relevant video retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision...

    2023

  3. [11]

    Bo Fang, Wenhao Wu, Chang Liu, Yu Zhou, Yuxin Song, Weiping Wang, Xiangbo Shu, Xiangyang Ji, and Jingdong Wang. 2023. Uatvr: Uncertainty-adaptive text-video retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13723--13733

    2023

  4. [12]

    Zixian Gao, Xun Jiang, Xing Xu, Fumin Shen, Yujie Li, and Heng Tao Shen. 2024. Embracing unimodal aleatoric uncertainty for robust multimodal fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26876--26885

    2024

  5. [13]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778

    2016

  6. [14]

    Xun Jiang, Zhiguo Chen, Xing Xu, Fumin Shen, Zuo Cao, and Xunliang Cai. 2023. Progressive event alignment network for partial relevant video retrieval. In 2023 IEEE International Conference on Multimedia and Expo (ICME), pages 1973--1978

    2023

  7. [15]

    Peng Jin, Jinfa Huang, Fenglin Liu, Xian Wu, Shen Ge, Guoli Song, David Clifton, and Jie Chen. 2022. Expectation-maximization contrastive learning for compact video-and-language representations. Advances in neural information processing systems, 35:30291--30306

    2022

  8. [16]

    Peng Jin, Jinfa Huang, Pengfei Xiong, Shangxuan Tian, Chang Liu, Xiangyang Ji, Li Yuan, and Jie Chen. 2023. Video-text as game players: Hierarchical banzhaf interaction for cross-modal representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2023

  9. [17]

    Diederik P Kingma. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

    2013 arXiv

  10. [18]

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017. Dense-captioning events in videos. In Proceedings of the IEEE International Conference on Computer Vision, pages 706--715

    2017

  11. [19]

    Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. 2020. Tvr: A large-scale dataset for video-subtitle moment retrieval. In European Conference on Computer Vision, pages 447--463

    2020

  12. [20]

    Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, and Yu Qiao. 2023. Unmasked teacher: Towards training-efficient video foundation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19948--19960

    2023

  13. [21]

    Pandeng Li, Chen-Wei Xie, Hongtao Xie, Liming Zhao, Lei Zhang, Yun Zheng, Deli Zhao, and Yongdong Zhang. 2024. Momentdiff: Generative video moment retrieval from random to real. Advances in neural information processing systems, 36

    2024

  14. [22]

    Zhouhan Lin, Minwei Feng, C \' cero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In 5th International Conference on Learning Representations, ICLR

    2017

  15. [23]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  16. [24]

    Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. 2022. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputing, 508:293--304

    2022

  17. [25]

    Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zisserman. 2020. End-to-end learning of visual representations from uncurated instructional videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pa...

    2020

  18. [26]

    Taichi Nishimura, Shota Nakada, and Masayoshi Kondo. 2023. Large-scale vision-language models learn super images for efficient and high-performance partially relevant video retrieval. arXiv preprint arXiv:2312.00414

    2023 arXiv

  19. [27]

    Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, and Andrew C

    Seong Joon Oh, Kevin P. Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, and Andrew C. Gallagher. 2019. Modeling uncertainty with hedged instance embeddings. In 7th International Conference on Learning Representations, ICLR , New Orleans, LA, USA, May 6-9, 2019

    2019

  20. [28]

    Haowen Pan, Yixin Cao, Xiaozhi Wang, Xun Yang, and Meng Wang. 2024. Finding and editing multi-modal neurons in pre-trained transformers. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1012--1037

    2024

  21. [29]

    Peipei Song, Dan Guo, Xun Yang, Shengeng Tang, and Meng Wang. 2024 a . Emotional video captioning with vision-based emotion interpretation network. IEEE Transactions on Image Processing, 33:1122--1135

    2024

  22. [30]

    Peipei Song, Long Zhang, Long Lan, Weidong Chen, Dan Guo, Xun Yang, and Meng Wang. 2025. Towards efficient partially relevant video retrieval with active moment discovering. IEEE Transactions on Multimedia, pages 1--12

    2025

  23. [31]

    Peipei Song, Yuanen Zhou, Xun Yang, Daqing Liu, Zhenzhen Hu, Depeng Wang, and Meng Wang. 2024 b . Efficiently gluing pre-trained language and vision models for image captioning. ACM Transactions on Intelligent Systems and Technology, 15(6):1--16

    2024

  24. [32]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  25. [33]

    Jiamian Wang, Guohao Sun, Pichao Wang, Dongfang Liu, Sohail Dianat, Majid Rabbani, Raghuveer Rao, and Zhiqiang Tao. 2024 a . Text is mass: Modeling as stochastic embedding for text-video retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  26. [34]

    Yi Wang, Conrad M Albrecht, Nassim Ait Ali Braham, Chenying Liu, Zhitong Xiong, and Xiao Xiang Zhu. 2024 b . Decoupling common and unique representations for multimodal self-supervised learning. In European Conference on Computer Vision, pages 286--303. Springer

    2024

  27. [35]

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Zun Wang, Yansong Shi, and 1 others. 2024 c . Internvideo2: Scaling foundation models for multimodal video understanding. In European Conference on Computer Vision, pages 396--416. Springer

    2024

  28. [36]

    Yun Wang, Long Zhang, Jingren Liu, Jiaqi Yan, Zhanjie Zhang, Jiahao Zheng, Xun Yang, Dapeng Wu, Xiangyu Chen, and Xuelong Li. 2025. Episodic memory representation for long-form video understanding. arXiv preprint arXiv:2508.09486

    2025

  29. [37]

    Yuting Wang, Jinpeng Wang, Bin Chen, Tao Dai, Ruisheng Luo, and Shu-Tao Xia. 2024 d . Gmmformer v2: An uncertainty-aware framework for partially relevant video retrieval. arXiv preprint arXiv:2405.13824

    2024 arXiv

  30. [38]

    Yuting Wang, Jinpeng Wang, Bin Chen, Ziyun Zeng, and Shu-Tao Xia. 2024 e . Gmmformer: Gaussian-mixture-model based transformer for efficient partially relevant video retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5767--5775

    2024

  31. [39]

    Ziyang Wang, Yi-Lin Sung, Feng Cheng, Gedas Bertasius, and Mohit Bansal. 2023. Unified coarse-to-fine alignment for video-text retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2816--2827

    2023

  32. [40]

    Wenhao Wu, Haipeng Luo, Bo Fang, Jingdong Wang, and Wanli Ouyang. 2023. Cap4video: What can auxiliary captions do for text-video retrieval? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10704--10713

    2023

  33. [41]

    Xun Yang, Tianyu Chang, Tianzhu Zhang, Shanshan Wang, Richang Hong, and Meng Wang. 2024 a . Learning hierarchical visual transformation for domain generalizable visual matching and recognition. International Journal of Computer Vision, 132(11):4823--4849

    2024

  34. [42]

    Xun Yang, Fuli Feng, Wei Ji, Meng Wang, and Tat-Seng Chua. 2021. Deconfounded video moment retrieval with causal intervention. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 1–10

    2021

  35. [43]

    Xun Yang, Shanshan Wang, Jian Dong, Jianfeng Dong, Meng Wang, and Tat-Seng Chua. 2022. Video moment retrieval with cross-modal neural architecture search. IEEE Transactions on Image Processing, 31:1204--1216

    2022

  36. [44]

    Xun Yang, Jianming Zeng, Dan Guo, Shanshan Wang, Jianfeng Dong, and Meng Wang. 2024 b . Robust video question answering via contrastive cross-modality representation learning. Science China Information Sciences, 67(10):202104

    2024

  37. [45]

    Shukang Yin, Sirui Zhao, Hao Wang, Tong Xu, and Enhong Chen. 2024. Exploiting instance-level relationships in weakly supervised text-to-video retrieval. ACM Transactions on Multimedia Computing, Communications and Applications, 20(10):1--21

    2024

  38. [46]

    Biao Zhang and Peter Wonka. 2021. Point cloud instance segmentation using probabilistic embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8883--8892

    2021

  39. [47]

    Gengyuan Zhang, Jisen Ren, Jindong Gu, and Volker Tresp. 2023. Multi-event video-text retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22113--22123

    2023

  40. [48]

    Hao Zhang, Aixin Sun, Wei Jing, Guoshun Nan, Liangli Zhen, Joey Tianyi Zhou, and Rick Siow Mong Goh. 2021. Video corpus moment retrieval with contrastive learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieva...

    2021

  41. [49]

    Jing Zhang, Dan Guo, Xun Yang, Peipei Song, and Meng Wang. 2024. Visual-linguistic-stylistic triple reward for cross-lingual image captioning. ACM Transactions on Multimedia Computing, Communications and Applications, 20(4):1--23

    2024

  42. [50]

    Long Zhang, Peipei Song, Zhangling Duan, Shuo Wang, Xiaojun Chang, and Xun Yang. 2025 a . Video corpus moment retrieval with query-specific context learning and progressive localization. IEEE Transactions on Circuits and Systems for Video Technology, 35(6):5659--5670

    2025

  43. [51]

    Qun Zhang, Chao Yang, Bin Jiang, and Bolin Zhang. 2025 b . Multi-grained alignment with knowledge distillation for partially relevant video retrieval. ACM Transactions on Multimedia Computing, Communications and Applications

    2025

  44. [52]

    Sheng Zhou, Junbin Xiao, Qingyun Li, Yicong Li, Xun Yang, Dan Guo, Meng Wang, Tat-Seng Chua, and Angela Yao. 2025 a . Egotextvqa: Towards egocentric scene-text aware video question answering. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 3363--3373

    2025

  45. [53]

    Sheng Zhou, Junbin Xiao, Xun Yang, Peipei Song, Dan Guo, Angela Yao, Meng Wang, and Tat-Seng Chua. 2025 b . Scene-text grounding for text-based video question answering. IEEE Transactions on Multimedia

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.