Pith. sign in

REVIEW 7 cited by

CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.06430 v4 pith:K3QLGRWC submitted 2022-09-14 cs.CV

Hongwei Xue , Yuchong Sun , Bei Liu , Jianlong Fu , Ruihua Song , Houqiang Li , Jiebo Luo This is my paper · ORCID
classification cs.CV
keywords clipclip-vippre-trainedimage-textmodelrepresentationresultsvideo-language
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The pre-trained image-text models, like CLIP, have demonstrated the strong power of vision-language representation learned from a large scale of web-collected image-text data. In light of the well-learned visual features, some existing works transfer image representation to video domain and achieve good results. However, how to utilize image-language pre-trained model (e.g., CLIP) for video-language pre-training (post-pretraining) is still under explored. In this paper, we investigate two questions: 1) what are the factors hindering post-pretraining CLIP to further improve the performance on video-language tasks? and 2) how to mitigate the impact of these factors? Through a series of comparative experiments and analyses, we find that the data scale and domain gap between language sources have great impacts. Motivated by these, we propose a Omnisource Cross-modal Learning method equipped with a Video Proxy mechanism on the basis of CLIP, namely CLIP-ViP. Extensive results show that our approach improves the performance of CLIP on video-text retrieval by a large margin. Our model also achieves SOTA results on a variety of datasets, including MSR-VTT, DiDeMo, LSMDC, and ActivityNet. We will release our code and pre-trained CLIP-ViP models at https://github.com/microsoft/XPretrain/tree/main/CLIP-ViP.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CoVR-R:Reason-Aware Composed Video Retrieval

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Zero-shot LMM reasoning over edit after-effects (states, phases, camera, tempo) plus a new CoVR-R benchmark yields large recall gains on implicit-effect composed video retrieval without task-specific training.

  2. Beyond Simple Edits: Composed Video Retrieval with Dense Modifications

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A new benchmark with much longer, denser modification texts, plus a single-encoder fusion model, raises composed video retrieval Recall@1 by 3.4 points on its own test set.

  3. Are Synthetic Videos Useful? A Benchmark for Retrieval-Centric Evaluation of Synthetic Videos

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Synthetic videos rated high on semantic alignment, especially objects and actions, improve text-to-video retrieval models when used as training data.

  4. Knowledge-guided Disentanglement with Atomic Actions for Action Recognition

    cs.CV 2026-07 conditional novelty 5.0 of 10

    LLM-generated atomic action descriptions injected into scene-graph node features improve prompt-guided disentanglement for multi-label action recognition, with reported gains on Charades-oracle and SportsHHI but not o...

  5. IntentVCNet: Bridging Spatio-Temporal Gaps for Intention-Oriented Controllable Video Captioning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    IntentVCNet uses per-frame object coordinates, red-box visual prompts, and a lightweight box adapter to make video captioning focus on a user-selected object, reporting 225.19 CIDEr on the IntentVC public test set.

  6. MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MoMa adapts frozen CLIP to video by injecting Mamba-computed scale and bias into each layer, improving accuracy and efficiency on multiple action recognition benchmarks.

  7. Video-Text Temporal Localization via Multi-Scale Convolution and Dynamic Routing

    cs.CV 2026-07 conditional novelty 4.5 of 10

    Multi-scale temporal convolutions plus capsule routing improve video-text moment localization to 42.9% R@0.5 and 41.1% mIoU on ActivityNet Captions.

Pith tools