REVIEW 7 cited by
Correlation-Guided Query-Dependency Calibration for Video Temporal Grounding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Temporal Grounding is to identify specific moments or highlights from a video corresponding to textual descriptions. Typical approaches in temporal grounding treat all video clips equally during the encoding process regardless of their semantic relevance with the text query. Therefore, we propose Correlation-Guided DEtection TRansformer (CG-DETR), exploring to provide clues for query-associated video clips within the cross-modal attention. First, we design an adaptive cross-attention with dummy tokens. Dummy tokens conditioned by text query take portions of the attention weights, preventing irrelevant video clips from being represented by the text query. Yet, not all words equally inherit the text query's correlation to video clips. Thus, we further guide the cross-attention map by inferring the fine-grained correlation between video clips and words. We enable this by learning a joint embedding space for high-level concepts, i.e., moment and sentence level, and inferring the clip-word correlation. Lastly, we exploit the moment-specific characteristics and combine them with the context of each video to form a moment-adaptive saliency detector. By exploiting the degrees of text engagement in each video clip, it precisely measures the highlightness of each clip. CG-DETR achieves state-of-the-art results on various benchmarks for temporal grounding. Codes are available at https://github.com/wjun0830/CGDETR.
Forward citations
Cited by 7 Pith papers
-
DisTime: Distribution-based Time Representation for Video Large Language Models
A single learnable time token, decoded into a probability distribution over time bins, improves temporal grounding in Video-LLMs and is trained partly on a new 1.25M-event pseudo-labeled dataset.
-
ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding
A coarse-to-fine video-grounding framework improves temporal boundary accuracy by densely re-examining frames around coarse start/end predictions, achieving SOTA on HC-STVGv1/v2 and VidSTG.
-
AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
A new audio-visual reasoning benchmark and a two-part metric (factual consistency, core inference) claim to expose a gap between answer accuracy and reasoning quality in AV-LLMs.
-
MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learning
MS-DETR improves moment retrieval and highlight detection by disentangling motion and semantic video features, sharing task information between the two tasks, and training on generated auxiliary captions.
-
SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM
SMART, an audio-enhanced MLLM with shot-aware token compression, reports new state-of-the-art moment retrieval accuracy on Charades-STA and QVHighlights.
-
BSN-II: The First Light Curve Study of Eight Total Eclipsing Contact Binary Stars with Shallow Fillout Factors
The claimed photometric study of eight contact binaries cannot be verified because the submitted full text is a different paper on video grounding.
-
TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding
TimeExpert applies dynamic mixture-of-experts routing to video temporal grounding, reporting small state-of-the-art gains over TRACE on dense video captioning, moment retrieval, and highlight detection.
Discussion (0). Sign in to comment.