{"id":"f972b94c-5082-4c32-bad3-9bed124fb2fd","arxiv_id":"2412.13441","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FlashVTG improves moment retrieval and highlight detection by replacing the DETR decoder with a temporal feature pyramid and adaptively refining confidence scores from intra- and inter-scale features.","lead":"FlashVTG is a new network for video temporal grounding that uses multiple temporal scales and a context-aware confidence score to find and rank video moments matching a text query. It reports state-of-the-art results on four benchmarks, with especially large gains on short moments, though the biggest numbers come from a stronger video feature backbone.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Clip-Aware Score Loss is dimensionally inconsistent: cfinal has ~2L entries while sgt has L, so Eq. (10) is undefined without an unspecified alignment step.","rationale":"The reader's backbone-matching concern is legitimate but not the most load-bearing: the paper also reports SlowFast+CLIP results that exceed other SlowFast+CLIP methods (e.g., +3.5 mAP over LLMEPET on QVHighlights test), so the architecture retains a same-backbone advantage. The Clip-Aware Score Loss, however, is a named contribution and a key training signal; if it is dimensionally ill-defined, the method cannot be implemented as described and the empirical results are not trustworthy from the manuscript alone. This is a concrete, internal inconsistency that would be settled by inspecting the released code. The verdict should remain conditional: acceptance is warranted only if the code or supplementary clarifies the alignment between cfinal and sgt and the clarified loss reproduces the reported numbers. I disagree with the reader's identification of the weakest assumption; the weaker point is the loss specification, not backbone matching.","tokens_in":16141,"tokens_out":11719,"duration_ms":98946,"concrete_test":"Download the official code from the provided GitHub link and locate the implementation of Clip-Aware Score Loss. Check the tensor shapes: if cfinal is reshaped, gathered, or pooled to length L before computing MSE with sgt, document the exact alignment and re-implement Eq. (10) strictly as written (no implicit alignment) on QVHighlights. If training fails or the reported mAP changes materially, the published loss is incomplete and the SOTA claim is not reproducible from the paper alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim to SOTA MR/HD rests on training with the proposed Clip-Aware Score Loss (§3.6, Eq. 10). As defined in §3.5, cfinal = x·cintra + (1−x)·cinter; c_k = ScoreHead1(F_k) ∈ R^{L/2^{k-1}}, so cintra = Concat(c_1,...,c_K) has length Σ_k L/2^{k-1} ≈ 2L, and cinter is obtained similarly on the concatenated pyramid, also ≈2L. Thus cfinal contains one score per pyramid position (≈2L scores), while the target sgt is the clip-level saliency sequence of length L. Eq. (10) then writes LCAS = MSE(normalize(cfinal), normalize(sgt)) with no alignment, pooling, or upsampling from pyramid positions to original video clips. The two tensors have incompatible shapes, so the loss as stated cannot be computed. The paper never specifies how multi-scale moment confidence scores are mapped to per-clip saliency labels; the supplementary is not included, and Table 8 does not ablate this loss. Consequently the fine-grained supervision highlighted in the abstract is not actually defined, and the reported gains cannot be attributed to a reproducible training objective. This is an internal inconsistency, not a matter of consensus, and it is load-bearing for the central claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FlashVTG proposes a Temporal Feature Layering (TFL) module that replaces DETR-style decoders with a multi-scale convolutional pyramid, and an Adaptive Score Refinement (ASR) module that combines intra-scale and inter-scale confidence scores. The paper also introduces a Clip-Aware Score Loss that uses highlight-detection saliency labels to supervise moment-retrieval confidence scores. Experiments on QVHighlights, TACoS, Charades-STA, TVSum, and YouTube-HL report state-of-the-art or competitive results in moment retrieval and highlight detection, with the largest gains obtained when the model is combined with an InternVideo2 backbone. The paper claims a 5.8% MR mAP and 3.3% HD mAP improvement on QVHighlights, and 125% of previous SOTA in short-moment retrieval mAP, alongside a smaller same-backbone gain of 1.4% MR mAP.","tokens_in":16422,"tokens_out":3026,"duration_ms":28084,"significance":"If the central claims survive scrutiny, the paper would make a useful contribution: the TFL module is a simple and plausible alternative to learned decoder queries, and the ASR module addresses a real ranking problem. The same-backbone gains on QVHighlights (1.4% MR mAP), Charades-STA, and TACoS are nontrivial, and the authors provide code, which aids reproducibility. However, the paper currently overstates the architectural benefit by advertising gains that come from a stronger backbone, and the main new training objective, the Clip-Aware Score Loss, is not well-defined as written. These issues are load-bearing for the paper's central empirical claim, so the manuscript needs substantive revision before the results can be accepted as reported.","major_comments":[{"comment":"The Clip-Aware Score Loss as defined cannot be computed. From Eq. (6), each c_k has length L_v / 2^{k-1}, so Eq. (7) gives c_intra of total length sum_{k=1}^K L_v / 2^{k-1} ≈ 2L_v. Eq. (8) states that c_inter matches the dimensions of the intra-scale output, and Eq. (9) therefore produces c_final of length ≈ 2L_v. The target s_gt is the clip-level saliency sequence of length L_v. Eq. (10) then takes an MSE between normalized c_final and normalized s_gt with no specified alignment, pooling, upsampling, or indexing from pyramid positions to video clips. The manuscript provides no supplementary material, and Table 8 does not ablate this loss. Consequently, the fine-grained supervision highlighted in the introduction and abstract is not actually defined, and the reported gains cannot be attributed to a reproducible training objective. The authors must specify how c_final is mapped to per-clip saliency labels, or revise the loss definition, and should report an ablation of the loss.","section":"§3.6, Eq. (10); §3.5, Eqs. (6)–(9)"},{"comment":"The headline improvements of 5.8% MR mAP and 3.3% HD mAP on QVHighlights are obtained with the InternVideo2 backbone, not with the same backbone as the prior SOTA. Table 1 shows the same-backbone gain is 1.4% MR mAP relative to R2-Tuning, and Table 4 shows the same-backbone HD gain is smaller (41.07 vs. 40.75 for R2-Tuning). The abstract states these boosts without disclosing the backbone mismatch, and §4.4 similarly says 'a 5.8% increase in mAP on the test set' before separately noting the same-backbone improvement. This presentation overstates the contribution of the proposed architecture. The short-moment claim of 125% of previous SOTA (Table 7) is also confounded by backbone choice, since R2-Tuning there uses only CLIP features while FlashVTG uses SlowFast+CLIP. The authors should either consistently report backbone-matched comparisons in the abstract and main claims, or clearly label which numbers use the stronger backbone.","section":"Abstract; §4.4; Tables 1 and 4"},{"comment":"The ablation study does not isolate the contribution of the ASR module as cleanly as the text suggests. Table 8 adds TFL and then ASR together with the Clip-Aware Score Loss, but the text in §4.5 says the saliency label from HD is used as a supervision signal and that this 'indirectly boosted HD performance.' Since the loss term is undefined as written (see the first major comment), the 0.37 MR mAP and 0.52 HD mAP differences between the TFL-only and TFL+ASR rows cannot be interpreted as purely due to score refinement. A complete ablation should show TFL alone, TFL+ASR without the clip-aware loss, and TFL+ASR with the loss, and should clarify what the loss actually computes.","section":"§3.5 and Table 8"}],"minor_comments":[{"comment":"The sentence 'performance improvements of 2.7%, 2.2%, and 11.9% respectively in MR' does not specify the comparison baseline or the backbone setting; please make the comparison explicit.","section":"§1"},{"comment":"The abstract says 'four widely adopted datasets' but §4.1 lists and the experiments evaluate five datasets (QVHighlights, TACoS, Charades-STA, TVSum, YouTube-HL); this should be corrected.","section":"§4.1 vs. Abstract"},{"comment":"The R2-Tuning validation mAP is reported as 47.86 in Table 7 but 47.59 in Table 1; these numbers should be reconciled.","section":"Table 7 vs. Table 1"},{"comment":"The implementation details state 'K = 4, 5 layers in temporal feature layering'; this is ambiguous and should say whether K=4 or K=5 is used, and whether different datasets use different values.","section":"§4.3"},{"comment":"The notation in Eq. (5), with transposes applied to the output of a 1D convolution, is confusing; a clearer statement of how Conv1D acts on the feature dimension would improve readability.","section":"§3.4, Eq. (5)"},{"comment":"The related work and citations are not always consistent in formatting (e.g., 'R2-Tunning' vs. 'R2-Tuning'); please unify spelling and reference style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2412.13441. First, the core architecture is genuinely new and the experiments are extensive: replacing the DETR decoder with a temporal feature pyramid (TFL) and refining confidence scores with intra- and inter-scale features (ASR) gives consistent gains across four benchmarks, and the ablations in Table 8 show each module helps. Second, the paper has a load-bearing technical flaw: the Clip-Aware Score Loss in Eq. (10) is undefined as written. c_final has roughly 2L entries (sum of L/2^{k-1} over K scales) while s_gt has L entries, and there is no described alignment, pooling, or upsampling to reconcile the shapes. The supplementary, which presumably would clarify this, is not included. This matters because the abstract highlights this loss as the source of fine-grained supervision for short moments, yet Table 8 does not even ablate it. So the training objective as stated is not reproducible.\n\nWhat the paper does well: the TFL module is a sensible departure from sparse DETR queries, and the ablations show a ~6% MR mAP gain from TFL alone, which is real. The ASR refinement also contributes. The comparisons are mostly fair; Tables 1 and 4 use the † symbol for InternVideo2, and Section 4.4 explicitly acknowledges that same-backbone gain is 1.4%. The abstract, however, leads with the 5.8%/3.3% numbers from the stronger backbone, which overstates the architectural benefit. The short-moment claim (125% of SOTA) also compares against R2-Tuning with only CLIP, not the SlowFast+CLIP used by others; that comparison is not fully matched.\n\nSoft spots, in order: (1) the undefined loss is the biggest; (2) abstract overstates backbone contribution; (3) the loss is not ablated; (4) minor typos like 'SampledNCE' misspelled and 'R2-Tunning'. The free-parameter list is fine; these are standard hyperparameters.\n\nShould a serious editor send this to peer review? Yes. The architectural idea is worth referee time, and the flaw can be fixed by specifying the alignment or removing the loss and reporting results without it. The empirical work is substantial. I would not recommend acceptance as-is, but conditional on a corrected Eq. (10) and a backbone-matched abstract, this becomes a solid contribution to video temporal grounding.\n\nWho is it for: researchers working on moment retrieval and highlight detection, especially those building on DETR-based architectures. I'd bring it to a reading group to discuss the TFL/ASR design. I would cite it if the loss is fixed, but for now I'd be hesitant to cite an undefined objective.","headline":"A genuinely new architecture with strong ablations, but the headline score loss is undefined as written and the abstract overstates backbone-matched gains.","tokens_in":16958,"tokens_out":3910,"would_cite":false,"duration_ms":34362,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FlashVTG claims that replacing the DETR decoder with a temporal feature pyramid and context-aware score refinement raises moment retrieval and highlight detection to state-of-the-art on multiple benchmarks, with short-moment mAP reaching…","keywords":["video temporal grounding","moment retrieval","highlight detection","short moment retrieval","temporal feature pyramid","adaptive score refinement","QVHighlights","transformer decoder"],"falsifier":"Running FlashVTG on QVHighlights with SlowFast+CLIP features and ablating the ASR module gives a concrete test: the paper predicts validation MR mAP falls from 52.84 to 52.47 without ASR and to 46.84 without both modules, so an external implementation should reproduce that gap. If a prior DETR-based method, under identical features, training schedule, and NMS settings, matches or exceeds 47.59 test mAP, the decoder-free claim is falsified.","tokens_in":15921,"feed_emoji":"🎬","tokens_out":8512,"duration_ms":71723,"temperature":0.7,"pith_summary":"FlashVTG is an architecture for video temporal grounding: given an untrimmed video and a natural-language query, it finds the moments that match the query and scores clips as highlights. The paper argues that DETR-style models fail on short moments because they rely on a small fixed set of decoder queries, and because they rank predictions from each predicted moment in isolation. To fix both, FlashVTG builds a multi-scale temporal feature pyramid with strided convolutions and reranks moments using neighboring and cross-scale context. On QVHighlights the authors report state-of-the-art moment retrieval and highlight detection, with short-moment mAP about 125% of the previous best, and claim these gains come without added training burden.","feed_headline":"Short-moment video retrieval jumps to 125% of the previous best","feed_subtitle":"A feature pyramid over time plus context-aware score refinement catches clips that sparse query decoders miss.","key_machinery":"Temporal Feature Layering (TFL) is a feature pyramid built by applying 1D convolutions with stride 2 to the fused video-text features, producing scales $F_k \\in \\mathbb{R}^{L_v/2^{k-1} \\times d}$ for $k=1,\\dots,K$; a shared moment-prediction head outputs start/end boundaries at every scale, replacing the decoder's sparse learned queries. Adaptive Score Refinement (ASR) computes intra-scale confidence scores $c_{intra}$ from a per-scale score head and an inter-scale score $c_{inter}$ from a second head over the concatenated pyramid, then combines them as $c_{final} = x\\,c_{intra} + (1-x)\\,c_{inter}$ with a learnable weight $x$. The Clip-Aware Score Loss transfers saliency labels from highlight detection into moment retrieval by min-max normalizing both confidence and saliency scores and penalizing their mean squared error, giving fine-grained clip-level supervision that mainly helps short moments.","core_discovery":"The paper's central claim is that a decoder-free design can outperform query-based DETR models on both moment retrieval and highlight detection, and that the clearest win is on short moments under ten seconds. Concretely, on QVHighlights the authors report a test mAP of 47.59 with the standard SlowFast+CLIP features and 52.00 when features come from InternVideo2, and they state that the same-backbone gain is 1.4% mAP. Their ablation attributes the gain to the Temporal Feature Layering module, which raises validation MR mAP from 46.84 to 52.47 by itself, and to the Adaptive Score Refinement module, which adds a further gain to 52.84. They also report short-moment retrieval mAP of 15.73 versus 12.62 for the previous best, and consistent improvements on TACoS, Charades-STA, TVSum, and YouTube-HL.","pith_inferences":["Editorial inference: the abstract's 5.8% MR and 3.3% HD figures compare FlashVTG with InternVideo2 features against baselines on weaker SlowFast+CLIP features; Section 4.4 states the matched-feature gain is 1.4% mAP, so the standalone architectural benefit is closer to the smaller number.","Editorial inference: the TFL module is a temporal analogue of image feature pyramids, but the paper does not ablate against alternative multi-scale temporal designs such as dilated convolutions or different stride choices, so the exact strided-convolution recipe is not isolated.","Editorial inference: the Clip-Aware Score Loss depends on having clip-level saliency labels; on datasets without such labels its benefit would likely shrink, and a clean test would be to run the method on MR-only data and measure the drop."],"forward_implications":["Short moments (under about 10 seconds) become retrievable without increasing the number of decoder queries, so the cost of finer localization does not grow with the number of candidates.","Confidence scores are no longer derived from a single isolated prediction; ranking improves because predictions are compared with their temporal neighbors and across scales, which should directly improve Recall@1.","Moment retrieval and highlight detection share the same clip-level supervision, so improvements in one task can propagate to the other within one training run.","The architecture composes with stronger video features: using InternVideo2 gives a further large jump on QVHighlights, suggesting the modules are complementary to feature quality.","Because the decoder is removed, the model trains with the same reported budget (about 1.5 hours on one RTX 4090 for QVHighlights), so the gains are not bought with longer training."],"supporting_citations":[{"why":"Supplies the QVHighlights dataset and the Moment-DETR baseline: the decoder-query architecture that FlashVTG replaces.","marker":"[16]"},{"why":"R2-Tuning is the previous state-of-the-art baseline for the headline comparisons and the source of the SampledNCE loss used for highlight detection.","marker":"[29]"},{"why":"InternVideo2 is the stronger video backbone used to produce the larger QVHighlights numbers.","marker":"[48]"},{"why":"CG-DETR is the source of the Adaptive Cross-Attention fusion module used to align video and query features.","marker":"[32]"},{"why":"SlowFast is one of the two standard video feature extractors used for matched comparisons.","marker":"[6]"},{"why":"CLIP supplies the text and image encoders that produce query and clip features in the standard setting.","marker":"[36]"},{"why":"QD-DETR provides a query-dependent representation baseline and a comparison point on Charades-STA and QVHighlights.","marker":"[33]"},{"why":"LLMEPET is a recent prior method used as a comparison and reference for several metrics.","marker":"[15]"}],"fun_headline_variants":["Decoder-free FlashVTG lifts short-moment retrieval to 125% of prior best","FlashVTG: New SOTA for video temporal grounding without extra training cost","Short-moment video grounding gets a 25% boost from FlashVTG's layered features","Zero extra training: FlashVTG tops video grounding benchmarks","Video grounding reinvented: FlashVTG excels at short moments without decoders"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage over prior methods is only meaningful when all compared models use the same video and text features; the largest headline numbers use InternVideo2 features that most baselines do not have.","fun_headline_variants_meta":{"raw":{"variants":["Decoder-free FlashVTG lifts short-moment retrieval to 125% of prior best","FlashVTG: New SOTA for video temporal grounding without extra training cost","Short-moment video grounding gets a 25% boost from FlashVTG's layered features","Zero extra training: FlashVTG tops video grounding benchmarks","Video grounding reinvented: FlashVTG excels at short moments without decoders"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000652,"raw_usage":{"total_tokens":3019,"prompt_tokens":1008,"completion_tokens":2011,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":1919}},"tokens_in":624,"tokens_out":2011,"duration_ms":12529,"temperature":1.0,"reasoning_tokens":1919,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:06:36.638358+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Running FlashVTG on QVHighlights with SlowFast+CLIP features and ablating the ASR module gives a concrete test: the paper predicts validation MR mAP falls from 52.84 to 52.47 without ASR and to 46.84 without both modules, so an external implementation should reproduce that gap. If a prior DETR-based method, under identical features, training schedule, and NMS settings, matches or exceeds 47.59 test mAP, the decoder-free claim is falsified.","supporting_citations":[{"cited_title":"Detecting moments and highlights in videos via natural language queries","cited_arxiv_id":null,"evidence_quote":"Supplies the QVHighlights dataset and the Moment-DETR baseline: the decoder-query architecture that FlashVTG replaces."},{"cited_title":"r2-tuning: Ef- ficient image-to-video transfer learning for video temporal grounding","cited_arxiv_id":null,"evidence_quote":"R2-Tuning is the previous state-of-the-art baseline for the headline comparisons and the source of the SampledNCE loss used for highlight detection."},{"cited_title":"Slowfast networks for video recognition","cited_arxiv_id":null,"evidence_quote":"SlowFast is one of the two standard video feature extractors used for matched comparisons."},{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"CLIP supplies the text and image encoders that produce query and clip features in the standard setting."},{"cited_title":"Query-dependent video representa- tion for moment retrieval and highlight detection","cited_arxiv_id":null,"evidence_quote":"QD-DETR provides a query-dependent representation baseline and a comparison point on Charades-STA and QVHighlights."},{"cited_title":"Prior knowledge integra- tion via LLM encoding and pseudo event regulation for video moment retrieval","cited_arxiv_id":null,"evidence_quote":"LLMEPET is a recent prior method used as a comparison and reference for several metrics."}],"review_version":1}