{"id":"19321bce-c9f3-4ab3-96f5-720111767c5f","arxiv_id":"2501.07972","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A tuning-free pipeline using frozen LLaMA-3, MiniGPT-v2, and Video-ChatGPT reports state-of-the-art zero-shot video moment retrieval on three benchmarks.","lead":"This paper assembles three off-the-shelf AI models into a no-training pipeline, Moment-GPT, that finds video segments matching a text query: LLaMA-3 rewrites the query, MiniGPT-v2 captions video frames, and Video-ChatGPT scores candidate clips. The method reports top results on standard video moment retrieval benchmarks without any fine-tuning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central SOTA claim may be an artifact of hyperparameters selected on the test sets; a fair zero-shot evaluation with fixed hyperparameters and error bars is needed.","rationale":"The reader's weakest assumption concerns caption fidelity and the oracle-bound gap (Table 17), which is a valid explanatory weakness but does not directly threaten the empirical claim: even imperfect captions could still yield SOTA zero-shot performance. The more load-bearing issue is that the reported hyperparameters appear to be selected on the Charades-STA test set, and the stochastic pipeline is evaluated without error bars. If that is true, the numerical advantage over SOTA zero-shot methods could be an artifact of tuning rather than the method's design. This is a correctable flaw, so the verdict remains CONDITIONAL pending a fair re-evaluation, but the condition should be made explicit: fixed a-priori hyperparameters and repeated trials. The reader mentioned test-set tuning in the rationale but did not make it the weakest assumption, hence partial agreement.","tokens_in":17019,"tokens_out":6089,"duration_ms":59769,"concrete_test":"Run five independent trials of Moment-GPT on Charades-STA and QVHighlights with hyperparameters fixed a priori (e.g., kappa=7, tau=5, lambda=0.2, sigma=0.9, Nd=3, selected from a validation split or from the QVHighlights val set), and compare mean +/- std R1@n and mIoU against the same baselines (Luo et al. and Wattasseril et al.) under identical settings. Also run a paired sweep of kappa on a validation split to measure how much of the reported gain is due to test-set selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 fixes eta=10, kappa=7, tau=5, lambda=0.2, sigma=0.9, Nd=3; Appendix B.3 shows these values were selected by ablating on Charades-STA, where R1@0.5 ranges from 37.0 to 38.4 across kappa and 36.9 to 38.4 across Nd. Charades-STA is reported on the same test split, so the reported 38.4 is the result of test-set hyperparameter tuning. The paper nowhere discloses this. The same tuned values are then transferred to QVHighlights and ActivityNet, which avoids direct test-set tuning there but still selects hyperparameters on a dataset from the same benchmark family. Because the pipeline involves stochastic MLLMs (temperature 0.2-0.3), and no error bars or multiple seeds are reported, the claimed margins over SOTA zero-shot methods (+4.8 R1@0.3 on Charades-STA, +5.8 R1@0.5 on QVHighlights val) may not be statistically robust. If the hyperparameters are re-chosen on validation data or fixed a priori, the advantage could shrink. This directly threatens the strongest claim of 'substantially outperforming' SOTA and most supervised methods.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Moment-GPT, a training-free zero-shot video moment retrieval pipeline that combines three frozen MLLMs: LLaMA-3 rephrases and corrects the input query to reduce language bias, MiniGPT-v2 captions individual frames, a histogram-based span generator converts frame-level cosine similarities into candidate spans, and Video-ChatGPT captions each span so that a LLaMA-3-based span scorer can rank candidates, followed by NMS. Experiments on QVHighlights, Charades-STA, and ActivityNet-Captions report large gains over prior zero-shot methods and level with or better than several supervised baselines, with component ablations and an oracle-bound analysis in the appendix.","tokens_in":17298,"tokens_out":9397,"duration_ms":82931,"significance":"If the empirical results survive a properly controlled evaluation, the paper is a useful demonstration that off-the-shelf frozen MLLMs can be assembled into a competitive zero-shot VMR system without fine-tuning. The query-debiasing step is simple and appears effective, and the adaptive span generator is computationally lighter than sliding-window alternatives. The paper also provides component-level ablations and an oracle-bound diagnostic, which help the reader locate where the pipeline loses accuracy. At present, however, the headline claim of substantially outperforming SOTA zero-shot methods is not fully supported because the evaluation methodology has unresolved issues: hyperparameters are selected on a test split without disclosure, and stochastic MLLM components are evaluated with single runs and no error bars. The paper's own oracle bound (Table 17, 68.5 vs 38.4 R1@0.5) further indicates large headroom, so the absolute performance should not be oversold.","major_comments":[{"comment":"The manuscript reports the hyperparameters (eta=10, kappa=7, tau=5, lambda=0.2, sigma=0.9, Nd=3) as fixed implementation details, but Appendix B.3 (Tables 12-16) shows that these values were selected by ablating on Charades-STA, the same test split whose results appear in Table 2. The R1@0.5 scores vary from 37.0 to 38.4 across kappa and from 36.9 to 38.4 across Nd, so the reported 38.4 is the selected maximum of a test-set search, not a fixed-configuration zero-shot result. This should be disclosed, and the evaluation should be repeated with hyperparameters chosen on a validation split or fixed a priori before computing test-set numbers; the transfer of the same values to QVHighlights and ActivityNet does not remove the concern, since they are selected on a dataset from the same benchmark family.","section":"Sec. 4.2 and Appendix B.3"},{"comment":"The pipeline is stochastic: LLaMA-3, MiniGPT-v2, and Video-ChatGPT are sampled at temperatures 0.3, 0.2, and 0.2, respectively, yet all tables report single runs with no error bars. The reported advantages over zero-shot baselines (+4.8 R1@0.3 on Charades-STA and +5.8 R1@0.5 on QVHighlights val) are modest relative to the run-to-run variability expected from stochastic decoding. Please report means and standard deviations over at least three independent runs and state whether the improvements are statistically significant.","section":"Sec. 4.2, Tables 1-2"},{"comment":"The claim of outperforming the state-of-the-art zero-shot method on Charades-STA and ActivityNet-Captions is supported by comparison with only one zero-shot baseline (Luo et al., 2023a). Table 1 identifies Wattasseril et al. (2023) as the strongest zero-shot method on QVHighlights, but that baseline is not evaluated on the other two datasets. Please include all applicable zero-shot baselines on these datasets, or explicitly justify their omission, before claiming SOTA results.","section":"Sec. 4.3, Table 2"}],"minor_comments":[{"comment":"The abstract's claim of outperforming 'most supervised methods' is too broad: on Charades-STA (Table 2), the fully supervised Moment-DETR and TimeChat obtain higher R1@0.5 (48.2 and 43.8 vs 38.4). Please qualify the claim or specify the datasets and metrics for which it holds.","section":"Abstract and Sec. 4.3"},{"comment":"Table 9 labels the zero-shot baseline as 'Wattasseril (Li et al. 2022a)', but the method is Wattasseril et al. (2023); also, Luo et al. 2023a and 2023b appear as separate references for the same work. Please correct these citation inconsistencies.","section":"Table 9 and References"},{"comment":"The number of histogram bins eta is said to be 'empirically fixed to 10', but no ablation for eta is reported in Appendix B.3. Please add a sensitivity analysis for eta or state the basis for this choice.","section":"Sec. 4.2 and Appendix B.3"},{"comment":"The out-of-distribution tables (Tables 10-11) lack protocol details: please define OOD-1/OOD-2, clarify whether R@0.5 denotes R1@0.5 or recall, and specify the exact dataset splits used.","section":"Appendix B.2"},{"comment":"The caption says 'We red out all biased and debiased words'; this should be 'highlighted in red' or similar.","section":"Figure 8 caption"},{"comment":"The sentence 'We will soon introduce tiny Moment-GPT' with the unfinished citation 'cite with tree of thoughts' is not appropriate for an archival paper; remove or rewrite as a concrete future-work statement, and consider noting that the paper does not release code.","section":"Appendix B.1 and C.1"},{"comment":"The 'poisoning test' mentioned for the MLLMs is vague; please specify what was checked and how.","section":"Appendix C.3"}],"recommendation":"major_revision","confidential_remarks":"The paper would be substantially strengthened by releasing code, exact prompts, and generated debiased-query examples. As it stands, the central SOTA claim is not yet supported because of test-set hyperparameter selection and missing variance reporting; a major revision addressing these points is appropriate. The bibliographic and presentation issues listed in the minor comments are easily fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: Moment-GPT is a sensible training-free pipeline for zero-shot video moment retrieval, and the paper's evidence for each design choice is much better than the typical MLLM paper. The new ingredient is the LLaMA-3 query-rewriting step, which fixes rare words and grammar errors before matching, plus the combination of MiniGPT-v2 frame captions and Video-ChatGPT span captions scored in LLaMA-3 text space. The ablations in Sec 4.4 and Appendix B are honest, and the oracle-bound table (Table 17) is a nice check on how far the pipeline is from ideal. The authors also state the main limitation clearly in C.1: converting video to text before similarity loses information.\n\nThe real soft spot is the evaluation protocol, and the stress-test note is on target. The hyperparameters (eta=10, kappa=7, tau=5, lambda=0.2, sigma=0.9, Nd=3) are chosen by ablating on Charades-STA, the same test split used for the headline numbers. That is test-set tuning, and it is not disclosed. The pipeline uses stochastic MLLMs at nonzero temperature, yet no error bars or multiple seeds are reported. So the claimed +4.8 R1@0.3 over Luo on Charades-STA and the SOTA zero-shot numbers on QVHighlights are not yet statistically grounded. This is not fatal—the gains are consistent across three datasets and the components all help—but it means the 'substantially outperforms' claim is conditional on a stricter evaluation.\n\nAlso minor: no code is released, which makes the stochastic pipeline harder to reproduce, and the comparison tables mix their own reproduced baselines with published numbers without always saying which.\n\nWho is this for: researchers working on zero-shot video grounding and on using off-the-shelf MLLMs as modules. They will get a clear, ablated pipeline and a useful reference point. It deserves a serious referee, but the referee should require the authors to fix hyperparameters on a validation split or report a grid over them, add error bars, release code, and soften the SOTA claim to 'competitive' until then.","headline":"A solid, well-ablated zero-shot VMR pipeline whose headline margins need re-validation once the test-set hyperparameter tuning and missing error bars are fixed.","tokens_in":17844,"tokens_out":2450,"would_cite":false,"duration_ms":25449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Moment-GPT is a tuning-free pipeline of frozen multimodal large language models that outperforms state-of-the-art zero-shot and most supervised video moment retrieval methods on three benchmarks.","keywords":["video moment retrieval","zero-shot learning","multimodal large language models","query debiasing","temporal localization","LLaMA-3","MiniGPT-v2","Video-ChatGPT"],"falsifier":"Replace the two captioning models with human-written dense captions for the same frames and candidate spans on the Charades-STA test set; if retrieval accuracy does not move substantially toward the paper's own oracle bound (R1@0.5 = 68.5 vs 38.4), then caption fidelity is not the main bottleneck and the explanation for the method's success would need to change.","tokens_in":16835,"feed_emoji":"🎯","tokens_out":10741,"duration_ms":91285,"temperature":0.7,"pith_summary":"This paper sets out to show that video moment retrieval — finding the segment of a video that answers a sentence query — can be solved without training on any retrieval data. Its proposed pipeline, Moment-GPT, chains three frozen off-the-shelf models: a large language model (LLaMA-3) corrects and rewrites the query to remove spelling, grammar, and rare-word biases; an image-language model (MiniGPT-v2) writes a caption for each frame; and a video-language model (Video-ChatGPT) writes a caption for each candidate segment. A text-only scorer then compares these captions with the rewritten queries and picks the best-matching spans. Across QVHighlights, Charades-STA, and ActivityNet-Captions, the paper reports results above previous zero-shot and MLLM-based methods and above most fully supervised baselines, which would make strong temporal localization available without costly annotated video datasets.","feed_headline":"Frozen multimodal LLM pipeline tops video moment retrieval","feed_subtitle":"Rewriting biased queries and scoring frame and span captions beats most supervised methods on three benchmarks.","key_machinery":"The engine of Moment-GPT is a caption-then-match loop: every visual unit is first converted into text by a frozen multimodal model, and relevance is computed as cosine similarity in the embedding space of LLaMA-3 between the debiased query and that text. The span generator, an adaptive-threshold rule, builds candidate segments from an inverse cumulative histogram of frame-query similarity scores: a segment starts when similarity crosses a threshold and ends after $\\tau$ consecutive low-scoring frames, with $\\eta$, $\\kappa$, and $\\tau$ controlling coarseness. A span-distance bonus and non-maximum suppression then refine the final selection.","core_discovery":"On its own terms, the central claim is that a tuning-free pipeline of frozen MLLMs forms a complete and accurate solution to zero-shot VMR: generate debiased query variants with LLaMA-3, derive frame-level captions with MiniGPT-v2, use LLaMA-3 text features to score each frame and adaptively build candidate spans, caption those spans with Video-ChatGPT, and finally score spans by text similarity with a distance bonus and non-maximum suppression. The paper reports that this pipeline substantially outperforms state-of-the-art zero-shot and MLLM-based methods and most fully supervised methods across QVHighlights, Charades-STA, and ActivityNet-Captions, and attributes the gain to the query-debiasing step and to using video-level MLLM captioning rather than frame-level or multimodal similarity for span selection.","pith_inferences":["The reported oracle-bound gap (R1@0.5 = 68.5 vs 38.4 on Charades-STA) suggests the current pipeline is far from its own ceiling; better candidate generation or scoring could yield large gains without changing the frozen-model strategy.","The caption-then-match recipe could transfer to other retrieval tasks such as text-video retrieval, temporal action localization, and video question answering, wherever an LLM can normalize the query and an MLLM can verbalize the visual content.","Because query debiasing is applied only at inference time, it could be grafted onto any existing VMR system, supervised or not, and should be stress-tested on noisy user-generated queries rather than benchmark annotations.","The paper's own limitation note points toward a single MLLM that jointly embeds video and text in one semantic space; such a model, if open-sourced, would likely outperform the current caption-and-match pipeline by removing the caption-fidelity bottleneck."],"forward_implications":["Zero-shot VMR can match or exceed fully supervised performance on QVHighlights, Charades-STA, and ActivityNet-Captions without any VMR-specific training data.","LLM-based query debiasing is a measurable accuracy component, especially on queries with rare words, spelling errors, and grammar errors, and it also improves video highlight detection.","Frozen MLLMs can be composed as a multi-agent pipeline, with each model doing the task it was trained for, avoiding the multi-stage fine-tuning used by prior MLLM-based VMR methods.","Because the scorers operate entirely on text, improving frame- and span-level caption quality should directly improve retrieval accuracy, making caption fidelity the key controllable variable.","The same inference-time pipeline transfers to video highlight detection and out-of-distribution settings, reporting state-of-the-art results without retraining."],"supporting_citations":[{"why":"Supplies LLaMA-3, used both to debias and rewrite queries and to extract the text features for frame and span scoring.","marker":"AI@Meta 2024"},{"why":"Supplies MiniGPT-v2, the frozen model that converts each frame into a detailed caption for the frame scorer.","marker":"Chen et al. 2023"},{"why":"Supplies Video-ChatGPT, the frozen model that captions each candidate span before the span scorer.","marker":"Maaz et al. 2023"},{"why":"Earlier zero-shot VMR method from frozen vision-language models; provides the baseline whose language-bias failure motivates query debiasing.","marker":"Luo et al. 2023a"},{"why":"The previous state-of-the-art zero-shot method that Moment-GPT compares against and surpasses on QVHighlights.","marker":"Wattasseril et al. 2023"},{"why":"Introduces the QVHighlights dataset and the Moment-DETR supervised baseline used in comparisons.","marker":"Lei et al. 2021"},{"why":"Contributes the Charades-STA dataset and query-span pairs used in the main experiments and ablations.","marker":"Gao et al. 2017"},{"why":"Provides the ActivityNet-Captions dataset used for the third benchmark.","marker":"Krishna et al. 2017"},{"why":"Zero-shot VMR approach based on shot detection and off-the-shelf models; a comparison baseline and the origin of the oracle-bound evaluation.","marker":"Diwan et al. 2023"}],"fun_headline_variants":["Frozen MLLM pipeline tops zero-shot video moment retrieval","Zero-shot video moment retrieval via frozen LLMs outperforms","Query rephrasing with LLMs boosts zero-shot moment retrieval","Off-the-shelf MLLMs beat supervised video moment retrieval","Tuning-free frozen MLLMs outperform on moment retrieval"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the written descriptions produced for each frame and for each candidate clip keep enough of what is visible that matching the rewritten query against those descriptions by text similarity can tell apart the right moments from the wrong ones.","fun_headline_variants_meta":{"raw":{"variants":["Frozen MLLM pipeline tops zero-shot video moment retrieval","Zero-shot video moment retrieval via frozen LLMs outperforms","Query rephrasing with LLMs boosts zero-shot moment retrieval","Off-the-shelf MLLMs beat supervised video moment retrieval","Tuning-free frozen MLLMs outperform on moment retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00076,"raw_usage":{"total_tokens":3356,"prompt_tokens":908,"completion_tokens":2448,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":2363}},"tokens_in":524,"tokens_out":2448,"duration_ms":17246,"temperature":1.0,"reasoning_tokens":2363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:29:05.815043+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the two captioning models with human-written dense captions for the same frames and candidate spans on the Charades-STA test set; if retrieval accuracy does not move substantially toward the paper's own oracle bound (R1@0.5 = 68.5 vs 38.4), then caption fidelity is not the main bottleneck and the explanation for the method's success would need to change.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies MiniGPT-v2, the frozen model that converts each frame into a detailed caption for the frame scorer."},{"cited_title":"I.; Shekhar, S.; Döllner, J.; and Trapp, M","cited_arxiv_id":null,"evidence_quote":"The previous state-of-the-art zero-shot method that Moment-GPT compares against and surpasses on QVHighlights."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the QVHighlights dataset and the Moment-DETR supervised baseline used in comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the Charades-STA dataset and query-span pairs used in the main experiments and ablations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ActivityNet-Captions dataset used for the third benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Zero-shot VMR approach based on shot detection and off-the-shelf models; a comparison baseline and the origin of the oracle-bound evaluation."}],"review_version":1}