{"id":"b8f3ebb5-1a04-4cae-857b-253e78214aa3","arxiv_id":"2608.08315","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A zero-training method that turns temporal grounding into a coarse-to-fine scan of binary clip questions beats direct timestamp prediction by 28 to 50 R@0.5 points on four frozen VLMs.","lead":"Instead of asking a video-language model to output start and end times, the authors ask it many yes/no questions about short clips and rank the answers. Across four frozen models this raises temporal grounding accuracy by 28 to 50 points, and the paper argues the failure was in the question format, not in what the model sees.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 6's 'identical input' claim is false: the scan sees 36 frames vs 16, and the paper's own F=3-to-5 ablation shows this evidence gap matters, so 'interface, not perception' is not yet controlled.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the two arms of Table 6 do not show the model comparable input evidence, and the paper's own ablation demonstrates that per-probe frame count is a material variable. This is the critical point because the paper's headline claim is causal: the failure is in the interface, not in perception. A causal claim requires holding the evidence fixed; the current design does not, and the F=3/F=5 inconsistency undermines confidence in the reported numbers. I agree with the reader that the central conclusion is plausible and likely correct in spirit—binary VQA ranking does seem to unlock localization ability in frozen models, and the paper provides a rich set of ablations and negative results that support the two-axis decomposition. However, the same-input control is the logical load-bearing pillar, and it is currently missing. The proposed test directly settles whether the 28–50 pp gap is attributable to the interface or confounded by a 2.25x difference in frame count and per-probe temporal context. I also note secondary concerns (test-set-swept W on long-video corpora, and the absence of code or data release), but these do not bear as directly on the causal claim as the input mismatch. Since the paper can likely address the concern with an additional controlled experiment, the appropriate verdict remains CONDITIONAL rather than ACCEPT or REJECT: conditional on a matched-input comparison, or on an explicit statement that the comparison is not controlled and the interface gain is an upper bound that includes richer evidence. The verdict is therefore unchanged from the reader's CONDITIONAL.","tokens_in":25781,"tokens_out":5667,"duration_ms":54041,"concrete_test":"Run a matched-evidence crossover on Charades-STA with Qwen2.5-VL: (a) feed the direct timestamp baseline the exact same 36 frames (12 clip-centred triples) that FV-Action consumes, presented through the native video interface; and (b) run FV-Action on exactly 16 single frames at the 16 regression locations. If the regression R@0.5 rises from 28.4 toward the scan's 56.8, or if the 16-frame scan collapses toward the regression baseline, the interface gap is partly an input-evidence artifact. If both variants move by less than 5 pp, the interface claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the failure 'lives in the interface, not in perception' rests on the controlled comparison in Table 6, whose caption states that both arms use 'identical input (16 uniformly sampled frames).' That statement is not true as implemented. The regression arm makes one forward pass over 16 frames at 16 temporal locations, while the FV-Action arm makes 12 coarse plus 24 fine forward passes, each reading 3 frames per probe—36 frames on the coarse stage alone (Sec. 4.1). The paper acknowledges the asymmetry and tries to dismiss it by redefining 'temporal resolution' as the number of distinct locations (12 vs 16), but Sec. 5.6 shows that per-probe evidence is not merely a coverage issue: increasing F from 3 to 5 frames while widening the probe extent improves R@0.5 by 5.7 pp on the full Charades-STA test set. Thus the extra frames do change what the model can judge, precisely at the point where the interface comparison is made. Because the scan arm receives 2.25 times as many frames and multiple local temporal contexts, Table 6's 28–50 pp gap confounds the output interface with the input evidence. The internal inconsistency—Sec. 3.3 specifies F=5, while Sec. 4.1 and all reported results use F=3, with F=5 shown to be materially better—makes the reported numbers ambiguous. Until a matched-input comparison is run, 'interface, not perception' is an overstatement; the direction of the effect may be right, but its magnitude and clean attribution are not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that multimodal LLMs' poor temporal grounding under direct timestamp prompting is a failure of the task interface, not of perception. It supports this with a diagnostic study showing that TVG-tuned models are confidently wrong on timestamp regression, an oracle discrimination test showing that frozen VLMs can separate ground-truth clips from negatives via binary VQA, and a training-free method (FV-Action) that scans a frozen VLM with yes/no questions and ranks the answers. On Charades-STA, FV-Action reaches 56.8% R@0.5 with Qwen2.5-VL-7B, outperforming the same backbone's native pipeline and prior training-free methods. The paper further decomposes residual failures into a perception axis (backbone-dependent clip discriminability) and a geometry axis (a fixed output window's analytically bounded overlap).","tokens_in":26056,"tokens_out":6060,"duration_ms":52218,"significance":"If the central claim holds, the paper would establish that frozen VLMs already contain substantial temporal grounding ability that is masked by the timestamp-output format, and that a ranking-based binary-VQA readout is a simple, training-free way to recover it. The paper has genuine strengths: the oracle discrimination test (AUROC 0.782 for InternVL2-8B) provides independent evidence for the perception side; the four-backbone consistency of the interface gains is impressive; and the geometric bound min(W,g)/max(W,g) is a clean, falsifiable prediction that correctly describes where fixed-window methods must fail. The extensive negative results and ablations are also transparent and useful. However, the central controlled comparison is currently not controlled: the two arms of Table 6 receive different input evidence, and the paper itself shows that this evidence difference changes accuracy substantially. The claim 'interface, not perception' is therefore an overstatement of what the present experiments establish, even though the direction of the effect is plausible.","major_comments":[{"comment":"The 'identical input' control is not identical. The regression arm makes one forward pass over 16 frames at 16 temporal locations, while the VQA arm runs the full FV-Action pipeline with Kc=12 coarse probes and Np×Kf=24 fine probes, each reading 3 frames, for at least 108 total frames and 36 distinct probe centers on Charades-STA. The caption's claim that both rows 'use identical input (16 uniformly sampled frames)' is therefore inaccurate. Section 5.6 shows this is not a cosmetic asymmetry: increasing frames per probe from 3 to 5 with a wider extent improves R@0.5 by 5.7pp on the full test set, and the paper attributes this to 'the evidence each probe sees.' Because the VQA arm receives richer per-probe evidence and more distinct locations, the 28–50pp gap in Table 6 confounds the output interface with the input evidence. The direction of the effect may be right, but the magnitude and the 'interface, not perception' attribution are not established until a matched-input comparison is run, e.g., presenting the same 16 frames to both arms or using the same probe structure with identical frame sets.","section":"§4.1, Table 6, §5.6"},{"comment":"The paper is internally inconsistent about F, the frames per probe. Section 3.3 states that 'Each clip is represented by F=5 frames' and that this setting 'sit[s] at a measured optimum,' while Section 4.1's implementation details and all reported results use 'F=3 frames are sampled around the clip centre.' Section 5.6 then reports that raising F to 5 while widening the probe extent improves R@0.5 from 56.7 to 62.3 (+5.7pp) on the full Charades-STA test set. The deployed configuration is therefore not the described optimum, the headline numbers are for a suboptimal variant, and readers cannot tell which configuration produced Table 6. This inconsistency must be resolved, and the main results should be re-reported under the configuration described in the method section.","section":"§3.3 vs §4.1 and §5.6"},{"comment":"The width W is a test-set-tuned hyperparameter on ActivityNet and TACoS, which undercuts the 'no tuning' and 'training-free' framing. The paper states that on Charades-STA and QVHighlights W equals the training-split mean event length, but for ActivityNet and TACoS the deployed values (80s and 20s) are not the training means (35.5s and 32s); they are chosen because the sensitivity sweep in Section 5.6 shows they lie on a plateau, with the training-mean alternative losing 3.1pp and about 1.7pp respectively. That is selection on the eval split. The geometric analysis in Figures 3–5 is consequently a post-hoc explanation of a configuration chosen on the test set, rather than a prediction made before any model runs. The r=0.998 in Figure 5 is also partly tautological, since the 'peak ceiling' is computed from the same actual peak positions that determine final R@0.5. Please rephrase the claims of prediction and add a validation-based selection procedure or justify the test-set choice explicitly as a reported design decision.","section":"§3.5, Table 13, §4.1"}],"minor_comments":[{"comment":"The captions of Figure 1 and Table 6 both say 'same input frames' or 'identical input'; these should be corrected to describe the actual inputs after addressing the corresponding major comment.","section":"Fig. 1, Table 6"},{"comment":"The model name is spelled inconsistently as 'LLaV A-NeXT-Video' in several places; the standard form is LLaVA-NeXT-Video.","section":"Throughout"},{"comment":"The first observation begins with 'An earlier snapshot of our results appeared to show...'; this historical anecdote should be removed in favor of the controlled comparison under identical window policy that follows.","section":"§5.3"},{"comment":"The term 'peak ceiling' is used in the figures before it is formally defined in the text; please define it in the caption or in the text before first use.","section":"Figures 4–5"},{"comment":"The statement that the two arms cover the timeline 'at a similar granularity' ignores the 24 fine-scan probes; please state the full probe and frame counts for the VQA arm so that the comparison is transparent.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The core empirical phenomenon—frozen VLMs localize far better through binary VQA ranking than through timestamp regression—is plausible and well supported by the oracle tests and the four-backbone consistency. The main barrier is the uncontrolled comparison in Table 6 and the F inconsistency between §3.3 and §4.1/§5.6. If the authors can run a genuinely matched-input control and re-report the results, the paper would be a strong contribution. I also recommend the editor ask for the test-set selection of W to be explicitly justified in the 'no tuning' discussion, or moved to a validation split."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [name],\n\nThis paper has a real result buried in an overstated headline. The core finding—frozen VLMs localize events far better when asked binary clip questions than when asked for timestamps—is plausible and mostly supported by the four-backbone comparison. But the claim that the failure 'lives in the interface, not in perception' rests on a control that is not actually controlled.\n\nWhat's genuinely new: a POPE-style binary interrogation turned into a temporal localizer, a four-backbone controlled comparison showing consistent gains, and an oracle-gap diagnostic that separates backbone discriminability from the output form. The paper is also unusually honest. It reports a long list of negative results, an explicit Limitations section, and an ablation (F=5 vs F=3) that directly undercuts its own 'identical input' claim. That kind of transparency is rare and should be credited.\n\nThe soft spots are real. Table 6 says both arms use 'identical input (16 uniformly sampled frames).' They don't. The regression arm makes one forward pass over 16 frames; the scan arm makes 12 coarse plus 24 fine passes, each reading 3 frames, so 36 frames on the coarse stage alone. The paper tries to redefine temporal resolution as number of distinct locations, but its own frames-per-probe ablation shows per-probe evidence matters: going from 3 to 5 frames per probe adds 5.7 points R@0.5 on Charades-STA. That means part of the 28–50 pp gap could be input evidence, not interface. There is also an internal contradiction: Section 3.3 specifies F=5, Section 4.1 says F=3, and all reported numbers use F=3. This needs to be fixed.\n\nTwo other issues are real but less damaging. The window width W on ActivityNet and TACoS is selected by sweeping on the test sets, which chips away at the 'no tuning' claim, even if the plateau argument helps. And the geometry axis (Eq. 5) is a definitional identity, not a prediction; the r=0.998 correlation in Fig. 5 uses measured peak positions, so it is descriptive, not predictive.\n\nOverall, the direction of the effect is probably right—binary scanning does help—but the magnitude and the clean attribution are not established. The paper deserves a serious referee, but it needs a matched-input control and a consistent frame count before the strongest claims can stand.\n\nMy recommendation: send it to peer review, and ask for those two things. It's worth reading for anyone in temporal grounding or VLM interfaces, even if the headline needs a qualifier.","headline":"A real and interesting training-free temporal grounding method, but the headline 'interface, not perception' claim is undercut by a mismatched input control.","tokens_in":26656,"tokens_out":3375,"would_cite":true,"duration_ms":28415,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frozen video-language models already know when events happen — the failure is the question format, not perception.","keywords":["temporal video grounding","training-free","binary VQA","frozen vision-language models","coarse-to-fine scan","timestamp regression","format hallucination","recall at IoU"],"falsifier":"Run the direct timestamp prompt on the identical 36-frame input the scan uses (12 clip-centred 3-frame probes presented as one video). If R@0.5 jumps toward the 53–57% range, the interface is not the bottleneck; if it stays near 3–28%, the paper's reading is supported. Alternatively, hold the scan to 16 frames total (one frame per temporal location, no shared clips) and check whether the +28 to +50 pp gain survives.","tokens_in":25428,"feed_emoji":"🎯","tokens_out":4952,"duration_ms":40516,"temperature":0.7,"pith_summary":"The paper tries to establish that multimodal LLMs' failure at temporal video grounding is a failure of the task interface, not of perception. Frozen video-language models that emit timestamps at near-zero accuracy can, when asked a binary yes-or-no question about a short clip, rank the correct clip above distractors most of the time. Replacing timestamp regression with a coarse-to-fine scan of such questions raises R@0.5 by 28 to 50 points across four frozen backbones, reaching 56.8% on Charades-STA without any temporal supervision. A sympathetic reader would care because the result says temporal grounding ability is already latent in off-the-shelf models and can be extracted by changing only how we ask. The paper further decomposes the residual errors into a backbone-dependent perception axis and a window-geometry axis that is analytically predictable.","feed_headline":"Ask yes or no, not timestamps: frozen VLMs find events","feed_subtitle":"Swapping timestamp regression for a binary-question scan lifts R@0.5 by 28–50 points with zero training.","key_machinery":"The central object is the interface swap: replace generative timestamp regression with a discriminative coarse-to-fine scan of binary \"Does this clip show {query}? Answer Yes or No\" questions, consuming only first-token yes/no probabilities as a within-video ranking. The load-bearing identity is the geometry bound IoU = min(W,g)/max(W,g), which predicts that a fixed window of width W cannot meet R@0.5 for events with ground-truth width g outside [W/2,2W], regardless of how well the peak is localized. The scan is carried by a pluggable frozen measurement layer (four backbones), an NMS peak-selection step with joint coarse/fine geometric-mean scoring, and a fixed-width window with a per-corpus width setting. Ranking rather than calibration is what the models satisfy, and the geometry bound explains residual failures before any model runs, organizing the two-axis analysis.","core_discovery":"The central claim is that the same frozen weights that regress timestamps at single-digit recall can separately discriminate event clips under binary VQA, so the bottleneck is the generative timestamp format rather than the model's knowledge. Decimal timestamps conditioned on a language query are largely absent from pre-training corpora, so the model falls back on language priors and produces plausible but ungrounded values with high confidence; binary VQA sidesteps the missing format, and consuming only the ranking of answers across clips is a weaker requirement than calibration. FV-Action operationalizes this by running a uniform coarse scan of binary questions, keeping the top candidate peaks under non-maximum suppression, re-examining each with a fine scan, combining the two resolutions with a geometric mean, and emitting a fixed-width window centred on the winning peak. On Charades-STA it reaches 56.8% R@0.5, above the same backbone's native grounding pipeline (53.6%) and above every prior training-free method, while improving over direct prediction on ActivityNet Captions and QVHighlights. The residual failures are decomposed into a perception axis that moves with the backbone and a geometry axis that is analytically predictable from the ratio of the output-window and event widths.","pith_inferences":["The paper's frame-per-probe ablation (raising from 3 to 5 frames gains 5.7 pp R@0.5) implies part of the interface gain may be conflated with richer per-probe evidence; a strictly controlled regression arm using the same 12 clip-centred probes as the scan, prompted for timestamps, would separate the pure interface effect from the evidence effect.","If \"interface, not perception\" generalizes, then any structured-output task where VLMs underperform — bounding-box regression, duration estimation, count prediction — could be re-expressed as a scan of binary questions, provided the model can rank candidate answers.","The geometry bound suggests that a query-conditioned window width, which the paper names as its own future direction, could recover long-tail events on ActivityNet and QVHighlights; a testable extension is to predict W from the query text's action or object words.","The oracle-gap diagnostic could be standardized as a cheap model-selection probe: run a few hundred binary questions on a small video sample and predict which frozen backbone will deploy better on a new corpus, without running the full localization pipeline."],"forward_implications":["Same backbone, same frozen weights: switching from timestamp regression to binary VQA scanning yields +28 to +50 pp R@0.5 on Charades-STA across InternVL2-8B, Qwen2.5-VL-7B, LLaVA-NeXT-Video-7B, and mPLUG-Owl3-7B.","FV-Action reaches 56.8% R@0.5 on Charades-STA with no temporal supervision, surpassing the same backbone's native grounding pipeline (53.6%) and the best prior training-free result (40.0%).","Confident failures are systematic, not epistemic: 77–80% of wrong timestamp predictions carry low output entropy, and entropy-based error detection stays below a random classifier (F1<0.36), so the remedy is not better uncertainty handling but a different output format.","Residual accuracy is governed by two separable axes: backbone per-clip discriminability (measurable by an oracle-gap diagnostic before deployment) and window-event geometry (predictable from the ratio alone).","The scan inherits future backbone improvements: any gain in frozen-VLM per-clip discrimination converts directly into grounding accuracy with no retraining."],"supporting_citations":[{"why":"Defines the Charades-STA benchmark and the temporal grounding task that FV-Action is evaluated on.","marker":"[7]"},{"why":"Supplies the same-backbone native temporal pipeline baseline (Qwen2.5-VL at 53.6% R@0.5) that FV-Action surpasses, and the large-scale supervised comparison point.","marker":"[33]"},{"why":"Provides the Qwen2.5-VL-7B frozen backbone used for the main results and the interface comparison.","marker":"[1]"},{"why":"Provides the InternVL2-8B frozen backbone, whose 3.76% to 53.39% jump under the VQA interface is the sharpest evidence for the interface bottleneck.","marker":"[4]"},{"why":"One of the TVG-tuned models whose confidently wrong low-entropy predictions are analyzed in the diagnostic study.","marker":"[11]"},{"why":"Another TVG-tuned model used in the entropy-based error-detection analysis and as a supervised baseline.","marker":"[24]"},{"why":"Third TVG-tuned model in the entropy analysis, contributing to the claim that 77–80% of wrong predictions carry low entropy.","marker":"[9]"},{"why":"POPE-style binary interrogation is the precedent that the scan's yes/no question format builds on.","marker":"[17]"},{"why":"Defines the ActivityNet Captions benchmark, where the geometry axis is demonstrated and where FV-Action improves over direct prediction.","marker":"[13]"},{"why":"Defines the TACoS benchmark, the out-of-domain cooking corpus where FV-Action surpasses every compared TVG-trained model zero-shot.","marker":"[23]"}],"fun_headline_variants":["Your VLM knows when: ask yes or no, not timestamps","Frozen VLM grounds time with binary questions, no training","Yes/no beats timestamps for training-free temporal grounding","VLM confidently wrong about timestamps? Try binary VQA","No training needed: VLM finds events via yes/no scan"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the gain comes from the interface assumes both arms give the model comparable evidence; the regression arm sees one 16-frame video while the scan sees 12 clip-centred probes of 3 frames each (36 frames total), and the paper's own ablation shows extra frames per probe improve accuracy — so if richer evidence contributes to the gain, the pure interface effect is smaller than claimed.","fun_headline_variants_meta":{"raw":{"variants":["Your VLM knows when: ask yes or no, not timestamps","Frozen VLM grounds time with binary questions, no training","Yes/no beats timestamps for training-free temporal grounding","VLM confidently wrong about timestamps? Try binary VQA","No training needed: VLM finds events via yes/no scan"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000705,"raw_usage":{"total_tokens":3232,"prompt_tokens":1054,"completion_tokens":2178,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":670,"completion_tokens_details":{"reasoning_tokens":2106}},"tokens_in":670,"tokens_out":2178,"duration_ms":16467,"temperature":1.0,"reasoning_tokens":2106,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:08:56.790874+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the direct timestamp prompt on the identical 36-frame input the scan uses (12 clip-centred 3-frame probes presented as one video). If R@0.5 jumps toward the 53–57% range, the interface is not the bottleneck; if it stays near 3–28%, the paper's reading is supported. Alternatively, hold the scan to 16 frames total (one frame per temporal location, no shared clips) and check whether the +28 to +50 pp gain survives.","supporting_citations":[{"cited_title":"Tall: Temporal activity localization via lan- guage query","cited_arxiv_id":null,"evidence_quote":"Defines the Charades-STA benchmark and the temporal grounding task that FV-Action is evaluated on."},{"cited_title":"Omnivtg: A large-scale dataset and training paradigm for open-world video tempo- ral grounding","cited_arxiv_id":null,"evidence_quote":"Supplies the same-backbone native temporal pipeline baseline (Qwen2.5-VL at 53.6% R@0.5) that FV-Action surpasses, and the large-scale supervised comparison point."},{"cited_title":"Qwen2.5- vl technical report, 2025","cited_arxiv_id":null,"evidence_quote":"Provides the Qwen2.5-VL-7B frozen backbone used for the main results and the interface comparison."},{"cited_title":"Internvl: Scal- ing up vision foundation models and aligning for generic visual-linguistic tasks","cited_arxiv_id":null,"evidence_quote":"Provides the InternVL2-8B frozen backbone, whose 3.76% to 53.39% jump under the VQA interface is the sharpest evidence for the interface bottleneck."},{"cited_title":"Lita: Language instructed temporal- localization assistant","cited_arxiv_id":null,"evidence_quote":"One of the TVG-tuned models whose confidently wrong low-entropy predictions are analyzed in the diagnostic study."},{"cited_title":"Timechat: A time-sensitive multimodal large language model for long video understanding","cited_arxiv_id":null,"evidence_quote":"Another TVG-tuned model used in the entropy-based error-detection analysis and as a supervised baseline."},{"cited_title":"Vtg-llm: Integrating timestamp knowledge into video llms for enhanced video tempo- ral grounding","cited_arxiv_id":null,"evidence_quote":"Third TVG-tuned model in the entropy analysis, contributing to the claim that 77–80% of wrong predictions carry low entropy."},{"cited_title":"Evaluating object hallucina- tion in large vision-language models","cited_arxiv_id":null,"evidence_quote":"POPE-style binary interrogation is the precedent that the scan's yes/no question format builds on."},{"cited_title":"Dense-captioning events in videos","cited_arxiv_id":null,"evidence_quote":"Defines the ActivityNet Captions benchmark, where the geometry axis is demonstrated and where FV-Action improves over direct prediction."},{"cited_title":"Grounding action descriptions in videos","cited_arxiv_id":null,"evidence_quote":"Defines the TACoS benchmark, the out-of-domain cooking corpus where FV-Action surpasses every compared TVG-trained model zero-shot."}],"review_version":1}