{"id":"3b1b1a06-6a78-4c49-9a28-428a248da8b0","arxiv_id":"2607.17994","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Highlight scores from an off-the-shelf detector become an attention bias that steers a frozen video-LLM toward important frames during summary generation, with modest benchmark gains and several evaluation gaps.","lead":"Video summarization usually deletes low-scoring frames before asking a model to write a summary. HAS instead keeps every frame and gently steers a frozen video-understanding model's attention toward the highlighted moments, reporting small gains in grounding and faithfulness on several benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's 'temporal salience' gain may be inherited from the undisclosed highlight prior H, not produced by the steering mechanism; no ablation separates input prior from steered output.","rationale":"The reader's weakest_assumption focuses on the reliability of the undisclosed highlight generator H. My concern is adjacent but distinct: even if H is reasonably reliable, Table 1 may not measure the steering mechanism at all. The paper's Eq. (3) and Algorithm 2 describe a plausible intervention, and Tables 2–3 provide some (if small) evidence for cross-modal and grounding improvements, so this does not justify rejection. But the temporal-salience claim — one of the paper's three advertised gains — rests on an undefined trajectory that is likely just the calibrated input prior. The paper explicitly concedes in §7 that 'the quality of the steering signal still depends on the reliability of the initial highlight estimate,' which is an honest limitation, but it does not address the attribution problem: which component produces the Table 1 correlation. This makes the current CONDITIONAL verdict appropriate: the central mechanism may be sound, but the submitted evidence does not yet demonstrate the temporal-salience component. A single well-posed ablation (shuffled steering vector or prior-only baseline) would settle whether Table 1 supports the claim or simply re-reports H's quality.","tokens_in":14232,"tokens_out":4037,"duration_ms":43006,"concrete_test":"Require the authors to specify exactly how the 'predicted importance trajectory' in Table 1 is computed, then rerun Table 1 with V replaced by a shuffled/random permutation of h while keeping the tuned head set, gates, and strengths fixed. If τ/ρ remain essentially unchanged, the table measures the input prior H, not the steering. Also add a 'prior-only' baseline that directly reports correlation of calibrated h against human annotations; if it matches HAS's numbers, the temporal-salience claim collapses to a property of H.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim includes 'improving temporal salience agreement' as one of three headline gains. The only evidence for this is Table 1, whose caption reports rank correlation between 'predicted importance trajectories' and human annotations. But the paper never defines what HAS's predicted importance trajectory is. Algorithm 1 returns only V = log(Repeat(h, P) + ε), with h derived directly from the off-the-shelf generator H(F, q) via interpolation and min-max normalization. If the Table 1 trajectory is this h (or a close transform of it), then Table 1 simply evaluates H, not the proposed attention-steering mechanism, and the comparison against V2Xum-LLaMA / LLMVS says nothing about whether steering improves salience ordering. Alternatively, if the trajectory is the post-steering attention mass, Fig. 4(B) states that this attention mass 'closely follows v,' which is a deterministic function of h; any correlation with human annotations would again be inherited from H. No ablation varies the prior's quality (e.g., random, shuffled, or uniform V), and H is never named or evaluated standalone. Thus the paper cannot rule out that the steering stage contributes nothing to Table 1, and one of the three central claims is currently unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HAS, an inference-time attention-steering method for multimodal LLM video summarization. Given a video and a query, an off-the-shelf highlight generator produces a raw temporal salience curve; HAS calibrates this curve into a smooth frame-level highlight distribution, lifts it into a token-level log-space steering vector V = log(Repeat(h,P)+ε), and injects a gated additive bias into selected cross-attention heads of a frozen video MLLM. The head-wise gates and steering strengths are chosen by minimizing teacher-forcing summary NLL on a held-out validation set, with a sparsity regularizer. The paper reports gains on SumMe/TVSum, VideoXum, VISTA, and MR.HiSum, and claims that HAS improves temporal salience agreement, cross-modal consistency, and grounded long-form summarization without updating the backbone.","tokens_in":14360,"tokens_out":7204,"duration_ms":64497,"significance":"The proposed mechanism is clean and non-circular: the highlight prior comes from an external generator, and the steering policy in Eq. (4) is optimized on validation summary NLL rather than on the benchmark metrics. If the claims hold, the contribution is a useful plug-and-play, backbone-agnostic inference-time module with broad benchmark coverage across several open-source video MLLMs. The derivation of Eqs. (2)–(4) is explicit and the method is falsifiable in principle. However, the evidence for the temporal-salience claim is currently confounded by the external prior, and the external highlight generator is never named or evaluated. These issues are central to the paper's main claims and need to be addressed before the results can be accepted.","major_comments":[{"comment":"The caption reports rank correlation between \"predicted importance trajectories\" and human annotations, but the manuscript never defines what HAS's predicted importance trajectory is. Algorithm 1 returns only V, a deterministic log/Repeat transform of the calibrated highlight prior h; Fig. 4(B) further states that post-steering attention mass \"closely follows v.\" Thus, if the curve scored is V or the attention mass, the reported τ/ρ largely measures the external generator H, not the steering operation in Eq. (3). If it is something else, it is not defined. No ablation (e.g., random, uniform, or shuffled V, or H alone) isolates the steering contribution, so the §4.2 claim that \"inference-time steering … improves global salience ordering\" is unsupported. Please specify the evaluated trajectory, report H-only, and add prior-quality ablations.","section":"§4.2, Table 1"},{"comment":"H(F,q) is load-bearing but never named, specified, or evaluated on the evaluation datasets. Since V is built directly from H, the injected bias in Eq. (3) adds a scaled version of H's output to attention logits at every decoding step, and §7 concedes that steering quality depends on the reliability of the initial highlight estimate. The paper should name the generator, report its standalone accuracy, and explicitly measure how HAS's metrics vary when H is replaced by corrupted or random priors. Without this, it is impossible to attribute observed gains to the attention-steering policy rather than to the external prior.","section":"§3.3, Algorithm 1"},{"comment":"The steering policy {a,β} is fit by minimizing teacher-forcing NLL of reference summaries on a \"held-out calibration/validation set,\" but the experimental section does not state the split, the dataset, or whether one policy is reused across benchmark families. This matters because the method is advertised as an inference-time plug-in; per-dataset validation-set tuning would weaken the zero-shot/generalization claims. Table 4 also says models are \"trained on SumMe\" even though HAS is a training-free intervention; the exact transfer protocol for HAS must be clarified.","section":"§3.5, Eq. (4) and §4.1"},{"comment":"The \"hard selection baseline\" is not identified, the budget T schedule is not defined, and the computation of fact-unit recall is not described. The central coverage claim—that soft steering defers saturation relative to discrete extraction—cannot be checked without this protocol. Please specify the baseline(s), budget grid, and metric definitions.","section":"§4.3, Fig. 2"}],"minor_comments":[{"comment":"There are several language issues: \"lost important information\" should be \"lose important information,\" \"continual attention guidance\" should be \"continuous attention guidance,\" and \"peace time steps\" should be \"non-highlight time steps.\"","section":"Abstract / Fig. 1 caption"},{"comment":"τ and ρ are reported as identical to two decimals for every method (e.g., 0.440/0.440 for LLMVS and 0.45/0.45 for HAS). This is suspicious for rank-correlation measures on the same data; please verify the underlying rankings and report standard deviations or significance tests.","section":"Table 4"},{"comment":"\"HAS (ours, averaged on selected backbones)\" is vague. Specify which backbones are included in the average and why those are selected; otherwise the comparison to individual zero-shot and fine-tuned baselines is hard to interpret.","section":"Table 3"},{"comment":"Only an \"averaged\" ΔFactVC is shown without per-backbone numerical values. Include a small table with the individual gains for each backbone and the number of runs used for averaging.","section":"Fig. 3"},{"comment":"The sentence \"Recent multimodal large language models (M-LLMs) [48]\" cites a deep-learning video summarization review, not a reference on MLLMs. The citation appears mismatched and should be corrected.","section":"§1, reference [48]"}],"recommendation":"major_revision","confidential_remarks":"The main obstacle is the Table 1 interpretation and the undisclosed highlight generator. I would ask for an ablation that separates H from the steering policy, a precise definition of the evaluated trajectory, and full disclosure of H. The other claims are more straightforward to verify once these issues are resolved. The paper is not ready for acceptance in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"HAS takes an off-the-shelf query-conditioned highlight detector, smooths and normalizes its frame-level score curve, lifts it to log space over visual tokens, and adds it as a bias to a few cross-attention heads of a frozen video MLLM during summary generation. That specific combination is not in the prior attention-steering literature I know, and the mechanism is simple, coherent, and genuinely inference-time. The math in Eqs. (2)–(4) checks out; the steering policy is fit to validation NLL, not to the benchmark metrics, so no circularity in the derivation. The authors also concede the obvious limitation in §7: the whole thing inherits the quality of the initial highlight estimate.\n\nWhat the paper does well: it targets a real gap. Existing video summarization pipelines mostly do score-then-select, and using a continuous highlight prior as a soft attention bias is a sensible alternative. The benchmark coverage is broad, they include several frozen backbones, and the zero-shot transfer is the right kind of stress test for a plug-and-play module.\n\nThe soft spots are in the evaluation, not the mechanism. Table 1 is the main problem. The paper never defines what HAS's “predicted importance trajectory” is. Algorithm 1 and Fig. 4(B) show the steered attention mass closely follows v, which is a deterministic transform of the highlight generator’s output h. So it looks like Table 1 is largely re-evaluating the highlight generator, not the attention steering. The authors need to define the trajectory, report H's stand-alone accuracy, and add ablations with random, uniform, or shuffled V. Without that, the temporal-salience claim is unsupported. Also, Table 4 has tau exactly equal to rho for every baseline to three decimals — implausible and needs correction. The headline numbers in Table 3 are “averaged on selected backbones” without per-backbone detail, and on SumMe rho HAS is below V2Xum-LLaMA while the text claims consistent gains. The budget protocol for Fig. 2 is also undefined.\n\nNone of this breaks the central idea, but the paper as submitted overstates its case. It deserves a serious referee, who should send it back with requests for the attribution and ablation fixes, the missing hyperparameters, and a concrete highlight generator. I’d bring it to a reading group to discuss exactly this kind of attribution error, but I wouldn’t cite it in its current form.","headline":"A lightweight steering idea worth a serious look, but the temporal-salience table as written appears to credit the highlight prior rather than the steering itself.","tokens_in":15049,"tokens_out":5600,"would_cite":false,"duration_ms":48100,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen video LLM can summarize better when guided by a continuous highlight curve instead of a discrete keyframe selection.","keywords":["video summarization","multimodal LLM","attention steering","inference-time control","highlight detection","temporal salience","factual grounding","long-form video"],"falsifier":"Run HAS with three steering vectors on the same videos: the real highlight prior, a uniform vector, and a scrambled vector. If uniform or scrambled vectors match the real-prior version on temporal salience and factual-consistency metrics, the reported gains come from the injection machinery rather than the highlight content; if an oracle human-annotation prior clearly outperforms the off-the-shelf prior, the unmeasured quality of H is the decisive factor.","tokens_in":13910,"feed_emoji":"🎬","tokens_out":4314,"duration_ms":35615,"temperature":0.7,"pith_summary":"This paper tries to establish that video summarization by a multimodal LLM improves when the model is steered at inference time by a continuous frame-level highlight score, rather than by first selecting a few keyframes and summarizing only those. The authors argue that hard selection discards context and breaks coherence, while soft guidance keeps the full video in front of the model and simply biases its cross-attention toward relevant moments. If correct, open-source video MLLMs can be made more faithful and coherent summarizers without any weight updates, just by adding a small attention bias derived from an existing highlight detector. The paper reports gains in temporal importance ranking, cross-modal consistency, and factual grounding on long-form scientific talks.","feed_headline":"Highlight curves beat keyframe picks for video summaries","feed_subtitle":"Steering attention with a highlight curve improves grounded, coherent summaries without retraining the video model.","key_machinery":"The central mechanism is the steering vector V, built from a query-conditioned highlight distribution h = [h_1, ..., h_T] produced by an off-the-shelf highlight generator, then temporally interpolated, min-max normalized to [0, 1], and lifted to token level by repeating each frame score over the P visual tokens of that frame. The log transform makes V usable as an additive bias on attention logits. During decoding, for each selected cross-attention head, the pre-softmax logits are modified row-wise as A_i,: + g_lm β_lm V, where g_lm = σ(a_lm) is a learned gate and β_lm a head-wise strength; only the gates and strengths are optimized on a validation set, while the MLLM stays frozen. This conv","core_discovery":"HAS claims that a frozen video MLLM M, guided by a steering vector V built from a calibrated highlight distribution h = H(F, q), produces better summaries O* = M_V(F, q) than score-then-select pipelines. The steering vector is constructed by lifting each frame's normalized highlight score to all P visual tokens extracted from that frame, V = log(Repeat(h, P) + ε), and adding it as a gated additive bias to the pre-softmax attention logits of a selected set of cross-attention heads during every decoding step. The intervention is soft: low-scored frames receive less attention, not zero, so the model retains dispersed evidence while concentrating on highlights. The authors present the method as","pith_inferences":["An implication the paper leaves implicit: the same steering recipe could be reused for any future video MLLM, so the long-term ceiling of the approach is set by the quality of the highlight prior, not by the steering injection itself.","A testable extension is to replace the single global highlight curve with per-section or per-query curves, steering different parts of the summary toward different evidence; the paper's own future-work section hints at a coarse-to-fine policy.","The method converts an explainability/saliency signal into a control signal, a pattern that could transfer to other long-context modalities where a global relevance curve is available, such as audio or document summarization.","Because the gating policy is tuned on reference summaries, the method may inherit the style of the validation set; zero-shot deployment to domains with very different summary conventions might require recalibrating the gate parameters."],"forward_implications":["Soft steering retains low-scored frames as weak attention targets, so summaries can keep using dispersed evidence that hard selection would irreversibly remove under a fixed frame budget.","Because the backbone is frozen and only a small gate/strength policy is learned, the method can be layered on any compatible open-source video MLLM without retraining.","The reported zero-shot transfer from one summarization benchmark to another suggests the steering mechanism generalizes across video distributions without target-domain adaptation.","Improvements in factual consistency and video–text alignment on long-form talks imply that attention-level control can reduce unsupported details in generated summaries."],"fun_headline_variants":["Steer video attention with highlight curves, not keyframes","Highlight curves guide attention for better video summaries","Attention steering outperforms keyframe selection in video summarization","Frozen MLLM summarizes videos better with highlight-guided attention","Continuous highlight distribution steers MLLM attention for summaries"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central claim rests on the reliability of the off-the-shelf highlight generator H: if its frame scores do not faithfully reflect what a summary should attend to, then adding them to a frozen model's attention logits injects the generator's noise into every decoding step.","fun_headline_variants_meta":{"raw":{"variants":["Steer video attention with highlight curves, not keyframes","Highlight curves guide attention for better video summaries","Attention steering outperforms keyframe selection in video summarization","Frozen MLLM summarizes videos better with highlight-guided attention","Continuous highlight distribution steers MLLM attention for summaries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1252,"prompt_tokens":810,"completion_tokens":442,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":363}},"tokens_in":554,"tokens_out":442,"duration_ms":4190,"temperature":1.0,"reasoning_tokens":363,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T16:26:23.237586+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HAS with three steering vectors on the same videos: the real highlight prior, a uniform vector, and a scrambled vector. If uniform or scrambled vectors match the real-prior version on temporal salience and factual-consistency metrics, the reported gains come from the injection machinery rather than the highlight content; if an oracle human-annotation prior clearly outperforms the off-the-shelf prior, the unmeasured quality of H is the decisive factor.","supporting_citations":[],"review_version":1}