{"id":"8d199728-b8ac-4ee1-acde-e5c6f2b1cd47","arxiv_id":"2505.24329","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A single learnable time token, decoded into a probability distribution over time bins, improves temporal grounding in Video-LLMs and is trained partly on a new 1.25M-event pseudo-labeled dataset.","lead":"DisTime gives video language models a single token that is decoded into a probability distribution over time, improving their ability to localize events in videos. The paper also releases InternVid-TG, a 1.25M-event pseudo-labeled dataset, and reports state-of-the-art results on moment retrieval and dense video captioning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zero-shot Charades-STA claim is not established: training data (ET-Instruct in Table 2; InternVid-TG pseudo-labels from Mr.Blip/UniMD) may leak benchmark annotations; a contamination audit or control retraining is needed.","rationale":"The paper's architectural contribution (single learnable time token, distribution-based decoding, iterative re-encoding) is coherent and supported by the ablations in Sec. 5.3; the time-decoder design is not the source of my concern. What is load-bearing is the generalization claim: 'surpasses all dedicated models in a zero-shot setting' on Charades-STA. That claim is the strongest evidence of temporal understanding beyond the training distribution. The reader's weakest assumption identified leakage through InternVid-TG pseudo-labels from Mr.Blip and UniMD, which are trained on Charades-STA and ANet-Caption. I agree that this is a real risk. I would add a more direct and even easier-to-check path: Table 2 lists ET-Instruct as 126k multi-event training data, and the paper never specifies whether it contains Charades-STA/ActivityNet instruction samples. ET-Instruct is an event-level instruction dataset, and if it draws from standard temporal grounding benchmarks, then the zero-shot claim fails regardless of the pseudo-label question. The 1k-query validation in Appendix A does not address contamination because it only measures quality of the ensemble annotations. The proposed control experiment would settle both paths at once: regenerate pseudo-labels with a model that has no Charades-STA exposure, and remove any potentially contaminated instruction data. If the 81.0 result persists, the zero-shot claim is credible; if it drops substantially, the claim must be reframed. I am not suggesting any intentional misrepresentation; the reporting simply leaves the contamination question open. The CONDITIONAL verdict from the reader remains appropriate, and my additional concern only strengthens the condition without changing its nature.","tokens_in":21057,"tokens_out":10635,"duration_ms":131196,"concrete_test":"Retrain DisTime-InternVL-8B with two changes: (a) replace InternVid-TG pseudo-labels with versions generated by TFVTG alone (a train-free model with no Charades-STA supervision), and (b) exclude any ET-Instruct samples that trace to Charades-STA, or remove ET-Instruct entirely if its source composition cannot be verified. Keep all other settings identical. If the Charades-STA R@1 IoU=0.3 score drops below the ~72.6 dedicated UniVTG baseline, or falls by more than 3 points from 81.0, the zero-shot claim collapses to a distillation result. This single control run isolates both the direct ET-Instruct path and the indirect pseudo-label path.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline SOTA claim rests on the Charades-STA zero-shot result in Table 5 (DisTime-InternVL-8B at 81.0 R@1 IoU=0.3). For this to be truly zero-shot, no Charades-STA annotation may enter training directly or indirectly, but the paper does not establish this. Direct route: Table 2 lists ET-Instruct [28] (126k) as multi-event training data, and its composition is never disclosed. If ET-Instruct contains Charades-STA-derived instruction samples, the zero-shot claim is simply false. Appendix D says the authors expanded on ET-Instruct's instruction data but never states that Charades-STA samples are excluded. Indirect route: Sec. 4.1 builds InternVid-TG using Mr.Blip and UniMD, both trained on Charades-STA/ANet-Caption, and the ensemble scoring selects among their outputs. This transfers benchmark-specific localization knowledge into the pseudo-labels, which then train DisTime. The 1k-query manual validation in Appendix A measures pseudo-label quality, not contamination. Without a precise audit of every training source or a control run that removes these leakage paths, the zero-shot framing and the resulting 'beats dedicated models' claim are unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DisTime, a lightweight temporal representation framework for Video-LLMs. A dedicated <TIME_STAMP> token is decoded by a small MLP into probability distributions over time bins, and timestamps are re-encoded into Gaussian-regularized token embeddings for input and for autoregressive refinement. Training combines next-token prediction, an IoU regression loss, and distribution focal loss. To address the scarcity of temporally labeled data, the authors construct InternVid-TG (1.25M events from 179k videos) by extracting events with GPT-4o, obtaining event boundaries from UniMD, Mr.Blip, and TFVTG, and ensembling their outputs via an InternVideo2-based similarity score. Experiments cover moment retrieval (Charades-STA, ANet-Caption, QVHighlights), dense video captioning (ANet-Caption, YouCook2), grounded VQA (NExT-GQA), and general video QA. The headline claim is state-of-the-art moment retrieval on Charades-STA in a zero-shot setting, e.g., 81.0 R@1 IoU=0.3 for the 8B model.","tokens_in":21364,"tokens_out":5044,"duration_ms":69531,"significance":"If the claims hold, DisTime is a clean, parameter-efficient alternative to dedicated temporal heads: distribution-based decoding is well motivated by boundary ambiguity, the single-token design is simple, and the re-encoding mechanism is a plausible way to maintain temporal context during autoregression. The authors provide clear ablations supporting the main architectural choices, and they release code and data. The InternVid-TG dataset is a potentially useful resource. However, the central zero-shot claim is not adequately supported because the pseudo-label generation pipeline uses models trained on the target benchmarks, and the composition of the other training data is not audited. The absence of a contamination audit, combined with hyperparameter selection on the evaluation benchmarks and no error bars, means the SOTA claim is currently not established.","major_comments":[{"comment":"The Charades-STA zero-shot claim is not established. Section 4.1 states that InternVid-TG pseudo-labels are generated by Mr.Blip, a model trained on Charades-STA, ANet-Caption, and QVHighlight, and by UniMD, a dedicated trained grounding model; the ensemble scoring then selects among these outputs. This supervision is used to train DisTime (Table 2). The Charades-STA evaluation in Table 5 is therefore not zero-shot with respect to benchmark-derived annotation knowledge. The 1k-query manual validation in Appendix A measures pseudo-label quality, not contamination. Please provide a training-data contamination audit, or a control experiment that removes this leakage path, or reframe the claim as a distillation result rather than zero-shot.","section":"Sec. 4.1, Table 5"},{"comment":"The direct training data may also leak Charades-STA annotations. Table 2 lists ET-Instruct (126k multi-event samples) as training data, but its composition is never disclosed. Appendix D says the authors expanded on ET-Instruct instruction data but does not state whether Charades-STA-derived samples are excluded. If ET-Instruct contains any Charades-STA-derived instructions, the zero-shot claim is false for that path as well. The manuscript must specify the provenance of every training source or demonstrate exclusion of target-benchmark samples.","section":"Table 2, Appendix D"},{"comment":"The time encoder uses N(st, δ^2) with δ=1 on normalized timestamps in [0,1]. A Gaussian with standard deviation 1 over the unit interval is nearly uniform, which would make re-encoded time tokens almost indistinguishable across different timestamps. This appears to contradict the claimed continuous temporal embedding in Sec. 3.1. Please clarify the intended value of δ and how discriminative temporal information survives this encoding; if δ=1 is a typo, state the correct value and rerun the affected ablations.","section":"Sec. 3.2, Eq. (6)"},{"comment":"Important hyperparameters (number of bins regmax, number of encoder/decoder layers) are selected using the evaluation benchmarks Charades-STA and YouCook2, with no held-out validation set reported. No error bars or multiple-seed results are given. Consequently, the reported margins over prior methods in Tables 5 and 6, and the ablation differences in Table 3 (e.g., 78.1 vs. 77.0 in R@1 IoU=0.3), may reflect selection bias or noise. Please report results with multiple seeds or a held-out validation split for hyperparameter selection.","section":"Appendix B, Tables B and C"}],"minor_comments":[{"comment":"The rows with checkmarks for Direct/Dist./Re-Enc. are not fully self-explanatory; please add a one-line description of each configuration, especially how the Direct regression baseline is implemented.","section":"Table 3"},{"comment":"There is a typo in the table header: \"ET-Instcuct\" should be \"ET-Instruct.\"","section":"Table 2"},{"comment":"The pipeline figure contains \"InternVideo23\" in the middle block; this appears to be a typo for \"InternVideo2.\"","section":"Fig. 5"},{"comment":"The paper states δ=1 for boundary uncertainty but provides no sensitivity analysis for δ; since the Gaussian width is central to the time-encoder design, please add an ablation over δ or justify the fixed value.","section":"Sec. 3.2"},{"comment":"The phrase \"in a zero-shot setting\" in Table 5 and the main text should be revised consistently once the contamination question is resolved, because the paper otherwise trains on ANet-Caption and InternVid-TG data that is closely related to the evaluation task.","section":"Sec. 5.4"}],"recommendation":"major_revision","confidential_remarks":"The zero-shot Charades-STA claim is the most attention-grabbing result, and the leakage concern is real and easily raised by reviewers. If the authors can provide a credible contamination audit or a control run removing the leakage paths, the paper would be substantially strengthened. Without that, the claim should be reframed. The architectural contribution itself appears sound and well ablated, so I do not see a reason to reject the manuscript outright."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"My take: this is a serious, well-executed paper on temporal grounding for Video-LLMs. The central trick, a single learnable token decoded into a softmax distribution over time bins and then re-encoded via a Gaussian-regularized encoder, is genuinely new and works surprisingly well. The ablations are clean: distribution decoding beats direct regression, re-encoding helps, and the InternVid-TG dataset with 1.25M events gives a real boost. The parameter overhead is negligible (0.34-0.84% of the backbone), and the method transfers across two different Video-LLM families. That is worth something.\n\nThe main claim, state-of-the-art on Charades-STA in a zero-shot setting, does not survive close reading. Training data includes ET-Instruct (126k) whose composition is never disclosed, and InternVid-TG pseudo-labels come from Mr.Blip and UniMD, both fine-tuned on Charades-STA and ANet-Caption; the ensemble scoring selects among their outputs. That is indirect supervision at best and likely leakage at worst. The 1k-query manual validation in Appendix A is about annotation quality, not contamination. Without an audit of every training source or a control run that removes these routes, the zero-shot label is not justified. This is load-bearing for the headline result.\n\nSmaller issues: hyperparameters (bins, layers, Gaussian spread) were tuned on the evaluation benchmarks, and no error bars are reported. Fixable, not fatal.\n\nTo the paper's credit, it is candid about weaker spots, acknowledging lower DVC scores on ANet-Caption and a small drop on VideoMME. That honesty makes me trust the rest of the numbers more.\n\nBottom line: the method and dataset are valuable for anyone working on video temporal grounding. The paper deserves a serious referee, but not acceptance as-is. The authors need to audit training data for benchmark overlap or drop the zero-shot framing. I would engage as a reviewer and ask for that change.","headline":"Solid distribution-based time token for Video-LLMs, but the zero-shot Charades-STA claim is undercut by likely benchmark leakage in the pseudo-label pipeline.","tokens_in":21864,"tokens_out":2479,"would_cite":true,"duration_ms":30279,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DisTime claims that one learnable token, decoded into a probability distribution over the timeline, gives Video-LLMs state-of-the-art temporal grounding without heavy modules.","keywords":["video large language models","temporal localization","moment retrieval","distribution-based time token","dense video captioning","grounded video question answering","automated annotation","temporal grounding dataset"],"falsifier":"Retrain DisTime on InternVid-TG annotations produced solely by TFVTG, the training-free annotator that has never seen Charades-STA or ANet-Caption, and measure Charades-STA R@1 IoU=0.3; if the score drops far below 78–81, the zero-shot result comes from the other annotators' benchmark exposure rather than the time-token mechanism. Also audit InternVid-TG for videos or queries overlapping the two benchmarks.","tokens_in":20888,"feed_emoji":"⏱️","tokens_out":11166,"duration_ms":120057,"temperature":0.7,"pith_summary":"DisTime sets out to give Video-LLMs a precise sense of when events happen. Its core proposal is to represent time with a single learnable token whose hidden state is decoded into two probability distributions over the normalized timeline, so start and end times emerge as expectation values rather than as text numerals or special-token classifications. On top of that, the authors build InternVid-TG, a 1.25M-event automatically annotated dataset produced by combining GPT-4o event captions with boundary proposals from three dedicated temporal localizers and an ensemble scoring step. They report state-of-the-art results on moment retrieval, dense video captioning, and grounded video QA—including a zero-shot 81.0 R@1 IoU=0.3 on Charades-STA with an 8B model—while keeping general video QA competitive.","feed_headline":"81.0 zero-shot: one token times video events","feed_subtitle":"A distribution-decoded time token gives Video-LLMs state-of-the-art temporal grounding on three tasks.","key_machinery":"The Distribution-based Time Token (<TIME STAMP>) is the load-bearing object: one extra vocabulary token whose hidden state the time decoder (MLP + softmax) turns into start/end probability vectors over regmax+1 anchors at positions i/regmax on the normalized timeline. Predictions are the expectation values of those distributions, which keeps time continuous and lets the distribution shape absorb boundary uncertainty; its inverse, the time encoder, converts any timestamp into a Gaussian-smoothed discretized distribution and then an MLP token, so decoded times can be re-injected for the next autoregressive step. Training couples distribution focal loss, a 1D-IoU regression loss, and next-token prediction, with LoRA used for the LLM backbone.","core_discovery":"The central claim is that the dominant failure modes of time in Video-LLMs—confusing time with numbers, under-trained special tokens, and boundary ambiguity—can all be handled by one Distribution-based Time Token. When the LLM emits <TIME STAMP>, its final-layer hidden state passes through a three-layer MLP with softmax to produce start and end distributions over regmax+1 uniformly spaced anchor bins on [0,1]; the predicted times are the anchor-weighted sums of those distributions. A companion time encoder maps any timestamp back into a Gaussian-regularized distribution and then an LLM embedding, and re-encodes the decoded times into the autoregressive context, giving what the paper calls Iterative Time Refinement. Trained with distribution focal loss plus a 1D-IoU regression loss and standard next-token prediction, the scheme lifts Charades-STA R@1 IoU=0.3 from 3.1 (baseline InternVL2.5-1B) to 78.1, and to 81.0 at 8B in a zero-shot setting, while also improving dense video captioning (YouCook2 F1 from 2.2 to 20.5 in ablation) and grounded VQA.","pith_inferences":["If the zero-shot number survives a data-leakage audit, the same distribution-decoder pattern should generalize to other continuous quantities that LLMs currently type as text—bounding-box coordinates, audio timestamps, or confidence scores.","The Gaussian re-encoding of decoded times behaves like a consistency regularizer between decoder and encoder; using learned anchors instead of the fixed i/regmax grid is a natural next probe.","Because InternVid-TG pseudo-labels are selected among three annotators, the dataset's ceiling is the union of those annotators; training on low-confidence ensemble cases may be where the model learns beyond any single annotator.","Charades-STA results are reached with only 16 sampled frames, suggesting the benchmark's queries are answerable from coarse temporal layout; the same method on denser sampling might close the ANet-Caption gap against dedicated models."],"forward_implications":["Video-LLMs can express precise continuous timestamps with a single added token, avoiding text-number confusion and the long-tail problem of many sparse special tokens.","The same time token transfers across backbone families—InternVL2.5 (1B, 8B) and LLaVA-OneVision (7B)—so the mechanism is model-agnostic rather than tied to one architecture.","Temporal supervision can be scaled automatically: 1.25M events over 179k videos, roughly 55 times ActivityNet-Caption, generated without shot-boundary or fixed-interval constraints.","Zero-shot moment retrieval on Charades-STA (81.0 R@1 IoU=0.3 at 8B) surpasses dedicated grounding models that were fine-tuned on the benchmark.","Iterative time re-encoding gives the largest gains in high-precision metrics; in ablations Charades-STA R@1 IoU=0.7 improves by 3 points and YouCook2 CIDEr by 11.6."],"supporting_citations":[{"why":"Supplies the distribution focal loss and the anchor-weighted expectation decoding that DisTime adapts from bounding boxes to time boundaries.","marker":"[22]"},{"why":"UniMD is one of the three dedicated temporal localizers whose boundary proposals populate InternVid-TG, and it is retrained on InternVideo2 features for domain fit.","marker":"[52]"},{"why":"Mr.Blip provides high-precision pseudo-label boundaries in the annotation ensemble and is the strongest dedicated-model baseline in the moment-retrieval comparisons.","marker":"[32]"},{"why":"TFVTG contributes training-free zero-shot boundary proposals to the ensemble, the only annotator without direct benchmark fine-tuning.","marker":"[55]"},{"why":"GPT-4o generates the dense event captions from which InternVid-TG queries are drawn.","marker":"[1]"},{"why":"Cited as the scoring model whose video-text similarity picks the ensemble output for each event; InternVideo2.5 also appears as a comparison baseline.","marker":"[44]"},{"why":"InternVL2.5 is the backbone for the 1B and 8B DisTime variants that carry the main results.","marker":"[5]"},{"why":"LLaVA-OneVision is the 7B backbone used to show the method transfers across model families.","marker":"[18]"},{"why":"Charades-STA is the benchmark where the zero-shot state-of-the-art claim (81.0 R@1 IoU=0.3) is made.","marker":"[9]"},{"why":"ANet-Caption is the second moment-retrieval and dense-captioning benchmark used for the main comparisons.","marker":"[16]"}],"fun_headline_variants":["Single distribution token: 81.0 zero-shot for video timing","One time token gives LLMs precise intervals at 81.0 zero-shot","Distribution-based time token: SOTA video grounding, 81.0 zero-shot","Video LLMs now time events with one token, hitting 81.0 zero-shot"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The zero-shot scores on Charades-STA and ANet-Caption rest on the assumption that InternVid-TG pseudo-labels do not count as benchmark training—yet the three annotating models were trained on moment-retrieval data that includes those benchmarks, so the indirect supervision may be the real source of the gains.","fun_headline_variants_meta":{"raw":{"variants":["Single distribution token: 81.0 zero-shot for video timing","One time token gives LLMs precise intervals at 81.0 zero-shot","Distribution-based time token: SOTA video grounding, 81.0 zero-shot","Video LLMs now time events with one token, hitting 81.0 zero-shot"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000999,"raw_usage":{"total_tokens":4276,"prompt_tokens":1043,"completion_tokens":3233,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":3149}},"tokens_in":659,"tokens_out":3233,"duration_ms":28745,"temperature":1.0,"reasoning_tokens":3149,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:25:12.123916+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain DisTime on InternVid-TG annotations produced solely by TFVTG, the training-free annotator that has never seen Charades-STA or ANet-Caption, and measure Charades-STA R@1 IoU=0.3; if the score drops far below 78–81, the zero-shot result comes from the other annotators' benchmark exposure rather than the time-token mechanism. Also audit InternVid-TG for videos or queries overlapping the two benchmarks.","supporting_citations":[{"cited_title":"Generalized focal loss: Learning qualified and distributed bounding boxes for dense 9 object detection","cited_arxiv_id":null,"evidence_quote":"Supplies the distribution focal loss and the anchor-weighted expectation decoding that DisTime adapts from bounding boxes to time boundaries."},{"cited_title":"Unimd: Towards unifying moment retrieval and temporal ac- tion detection","cited_arxiv_id":null,"evidence_quote":"UniMD is one of the three dedicated temporal localizers whose boundary proposals populate InternVid-TG, and it is retrained on InternVideo2 features for domain fit."},{"cited_title":"Training-free video temporal grounding using large-scale pre-trained models","cited_arxiv_id":null,"evidence_quote":"TFVTG contributes training-free zero-shot boundary proposals to the ensemble, the only annotator without direct benchmark fine-tuning."},{"cited_title":"Tall: Temporal activity localization via language query","cited_arxiv_id":null,"evidence_quote":"Charades-STA is the benchmark where the zero-shot state-of-the-art claim (81.0 R@1 IoU=0.3) is made."}],"review_version":1}