{"id":"4bffb9bb-bdfd-42de-8e3b-871c22adc5de","arxiv_id":"2508.18634","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"OwlCap balances motion and detail in video captions using the HMD-270K dataset and a Caption Set Equivalence Reward, improving VDC accuracy and DREAM-1K recall over baselines.","lead":"This paper introduces OwlCap, a video captioning model trained on a new 270K video-caption dataset and a reinforcement learning reward that balances motion and detail. It reports gains over existing models on detail and motion benchmarks, and better text-to-video generation quality.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CSER and HMD-270K are both built on MLLM judges; without demonstrated human agreement, the reward may optimize a model-specific proxy rather than caption quality.","rationale":"The reader identified the same load-bearing assumption: MLLM judges underpin both the HMD-270K filter and the CSER reward. I agree with that identification. The paper is internally coherent: the pipeline is described in enough detail to reproduce, the ablations in Tables 5-7 show monotone contributions from MDF/FGE and from the two reward components, and the final model is evaluated on human-annotated benchmarks, which rules out the simplest form of same-benchmark overfitting. However, the load-bearing premise is unverified: no human agreement statistics are provided for the unit judgments, despite the FGE section explicitly referring to planned checks. Because the same kind of semantic-equivalence judgment is used to build the SFT data and to compute the RL reward, a systematic deviation from human judgment would not be visible in the final benchmark tables alone; it would be absorbed into the training distribution and the reward, so the model could score well while optimizing a self-consistent surrogate. This is not a claim of misconduct; it is an empirical validation gap in the central argument. A human-correlation study on a few hundred units would settle whether the concern lands, and if it does, the claimed motion-detail balance would need to be re-estimated. Since the paper already received a CONDITIONAL verdict, my reading does not move the verdict; the condition should explicitly include demonstrating human agreement with the MLLM judge chain.","tokens_in":13286,"tokens_out":6149,"duration_ms":61626,"concrete_test":"Collect 300 candidate captions generated by the SFT model during GRPO rollouts. Have three human annotators score each caption using the same unit-level correctness and completeness criteria as CSER, and compute the rank correlation (e.g., Spearman) between the human scores and the Qwen3-32B-based CSER scores. If the correlation is below about 0.8, retrain a small GRPO run using human-scored units on this subset and compare VDC and DREAM-1K gains; if the gains are not reproduced with the human-calibrated reward, the reported improvements are attributable to optimizing the MLLM judge proxy rather than human caption quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the assumption that the unit-level judgments used in two places are aligned with human caption quality. In the Fine-Grained Examination stage, InternVL2.5-78B decomposes generated captions into units and filters HMD-270K at 90% unit accuracy. In CSER, Qwen3-32B both decomposes predicted and GT captions and judges the relevance needed for Eqs. (4) and (5). The same style of decomposition and semantic-equivalence judgment is used to create the training signal and to define the reward, so any systematic bias in these MLLMs is amplified: SFT and GRPO can become more fluent at matching the judge's preferences without improving ground-truth caption quality. The paper states in the FGE section that the appendix includes three complementary checks, but none of these checks are reported in the main text, and the main evaluation relies only on final benchmark numbers. The benchmarks VDC and DREAM-1K are human-annotated, which makes the concern testable; if the judge chain were well calibrated, the gains would be credible, but the paper does not yet show that calibration.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the motion-detail imbalance in video captioning by contributing (1) HMD-270K, a 270K-pair dataset built through a two-stage Motion-Detail Fusion (MDF) and Fine-Grained Examination (FGE) pipeline, and (2) the Caption Set Equivalence Reward (CSER), a GRPO-based reward that scores predicted captions against ground-truth captions through unit-level correctness and completeness measures. The resulting model, OwlCap, is a Qwen2.5-VL-7B base fine-tuned with HMD-270K SFT and CSER-based RL. The authors report improvements on the VDC benchmark (+4.2 accuracy) and DREAM-1K (+4.6 F1) relative to the Qwen2.5-VL-7B baseline, and show additional gains in a small text-to-video (T2V) evaluation. The paper claims state-of-the-art performance on both benchmarks and plans to release the dataset and model.","tokens_in":13668,"tokens_out":5779,"duration_ms":52309,"significance":"If the results hold, the paper makes a useful engineering contribution: it demonstrates that a single 7B model can be trained to capture both motion and static detail, with positive downstream effects on text-to-video generation. Strengths include a clearly described dataset construction pipeline, ablations isolating the contributions of HMD-270K SFT and CSER components, and experiments across several Qwen-family base models showing consistent improvements. The planned public release of HMD-270K and OwlCap would benefit the community. However, the central claims are currently supported by single-run benchmark numbers without uncertainty quantification, and the training signal relies on MLLM judges whose human alignment is not demonstrated in the main text. These limitations temper the significance until addressed.","major_comments":[{"comment":"The abstract claims 'significant improvements' on DREAM-1K, but the reported F1 margin over Tarsier-7B is 0.1 (34.7 vs. 34.6). No error bars, multiple seeds, or statistical significance tests are reported for any table. Without such uncertainty quantification, the claim of outperforming all existing models on the motion-focused benchmark is not supported. The authors should either provide repeated-run statistics or revise the wording to 'numerical improvements.'","section":"Table 3 and Abstract"},{"comment":"Both the HMD-270K filtering and the CSER reward rely on MLLM judges: InternVL2.5-78B for FGE unit verification and Qwen3-32B for the relevance judgments that define the correctness and completeness scores in Eqs. (4) and (5). The paper states that three validation checks (cross-model consistency, threshold sensitivity, human re-evaluation) are in the appendix, but none of their results appear in the main text. Since the RL training set is also selected using CSER variance computed with the same Qwen3-32B judge, any systematic bias in these judges is amplified across data construction and reward optimization. The authors should report the human-agreement and cross-model consistency results in the main text, or at least prominently summarize them, to establish that the judge chain is aligned with human caption quality.","section":"Fine-Grained Examination Stage and Equations (4)-(5)"},{"comment":"The claim of outperforming 'all existing models on mainstream video captioning benchmarks' is broader than the evidence in the main text, which reports only VDC and DREAM-1K. Additional benchmarks (VidCapBench-AE, CaReBench) are mentioned but their results are deferred to the appendix. The T2V experiment uses only 200 videos and five captioning models, and the 'significant advantages' in Table 4 are asserted without error bars or human evaluation. The authors should either include the additional benchmark results in the main text or restrict the claim to the benchmarks actually reported.","section":"Abstract and Experiments (T2V evaluation)"}],"minor_comments":[{"comment":"The model name 'Taesier' in the video captioning paragraph should be 'Tarsier.'","section":"Related Work"},{"comment":"In the paragraph comparing CSER with VideoCap-R1's Event Score, 'CESR' is a typo and should be 'CSER' for consistency.","section":"Caption Set Equivalence Reward"},{"comment":"The caption says 'HDM-270K' but the dataset name is 'HMD-270K.'","section":"Figure 2 caption"},{"comment":"The VILA citation appears as 'Lin et al. 2023' in Table 3 and 'Lin et al. 2024' in Table 2, and the model name is printed as 'LLaV A-OneVision' (with an extra space) in both tables; these should be made consistent.","section":"Tables 2 and 3"},{"comment":"The phrase 'as the data radio grows' should be 'as the data ratio grows.'","section":"Ablation study about different data sizes"},{"comment":"The x-axis labels are malformed ('0% 100%1% 10% 30% 70%'); they should be clearly separated and ordered.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with a potentially useful dataset and model release, but the current evidence is not yet convincing enough for the stated claims. The thin margin on DREAM-1K F1 and the unvalidated reliance on MLLM judges are the main concerns. I would encourage the editor to require the appendix validation checks to be surfaced in the main text and to ask for at least basic uncertainty quantification (multiple seeds or significance tests) before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nQuick take: this is a solid, workmanlike video captioning paper that does what it says: builds a 270K motion-detail balanced dataset, adds a fine-grained set-equivalence reward for GRPO, and reports gains on both a detail benchmark and a motion benchmark. The central claim is likely true in direction, but the thin margin on DREAM-1K and the unvalidated MLLM judge chain mean the \"significant improvements\" phrasing is stronger than the evidence.\n\nWhat is genuinely new: HMD-270K is a real contribution. The two-stage pipeline — MDF to fuse motion and detail captions, then FGE to decompose and verify units — is sensible, and the MDB statistic is a simple, useful way to characterize caption balance. Table 1 shows their captions are denser and better balanced than Panda-70M, ShareGPT4Video, Vript, and Tarsier-585K. CSER, the bidirectional unit-to-set reward, is a reasonable extension of VideoCap-R1's Event Score, and Table 7 shows it beats Event Score under the same base model. The ablation in Table 6 is honest: SFT alone gives little, RL alone gives little, but both together work. That is believable. Table 8, showing consistent gains across three Qwen base models, is a nice robustness check.\n\nSoft spots. First, the DREAM-1K headline margin is 34.7 vs Tarsier's 34.6 F1 — 0.1 points — and no significance test is reported. Calling that a \"significant improvement\" in the abstract is an overstatement. Second, and more important: both the dataset filter (FGE with InternVL2.5-78B at 90% unit accuracy) and the CSER reward (Qwen3-32B unit relevance) depend on MLLM judge judgments that are not validated against humans in the main text. The paper says the appendix includes a cross-model check, a threshold sensitivity analysis, and a human re-evaluation of 100 rejected samples, but none of those numbers appear in the main text. Since the reward and the training data share the same style of decomposition, systematic judge bias would amplify in training even though the evaluation benchmarks are human-annotated. The concern is testable, and the paper should show the calibration. Third, the T2V evaluation is small (200 videos) and uses a non-standard protocol; treat it as illustrative, not evidence of general video-generation gains.\n\nOn balance: the method is coherent, the components are ablated, and the dataset is a potential community resource if released. The overclaim on DREAM-1K and the missing judge-calibration data should be fixed in revision. This deserves a serious peer review — I would send it out. Whether the final numbers hold up is secondary; the dataset and the reward design are worth engaging with regardless.","headline":"Solid, incremental video captioning paper with a useful new dataset and reward; main results likely real but the DREAM-1K margin is razor-thin and the MLLM judge chain needs human calibration.","tokens_in":14020,"tokens_out":2628,"would_cite":true,"duration_ms":22547,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a 7B video captioning model can capture both motion and detailed content, beating prior models on both the detail-focused VDC and motion-focused DREAM-1K benchmarks.","keywords":["video captioning","motion-detail balance","HMD-270K","Caption Set Equivalence Reward","GRPO","multimodal large language model","text-to-video generation","detailed video captioning"],"falsifier":"Take 500 video-caption pairs produced by the HMD-270K pipeline and have human annotators mark each decomposed unit as correct, wrong, or ambiguous, then compare with the judge model's unit labels. If human agreement on unit correctness falls below roughly the 90% threshold the pipeline uses, the filtering and reward are built on a weaker signal than claimed. Alternatively, retrain OwlCap with CSER scores computed by a different judge model that disagrees with the current one; if the benchmark gains vanish, the reward's effect depends on the judge choice.","tokens_in":13119,"feed_emoji":"🎬","tokens_out":6945,"duration_ms":60373,"temperature":0.7,"pith_summary":"This paper claims that video captioning models fail because they specialize: some capture motion and miss detail, while others capture detail and miss motion. It proposes a two-part fix: a 270K-video dataset built by merging motion-oriented and detail-oriented captions and then filtering them through a fine-grained unit check, plus a reinforcement-learning reward that pushes predicted captions toward semantic equivalence with a reference in both directions. Applied to Qwen2.5-VL-7B, this produces OwlCap, which the paper reports improves the detail-focused VDC benchmark by +4.2 accuracy and the motion-focused DREAM-1K benchmark by +4.6 F1 over the baseline. If true, a single 7B model can supply captions detailed and motion-aware enough for both video understanding and text-to-video generation.","feed_headline":"One caption model tops both motion and detail tests","feed_subtitle":"OwlCap's 270K balanced dataset and set-equivalence reward lift a 7B model past prior specialists.","key_machinery":"The load-bearing mechanism is the Caption Set Equivalence Reward (CSER), built on the idea that a predicted caption and a ground-truth caption should be semantically equivalent sets of facts. Predicted and reference captions are decomposed into minimal semantic units, then two scores are computed: $S_{\\text{correctness}} = \\frac{1}{n}\\sum_i I(U_i \\in C_{\\text{gt}})$, the fraction of predicted units that match the reference, and $S_{\\text{completeness}} = \\frac{1}{m}\\sum_j I(F_j \\in C_{\\text{pred}})$, the fraction of reference facts covered by the prediction. These are added to a format score and optimized with GRPO, which compares candidate captions within a group. The same unit-decomposition idea appears earlier in the dataset pipeline, where Fine-Grained Examination verifies each caption unit against the video at a 90% threshold; CSER then coerces the model toward bidirectional equivalence, discouraging both fabricated details and missing actions.","core_discovery":"The central claim is that motion-detail imbalance in video captioning can be resolved at two levels. At the data level, HMD-270K is constructed by first prompting a motion-specialized MLLM to write a temporal caption, then asking a general MLLM to complete it with static details, and finally decomposing each caption into atomic units and retaining only pairs whose units are verified against the video by a judge MLLM at at least 90% unit accuracy. At the optimization level, CSER turns caption quality into a set-equivalence check: it decomposes predicted and ground-truth captions into units, rewards the fraction of predicted units entailed by the reference as correctness, rewards the fraction of reference facts covered by the prediction as completeness, and optimizes the sum through GRPO. The paper reports that this recipe yields consistent gains over the Qwen2.5-VL-7B baseline on both VDC and DREAM-1K, and that OwlCap improves text-to-video generation metrics when its captions are fed to a video generator.","pith_inferences":["The paper leaves implicit that the same unit-to-set, bidirectional reward could transfer to image captioning or dense video grounding, wherever captions can be decomposed into atomic facts; that transfer is not tested here.","The MDB statistic suggests a cheap diagnostic for other caption datasets: captions low in words per second or verbs per second are likely to be detail-poor or motion-poor, so MDB could flag datasets that need rebalancing before training.","A stress test not run in the paper: replace the InternVL2.5-78B judge with a substantially smaller judge model; if the benchmark gains vanish, the judge's quality, rather than the set-equivalence formulation, is the active ingredient.","The T2V probe with a fixed generator implies that caption quality is a limiting factor for text-to-video consistency; if true, captioning benchmarks should routinely pair with a generation metric, which the current leaderboards do not."],"forward_implications":["If OwlCap's reported gains hold, text-to-video systems that consume generated captions should produce more faithful videos: the paper's T2V probe with HunyuanVideo shows higher SSIM and PSNR and lower FID than captions from prior models.","A single 7B model can top both a detail-focused leaderboard and a motion-focused leaderboard, so practitioners no longer need to choose between specialist captioners.","The ablation pattern implies that HMD-270K supervised fine-tuning is a prerequisite: reinforcement learning with CSER alone yields small gains, while the two phases together give the large improvements.","CSER is a drop-in replacement for the Event Score reward used by VideoCap-R1, keeping roughly the same training time while improving both benchmarks.","Because HMD-270K will be publicly released, other video captioners can be fine-tuned on the same balanced data and evaluated with the same motion-detail balance lens."],"supporting_citations":[{"why":"Supplies Tarsier, the motion-oriented captioner used in Motion-Detail Fusion, and DREAM-1K, the motion-focused benchmark that defines one side of the imbalance.","marker":"Wang et al. 2024"},{"why":"Supplies AuroraCap and the Video Detailed Captions benchmark, defining the detail-focused side that OwlCap must beat.","marker":"Chai et al. 2025"},{"why":"Provides Qwen2.5-VL, the base model used both for detail completion in the MDF stage and as the backbone for OwlCap training.","marker":"Bai et al. 2025"},{"why":"Provides GRPO, the group-relative policy-optimization algorithm on which the CSER reward is optimized.","marker":"Shao et al. 2024"},{"why":"Defines VideoCap-R1 and its single-sided Event Score, the reward baseline that CSER is compared against and outperforms.","marker":"Meng et al. 2025"},{"why":"Provides HunyuanVideo, the fixed text-to-video generator used to show that OwlCap captions improve downstream generation quality.","marker":"Kong et al. 2024"}],"fun_headline_variants":["OwlCap: one model, both motion and detail tests topped","Motion-detail balance: OwlCap wins VDC and DREAM-1K","OwlCap's 270K set and equivalence reward beat tradeoff","From imbalance to balance: OwlCap's dual fix for captions","OwlCap: harmonizing motion and detail via 270K and CSER"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes that the automated judge models used to filter captions and score rewards grade caption units the way a human grader would; if those judgments are systematically biased, both the training data and the reward are biased in the same direction.","fun_headline_variants_meta":{"raw":{"variants":["OwlCap: one model, both motion and detail tests topped","Motion-detail balance: OwlCap wins VDC and DREAM-1K","OwlCap's 270K set and equivalence reward beat tradeoff","From imbalance to balance: OwlCap's dual fix for captions","OwlCap: harmonizing motion and detail via 270K and CSER"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1603,"prompt_tokens":1043,"completion_tokens":560,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":462}},"tokens_in":659,"tokens_out":560,"duration_ms":5612,"temperature":1.0,"reasoning_tokens":462,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:53:39.709431+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take 500 video-caption pairs produced by the HMD-270K pipeline and have human annotators mark each decomposed unit as correct, wrong, or ambiguous, then compare with the judge model's unit labels. If human agreement on unit correctness falls below roughly the 90% threshold the pipeline uses, the filtering and reward are built on a weaker signal than claimed. Alternatively, retrain OwlCap with CSER scores computed by a different judge model that disagrees with the current one; if the benchmark gains vanish, the reward's effect depends on the judge choice.","supporting_citations":[],"review_version":2}