Pith. sign in

REVIEW 4 major objections 6 minor 91 references

MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning

T0 review · 4 major / 6 minor · reviewed 2026-07-31 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new marine-video benchmark and a tool-calling model push event understanding beyond current VLMs

desk verdict MarineEVT is a genuinely useful new event-centric marine video benchmark, but the paper's central performance claim for EVT-R1 over standard GRPO is not supported by its own matched comparison. read the letter →

arxiv 2607.24064 v1 pith:H2BY27ZA submitted 2026-07-27 cs.CV cs.AI

Tuan-An To , Yuk-Kwan Wong , Tuan-Anh Vu , Ziqiang Zheng , Sai-Kit Yeung This is my paper · ORCID
classification cs.CVcs.AI
keywords MarinevideounderstandingEvent-centricreasoningVisualtoolusebenchmarkVision-languagemodelReinforcementlearningquestionansweringSpatio-temporal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces MarineEVT, claimed to be the first event-centric marine video understanding dataset, with 20,000 question-answer pairs across 20 reasoning dimensions. It also proposes EVT-R1, a training method that lets a vision-language model call visual tools—such as grounding, depth, and orientation modules—during multi-turn reasoning to locate sparse, critical events in underwater footage. On the dataset's 2,000-pair test set, EVT-R1 reaches 48.89% average accuracy, exceeding the best open-source model by 5.22 points and the best commercial model by 11.09 points. The authors argue that tool-integrated reasoning is a promising direction for domain-specific video understanding, where events are rare and unevenly distributed.

What carries the argument

The central mechanism is EVT-R1, a reinforcement-learning framework built on GRPO-like optimization but with a dual reward: one term scores whether the model invokes the correct tool at the correct step (and whether the tool's output matches ground truth), and another scores final-answer correctness by format compliance and semantic similarity. The toolbox includes SAM3 for object grounding, DepthAnythingV2 for depth, and OrientAnything for orientation. The model is allowed to replace its visual input with tool-enhanced versions during inference, enabling it to focus on sparse events.

What would settle it

Conduct an independent human-expert evaluation of a random sample of, say, 400 test-set pairs from MarineEVT, comparing EVT-R1's answers to expert judgments. If the model's accuracy on this expert-verified subset is not significantly higher than the best open-source baseline, the central claim of superiority would be undermined.

Watch

Extended reading notes

Core claim

The paper claims that decomposing event-centric marine video understanding into a multi-turn process in which a VLM decides when to invoke grounding, depth, or orientation tools yields substantially better answers than standard zero-shot, fine-tuned, or token-compression approaches. The evidence is a 20K-pair dataset (MarineEVT) and a method (EVT-R1) that trains the VLM with separate rewards for tool-use correctness and answer accuracy. The reported test accuracy is 48.89%, versus 43.67% for the best open VLM without tools, 40.35% for the best commercial VLM with tools, and 45.27% for supervised fine-tuning alone.

Load-bearing premise

The benchmark's ground-truth labels and tool-use annotations were produced by automated pipelines (LLM generation, vision-tool grounding) with limited human oversight, so the reported accuracy gains could reflect agreement with the generator rather than true marine video understanding.

Editorial extensions

If this is right

  • If the reported numbers hold, tool-integrated reasoning could be a practical way to adapt general VLMs to other sparse-event video domains like wildlife monitoring or surveillance.
  • The dual-reward design suggests that reward models should evaluate intermediate tool calls, not just final answers, to avoid the instability seen with GRPO-only training.
  • The dataset provides a new benchmark for future work on video-level temporal and causal reasoning in marine ecology, potentially supporting automated monitoring and educational tools.
  • The finding that token-compression algorithms underperform on marine videos indicates that naive redundancy reduction is insufficient for event-localization tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A likely implication the paper does not fully explore is that the performance gap might partly reflect the benchmark's synthetic construction: because QA pairs and tool-use labels were generated by LLMs and vision tools with only partial human verification, some of the gains could measure alignment with the generator's reasoning style rather than ecological knowledge.
  • One testable extension would be to run the same EVT-R1 training on a human-only-verified subset of MarineEVT and compare accuracy; if the drop is small, the synthetic pipeline is more trustworthy.
  • A second connection is to other sparse-domain benchmarks (e.g., surgical or surveillance video): if the method transfers, it would strengthen the claim that tool-integrated reasoning is a general domain-adaptation strategy.
  • The authors' adversarial-input experiment only used GPT-5-Mini, so it remains unclear whether EVT-R1 itself relies on language priors; a direct test on EVT-R1 would clarify this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MarineEVT, which it claims is the first event-centric marine video understanding dataset, comprising 20K multi-task video QA pairs across 20 dimensions, generated through a two-stage pipeline that combines GPT-5, QwenVL-Max, SAM3, DepthAnythingV2, and OrientAnything with two-tier human verification. The paper further proposes EVT-R1, a visual tool-integrated reasoning framework that decomposes marine video understanding into multi-turn tool invocation steps and trains a VLM with a dual-reward GRPO objective: one reward for tool-use validity/accuracy and one for final answer correctness. Experiments compare EVT-R1 against 11 open- and closed-source VLMs, reporting that it outperforms the best open-source and commercial baselines by 5.22 and 11.09 points respectively, with ablations showing gains over token compression, temporal localization, and standard GRPO. The central claims are that the dataset is a valid benchmark for event-centric marine video understanding and that the proposed dual-reward RL paradigm is the source of the improvement.

Significance. If the claims hold, the paper makes a useful contribution: MarineEVT addresses a genuine gap, since existing marine VLMs are predominantly image-level and general-purpose video VLMs are weak at sparse event localization. The decomposition of video understanding into tool-invocation turns with separate rewards for tool use and answer accuracy is a plausible extension of VTool-R1-style RL to a domain-specific temporal task. The paper also includes a commendable sanity analysis of language-prior reliance (Table 7), and the multi-stage human-verification pipeline is a positive step for dataset construction. However, the significance is conditional on two issues that are partially acknowledged in the text: the headline gain over commercial models is dominated by in-domain supervised fine-tuning, and the benchmark ground truth is generated by the same class of models being evaluated, so the reported margins may partly measure agreement with generator priors rather than marine video understanding. With proper matched comparisons and external validation, the dataset and training recipe could be valuable to the community.

major comments (4)
  1. [§4.2, Table 2 vs Table 8] The central comparison is not matched: EVT-R1 (48.89) is fine-tuned on MarineEVT, while the open-source (43.67) and closed-source (40.35 with tools) baselines are zero-shot. A same-setting SFT-with-tools baseline reaches 45.27, so most of the reported margin over commercial models comes from in-domain supervision. The only matched comparison isolating the dual-reward mechanism is Table 8, where EVT-R1 exceeds GRPO by only 0.69 (SFT+RL) and 0.94 (RL-only). On a 2,000-item test set, the standard error of such a difference is on the order of 1 point; no error bars, variance, or multiple seeds are reported. The paper should provide matched SFT+RL baselines for all headline comparisons and report uncertainty estimates; otherwise the claimed effectiveness of EVT-R1's dual rewards is not statistically supported.
  2. [Abstract vs §4.2, Tables 2–3] The abstract states EVT-R1 outperforms the top commercial model by 11.09, but §4.2 reports a gain of +8.54 over GPT-5-Mini with tool invocation (40.35). The 11.09 number appears to refer to the closed-source model without tools (37.71), which is a different and less relevant comparison. The numeric inconsistency and the ambiguity about which baseline is used should be corrected, and the abstract should report the matched or properly labeled comparison.
  3. [§3.1 and §4.1] MarineEVT's QA pairs are generated by GPT-5 and QwenVL-Max, grounded by SAM3, DepthAnythingV2, and OrientAnything; the intermediate tool-use labels used to supervise R_tool are produced by the same pipeline, and the 2,000 test pairs also come from this pipeline. This creates a circularity risk: training to match those labels can recover generator priors, and the benchmark may reward models that mimic the generator rather than models that understand marine events. The paper should report the exact human verification protocol, inter-annotator agreement on the test set, and ideally an externally validated or human-annotated subset where the main claims are re-evaluated. Without this, the benchmark's validity is not fully established.
  4. [§4.3, Table 9] The reward coefficient λ is ablated on what appears to be the same 2,000-pair test set used for the main results. Selecting hyperparameters directly on the test set can inflate results and makes the reported gains optimistic. Please specify whether a separate validation set was used, or report sensitivity with a clear validation split. This is especially important given the small observed differences in Table 8.
minor comments (6)
  1. [Abstract / §4.2] The abstract's '11.09' margin is inconsistent with the body's '+8.54' over the best closed-source tool-augmented model. Please harmonize these numbers and clearly state the baseline setting.
  2. [Table 1] The column headers are difficult to parse because 'S' appears both as an attribute and as a dataset characteristic, and several entries are compressed. Reformatting would improve readability.
  3. [§4.1] The fine-tuning protocol is ambiguous: the text says 'SFT warm-up with one epoch, followed by RL with 4 epochs' and then 'we also perform SFT and GRPO for 5 epochs.' Clarify whether the GRPO-only setting includes an SFT warmup or exactly how epochs are counted across settings.
  4. [Eq. (3)] The indicator functions I[tool_turn] and I[answer_turn] are not formally defined. Defining the turn types and how they are determined would make the reward model easier to reproduce.
  5. [Algorithm 1] The function process(V, H) is used but never defined. It should be specified, since it appears to control how visual input is replaced or augmented during tool invocation.
  6. [Table 6] The temporal IoU improvement is reported as +0.12 (14.88 to 15.00), which is far smaller than the spatial IoU gain. This near-negligible change should be discussed or acknowledged, since it tempers the claim that RL improves temporal localization.

Circularity Check

1 steps flagged · score 4.0 of 10

Evaluation and reward oracles share the same GPT-5/QwenVL-Max/SAM3 generation pipeline; headline gains partly measure generator reproduction and are not isolated from in-domain SFT.

  1. fitted input called prediction [§3.1 (MarineEVT Construction Pipeline), §3.4 (Reward Model), §4.1 (Datasets & metrics)]
    "structured prompts for QA generation via QwenVL-Max [8]. ... we synthesize intermediate reasoning steps to enable VLMs to localize critical information. GPT-5 [53] validates the correlation between sub-scene descriptions and QA pairs, introducing temporal grounding steps when necessary. Subsequently, SAM3 [12] generates spatial bounding boxes based on the question’s key intent."

    These GPT-5/QwenVL-Max/SAM3 outputs are used both as training supervision and as the evaluation oracle: R_tool in Eq. (3) scores 'whether the tool invocation matches the ground truth' and 'the tool’s visual output matches the ground truth' (§3.4), and the 2,000-item test set is drawn from the same MarineEVT pipeline (§4.1). A policy optimized to maximize R_tool/R_ans on MarineEVT is therefore trained to reproduce the generator's QA labels and SAM3 bounding boxes; reporting accuracy on a holdout from the same pipeline measures agreement with that generator, not an independently established ecological ground truth. Human verification is described only for QA pairs, while the synthesized tool steps enter R_tool directly, so part of the loop is closed by construction. The matched GRPO comparis

full rationale

EVT-R1's optimization machinery is not circular: Eq. (2) is a standard RL objective and Eq. (3) is a genuine dual-reward extension of GRPO. No load-bearing self-citation or imported uniqueness theorem is used; MarineEval/MarineGPT/MarineInst are cited only as prior benchmarks, and VTool-R1 is an external baseline that EVT-R1 extends. The score is raised by benchmark-construction circularity: the QA and tool-step ground truth are produced by GPT-5/QwenVL-Max/SAM3, then used both to define R_tool/R_ans during training and to score the 2,000-pair test set. This makes the central 'understanding' claim partly a measure of agreement with generator priors and tool artifacts. The paper mitigates this with two-tier automated filtering, three human annotators for QA pairs, and a held-out split, so the reduction is not definitional; however, tool-use labels are not human-verified, and the headline comparison is not isolated from in-domain SFT (the controlled GRPO contrast is only +0.69 without error bars). Hence 4 rather than 0 or 8.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on a self-built benchmark whose labels are LLM-generated and human-verified; the method adds one tuned hyperparameter (λ) and standard RL/LoRA hyperparameters; no new entities are introduced.

free parameters (2)
  • reward coefficient λ = 0.75
    Table 9 selects λ=0.75 on the test set (avg 39.87), with no separate validation split; other values 0.00–1.00 are tested.
  • GRPO hyperparameters ε, β and LoRA rank = not fully reported
    The GRPO objective depends on clip parameter ε and KL coefficient β; LoRA rank/α are not given. Only learning rate 5e−6 and weight decay 1e−2 are stated, so the method cannot be reproduced exactly.
assumptions (5)
  • domain assumption The LLM-generated and human-verified QA pairs in MarineEVT are correct ecological ground truth.
    Section 3.1: QA pairs are generated by GPT-5/QwenVL-Max and filtered by three LLMs plus three human annotators; there is no independent ecological gold standard.
  • domain assumption The visual tools (SAM3, DepthAnythingV2, OrientAnything) provide reliable localization evidence for sparse marine events.
    Section 3.1 uses these tools to produce 94,028 object-grounding annotations; sparse or camouflaged marine organisms may not be detected reliably.
  • domain assumption The 2,000 held-out test pairs are a statistically meaningful sample of marine video understanding.
    Section 4.1: testing set is 2,000 QA pairs from the same MarineEVT pipeline; per-cell accuracies in Tables 2–6 have no confidence intervals aside from Table 7.
  • domain assumption Baseline models were evaluated under settings comparable to EVT-R1.
    Section 4.1 says 'different settings' for tool-integrated vs tool-free evaluation; exact prompts, frame sampling, and answer parsing for baselines are not given, so fairness is not verifiable.
  • domain assumption The training and test splits do not overlap in videos or generated content.
    Section 4.1 states 2,000 QA pairs are 'for evaluation only' but does not describe how videos were partitioned; same-source contamination from the generation pipeline is a plausible risk.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning." pith.science (2026). https://pith.science/paper/H2BY27ZA

@misc{pith2026260724064,
  author       = {Pith},
  title        = {Pith review of: MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2BY27ZA}},
  note         = {Machine review of arXiv:2607.24064}
}
read the original abstract

Recent Vision-Language Models (VLMs) have achieved remarkable success in visual understanding, driven by the growing availability of high-quality image-text pairs. However, the performance of VLMs often degrades in the video domain due to the essential need for temporal understanding and the scarcity of large-scale annotated video data. In this work, we focus on marine video understanding, which brings further challenges: first, it requires substantial domain expertise; and video VLMs usually struggle with localizing and interpreting critical information from marine videos, as the informative events are typically sparse, unpredictable, and unevenly distributed. To address these challenges, we carefully curate the first event-centric marine video understanding dataset called MarineEVT, which features 20K multi-task, video-level visual question-answering pairs spanning multiple dimensions of marine understanding and analysis. Meanwhile, based on MarineEVT, we decompose marine video understanding as an Event-centric Visual Tool-integrated Reasoning process EVT-R1 for short, where we leverage powerful visual tools to drive the model to localize and interpret critical information aligned with visual questions and human intent. To demonstrate its effectiveness, we compare EVT-R1 against 11 SOTA VLMs in different settings. EVT-R1 outperforms the top open-source and top commercial models by 5.22 and 11.09, respectively. MarineEVT and EVT-R1 lay the foundation for ecological discovery and marine education, fostering the development of VLMs capable of interpreting marine dynamics, reasoning about ecological interactions, and supporting sustainable ocean video understanding and analysis.

Figures

Figures reproduced from arXiv: 2607.24064 by the authors.

Figure 1
Figure 1. We propose MarineEVT, the first hierarchical and comprehensive event-centric marine video dataset. Based on MarineEVT, we propose EVT-R1, integrating visual tool reasoning into the VLM for more reliable marine video understanding. Abstract. Recent Vision-Language Models (VLMs) have achieved re￾markable success in visual understanding, driven by the growing avail￾ability of high-quality image-text pairs. However, the… view at source ↗
Figure 2
Figure 2. An example question for evaluating event summarization tested on GPT-5.0 [53] with human response provided for comparison. general-purpose VLMs primarily emphasize scene summarization, overlooking the fine-grained visual dynamics that better align with domain requirements. Thus, marine video understanding demands a significant shift toward precise, event-centric descriptions that capture specific entities (e.g., mar… view at source ↗
Figure 3
Figure 3. We propose a data construction pipeline that systematically transforms pub￾licly available marine videos into high-quality, human-verified annotations, ensuring reliability and versatility for diverse VLM training tasks. aligned instructions. This expert integration enables the model to better inter￾pret biologically meaningful cues and reduce ambiguity: Two-stage data generation from coarse-descriptions to fine-ann… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The training and inference processes of EVT-R1. 3.3 EVT-R1 Overview The proposed EVT-R1 leverages reinforcement learning (RL) to optimize VLMs for flexible reasoning and strategic visual tool invocation. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Compared with GRPO, EVT-R1 devises separate rewards for tool usage and answer accuracy, providing more informative intermediate feedback. tools effectively. Specifically, we propose a dual-component reward model that corrects final answers while explicitly encouraging …
Figure 6
Figure 6. Figure 6: Experimental result produced by open-source (general-purpose token￾compression), closed-source, and our EVT-R1 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Attention activation visualization of various models produced by TAM [38] [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: We compare the reward score curves of GRPO and EVT-R1 [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

91 extracted references · 32 linked inside Pith

  1. [1]

    Discovery video-source webpage,https://www.discovery.com/

  2. [2]

    Instagram video-source webpage,https://www.instagram.com/

  3. [3]

    Mbari video-source webpage,https://www.mbari.org/

  4. [4]

    National geographic video-source webpage,https://www.nationalgeographic. com/

  5. [5]

    Youtube video-source webpage,https://www.youtube.com/

  6. [6]

    In: ACCV (2022)

    Alawode, B., Guo, Y., Ummar, M., Werghi, N., Dias, J., Mian, A., Javed, S.: Utb180: A high-quality benchmark for underwater tracking. In: ACCV (2022)

    2022

  7. [7]

    arXiv preprint arXiv:2308.12966 (2023)

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J.: Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 (2023)

    arXiv 2023

  8. [8]

    arXiv preprint arXiv:2511.21631 (2025)

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    arXiv 2025

Show all 91 references
  1. [9]

    arXiv preprint arXiv:2502.13923 (2025)

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical ...

    2025 arXiv

  2. [10]

    arXiv preprint arXiv:2501.13106 (2025),https://arxiv.org/abs/2501.13106

    Boqiang Zhang, Kehan Li, Z.C.: Videollama 3: Frontier multimodal founda- tion models for image and video understanding. arXiv preprint arXiv:2501.13106 (2025),https://arxiv.org/abs/2501.13106

    2025 arXiv

  3. [11]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2015)

    Caba Heilbron, F., Escorcia, V., Ghanem, B., Carlos Niebles, J.: Activitynet: A large-scale video benchmark for human activity understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2015)

    2015

  4. [12]

    org/abs/2511.16719

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala, K.V., Khedr, H., Huang, A., Lei, J., Ma, T., Guo, B., Kalla, A., Marks, M., Greer, J., Wang, M., Sun, P., Rädle, R., Afouras, T., Mavroudi, E., Xu, K., Wu, T.H., Zhou, Y., Momeni, L., Hazra...

    2025 arXiv

  5. [13]

    Chen, L., Li, J., Dong, X., Zhang, P., Zang, Y., Chen, Z., Duan, H., Wang, J., Qiao, Y., Lin, D., Zhao, F.: Are we on the right way for evaluating large vision-language models? (2024),https://arxiv.org/abs/2403.20330

    2024 arXiv

  6. [14]

    arXiv preprint arXiv:2412.05271 (2024)

    Chen, Z., Wang, W., Cao, Y., Liu, Y., Gao, Z., Cui, E., Zhu, J., Ye, S., Tian, H., Liu, Z., et al.: Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271 (2024)

    2024 arXiv

  7. [15]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al.: Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  8. [16]

    Open Textbook (2024),https://oceancv.org/

    Contributors, V.: Computer Vision Across the Marine Sciences. Open Textbook (2024),https://oceancv.org/

    2024

  9. [17]

    Damen, D., Doughty, H., Farinella, G.M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D., Munro, J., Perrett, T., Price, W., Wray, M.: Scaling egocentric vision: The epic-kitchens dataset (2018),https://arxiv.org/abs/1804.02748

    2018 arXiv

  10. [18]

    In: European Conference on Computer Vision (ECCV)

    Fan, B.,Chen, W., Cong, Y., Tian,J.: Dual refinementunderwater objectdetection network. In: European Conference on Computer Vision (ECCV). pp. 275–291. Springer (2020)

    2020

  11. [19]

    Feichtenhofer, C., Fan, H., Malik, J., He, K.: Slowfast networks for video recogni- tion (2019),https://arxiv.org/abs/1812.03982

    2019 arXiv

  12. [20]

    Ghazanfari, S., Croce, F., Flammarion, N., Krishnamurthy, P., Khorrami, F., Garg, S.: Chain-of-frames: Advancing video understanding in multimodal llms via frame- aware reasoning (2026),https://arxiv.org/abs/2506.00318

    2026 arXiv

  13. [21]

    ICLR / Nature Methods (2024),https://openreview.net/forum?id= d5HUnyByAI

    Gong, Z., et al.: CLIBD: Bridging vision and genomics for biodiversity monitoring at scale. ICLR / Nature Methods (2024),https://openreview.net/forum?id= d5HUnyByAI

    2024

  14. [22]

    ACM Computing Surveys56(4) (2023).https://doi.org/10.1145/3578516, https://dl.acm.org/doi/full/10.1145/3578516

    González-Sabbagh, S.P., Robles-Kelly, A.: A survey on underwater computer vi- sion. ACM Computing Surveys56(4) (2023).https://doi.org/10.1145/3578516, https://dl.acm.org/doi/full/10.1145/3578516

    2023 doi

  15. [23]

    Guo, D., et .al, Y.: Deepseek-r1 incentivizes reasoning in llms through reinforce- mentlearning.Nature645(8081),633–638(Sep2025).https://doi.org/10.1038/ s41586-025-09422-z,http://dx.doi.org/10.1038/s41586-025-09422-z

    doi

  16. [24]

    Han, H., Wang, W., Zhang, G., Li, M., Wang, Y.: Coralvqa: A large-scale visual question answering dataset for coral reef image understanding (2025),https:// arxiv.org/abs/2507.10449

    2025

  17. [25]

    Han, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y., Liu, S.: Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection (2024),https://arxiv.org/abs/2411.14794

    2024 arXiv

  18. [26]

    IEEE Transactions on Image Processing (TIP) (2023)

    Hong, L., Wang, X., Zhang, G., Zhao, M.: Usod10k: a new benchmark dataset for underwater salient object detection. IEEE Transactions on Image Processing (TIP) (2023)

    2023

  19. [27]

    org/abs/2106.09685

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models (2021),https://arxiv. org/abs/2106.09685

    2021 arXiv

  20. [28]

    Ecology99(2) (2018)

    Hughes, T.P., Kerry, J.T., Simpson, T.: Large-scale bleaching of corals on the great barrier reef. Ecology99(2) (2018)

    2018

  21. [29]

    IEEE Transactions on Image Processing (2023),https://ieeexplore.ieee.org/document/XXXXXXX

    Jalal, A., et al.: Fish detection and species classification in underwater environ- ments using deep learning with temporal information. IEEE Transactions on Image Processing (2023),https://ieeexplore.ieee.org/document/XXXXXXX

    2023

  22. [30]

    In: IEEE/CVF International Conference on Computer Vision

    Jang, J., Park, J., Kim, J., Kwon, H., Sohn, K.: Knowing where to focus: Event- aware transformer for video grounding. In: IEEE/CVF International Conference on Computer Vision. pp. 13846–13856 (2023)

    2023

  23. [31]

    In: ECCV Workshops / arXiv preprint (2025), https://www.researchgate.net/publication/389540473

    Katona, Z., et al.: MARINE: A computer vision model for detecting rare predator– prey interactions in animal videos. In: ECCV Workshops / arXiv preprint (2025), https://www.researchgate.net/publication/389540473

    2025

  24. [32]

    In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Khan, F.F., Li, X., Temple, A.J., Elhoseiny, M.: Fishnet: A large-scale dataset and benchmark for fish recognition, detection, and functional trait prediction. In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 20496–20506 (October 2023)

    2023

  25. [33]

    arXiv preprint arXiv:2509.18802 (2025) 18 A

    Kim, G., Jeong, T.K., Park, J.: Surgical video understanding with label interpola- tion. arXiv preprint arXiv:2509.18802 (2025) 18 A. To et al

    2025

  26. [34]

    arXiv preprint arXiv:2511.07923 (2025)

    Li, B., Huo, T., Zhang, D., Zhao, Z., Gao, J., Li, X.: Exploring the underwater world segmentation without extra training. arXiv preprint arXiv:2511.07923 (2025)

    2025

  27. [35]

    Li, P., Lu, Y., Song, P., Li, W., Yao, H., Xiong, H.: Eventvl: Understand event streams via multimodal large language model (2025),https://arxiv.org/abs/ 2501.13707

    2025

  28. [36]

    Li, P., Abdullaeva, I., Gambashidze, A., Kuznetsov, A., Oseledets, I.: Maxinfo: A training-free key-frame selection method using maximum volume for enhanced video understanding (2025),https://arxiv.org/abs/2502.03183

    2025

  29. [37]

    Li, Y., Yang, X., Xu, D., Yu, Y., Zhao, L., Hu, X., Li, J., Heng, P.A.: Surgpub- video: A comprehensive surgical video dataset for enhanced surgical intelligence in vision-language model (2025),https://arxiv.org/abs/2508.10054

    2025

  30. [38]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Li, Y., Wang, H., Ding, X., Wang, H., Li, X.: Token activation map to visually ex- plain multimodal llms. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 48–58 (October 2025)

    2025

  31. [39]

    In: IEEE/CVF International Conference on Computer Vision (ICCV)

    Lian, S., Li, H., Cong, R., Li, S., Zhang, W., Kwong, S.: Watermask: Instance segmentation for underwater imagery. In: IEEE/CVF International Conference on Computer Vision (ICCV). pp. 1305–1315 (2023)

    2023

  32. [40]

    Lin, B., Ye, Y., Zhu, B., Cui, J., Ning, M., Jin, P., Yuan, L.: Video-llava: Learn- ing united visual representation by alignment before projection (2024),https: //arxiv.org/abs/2311.10122

    2024 arXiv

  33. [41]

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning (2023)

    2023

  34. [42]

    In: Proceed- ings of the Computer Vision and Pattern Recognition Conference

    Liu, S., Li, J., Zhao, G., Zhang, Y., Meng, X., Yu, F.R., Ji, X., Li, M.: Eventgpt: Event stream understanding with multimodal large language models. In: Proceed- ings of the Computer Vision and Pattern Recognition Conference. pp. 29139–29149 (2025)

    2025

  35. [43]

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization (2019),https: //arxiv.org/abs/1711.05101

    2019 arXiv

  36. [44]

    Frontiers in Environmental Science (2026),https:// www.frontiersin.org/journals/environmental- science/articles/10.3389/ fenvs.2025.1689855/full

    Minglong, W., et al.: A machine learning-driven framework for enhancing under- water ecological monitoring. Frontiers in Environmental Science (2026),https:// www.frontiersin.org/journals/environmental- science/articles/10.3389/ fenvs.2025.1689855/full

    2026

  37. [45]

    Mukherjee, R., Singh, S., McWilliams, J., Sattar, J.: The common objects un- derwater (cou) dataset for robust underwater object detection (02 2025).https: //doi.org/10.48550/arXiv.2502.20651

    2025 doi arXiv

  38. [46]

    Murphy, K.: Reinforcement learning: An overview (2025),https://arxiv.org/ abs/2412.05265

    2025

  39. [47]

    Pantazis, O.: Data-Efficient Computer Vision for Biodiversity Monitoring. Ph.D. thesis, University College London (2023),https://discovery.ucl.ac.uk/id/ eprint/10200299/

    2023

  40. [48]

    In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://arxiv.org/abs/2305.18290

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Di- rect preference optimization: Your language model is secretly a reward model. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://arxiv.org/abs/2305.18290

    2023 arXiv

  41. [49]

    In: IEEE International Conference on Computer Vision

    Ramanathan, V., Liang, P., Fei-Fei, L.: Video event understanding using natural language descriptions. In: IEEE International Conference on Computer Vision. pp. 905–912 (2013)

    2013

  42. [50]

    Sanders, K., Van Durme, B.: A survey of video datasets for grounded event under- standing.In:IEEE/CVFConferenceonComputerVisionandPatternRecognition. pp. 7314–7327 (2024) MarineEVT 19

    2024

  43. [51]

    IEEE Journal of Oceanic Engineering47(4), 959–974 (2022)

    Shi, Z., Guan, C., Li, Q., Liang, J., Cao, L., Zheng, H., Gu, Z., Zheng, B.: Detecting marineorganismsviajointattention-relationlearningformarinevideosurveillance. IEEE Journal of Oceanic Engineering47(4), 959–974 (2022)

    2022

  44. [52]

    Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recog- nition in videos (2014),https://arxiv.org/abs/1406.2199

    2014 arXiv

  45. [53]

    Singh, A., et al.: Openai gpt-5 system card (2025),https://arxiv.org/abs/2601. 03267

    2025

  46. [54]

    arXiv preprint arXiv:1906.03363 (2019)

    Souček, T., Moravec, J., Lokoč, J.: Transnet: A deep network for fast detection of common shot transitions. arXiv preprint arXiv:1906.03363 (2019)

    1906 arXiv

  47. [55]

    Tang, X., Qiu, J., Xie, L., Tian, Y., Jiao, J., Ye, Q.: Adaptive keyframe sampling for long video understanding (2025),https://arxiv.org/abs/2502.21271

    2025 arXiv

  48. [56]

    Tang, Y.Y., Bi, J., Xu, S., Song, L., Liang, S., Wang, T., Zhang, D., An, J., Lin, J., Zhu, R., Vosoughi, A., Huang, C., Zhang, Z., Liu, P., Feng, M., Zheng, F., Zhang, J., Luo, P., Luo, J., Xu, C.: Video understanding with large language models: A survey (2025),https://arxiv....

    2025

  49. [57]

    Team, G.: Gemini: A family of highly capable multimodal models (2025),https: //arxiv.org/abs/2312.11805

    2025 arXiv

  50. [58]

    arXiv preprint arXiv:2409.12191 (2024)

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., Fan, Y., Dang, K., Du, M., Ren, X., Men, R., Liu, D., Zhou, C., Zhou, J., Lin, J.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint...

    2024 arXiv

  51. [59]

    Wang, S., Chen, G., an Huang, D., Li, Z., Li, M., Liu, G., Alvarez, J.M., Zhang, L., Yu, Z.: Videoitg: Multimodal video understanding with instructed temporal grounding (2026),https://arxiv.org/abs/2507.13353

    2026

  52. [60]

    arXiv preprint arXiv:2508.18265 (2025)

    Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., et al.: Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025)

    2025 arXiv

  53. [61]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, X., Wang, S., Tang, C., Zhu, L., Jiang, B., Tian, Y., Tang, J.: Event stream- based visual object tracking: A high-resolution benchmark dataset and a novel baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19248–19257 (2024)

    2024

  54. [62]

    arXiv preprint arXiv:2307.06942 (2023)

    Wang, Y., He, Y., Li, Y., Li, K., Yu, J., Ma, X., Li, X., Chen, G., Chen, X., Wang, Y., et al.: Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942 (2023)

    2023 arXiv

  55. [63]

    arXiv:2412.18605 (2024)

    Wang, Z., Zhang, Z., Pang, T., Du, C., Zhao, H., Zhao, Z.: Orient any- thing: Learning robust object orientation estimation from rendering 3d models. arXiv:2412.18605 (2024)

    2024 arXiv

  56. [64]

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., Zhou, D.: Chain-of-thought prompting elicits reasoning in large language models (2023),https://arxiv.org/abs/2201.11903

    2023 arXiv

  57. [65]

    arXiv preprint arXiv:2508.18729 (2025)

    Wille, M., Fischer, T., Raine, S.: Are all marine species created equal? performance disparities in underwater object detection. arXiv preprint arXiv:2508.18729 (2025)

    2025

  58. [66]

    Nature ecology & evolution4(11), 1451–1458 (2020)

    Winther, J.G., Dai, M., Rist, T., Hoel, A.H., Li, Y., Trice, A., Morrissey, K., Juinio- Meñez, M.A., Fernandes, L., Unger, S., et al.: Integrated ocean management for a sustainable ocean economy. Nature ecology & evolution4(11), 1451–1458 (2020)

    2020

  59. [67]

    arXiv preprint arXiv:2512.21150 (2025)

    Wong, Y.K., Liang, H., Ma, Z., Chen, Y., Zheng, Z., Gotama, R., Sebastian, P., Sparks, L.D., Yeung, S.K.: Orca: Object recognition and comprehension for archiv- ing marine species. arXiv preprint arXiv:2512.21150 (2025)

    2025

  60. [68]

    arXiv preprint arXiv:2512.21126 (2025) 20 A

    Wong, Y.K., To, T.A., Zhang, J., Zheng, Z., Yeung, S.K.: Marineeval: Assessing the marine intelligence of vision-language models. arXiv preprint arXiv:2512.21126 (2025) 20 A. To et al

    2025

  61. [69]

    Wu, M., Yang, J., Jiang, J., Li, M., Yan, K., Yu, H., Zhang, M., Zhai, C., Nahrst- edt, K.: Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use (2025),https://arxiv.org/abs/2505.19255

    2025

  62. [70]

    xAI: Grok 4 - xai.https://x.ai/news/grok-4(July 2025), accessed: 2026-02-27

    2025

  63. [71]

    In: IEEE/CVF conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Xie, Y., Kong, L., Chen, K., Zheng, Z., Yu, X., Yu, Z., Zheng, B.: Uveb: A large- scale benchmark and baseline towards real-world underwater video enhancement. In: IEEE/CVF conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    2024

  64. [72]

    In: Moens, M.F., Huang, X., Specia, L., Yih, S.W.t

    Xu, H., Ghosh, G., Huang, P.Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettle- moyer, L., Feichtenhofer, C.: VideoCLIP: Contrastive pre-training for zero-shot video-text understanding. In: Moens, M.F., Huang, X., Specia, L., Yih, S.W.t. (eds.) Proceedings of the 2021 Conferen...

    2021 doi

  65. [73]

    arXiv preprint arXiv:2510.27481 (2025)

    Xu, W., Wang, C., Liang, D., Zhao, Z., Jiang, X., Zhang, P., Bai, X.: Nautilus: A large multimodal model for underwater scene understanding. arXiv preprint arXiv:2510.27481 (2025)

    2025

  66. [74]

    Xue, X., Zhou, Y., Yan, D., Tao, L., Li, J., Li, Y., Zhang, H., Xiao, R.: Uvlm: Benchmarking video language model for underwater world understanding (2025), https://arxiv.org/abs/2507.02373

    2025

  67. [75]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Yang, C., Dong, X., Zhu, X., Su, W., Wang, J., Tian, H., Chen, Z., Wang, W., Lu, L., Dai, J.: Pvc: Progressive visual token compression for unified image and video processing in large vision-language models. In: Proceedings of the Computer Vision and Pattern Recognition Confer...

    2025

  68. [76]

    arXiv:2406.09414 (2024)

    Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. arXiv:2406.09414 (2024)

    2024 arXiv

  69. [77]

    arXiv preprint arXiv:2412.04467 (2024)

    Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., Jia, J.: Visionzip: Longer is better but not necessary in vision language models. arXiv preprint arXiv:2412.04467 (2024)

    2024

  70. [78]

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., Liu, X., Lin, H., Lin, Z., Ma, B., Sheng, G., Tong, Y., Zhang, C., Zhang, M., Zhang, W., Zhu, H., Zhu, J., Chen, J., Chen, J., Wang, C., Yu, H., Song, Y., Wei, X., Zhou, H., Liu, J., Ma...

    2025 arXiv

  71. [79]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Zang, C., Wang, H., Pei, M., Liang, W.: Discovering the real association: Mul- timodal causal reasoning in video question answering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 19027–19036 (June 2023)

    2023

  72. [80]

    CoRRabs/2106.02636 (2021),https://arxiv.org/abs/2106.02636

    Zellers, R., Lu, X., Hessel, J., Yu, Y., Park, J.S., Cao, J., Farhadi, A., Choi, Y.: MERLOT: multimodal neural script knowledge models. CoRRabs/2106.02636 (2021),https://arxiv.org/abs/2106.02636

    2021 arXiv

  73. [81]

    Zeng, Z., Zhuo, Z., Jia, X., Zhang, E., Wu, J., Zhang, J., Wang, Y., Low, C.H., Jiang, J., Zheng, Z., Cao, X., Ban, Y., Dou, Q., Liu, Y., Jin, Y.: Surgvlm: A large vision-language model and systematic evaluation benchmark for surgical intelli- gence (2025),https://arxiv.org/ab...

    2025 arXiv

  74. [82]

    Zhang, C., Liu, L., Huang, G., Wen, H., Zhou, X., Wang, Y.: Webuot-1m: Ad- vancing deep underwater object tracking with a million-scale benchmark (2024), https://arxiv.org/abs/2405.19818 MarineEVT 21

    2024 arXiv

  75. [83]

    Zhang, D., Rong, C., Li, B., Wang, F., Zhao, Z., Gao, J., Li, X.: Uwbench: A comprehensive vision-language benchmark for underwater understanding (2025), https://arxiv.org/abs/2510.18262

    2025

  76. [84]

    Zhang, P., Yan, T., Liu, Y., Lu, H.: Fantastic animals and where to find them: Segment any marine animal with dual sam (2024),https://arxiv.org/abs/2404. 04996

    2024

  77. [85]

    Zhang, Y., Li, B., Liu, h., Lee, Y.j., Gui, L., Fu, D., Feng, J., Liu, Z., Li, C.: Llava- next: A strong zero-shot video understanding model (April 2024),https://llava- vl.github.io/blog/2024-04-30-llava-next-video/

    2024

  78. [86]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Zhao, Y., Zhang, H., Xie, L., Hu, T., Gan, G., Long, Y., Hu, Z., Chen, W., Li, C., Xu, Z., et al.: Mmvu: Measuring expert-level multi-discipline video understanding. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 8475–8489 (2025)

    2025

  79. [87]

    Zheng, C., Liu, S., Li, M., Chen, X.H., Yu, B., Gao, C., Dang, K., Liu, Y., Men, R., Yang, A., Zhou, J., Lin, J.: Group sequence policy optimization (2025),https: //arxiv.org/abs/2507.18071

    2025 arXiv

  80. [88]

    ECCV (2024)

    Zheng, Z., Chen, Y., Zeng, H., Vu, T.A., Hua, B.S., Yeung, S.K.: Marineinst: A foundation model for marine image analysis with instance visual description. ECCV (2024)

    2024

  81. [89]

    Zheng, Z., Zhang, J., Vu, T.A., Diao, S., Tim, Y.H.W., Yeung, S.K.: Marinegpt: Unlocking secrets of ocean to the public (2023),https://arxiv.org/abs/2310. 13596

    2023

  82. [90]

    In: IEEE/CVF Winter Conference on Applications of Com- puter Vision (WACV)

    Zhong, J., Li, M., Zhang, H., Qin, J.: Combining photogrammetric computer vi- sion and semantic segmentation for fine-grained understanding of coral reef growth under climate change. In: IEEE/CVF Winter Conference on Applications of Com- puter Vision (WACV). pp. 186–195 (2023)

    2023

  83. [91]

    arXiv preprint arXiv:2504.10479 (2025)

    Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al.: Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 (2025)

    2025 arXiv

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.