Pith. sign in

REVIEW 4 major objections 6 minor 42 references

SV3.3B: A Sports Video Understanding Model for Action Recognition

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 3.3B-parameter sports video model claims to outscore GPT-4o at describing basketball plays.

desk verdict A credible system with a broken comparison: the released model and data are useful, but the 'beats GPT-4o' claim is unsupported as stated. read the letter →

arxiv 2507.17844 v1 pith:UYRQJAXR submitted 2025-07-23 cs.CV cs.AI

Sai Varun Kodathala , Yashwanth Reddy Vutukoori , Rakesh Vunnam This is my paper
classification cs.CVcs.AI
keywords sportsvideounderstandingactionrecognitionkeyframeextractionself-supervisedlearningJEPA2DWT-VGG16-LDAsamplingmultimodalLLMbasketballdescription
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SV3.3B is a 3.3B-parameter vision-language model built specifically to describe sports actions in fine-grained, technically detailed language. The paper claims that, on a 1,315-clip subset of the NSVA basketball dataset, SV3.3B outperforms larger closed-source models including GPT-4o variants on both traditional text-generation metrics and sports-specific analytics metrics, achieving a 29.2% higher Ground Truth Validation Score. The authors argue that the key is not scale but temporal sampling: a DWT-VGG16-LDA pipeline selects 16 keyframes that capture biomechanical phases such as preparation, execution, and follow-through, which uniform sampling misses. If right, the result would mean that specialized, efficient models can displace much larger general-purpose systems for professional-grade sports analysis on ordinary devices.

What carries the argument

The machinery is the DWT-VGG16-LDA keyframe extraction combined with the V-DWT-JEPA2 encoder. The sampler computes Haar wavelet approximation coefficients at level 2, feeds appearance frames and wavelet-difference motion maps through dual VGG-16 streams, fuses the features, clusters them with K-means, projects with Linear Discriminant Analysis, and selects the frame nearest each cluster center as one of 16 keyframes. These keyframes represent distinct phases of an athletic movement and are the input to a ViT-L encoder trained by JEPA2 mask-denoising, whose frozen representations are mapped by an MLP into the LLaMA-3.2-3B embedding space for description generation.

What would settle it

Fine-tune a GPT-class baseline on the exact same 1,050 training clips, feed it the same DWT-VGG16-LDA keyframes, and evaluate with human expert ratings on the 265 validation clips; if the SV3.3B advantage disappears or reverses, the claim of superiority collapses. A second decisive check is testing whether any of the 12,000 pretraining clip-views overlap the validation videos.

Watch

Extended reading notes

Core claim

The central claim is that domain-specialization plus principled temporal sampling can beat raw scale for sports video understanding. Concretely, SV3.3B uses a frozen V-DWT-JEPA2 video encoder (a 300M ViT-L pretrained with mask-denoising on unlabeled sports clips) feeding a LLaMA-3.2-3B decoder fine-tuned with LoRA, with the DWT-VGG16-LDA keyframe sampler choosing the 16 most representative frames. On the NSVA basketball subset, the paper reports a Ground Truth Validation Score of 2.123 versus 1.643 for GPT-4o, and a Combined Performance Score of 162.821 versus 88.956, with large Cohen's d effect sizes. The authors interpret this as evidence that capturing multi-phase athletic motion through keyframe selection is more important than model size, and that self-supervised pretraining plus parameter-efficient adaptation can produce coach-level descriptions affordably.

Load-bearing premise

The central comparison is only meaningful if evaluating closed-source baselines zero-shot with uniform sampling against SV3.3B fine-tuned on the same NSVA distribution, and scoring with author-defined metrics without human validation, is a fair test of the claimed 29.2% advantage.

Editorial extensions

If this is right

  • A domain-specialized 3.3B model can match or beat much larger closed models on a narrow, high-stakes video-understanding task, under the paper's evaluation setup.
  • The DWT-VGG16-LDA sampler is the load-bearing component: with 16 keyframes per clip, the encoder sees the full biomechanical phase structure instead of uniformly subsampled frames.
  • Self-supervised JEPA2 pretraining on unlabeled sports clips plus LoRA fine-tuning of a 3B LLM is sufficient to produce dense, analytically rich descriptions, avoiding the cost of full large-model training.
  • The framework is positioned for on-device use, with frozen encoder and parameter-efficient adaptation keeping inference cost low enough for amateur coaching environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper does not run is to give the same DWT-VGG16-LDA keyframes to the closed-source baselines; that would isolate whether the advantage comes from the sampling or from the V-DWT-JEPA2 encoder.
  • The generalizability claim rests entirely on a single basketball dataset; whether the 16-keyframe budget is right for slower or faster sports is an open question.
  • The author-defined richness metrics reward verbosity and technical vocabulary, so scores favoring SV3.3B may partly reflect style differences rather than factual accuracy; human expert evaluation would be needed to confirm coaching value.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SV3.3B, a 3.3B-parameter video understanding model for sports action description, combining a DWT-VGG16-LDA keyframe sampler, a V-DWT-JEPA2 pretrained ViT-L encoder, and a LoRA-fine-tuned LLaMA-3.2-3B decoder. The model is evaluated on a 1,315-clip subset of the NSVA basketball dataset, with 1,050 training and 265 validation clips. The paper claims that SV3.3B outperforms larger closed-source models including GPT-4o variants on traditional text-generation metrics and author-defined sports-specific richness metrics, achieving a 29.2% improvement in Ground Truth Validation Score over GPT-4o and reporting a Combined Performance Score of 162.821 versus 88.956 for GPT-4o as printed in Table III.

Significance. If the headline comparison were supported by a controlled, validated evaluation, a 3.3B parameter model with 300M-parameter video encoder and efficient keyframe sampling would be a practically useful lightweight alternative to much larger closed video-language models for sports description, especially for on-device deployment. The paper has concrete strengths: the architecture is described in enough detail to be reproduced, the training-loss curves are reported, the model is released on Hugging Face, and the problem domain is well motivated. However, the current evidence does not establish the central claim: the baseline comparison is structurally asymmetric, the reported table values are internally inconsistent, and the primary metrics are author-defined without human validation or standard-benchmark grounding.

major comments (4)
  1. [Section IV-E and Tables I-III] The comparison is uncontrolled and cannot support the superiority claim. SV3.3B is fine-tuned on 1,050 clips from the same NSVA distribution and annotation format as the 265-clip validation set (Section IV-A), while the GPT baselines are deployed zero-shot with uniformly sampled frames (Section IV-E). The reported 29.2% Ground Truth improvement (2.123 vs. 1.643, Table I) and the combined-score margins therefore conflate model capability with a training-data advantage. To substantiate the claim, the authors must either fine-tune or otherwise adapt an open-weight baseline on the identical training split with the same annotation format, or explicitly and honestly reframe the comparison as 'fine-tuned domain model vs. zero-shot general baselines.'
  2. [Table III] The baseline rows in Table III are permuted relative to Tables I and II. The GT Validation Score printed for GPT-o4 Mini (1.6431) is Table I's value for GPT-4o; the value printed for GPT-4o Mini (1.7134) is Table I's value for GPT-o4 Mini; and the value printed for GPT-4o (1.3056) is Table I's value for GPT-4o Mini. The Information Richness values are shifted in the same way. After correcting the row labels, the combined-score margin over GPT-4o is approximately 25.1 points (162.821 vs. about 137.68), not the 73.9-point margin implied by the printed rows. This must be corrected and all numerical claims in the abstract and Section IV-D revisited.
  3. [Section IV-C and Table II] The sports-specific metrics are author-defined and have no human validation or link to any established benchmark. In Table II, Vocabulary Richness, a raw, length-sensitive count (151.6 for SV3.3B), contributes about 151.6 of the 160.7 Information Richness points, so the Information Richness ranking largely reflects output length or lexical variety rather than analytical correctness or density. The authors should (a) normalize Vocabulary Richness by response length or report per-token values, (b) validate the composite scores against human ratings on a sample of the 265 validation clips, and (c) report the correlation between output length and each metric to rule out length-driven confounds.
  4. [Section III-C(a) and Section IV-A] The encoder pretraining corpus is never specified, and the pretraining budget creates a data-leakage risk that is not addressed. Section III-C(a) states 3,000 training iterations at batch size 4, i.e., 12,000 clip-views, which is an order of magnitude larger than the 1,315-clip dataset. The paper does not state where the unlabeled pretraining videos came from or whether they overlap with the 265 validation clips. The authors must disclose the pretraining source, describe any overlap filtering, and confirm that the validation clips were excluded from pretraining.
minor comments (6)
  1. [Section IV-A] The citation for the original NSVA dataset appears as '[reference]' and should be replaced with a proper bibliographic entry; reference [42] is only the Hugging Face subset.
  2. [Section IV-E] The prompt template, number of uniformly sampled frames, and any few-shot examples used for the GPT baselines are not described; this information is necessary for reproducibility and for assessing the fairness of the zero-shot condition.
  3. [Section III-B and Figures 3-5] The paper asserts that DWT-VGG16-LDA keyframe sampling 'significantly outperformed' uniform sampling and color histogram methods, but Figures 3-5 are anecdotal visualizations and no quantitative comparison is provided for those claims.
  4. [Section III-B] The choice of K=16 keyframes is presented as aligning with the ViT-L input, but no ablation varying K is reported, so the sensitivity of the results to this central hyperparameter is unknown.
  5. [Tables I-II] The model name 'GPT-o4 Mini' appears in Tables I and II while 'GPT-4o Mini' appears in Table III and elsewhere; this inconsistency should be resolved.
  6. [Table IV] The column header 'GT Score Validation' is confusing; it appears to be the difference in Ground Truth Validation Score between models, which should be labeled clearly as a difference.

Circularity Check

1 steps flagged · score 4.0 of 10

Superiority claim reduces to a fine-tuned-versus-zero-shot comparison rather than to the proposed architecture.

  1. fitted input called prediction [Section IV-A (Dataset) and Section IV-E (Comparison to State-Of-The-Art)]
    "For our SV3.3B training, we created a subset of NSVA [42] due to computational resource constraints, resulting in 1,315 video clips split into 1,050 training samples and 265 validation samples. ... To evaluate the model's performance, baseline comparisons were conducted using GPT-o4 Mini, GPT-4o Mini, and GPT-4o models with uniformly sampled video frames, while our SV3.3B model utilized the proposed DWT-based temporal sampling approach."

    The headline claim of 29.2% improvement over GPT-4o (GT Validation Score 2.123 vs 1.643) and the combined-score margin (162.821 vs 88.956) are produced by comparing a model fine-tuned on 1,050 NSVA clips, drawn from the same distribution and annotation format as the 265-clip validation set, against closed baselines that are described as zero-shot with uniformly sampled frames and no fine-tuning. The reported superiority is therefore forced by the training-status asymmetry: the result is the difference between a model fitted to the target data distribution and models excluded from training.

full rationale

No equation-level circularity is present: the text targets are external NBA play-by-play annotations, and no metric is directly fitted to a parameter. The central circularity is at the evaluation-design level. SV3.3B is fine-tuned on 1,050 clips from the same NSVA distribution and annotation format used for the 265-clip validation set, while the GPT baselines are deployed zero-shot with uniform sampling (Section IV-A and IV-E). The headline superiority margin therefore reduces, by construction, to a trained-versus-untrained comparison rather than providing evidence for the proposed architecture or sampling method. Additionally, the Information Richness Score is dominated by an unnormalized Vocabulary Richness component (about 151.6 of the 160.7 points for SV3.3B in Table II), so 'analytically rich' largely reflects vocabulary count or response length rather than validated correctness; and Table III permutes baseline rows relative to Tables I and II, which weakens the numerical presentation. These are partly correctness and validity concerns, but they make the central superiority claim an artifact of the setup to a substantial degree. The model still appears to lead even after correcting the row permutation, so the circularity is partial rather than total (score 4). The missing dataset citation '[reference]' in Section IV-A and the self-published NSVA subset [42] further limit external validation but are not by themselves circular.

Assumptions & free parameters 7 free parameters · 5 assumptions · 2 invented entities

The central claim depends on hand-chosen system constants (16 keyframes, DWT level 2, masking scales, LoRA rank), on unstated data choices (the pretraining corpus, the subset construction), and on domain assumptions about what features and self-supervised objectives suffice for sports understanding. The composite evaluation metrics are equal-weight sums of components on incompatible scales, which is a design choice that shapes the reported winner.

free parameters (7)
  • Number of keyframes K = 16
    Section III-B sets K to 16 to match the ViT-L input; no ablation over K is reported, and K directly controls how much temporal information the decoder sees.
  • DWT decomposition level and wavelet = L=2, Haar
    Section III-B chooses level 2 with the Haar wavelet 'to preserve fine-grained motion details'; the choice is motivated but never varied or tested.
  • K-means cluster count for pseudo-labels = Unstated
    Section III-B applies K-means to the fused feature space but never states the number of clusters, which must equal the number of selected keyframes; the procedure cannot be reproduced as written.
  • JEPA2 masking configuration = 8 blocks at [0.15,0.15]; 2 blocks at [0.7,0.7]; temporal [1.0,1.0]
    Section III-C(a); hand-chosen masking scales with no ablation, and temporal scale [1.0,1.0] means the 16 selected frames are never masked temporally.
  • Equal-weight composite score sums = Unit weights
    Section IV-C defines Ground Truth Validation and Information Richness Scores as plain sums; the unit weights are never justified and Vocabulary Richness (~151.6) dominates the richness total (~160.7).
  • LoRA rank and learning rate = rank 16, lr 1e-4
    Section III-C(b); standard PEFT hyperparameters reported without ablations; they affect the efficiency claim that anchors the paper's motivation.
  • Encoder pretraining budget = 3,000 iterations at batch size 4 (12,000 clip-views)
    Section III-C(a); this budget exceeds the 1,315-clip dataset, so the pretraining corpus must be the same subset reused, unstated external data, or include validation videos; none is specified.
assumptions (5)
  • domain assumption VGG-16 features over RGB frames and wavelet-LL difference maps capture appearance and motion well enough for representative keyframe selection.
    Invoked in Section III-B; the sufficiency of these hand-picked features is never measured, and the claimed advantage over uniform sampling and color histograms is shown only by example frames.
  • domain assumption JEPA2 mask-denoising pretraining on unlabeled sports video transfers to basketball action description.
    Invoked in Sections I and III-C(a); a standard self-supervised transfer assumption, but the pretraining corpus is never identified and the budget (12,000 clip-views) is tiny relative to V-JEPA2-scale pretraining.
  • domain assumption The NSVA subset ground truth (NBA play-by-play text) is accurate and the 265 validation clips are representative for quality comparison.
    Section IV-A; subset construction criteria are not stated and no human re-check or inter-annotator agreement is reported.
  • standard math Standard tools (DWT, K-means, LDA) are applied correctly and their outputs are used as described.
    Section III-B; the step sequence is plausible but the K-means cluster count and LDA output dimension are unstated, so the procedure cannot be verified as described.
  • ad hoc to paper Validation videos were not included in encoder pretraining.
    Sections III-C(a) and IV-A; pretraining volume (12,000 clip-views) exceeds the 1,315-clip dataset, and the paper neither identifies the pretraining data nor rules out overlap with the 265 validation clips.
invented entities (2)
  • V-DWT-JEPA2 video encoder (300M ViT-L pretrained with JEPA2 mask-denoising) independent evidence
    purpose: Produces 1024-dimensional spatiotemporal representations that the LLaMA decoder converts into action descriptions.
    Weights are released at the HuggingFace URL, so the encoder's outputs can be inspected and compared; however, the claimed benefit over uniform sampling is not independently evidenced in the paper.
  • DWT-VGG16-LDA keyframe sampler
    purpose: Reduces each clip to 16 representative frames said to capture biomechanical phases.
    Described procedurally in Section III-B with example frames only (Figs. 3-5); no code, no quantitative ablation, and no failed-case analysis, so the claimed advantage is not checkable from the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SV3.3B: A Sports Video Understanding Model for Action Recognition." pith.science (2026). https://pith.science/paper/UYRQJAXR

@misc{pith2026250717844,
  author       = {Pith},
  title        = {Pith review of: SV3.3B: A Sports Video Understanding Model for Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UYRQJAXR}},
  note         = {Machine review of arXiv:2507.17844}
}
read the original abstract

This paper addresses the challenge of automated sports video analysis, which has traditionally been limited by computationally intensive models requiring server-side processing and lacking fine-grained understanding of athletic movements. Current approaches struggle to capture the nuanced biomechanical transitions essential for meaningful sports analysis, often missing critical phases like preparation, execution, and follow-through that occur within seconds. To address these limitations, we introduce SV3.3B, a lightweight 3.3B parameter video understanding model that combines novel temporal motion difference sampling with self-supervised learning for efficient on-device deployment. Our approach employs a DWT-VGG16-LDA based keyframe extraction mechanism that intelligently identifies the 16 most representative frames from sports sequences, followed by a V-DWT-JEPA2 encoder pretrained through mask-denoising objectives and an LLM decoder fine-tuned for sports action description generation. Evaluated on a subset of the NSVA basketball dataset, SV3.3B achieves superior performance across both traditional text generation metrics and sports-specific evaluation criteria, outperforming larger closed-source models including GPT-4o variants while maintaining significantly lower computational requirements. Our model demonstrates exceptional capability in generating technically detailed and analytically rich sports descriptions, achieving 29.2% improvement over GPT-4o in ground truth validation metrics, with substantial improvements in information density, action complexity, and measurement precision metrics essential for comprehensive athletic analysis. Model Available at https://huggingface.co/sportsvision/SV3.3B.

Figures

Figures reproduced from arXiv: 2507.17844 by the authors.

Figure 1
Figure 1. SV3.3B Architecture Overview. The framework operates through three main phases: (a) Video Encoder Pre-Training employs DWT-VGG16-LDA sampling to extract 16 keyframes from input video sequences, processes appearance and motion features through dual VGG-16 pathways, applies K-means clustering and LDA for feature fusion, and trains a ViT-L encoder using JEPA2 self-supervised learning; (b) LLM as Text Decoder Post-Train… view at source ↗
Figure 2
Figure 2. PCA Projection of Fused Feature Space with Selected Keyframes. maximizing the separation between the identified clusters while preserving their internal coherence. Within this LDA￾transformed space, the algorithm computes the center of each cluster and selects the frame whose feature representation is closest to each center as a keyframe as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Extracted key frames using DWT-VGG16-LDA [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Extracted key frames using uniform sampling [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Extracted key frames using color histogram [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Training loss curves for pre-training and post-training, showing raw losses (blue) and smoothed trends (red). IV. EMPIRICAL EVALUATION A. Dataset We utilize a subset of the NSVA (NBA Sports Video Analysis) dataset [reference], which was originally collected by leveragi…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 37 canonical work pages

  1. [42]

    NSVA Subset: Basketball Video -Text Dataset,

    Sportsvision, "NSVA Subset: Basketball Video -Text Dataset," Hugging Face, 2025. [Online]. Available: https://huggingface.co/datasets/sportsvision/nsva_subset

    work page 2025

  2. [1]

    Review on wearable technology in sports: Concepts, challenges and opportunities,

    A. Ç. Seçkin, B. Ateş, and M. Seçkin, "Review on wearable technology in sports: Concepts, challenges and opportunities," Applied Sciences, vol. 13, no. 18, p. 10399, 2023, doi: 10.3390/app131810399

    work page doi:10.3390/app131810399 2023

  3. [2]

    Video -ChatGPT: Towards detailed video understanding via large vision and language models,

    M. Maaz, H. Rasheed, S. Khan, and F. S. Khan, "Video -ChatGPT: Towards detailed video understanding via large vision and language models," arXiv preprint arXiv:2306.05424, 2023

    arXiv 2023

  4. [3]

    LLaMA-VID: An image is worth 2 tokens in large language models,

    Y. Li, C. Wang, and J. Jia, "LLaMA-VID: An image is worth 2 tokens in large language models," arXiv preprint arXiv:2311.17043, 2023

    arXiv 2023

  5. [4]

    A path towards autonomous machine intelligence,

    Y. LeCun, "A path towards autonomous machine intelligence," OpenReview, 2022. [Online]. Available: https://openreview.net/pdf?id=BZ5a1r-kVsf

    work page 2022

  6. [5]

    Self-supervised learning from images with a joint- embedding predictive architecture,

    M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, and N. Ballas, "Self-supervised learning from images with a joint- embedding predictive architecture," in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2023, pp. 15619–15629

    work page 2023

  7. [6]

    V -JEPA: Latent video prediction for visual representation learning,

    A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y. LeCun, M. Assran, and N. Ballas, "V -JEPA: Latent video prediction for visual representation learning," OpenReview, 2024. [Online]. Available: https://openreview.net/forum?id=WFYbBOEOtv

    work page 2024

  8. [7]

    UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity

    Y. Fu, R. Anantha, P. Vashisht, J. Cheng, and E. Littwin, "UI -JEPA: Towards Active Perception of User Intent through Onscreen User Activity," arXiv preprint arXiv:2409.04081, 2024

    work page Pith review arXiv 2024

Show all 42 references
  1. [8]

    V -jepa 2: Selfsupervised video models enable understanding, prediction and planning,

    M. Assran, A. Bardes, D. Fan, Q. Garrido, R. Howes, M. Muckley, A. Rizvi, C. Roberts, K. Sinha, A. Zholus, et al., “V -jepa 2: Selfsupervised video models enable understanding, prediction and planning,” arXiv preprint arXiv:2506.09985, 2025

    2025 arXiv

  2. [9]

    Computer vision for sports: Current applications and research topics,

    G. Thomas, R. Gade, T. B. Moeslund, P. Carr, and A. Hilton, "Computer vision for sports: Current applications and research topics," Computer Vision and Image Understanding, vol. 159, pp. 3–18, 2017

    2017

  3. [10]

    Soccernet-v2: A dataset and benchmarks for holistic understanding of broadcast soccer videos,

    A. Deliege, A. Cioppa, S. Giancola, M. J. Seikavandi, J. V. Dueholm, K. Nasrollahi, B. Ghanem, T. B. Moeslund, and M. Van Droogenbroeck, "Soccernet-v2: A dataset and benchmarks for holistic understanding of broadcast soccer videos," in Proc. IEEE Conf. Com puter Vision and Pat...

    2021

  4. [11]

    Soccernet: A scalable dataset for action spotting in soccer videos,

    S. Giancola, M. Amine, T. Dghaily, and B. Ghanem, "Soccernet: A scalable dataset for action spotting in soccer videos," in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2018, pp. 1711 – 1721

    2018

  5. [12]

    Fine-grained action recognition on a novel basketball dataset,

    X. Gu, X. Xue, and F. Wang, "Fine-grained action recognition on a novel basketball dataset," in International Conference on Acoustics, Speech, and Signal Processing, 2020, pp. 2563–2567

    2020

  6. [13]

    Soccernet caption: Dense video captioning for soccer broadcasts commentaries,

    H. Mkhallati, A. Cioppa, S. Giancola, B. Ghanem, and M. Van Droogenbroeck, "Soccernet caption: Dense video captioning for soccer broadcasts commentaries," in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2023, pp. 5074–5085

    2023

  7. [14]

    Sports video captioning via attentive motion representation and group relationship modeling,

    M. Qi, Y. Wang, A. Li, and J. Luo, "Sports video captioning via attentive motion representation and group relationship modeling," IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 8, pp. 2617–2633, 2019

    2019

  8. [15]

    Matchtime: Towards automatic soccer game commentary generation,

    J. Rao, H. Wu, C. Liu, Y. Wang, and W. Xie, "Matchtime: Towards automatic soccer game commentary generation," in Proc. Conf. Empirical Methods in Natural Language Processing, 2024

    2024

  9. [16]

    Knowledge graph supported benchmark and video captioning for basketball,

    Z. Xi, G. Shi, L. Wu, X. Li, J. Yan, L. Wang, and Z. Liu, "Knowledge graph supported benchmark and video captioning for basketball," arXiv preprint arXiv:2401.13888, 2024

    2024 arXiv

  10. [17]

    Fine-grained video captioning for sports narrative,

    H. Yu, S. Cheng, B. Ni, M. Wang, J. Zhang, and X. Yang, "Fine-grained video captioning for sports narrative," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018, pp. 6006–6015

    2018

  11. [18]

    Finegym: A hierarchical video dataset for fine -grained action understanding,

    D. Shao, Y. Zhao, B. Dai, and D. Lin, "Finegym: A hierarchical video dataset for fine -grained action understanding," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2020, pp. 2616–2625

    2020

  12. [19]

    Finediving: A fine - grained dataset for procedure-aware action quality assessment,

    J. Xu, Y. Rao, X. Yu, G. Chen, J. Zhou, and J. Lu, "Finediving: A fine - grained dataset for procedure-aware action quality assessment," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2022, pp. 2949 – 2958

    2022

  13. [20]

    Tacticai: An AI assistant for football tactics,

    Z. Wang, P. Veličković, D. Hennes, N. Tomašev, L. Prince, M. Kaisers, Y. Bachrach, R. Elie, L. K. Wenliang, F. Piccinini, et al., "Tacticai: An AI assistant for football tactics," Nature Communications, vol. 15, no. 1, pp. 1–13, 2024

    2024

  14. [21]

    VARS: Video assistant referee system for automated soccer decision making from multiple views,

    J. Held, A. Cioppa, S. Giancola, A. Hamdi , B. Ghanem, and M. Van Droogenbroeck, "VARS: Video assistant referee system for automated soccer decision making from multiple views," in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2023, pp. 5085 – 5096

    2023

  15. [22]

    X -VARS: Introducing explainability in football refereeing with multimodal large language models,

    J. Held, H. Itani, A. Cioppa, S. Giancola, B. Ghanem, and M. Van Droogenbroeck, "X -VARS: Introducing explainability in football refereeing with multimodal large language models," in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2024, pp. 3267– 3279

    2024

  16. [23]

    Sports-QA: A large -scale video question answering benchmark for complex and professional sports,

    H. Li, A. Deng, Q. Ke, J. Liu, H. Rahmani, Y. Guo, B. Schiele, and C. Chen, "Sports-QA: A large -scale video question answering benchmark for complex and professional sports," arXiv preprint arXiv:2401.01505, 2024

    2024

  17. [24]

    SportQA: A benchmark for sports understanding in large language models,

    H. Xia, Z. Yang, Y. Wang, R. Tracy, Y. Zhao, D. Huang, Z. Chen, Y. Zhu, Y. Wang, and W. Shen, "SportQA: A benchmark for sports understanding in large language models," in Proc. Conf. North American Chapter of the Association for Computational Linguistics, 2024

    2024

  18. [25]

    SportU: A comprehensive sports understanding benchmark for multimodal large language models,

    H. Xia, Z. Yang, J. Zou, R. Tracy, Y. Wang, C. Lu, C. Lai, Y. He, X. Shao, Z. Xie, et al., "SportU: A comprehensive sports understanding benchmark for multimodal large language models," arXiv preprint arXiv:2410.08474, 2024

    2024 arXiv

  19. [26]

    Flamingo: a visual language model for few -shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al., "Flamingo: a visual language model for few -shot learning," in Advances in Neural Information Processing Systems, 2022, pp. 23716–23736

    2022

  20. [27]

    BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, "BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation," in Proc. International Conference on Machine Learning, 2022, pp. 12888 – 12900

    2022

  21. [28]

    BLIP -2: Bootstrapping language- image pre -training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, "BLIP -2: Bootstrapping language- image pre -training with frozen image encoders and large language models," in Proc. International Conference on Machine Learning, 2023, pp. 19730–19742

    2023

  22. [29]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., "Learning transferable visual models from natural language supervision," in Proc. International Conference on Machine Learning, 2021

    2021

  23. [30]

    Sigmoid loss for language image pre -training,

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, "Sigmoid loss for language image pre -training," in Proc. International Conference on Computer Vision, 2023, pp. 11975–11986

    2023

  24. [31]

    MVBench: A comprehensive multi-modal video understanding benchmark,

    K. Li, Y. Wang, Y. He, Y. Li, Y. Wang, Y. Liu, Z. Wang, J. Xu, G. Chen, P. Luo, L. Wang, and Y. Qiao, "MVBench: A comprehensive multi-modal video understanding benchmark," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2024, pp. 22195–22206

    2024

  25. [32]

    Llama -vid: An image is worth 2 tokens in large language models,

    Y. Li, C. Wang, and J. Jia, "Llama -vid: An image is worth 2 tokens in large language models," in Proc. European Conference on Computer Vision, 2024

    2024

  26. [33]

    Video-llama: An instruction-tuned audio- visual language model for video understanding,

    H. Zhang, X. Li, and L. Bing, "Video-llama: An instruction-tuned audio- visual language model for video understanding," in Proc. Conf. Empirical Methods in Natural Language Processing, 2023

    2023

  27. [34]

    Temporal alignment networks for long-term video,

    T. Han, W. Xie, and A. Zisserman, "Temporal alignment networks for long-term video," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2022, pp. 2906–2916

    2022

  28. [35]

    Multi-sentence grounding for long -term instructional video,

    Z. Li, Q. Chen, T. Han, Y. Zhang, Y. Wang, and W. Xie, "Multi-sentence grounding for long -term instructional video," in Proc. European Conference on Computer Vision, 2024.X

    2024

  29. [36]

    Panda- 70M: Captioning 70M videos with multiple cross -modality teachers,

    T.-S. Chen, A. Siarohin, W. Menapace, E. Deyneka, H. -W. Chao, B. E. Jeon, Y. Fang, H.-Y. Lee, J. Ren, M.-H. Yang, and S. Tulyakov, "Panda- 70M: Captioning 70M videos with multiple cross -modality teachers," in Proc. IEEE Conf. Computer Vision and Pattern Rec ognition (CVPR), 2024

    2024

  30. [37]

    Vid2seq: Large -scale pretraining of a visual language model for dense video captioning,

    A. Yang, A. Nagrani, P. H. Seo, A. Miech, J. Pont -Tuset, I. Laptev, J. Sivic, and C. Schmid, "Vid2seq: Large -scale pretraining of a visual language model for dense video captioning," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2023, pp. 10714–10726

    2023

  31. [38]

    Streaming dense video captioning,

    X. Zhou, A. Arnab, S. Buch, S. Yan, A. Myers, X. Xiong, A. Nagrani, and C. Schmid, "Streaming dense video captioning," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2024

    2024

  32. [39]

    Autoad: Movie description in context,

    T. Han, M. Bain, A. Nagrani, G. Varol, W. Xie, and A. Zisserman, "Autoad: Movie description in context," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2023, pp. 18930–18940

    2023

  33. [40]

    Autoad II: The sequel —who, when, and what in movie audio description,

    T. Han, M. Bain, A. Nagrani, G. Varol, W. Xie, and A. Zisserman, "Autoad II: The sequel —who, when, and what in movie audio description," in Proc. International Conference on Computer Vision, 2023, pp. 13645–13655

    2023

  34. [41]

    Autoad III: The prequel —back to the pixels,

    T. Han, M. Bain, A. Nagrani, G. Varol, W. Xie, and A. Zisserman, "Autoad III: The prequel —back to the pixels," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2024, pp. 18164–18174

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.