Pith. sign in

REVIEW 3 major objections 5 minor 44 references

Dense video captioning reaches a new state of the art by weighting frames by importance and retrieving captions from scene-aligned segments.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 05:55 UTC pith:JPJXEMA6

load-bearing objection Solid incremental DVC paper with clean ablations and honest limitations; the ViTT CIDEr inconsistency needs fixing before the SOTA margin is trusted. the 3 major comments →

arxiv 2509.04602 v1 pith:JPJXEMA6 submitted 2025-09-04 cs.CV

Sali4Vid: Saliency-Aware Video Reweighting and Adaptive Caption Retrieval for Dense Video Captioning

MinJu Jeon , Si-Woo Kim , Ye-Chan Kim , HyunGee Kim , Dong-Jin Kim This is my paper
classification cs.CV
keywords dense video captioningsaliency-aware video reweightingtimestamp supervisionadaptive segmentationcaption retrievalYouCook2ViTTvideo-language model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Dense video captioning — localizing every event in a long, untrimmed video and describing each one — has two blind spots in today's end-to-end models: annotated event timestamps are used only to supervise the generated text, so the video encoder is trained as if every frame matters equally, and auxiliary caption retrieval operates on fixed-size clips that can bundle several unrelated events. Sali4Vid claims both blind spots can be fixed with cheap, model-free operations. It turns timestamps into smooth sigmoid weights that rescale video features during training, emphasizing the interior of each annotated event while keeping soft boundary transitions, and it builds retrieval segments by detecting sustained frame-to-frame similarity changes rather than using a fixed window. On the YouCook2 and ViTT instructional-video benchmarks the combined framework beats the previous state of the art by +3.96 and +2.58 CIDEr (a standard caption-quality score), while also improving event localization. The paper's central claim is that explicit temporal supervision on the visual side and semantic alignment in retrieval are complementary, and neither requires changing the underlying sequence-to-sequence architecture.

Core claim

The paper's central claim is that two supervision-shaped interventions jointly lift dense video captioning. Saliency-aware Video Reweighting converts ground-truth event timestamps into a per-frame importance weight — a left sigmoid rising toward the event start multiplied by a right sigmoid falling after the event end, with the max taken across overlapping events — and multiplies frame features by it during training, giving the visual encoder explicit temporal supervision. Semantic-based Adaptive Caption Retrieval cuts the video where accumulated frame-to-frame cosine differences cross an adaptive threshold and retrieves Top-k captions per segment, replacing fixed-size chunks that mix unrela

What carries the argument

Two mechanisms carry the argument. The first is the sigmoid importance weight Wn(i) = Sigmoid(α·(i/T − ts_n/d)) × Sigmoid(α·(te_n/d − i/T)), with the maximum taken across all annotated events: a product of a sigmoid rising as the frame approaches the event start and a sigmoid falling as the frame passes the event end, producing a continuous 'center-skew' mask that up-weights the event interior while keeping soft boundary transitions, with sharpness controlled by α. The second is the adaptive segmenter, which computes the cosine difference between consecutive frames, maintains a running average segment feature, and closes a segment only when the similarity to the incoming frame falls below an

Load-bearing premise

The method assumes that frame-similarity dips align with real event boundaries, so when the visual signal is noisy the retrieved captions can describe the wrong events — a failure the paper's own limitation section admits still occurs.

What would settle it

A concrete check: give segment-level retrieval the ground-truth event timestamps instead of similarity-derived boundaries (an oracle-segmentation version). If oracle segmentation does not clearly beat the similarity-based segmenter, then segmentation is not what drives the gain. A second check uses videos with a static background and hands-only activity, where events change but scenes do not; if the accumulated cosine difference never crosses the adaptive threshold, several events collapse into one segment and retrieval should fall back toward the fixed-size baseline behavior.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • The fixed-size clip-level retrieval of prior memory-augmented models can be dropped: adaptive segmentation beats it on CIDEr and F1 while removing the need to tune the window size per dataset.
  • Retrieval does not hinge on an in-domain datastore: COCO and CC3M captions land within about half a CIDEr point of in-domain captions, so the gain is driven by segment alignment rather than datastore coverage.
  • The reweighting operates only during training, yet it improves the model's inference-time attention inside annotated segments, so timestamp supervision survives as better learned features even though the weights are removed at test time.
  • Captioning and event localization improve together, with the largest localization gain in precision (+4.76 on YouCook2, +2.57 on ViTT), indicating that visual saliency supervision sharpens boundary decisions rather than only improving description quality.
  • Retrieval cost can be cut sharply with almost no loss: a 10% subset of the datastore reaches 74.92 CIDEr at roughly 2 ms per video, versus 75.80 with the full datastore.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the sigmoid product is essentially a soft-edged window, the paper's results suggest a testable hypothesis: any smooth boundary-preserving mask (raised-cosine, trapezoidal) should land close to the sigmoid's +8.43 CIDEr gain, implying the soft boundary is what matters, not the logistic shape itself.
  • The model already consumes speech transcripts with time tokens, so fusing transcript boundary cues with frame-similarity cut points would directly attack the noisy segments the limitation section acknowledges, since the two transition signals fail independently.
  • The reweighting's dependence on timestamps points to a weakly supervised loop: use the model's own predicted boundaries as pseudo-timestamps to bootstrap reweighting on unannotated video, which the paper itself lists as future work.
  • The same pair of interventions should transfer to other long-video language tasks where events are delineated by scene changes — egocentric activity narration or sports commentary — where fixed-size chunking is a known poor fit; the segmentation mechanism is content-agnostic even if the retrieval datastore would need to change.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Sali4Vid, a dense video captioning framework built on Vid2Seq with two additions: (1) sigmoid-based reweighting of video features using ground-truth event timestamps during training, and (2) an adaptive, frame-similarity-based segmentation module for retrieving auxiliary captions from a training-caption datastore at segment level. The authors report state-of-the-art CIDEr scores of 75.80 on YouCook2 and 53.87 on ViTT, with consistent improvements in localization F1 in Table 2. They provide component ablations, hyperparameter sensitivity plots, attention-map analysis, and qualitative examples. The paper is clearly written and the method is simple and reproducible in principle, with code released.

Significance. If the reported numbers hold, the paper makes a modest but useful contribution: it demonstrates that timestamp supervision can be injected on the visual side during training, and that adaptive segmentation improves retrieval-augmented dense video captioning over fixed-size clip retrieval. The ablations are systematic and the design choices are reasonable. The main strengths are the simplicity of the approach, the clear ablation structure, and the release of code. However, the reported ViTT CIDEr value is internally inconsistent (53.32 vs. 53.87), and the main results are single-run without error bars, so the precise SOTA margin and its statistical reliability are not yet established.

major comments (3)
  1. [Section 1 / Table 1 / Section 4.1] The ViTT CIDEr score is reported as 53.32 in the Introduction (with a claimed +2.58 improvement over HiCM2's 51.29), but Table 1 and Section 4.1 report 53.87. Since 51.29 to 53.32 is +2.03, not +2.58, the claimed SOTA margin depends on which number is correct. This must be reconciled, and the released code should reproduce the final number under the official evaluation tool. As written, the central SOTA claim on ViTT is not fully verified.
  2. [Section 4, Tables 1-8] All reported results are from a single run, and hyperparameters alpha, beta, and k are selected on the YouCook2 validation set. The claimed gains over HiCM2, especially the ViTT +2.58 CIDEr and YouCook2 +3.96 CIDEr, have no variance estimates. Please provide multiple seed runs or confidence intervals for the main results, or at minimum report the number of seeds and justify that the margins exceed run-to-run variation.
  3. [Section 3.2, Eq. (7)] Eq. (7) defines D'(i) = 1 - z_cur · x_spat_{i+1}, calling this a semantic difference based on cosine similarity. The dot product equals cosine similarity only if both vectors are unit-normalized. The moving-average update in Eq. (8) does not by itself preserve unit norm, unless features are normalized after averaging. Please specify the normalization step used for z_cur and x_spat, since the adaptive segmentation behavior depends directly on this definition.
minor comments (5)
  1. [Section 2.1] Typo: 'HICM2' should be 'HiCM2' for consistency with the reference list and the rest of the paper.
  2. [Table 2] The HiCM2 row lists F1, Recall, and Precision all equal to 32.51 on YouCook2. This is unusual and should be verified; it may be a copying or formatting error.
  3. [Section 4.1 / Table 10] The text says 'We will revise our manuscript to include this segmentation time report,' but Table 10 already reports the numbers. This phrase appears to be a leftover editing note and should be removed or converted into a direct statement of the measured time.
  4. [Section 4.1] The training time statement '1h 20m total (4m 20s/epoch)' is inconsistent for 20 epochs: 4m20s × 20 = 1h26m40s. Please adjust one of the figures.
  5. [Figures 6-7] The y-axis label 'Precesion' is misspelled; it should be 'Precision'.

Circularity Check

0 steps flagged

No circular derivation: training-time reweighting and retrieval datastore do not leak test labels, and the SOTA claim rests on external benchmarks.

full rationale

The derivation chain is self-contained in the relevant sense. Saliency-aware Video Reweighting (Eqs. 1-5) uses ground-truth timestamps as training supervision, but Section 3.3 states explicitly that 'we perform inference without timestamp supervision and do not apply any weighting to the video features'; the reported captioning/localization numbers are produced at inference on held-out YouCook2 val and ViTT test, so the saliency weights are a training regularizer, not the predicted quantity. Semantic-based Adaptive Caption Retrieval (Eqs. 6-10) segments by cosine similarity of frame features and an adaptive threshold, with no reference to ground-truth captions or timestamps; the datastore is 'constructed from the training captions only,' and evaluation is against held-out annotations, so retrieval is not a disguised replay of the test labels. The method is benchmarked against externally developed systems (Vid2Seq, CM2, HiCM2), and the self-citations that do appear are in the concluding 'readily extended' list, not load-bearing premises. The acknowledged limitation in Section 6 (noisy segments) and the mismatch between the Abstract/Introduction ViTT CIDEr (53.32) and Table 1 (53.87) are correctness/reproducibility concerns, not circularity; no equation reduces to its own input, no fitted parameter is renamed a prediction, and no uniqueness argument is imported from the authors' prior work.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical or conceptual entities are introduced. The method's parameters are all standard hyperparameters. The main assumptions are the reliability of frame-similarity for segmentation and the transfer of training-time reweighting to inference.

free parameters (3)
  • alpha (sigmoid sharpness) = 10.0
    Controls the steepness of the sigmoid reweighting (Eq. 1-3); selected by ablation on YouCook2 validation (Figure 6).
  • beta (adaptive threshold scaling) = 1.0
    Sets the adaptive segmentation threshold tau_adap = mu + beta*sigma (Section 3.2); selected by ablation on validation (Figures 6-7).
  • k (number of retrieved captions per segment) = 10
    Top-k retrieved caption embeddings per segment (Eq. 9); selected by ablation on validation (Table 6).
axioms (4)
  • domain assumption Cosine similarity between consecutive frames is a reliable proxy for semantic event boundaries.
    Section 3.2 uses D(i) = 1 - sim(x_i, x_i+1) to detect scene transitions; the adaptive segmentation relies on this without independent validation, and the Limitation section admits segments can be noisy.
  • domain assumption Training with ground-truth timestamp reweighting produces features that improve inference without reweighting.
    Section 3.3 states no weighting is applied at inference; Section 4 provides attention map analysis (Table 9) as indirect evidence, but the transfer is assumed rather than proven.
  • domain assumption Retrieval from a datastore of training captions provides useful auxiliary semantic guidance at test time.
    Inherited from CM2/HiCM2 (Kim et al. 2024b, 2025a), Section 3.2; the paper assumes retrieved captions improve decoding, and Table 7 shows robustness across datastores.
  • domain assumption The Vid2Seq pretrained model is a valid base and its reproduced baseline numbers are reliable.
    The method builds on Vid2Seq (Section 3 Preliminaries); the comparison in Table 1 relies on the official reproduction of Vid2Seq (dagger).

pith-pipeline@v1.4.0-alltime-deepseek-medium · 13753 in / 10068 out tokens · 87287 ms · 2026-08-05T05:55:31.478592+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Sali4Vid: Saliency-Aware Video Reweighting and Adaptive Caption Retrieval for Dense Video Captioning." pith.science (2026). https://pith.science/paper/JPJXEMA6

@misc{pith2026250904602,
  author       = {Pith},
  title        = {Pith review of: Sali4Vid: Saliency-Aware Video Reweighting and Adaptive Caption Retrieval for Dense Video Captioning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JPJXEMA6}},
  note         = {Machine review of arXiv:2509.04602}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Dense video captioning aims to temporally localize events in video and generate captions for each event. While recent works propose end-to-end models, they suffer from two limitations: (1) applying timestamp supervision only to text while treating all video frames equally, and (2) retrieving captions from fixed-size video chunks, overlooking scene transitions. To address these, we propose Sali4Vid, a simple yet effective saliency-aware framework. We introduce Saliency-aware Video Reweighting, which converts timestamp annotations into sigmoid-based frame importance weights, and Semantic-based Adaptive Caption Retrieval, which segments videos by frame similarity to capture scene transitions and improve caption retrieval. Sali4Vid achieves state-of-the-art results on YouCook2 and ViTT, demonstrating the benefit of jointly improving video weighting and retrieval for dense video captioning

Figures

Figures reproduced from arXiv: 2509.04602 by Dong-Jin Kim, HyunGee Kim, MinJu Jeon, Si-Woo Kim, Ye-Chan Kim.

Figure 1
Figure 1. Figure 1: (a) Previous works incorporate timestamps [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (Left) The previous caption retrieval approach overlooks the scene transition, leading to redundant or [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of our Sali4Vid framework for dense video captioning. Sali4Vid enhances dense video captioning by combining Saliency-Aware Video Reweighting with Semantic-based Adaptive Caption Retrieval. Specifically, we utilize timestamp supervision to softly reweight video features in the training phase and retrieve relevant captions by clustering semantically similar video frames. The reweighted video feature… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of different weights with multiple-timestamps. Unlike Gaussian or binary, our sigmoid-based weight provides continuous importance weights while preserving the start and end boundaries. fel et al., 2020) as both the text encoder and de￾coder. Video frames are extracted at 1 FPS and sub-sampled or padded to a fixed length T = 100. The model is first trained for 10 epochs following Vid2Seq with a l… view at source ↗
Figure 6
Figure 6. Figure 6: Impact of hyper-parameter α for video reweighting and β for semantic-based caption retrieval on captioning performance. 2.5 5.0 7.5 10.0 12.5 15.0 32.0 32.5 33.0 33.5 34.0 F1 0.6 0.8 1.0 1.2 1.4 32.0 32.5 33.0 33.5 34.0 F1 HiCM2 F1, Precesion 34.0 34.5 35.0 35.5 36.0 Precesion 35.0 35.5 36.0 36.5 37.0 Precesion F1 and Precision Scores [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Impact of hyper-parameter α for video reweighting and β for semantic-based caption retrieval on localization performance. range of vision-language modeling tasks (Oh et al., 2024; Kim et al., 2025b; Cha et al., 2025; Kim et al., 2025d,c) beyond video captioning. 6 Limitation Our model, Sali4Vid, achieves state-of-the-art performance on dense video captioning through annotation-based video reweighting and s… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 34 canonical work pages · 4 internal anchors

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

  4. [4]

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European conference on computer vision, pages 213--229. Springer

    work page 2020

  5. [5]

    SeungJu Cha, Kwanyoung Lee, Ye-Chan Kim, Hyunwoo Oh, and Dong-Jin Kim. 2025. Verbdiff: Text-only diffusion models with enhanced interaction awareness. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8041--8050

    work page 2025

  6. [6]

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3558--3568

    work page 2021

  7. [7]

    Shizhe Chen, Jia Chen, Qin Jin, and Alexander Hauptmann. 2017. Video captioning with guidance of multimodal latent topics. In Proceedings of the 25th ACM international conference on Multimedia, pages 1838--1846

    work page 2017

  8. [8]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, and 1 others. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    Pith/arXiv arXiv 2020

  9. [9]

    Xuguang Duan, Wenbing Huang, Chuang Gan, Jingdong Wang, Wenwu Zhu, and Junzhou Huang. 2018. Weakly supervised dense event captioning in videos. Advances in Neural Information Processing Systems, 31

    work page 2018

  10. [10]

    Soichiro Fujita, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Soda: Story oriented dense video captioning evaluation framework. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part VI 16, pages 517--531. Springer

    work page 2020

  11. [11]

    Lianli Gao, Zhao Guo, Hanwang Zhang, Xing Xu, and Heng Tao Shen. 2017. Video captioning with attention-based lstm and semantic consistency. IEEE Transactions on Multimedia, 19(9):2045--2055

    work page 2017

  12. [12]

    Shiping Ge, Qiang Chen, Zhiwei Jiang, Yafeng Yin, Liu Qin, Ziyao Chen, and Qing Gu. 2025. Implicit location-caption alignment via complementary masking for weakly-supervised dense video captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3113--3121

    work page 2025

  13. [13]

    Gabriel Huang, Bo Pang, Zhenhai Zhu, Clara Rivera, and Radu Soricut. 2020. Multimodal pretraining for dense video captioning. arXiv preprint arXiv:2011.11760

    Pith/arXiv arXiv 2020

  14. [14]

    Vladimir Iashin and Esa Rahtu. 2020 a . A better use of audio-visual cues: Dense video captioning with bi-modal transformer. arXiv preprint arXiv:2005.08271

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  15. [15]

    Vladimir Iashin and Esa Rahtu. 2020 b . Multi-modal dense video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 958--959

    work page 2020

  16. [16]

    Dong-Jin Kim, Tae-Hyun Oh, Jinsoo Choi, and In So Kweon. 2024 a . Semi-supervised image captioning by adversarially propagating labeled data. IEEE Access, 12:93580--93592

    work page 2024

  17. [17]

    Minkuk Kim, Hyeon Bae Kim, Jinyoung Moon, Jinwoo Choi, and Seong Tae Kim. 2024 b . Do you remember? dense video captioning with cross-modal memory retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13894--13904

    work page 2024

  18. [18]

    Minkuk Kim, Hyeon Bae Kim, Jinyoung Moon, Jinwoo Choi, and Seong Tae Kim. 2025 a . Hicm ^2 : Hierarchical compact memory modeling for dense video captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 4293--4301

    work page 2025

  19. [19]

    Si-Woo Kim, MinJu Jeon, Ye-Chan Kim, Soeun Lee, Taewhan Kim, and Dong-Jin Kim. 2025 b . Sync: Synthetic image caption dataset refinement with one-to-many mapping for zero-shot image captioning. arXiv preprint arXiv:2507.18616

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  20. [20]

    Taewhan Kim, Soeun Lee, Si-Woo Kim, and Dong-Jin Kim. 2025 c . Vipcap: Retrieval text-based visual prompts for lightweight image captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 4320--4328

    work page 2025

  21. [21]

    Ye-Chan Kim, SeungJu Cha, Si-Woo Kim, Taewhan Kim, and Dong-Jin Kim. 2025 d . Sida: Synthetic image driven zero-shot domain adaptation. arXiv preprint arXiv:2507.18632

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    Giorgos Kordopatis-Zilos, Symeon Papadopoulos, Ioannis Patras, and Ioannis Kompatsiaris. 2019. Visil: Fine-grained spatio-temporal video similarity learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6351--6360

    work page 2019

  23. [23]

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017 a . Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision, pages 706--715

    work page 2017

  24. [24]

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017 b . Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision, pages 706--715

    work page 2017

  25. [25]

    Soeun Lee, Si-Woo Kim, Taewhan Kim, and Dong-Jin Kim. 2024. Ifcap: Image-like retrieval and frequency-based entity filtering for zero-shot captioning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20715--20727

    work page 2024

  26. [26]

    Yehao Li, Ting Yao, Yingwei Pan, Hongyang Chao, and Tao Mei. 2018. Jointly localizing and describing events for dense video captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7492--7500

    work page 2018

  27. [27]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer vision--ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pages 740--755. Springer

    work page 2014

  28. [28]

    Hassan Mkhallati, Anthony Cioppa, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2023. Soccernet-caption: Dense video captioning for soccer broadcasts commentaries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5074--5085

    work page 2023

  29. [29]

    Youngtaek Oh, Jae Won Cho, Dong-Jin Kim, In So Kweon, and Junmo Kim. 2024. Preserving multi-modal capabilities of pre-trained vlms for improving vision-linguistic compositionality. In EMNLP 2024-2024 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pages 19060--19076. Association for Computational Linguistics (ACL)

    work page 2024

  30. [30]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  31. [31]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748--8763. PmLR

    2021

  32. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  33. [33]

    Paul Hongsuck Seo, Arsha Nagrani, Anurag Arnab, and Cordelia Schmid. 2022. End-to-end generative pretraining for multimodal video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17959--17968

    work page 2022

  34. [34]

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566--4575

    2015

  35. [35]

    Bairui Wang, Lin Ma, Wei Zhang, and Wei Liu. 2018. Reconstruction network for video captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7622--7631

    work page 2018

  36. [36]

    Teng Wang, Ruimao Zhang, Zhichao Lu, Feng Zheng, Ran Cheng, and Ping Luo. 2021. End-to-end dense video captioning with parallel decoding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6847--6857

    work page 2021

  37. [37]

    Teng Wang, Huicheng Zheng, and Mingjing Yu. 2020. Dense-captioning events in videos: Sysu submission to activitynet challenge 2020. arXiv preprint arXiv:2006.11693

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  38. [38]

    Yiwei Wei, Shaozu Yuan, Meng Chen, Xin Shen, Longbiao Wang, Lei Shen, and Zhiling Yan. 2023. Mpp-net: multi-perspective perception network for dense video captioning. Neurocomputing, 552:126523

    work page 2023

  39. [39]

    Hao Wu, Huabin Liu, Yu Qiao, and Xiao Sun. 2024. Dibs: Enhancing dense video captioning with unlabeled videos via pseudo boundary enrichment and online refinement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18699--18708

    work page 2024

  40. [40]

    Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. 2023. Vid2seq: Large-scale pretraining of a visual language model for dense video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10714--10726

    work page 2023

  41. [41]

    a henb \

    Yue Zhao, Ishan Misra, Philipp Kr \"a henb \"u hl, and Rohit Girdhar. 2023. Learning video representations from large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6586--6597

    2023

  42. [42]

    Luowei Zhou, Chenliang Xu, and Jason Corso. 2018 a . Towards automatic learning of procedures from web instructional videos. In Proceedings of the AAAI conference on artificial intelligence, volume 32

    work page 2018

  43. [43]

    Luowei Zhou, Yingbo Zhou, Jason J Corso, Richard Socher, and Caiming Xiong. 2018 b . End-to-end dense video captioning with masked transformer. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8739--8748

    work page 2018

  44. [44]

    Xingyi Zhou, Anurag Arnab, Shyamal Buch, Shen Yan, Austin Myers, Xuehan Xiong, Arsha Nagrani, and Cordelia Schmid. 2024. Streaming dense video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18243--18252

    work page 2024