diveXplore 6.0: ITEC's Interactive Video Exploration System at VBS 2022

Andreas Leibetseder; Klaus Schoeffmann

REVIEW 4 major objections 4 minor 18 references

diveXplore 6.0: ITEC's Interactive Video Exploration System at VBS 2022

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that diveXplore 6.0 cures missed known-item searches by sampling one keyframe per second instead of one per ten seconds, and improves map retrieval, adds OCR and speech-to-text, and sharpens temporal search.

desk verdict A short VBS system description that makes plausible but unmeasured claims about video retrieval improvements; useful for the competition community, but it doesn't establish any of its central results. read the letter →

arxiv 2508.20687 v1 pith:CTY3CWRY submitted 2025-08-28 cs.MM

Andreas Leibetseder , Klaus Schoeffmann This is my paper · ORCID

classification cs.MM

keywords interactivevideosearchknown-itemshotsegmentationkeyframesamplingtemporalopticalcharacterrecognitionspeech-to-textretrieval

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

diveXplore 6.0 is an interactive video retrieval system refined for a live search competition. The paper's central claim is that the version 5.0 system failed known-item search tasks because its uniform ten-second shot segmentation skipped over very short target segments, and that sampling keyframes every second instead retrieves those targets. It also claims that map-based retrieval should move beyond simple feature-frequency matching, that OCR and speech-to-text are useful additions, and that temporal search needs to combine full-segment events with single-keyframe features. A sympathetic reader would care because these are concrete, mechanism-level choices: if the granularity change works, short and temporally specific video queries become solvable without changing the deep-feature vocabulary.

What carries the argument

The carrier of the argument is the shot segmentation: the procedure that uniformly divides every source video into consecutive scenes and assigns a representative keyframe to each. In version 5.0 this grid was ten seconds long; in version 6.0 it is one second. That grid determines which units can be retrieved, which keyframes have features extracted, and how map search aggregates feature frequencies across a video. Reducing the interval is the load-bearing change, with alternative map matching, OCR, speech-to-text, and temporal search as supporting additions.

What would settle it

Take the same corpus and run the same set of known-item search tasks under diveXplore 5.0's ten-second segmentation and 6.0's one-second segmentation, keeping the feature set, ranking, and interface fixed; if short-target success rates do not rise when the sampling interval shrinks, the claim fails. A complementary check: for each previously missed short target, verify whether any one-second keyframe actually falls inside the target's true interval and is retrievable by the target's distinguishing features.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that retrieval granularity, not feature quality, is the bottleneck for finding short moments in a large video corpus. With the old ten-second segmentation, a target segment that lasts a few seconds has no keyframe of its own and must be reached through neighboring shots, which often means it is never found. Version 6.0 therefore samples one keyframe per second, making every second a potential retrieval unit, and adds OCR and speech-to-text features that let users search visible text and spoken words. The paper further asserts that map search should not rely solely on how often a feature appears in a video's shots, and that temporal-context s

Load-bearing premise

The argument assumes that the reason some target segments are never found is that the ten-second shot grid skips over short moments, not that the feature vocabulary, ranking, or query interface hides them; if the misses have another cause, one-second sampling will not improve search results.

Editorial extensions

If this is right

Known-item search tasks aimed at segments shorter than roughly ten seconds should become directly retrievable, because the target itself now appears as a keyframe instead of being buried inside a ten-second shot.
The feature index grows about tenfold under one-second sampling, so the system trades storage and query cost for finer recall.
Adding OCR and speech-to-text means queries can match text visible on screen and spoken words, covering cases visual concepts cannot distinguish.
Replacing pure feature-frequency map matching with alternative similarity methods should improve map retrieval for queries where a feature appears in many unrelated shots.
Temporal search that combines full-segment action events with single-keyframe features should locate activities with sharper start and end points than event labels alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's rationale implies a sharper test than it runs: the one-second interval should help exactly the short-segment known-item searches that failed under ten-second sampling, and provide little or no gain for targets much longer than a second. That differential is what would confirm the mechanism.
If granularity is the real cause, a cheaper design could first retrieve coarse shots and then re-sample only the candidate videos at one-second resolution, avoiding a tenfold index increase.
The same finer sampling should also help ad-hoc video search for actions with short temporal extent, though the paper only frames the improvement for known-item search.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You can read this in ten minutes. It's a five-page competition system description from the diveXplore group, reporting what they changed for VBS 2022: one-second shot sampling instead of ten, alternative map matching, OCR and STT, and something they call temporal search. None of these claims are backed by any measurement. That is the whole story.

What's actually new: the specific 6.0 configuration is new, and the paper describes the architecture, interface, and feature set clearly enough for someone in the VBS orbit to understand what was built. The observation that the ten-second segmentation 'seems to greatly impede' KIS tasks is plausible, and reducing the sampling interval to one second is a reasonable engineering response. Credit is due for honesty: the paper uses hedged language and doesn't present unverified guesses as findings.

The soft spot is not subtle. Table 2 lists '1s segmentation' and 'temporal search' as improvements, but no evaluation is reported. The single rank from VBS2021 (12/18) has no context. The load-bearing assumption in Section 2.3 is that the sampling interval, not the features or ranking, caused the missed KIS items; the paper gives no evidence for that. Also, 'temporal search' in Table 2 is never actually described in Section 2.3, so there is no mechanism to evaluate. A one-second interval will multiply the number of candidate shots by an order of magnitude, which could hurt precision and response time just as easily as it helps recall; nobody can tell from this text.

This is a system description for a competition, not a research result. For someone building a VBS system, it's a useful data point about what one experienced team tried. For anyone expecting a finding, there is nothing here.

I would send it to a short-paper or workshop track if the venue expects system descriptions, and ask the authors to include even a minimal report of VBS2022 outcomes or a small ablation. As a journal contribution it would need real evaluation. Worth a serious referee for the workshop; not because it proves anything, but because VBS system descriptions are a real genre and this one is clear and honest.

Referee Report

4 major / 4 minor

Summary. The paper describes diveXplore 6.0, ITEC's interactive video retrieval system for the VBS 2022 competition. It presents the system's modular web-based architecture, two search modes (shot and map search), the set of deep features (Inception v3 concepts, YOLO objects, Moments in Time events, Places), and the user interface. The main contribution is Section 2.3 and Table 2, which list four improvements over diveXplore 5.0: reducing the keyframe sampling interval to one second, an alternative map-search mechanism, adding OCR/STT, and temporal context search. The paper contains no task-level evaluation or quantitative comparison with the previous version; the improvement claims are expressed in hedged, qualitative terms.

Significance. If the claims were substantiated, the paper would provide a useful data point for interactive video retrieval: the move to 1s shot sampling and the integration of temporal context features could inform other VBS systems. The paper also offers a clear description of a deployed, working interactive search system, including UI details and feature choices. However, none of the improvements are validated: there are no ablation results, no offline retrieval experiments, and no VBS 2022 outcome data. The paper is therefore best regarded as a system description with design hypotheses, not as an evaluation-driven contribution. The authors are transparent about the VBS 2021 rank (12/18), which is a strength, but the significance of the proposed changes remains unknown.

major comments (4)

[§2.3, Table 2] The central claims in Table 2—especially '1s segmentation ... Improving KIS performance'—are not supported by any evaluation. The only evidence cited is the hedged observation in §2.3 that 'the rather long 10s shot segmentation seems to greatly impede KIS tasks.' No VBS task-level results, no offline KIS experiment, and no comparison of diveXplore 5.0 vs. 6.0 are reported. As written, the improvement is a design hypothesis, not a finding.
[§2.3, Table 2] Even if one accepts that KIS misses occur, the paper does not establish the sampling interval as the cause. Missed targets could be due to non-discriminative features (Inception/YOLO/Places), to ranking that buries short segments, or to the map/shot interface. Moreover, reducing the interval from 10s to 1s multiplies the number of shots by about ten, which can hurt precision and increase browsing time. The paper should either provide an ablation isolating the sampling interval or qualify the claim as a hypothesis to be tested at VBS 2022.
[§2.3, Table 2] Two of the four listed improvements—'alternative map search' and 'temporal search'—are not actually described. The text says only 'we explore alternative ways to match those features' and 'the system requires more sophisticated means for temporal context search, which also will include single keyframe-based features.' No retrieval mechanism is given, and no precision/recall measurement supports 'Improving result precision.' Please specify the algorithms and provide at least a baseline comparison, or remove these entries from the table.
[Abstract and §1] The Abstract states that the v5.0 refactoring 'proved to be a sensible decision' because 'the new system showed increasing performance in VBS2021,' but §1 reports only the absolute rank 12/18. Without prior ranks (VBS2017–VBS2021) and ideally task-level scores, 'increasing performance' is unsubstantiated; VBS rank also depends on the number and skills of competing teams. Provide the underlying numbers or soften the claim.

minor comments (4)

[§2.1, §2.3] Please clarify whether '1s segmentation' changes shot boundaries or only the keyframe sampling interval. Section 2.1 says shots are 'ten-second scenes' and that events are computed on the full ten-second segments; Section 2.3 uses 'shot segmentation' and 'keyframe sampling interval' interchangeably.
[§2.3] The phrase 'which also will include single keyframe-based features' is ambiguous: does temporal search combine segment-level event features with per-second keyframe features? Please define the temporal context search pipeline.
[§2.2] Minor grammatical issue: 'As depicted the front end part of Figure 1' should be 'As depicted in the front-end part of Figure 1.'
[Table 2] The 'Impact' column values such as 'Improving KIS performance' are outcome claims; consider renaming the column 'Intended effect' to distinguish design goals from measured outcomes.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a system description with no derivation chain, fitted parameters, or numeric predictions to reduce to its inputs.

full rationale

diveXplore 6.0 is a short interactive-video-retrieval system description. It contains no equations, no fitted parameters, no quantitative evaluation, and no derivation chain that could be circular. The central claims in Table 2 and Section 2.3 — e.g., that reducing shot segmentation from 10s to 1s 'Improves KIS performance' and that 'temporal search' improves result precision — are assertions about intended or expected system behavior, not derived results. The text itself is hedged ('seems to greatly impede KIS tasks'), and no KIS/AVS scores are reported, so there is no statistically forced prediction or self-defined quantity. Citations to the authors' prior diveXplore papers (e.g., [4,5,6,11,15,16]) are used only to identify the system's history and components, not as load-bearing justification for a mathematical or empirical claim. The skeptical concern that the 1s segmentation improvement is unverified rests on missing evaluation evidence, which is a correctness/validation issue, not circularity. Under the specified rules, a lack of evaluation or unsupported empirical assertion does not constitute circularity unless a specific reduction to inputs can be exhibited; none can be here.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No derivation is attempted, so there are no fit parameters in the mathematical sense. The system has design choices, chiefly the sampling interval, that are hand-picked rather than justified by data. The central performance claims rely on unvalidated domain assumptions about the features and segmentation.

free parameters (1)

shot keyframe sampling interval = 1 second (10 seconds in earlier versions)
Chosen by hand based on anecdotal competition experience to improve KIS; no fitting or evaluation of alternative intervals is reported.

assumptions (3)

domain assumption Pre-trained deep models (Inception v3, YOLOv4, Moments in Time, Places365) produce discriminative features for V3C video retrieval.
The system's shot and map search rely entirely on these features (Table 1, Section 2.1); no validation on V3C is provided.
domain assumption Uniform sampling into fixed-duration shots yields representative keyframes for retrieval.
Section 2.1 states both search modes rely on custom shot segmentation that uniformly samples into ten-second scenes; version 6.0 merely shortens the interval rather than detecting shot boundaries.
domain assumption OCR (Tesseract) and STT (Kaldi) outputs are accurate and useful on V3C content.
Section 2.3 introduces these as 'proven successful features' citing the tool papers, but no V3C-specific accuracy or retrieval benefit is measured.

how reviews work

0 comments

Cite this review

Pith. "Pith review of diveXplore 6.0: ITEC's Interactive Video Exploration System at VBS 2022." pith.science (2026). https://pith.science/paper/CTY3CWRY

@misc{pith2026250820687,
  author       = {Pith},
  title        = {Pith review of: diveXplore 6.0: ITEC's Interactive Video Exploration System at VBS 2022},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CTY3CWRY}},
  note         = {Machine review of arXiv:2508.20687}
}

read the original abstract

Continuously participating since the sixth Video Browser Showdown (VBS2017), diveXplore is a veteran interactive search system that throughout its lifetime has offered and evaluated numerous features. After undergoing major refactoring for the most recent VBS2021, however, the system since version 5.0 is less feature rich, yet, more modern, leaner and faster than the original system. This proved to be a sensible decision as the new system showed increasing performance in VBS2021 when compared to the most recent former competitions. With version 6.0 we reconsider shot segmentation, map search and introduce new features for improving concept as well as temporal context search.

Figures

Figures reproduced from arXiv: 2508.20687 by the authors.

**Figure 1.** diveXplore architecture Generally, both of the diveXplore’s search modes rely on its custom shot segmentation of uniformly sampling the source datasets into ten-second scenes. However, they differ in their targeted amount of data: shot search is concerned with features belonging to individual shot keyframes, while map search operates on a video basis: results are matched by feature frequency in all shots of particu… view at source ↗

**Figure 2.** diveXplore interface components video summary maps, a user is able to quickly overview a video’s complete content. Only one map is inspected at any given moment and all the information in the corner areas always correspond to that map, which is the largest one centered in the middle of the screen. The other maps are previews of the previous and next maps for two exploration modes: horizontal navigation traverses t… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 15 canonical work pages

[1]

In: Proc

Berns, F., Rossetto, L., Schoeffmann, K., Beecks, C., Awad, G.: V3c1 dataset: An evaluation of content characteristics. In: Proc. of the 2019 on Intl. Conf. on Multimedia Retrieval. pp. 334–338. ACM (2019)

work page 2019
[2]

CoRR abs/2004.10934 (2020), https://arxiv.org/abs/2004.10934

Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. CoRR abs/2004.10934 (2020), https://arxiv.org/abs/2004.10934

arXiv 2004
[3]

Kay,A.:Tesseract:anopen-sourceopticalcharacterrecognitionengine.LinuxJour- nal 2007(159), 2 (2007)

work page 2007
[4]

In: Intl

Leibetseder, A., Kletz, S., Schoeffmann, K.: Sketch-based similarity search for col- laborative feature maps. In: Intl. Conf. on Multimedia Modeling. pp. 425–430. Springer (2018) 6 A. Leibetseder et al

work page 2018
[5]

In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W

Leibetseder, A., Münzer, B., Primus, J., Kletz, S., Schoeffmann, K.: divexplore 4.0: The itec deep interactive video exploration system at vbs2020. In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W. (eds.) MultiMedia Modeling. pp. 753–759. Springer Intl. Publishing, Cham (2020)

work page 2020
[6]

In: Intl

Leibetseder, A., Schoeffmann, K.: Less is more-divexplore 5.0 at vbs 2021. In: Intl. Conf. on Multimedia Modeling. pp. 455–460. Springer (2021)

work page 2021
[7]

IEEE Transactions on Multimedia pp

Lokoc, J., Bailer, W., Schoeffmann, K., Muenzer, B., Awad, G.: On influential trends in interactive video retrieval: Video browser show- down 2015-2017. IEEE Transactions on Multimedia pp. 1–1 (2018). https://doi.org/10.1109/TMM.2018.2830110

work page arXiv 2015
[8]

ACM Trans

Lokoč, J., Kovalčík, G., Münzer, B., Schöffmann, K., Bailer, W., Gasser, R., Vrochidis, S., Nguyen, P.A., Rujikietgumjorn, S., Barthel, K.U.: Interactive search or sequential browsing? a detailed analysis of the video browser showdown 2018. ACM Trans. Multimedia Comput. Commun. Appl.15(1), 29:1–29:18 (Feb 2019). https://doi.org/10.1145/3295663, http://doi...

work page doi:10.1145/3295663 2018

Show all 18 references

[9]

IEEE Trans- actions on Pattern Analysis and Machine Intelligence 42(2), 502–508 (2020), https://doi.org/10.1109/TPAMI.2019.2901464

Monfort, M., Vondrick, C., Oliva, A., Andonian, A., Zhou, B., Ramakrish- nan, K., Bargal, S.A., Yan, T., Brown, L.M., Fan, Q., Gutfreund, D.: Mo- ments in time dataset: One million videos for event understanding. IEEE Trans- actions on Pattern Analysis and Machine Intelligence...

2020
[10]

In: IEEE 2011 workshop on automatic speech recognition and understand- ing

Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hanne- mann, M., Motlicek, P., Qian, Y., Schwarz, P., et al.: The kaldi speech recognition toolkit. In: IEEE 2011 workshop on automatic speech recognition and understand- ing. No. CONF, IEEE Signal Proce...

2011
[11]

In: Intl

Primus, M.J., Münzer, B., Leibetseder, A., Schoeffmann, K.: The itec collabora- tive video search system at the video browser showdown 2018. In: Intl. Conf. on Multimedia Modeling. pp. 438–443. Springer (2018)

2018
[12]

IEEE Transactions on Multimedia 23, 243–256 (2021)

Rossetto, L., Gasser, R., Lokoč, J., Bailer, W., Schoeffmann, K., Muen- zer, B., Souček, T., Nguyen, P.A., Bolettieri, P., Leibetseder, A., Vrochidis, S.: Interactive video retrieval in the age of deep learning – detailed eval- uation of vbs 2019. IEEE Transactions on Multimed...

2019
[13]

arXiv preprint arXiv:2105.01475 (2021)

Rossetto, L., Schoeffmann, K., Bernstein, A.: Insights on the v3c2 dataset. arXiv preprint arXiv:2105.01475 (2021)

2021 arXiv
[14]

MultiMedia, IEEE 21(4), 8–13 (Oct 2014)

Schoeffmann, K.: A user-centric media retrieval competition: The video browser showdown 2012-2014. MultiMedia, IEEE 21(4), 8–13 (Oct 2014). https://doi.org/10.1109/MMUL.2014.56

2012 doi
[15]

In: Intl

Schoeffmann, K., Münzer, B., Leibetseder, A., Primus, J., Kletz, S.: Autopilot- ing feature maps: The deep interactive video exploration (divexplore) system at vbs2019. In: Intl. Conf. on Multimedia Modeling. pp. 585–590. Springer (2019)

2019
[16]

In: Intl

Schoeffmann, K., Primus, M.J., Muenzer, B., Petscharnig, S., Karisch, C., Xu, Q., Huerst, W.: Collaborative feature maps for interactive video search. In: Intl. Conf. on Multimedia Modeling. pp. 457–462. Springer (2017)

2017
[17]

In: Conf

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethink- ing the inception architecture for computer vision. In: Conf. on Com- puter Vision and Pattern Recognition. pp. 2818–2826. IEEE (2016), https://doi.org/10.1109/CVPR.2016.308

2016 doi
[18]

IEEE Transactions on Pattern Analysis and Machine Intelligence 40(6), 1452–1464 (2018), https://doi.org/10.1109/TPAMI.2017.2723009

Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(6), 1452–1464 (2018), https://doi.org/10.1109/TPAMI.2017.2723009

2018

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

In: Proc

Berns, F., Rossetto, L., Schoeffmann, K., Beecks, C., Awad, G.: V3c1 dataset: An evaluation of content characteristics. In: Proc. of the 2019 on Intl. Conf. on Multimedia Retrieval. pp. 334–338. ACM (2019)

work page 2019

[2] [2]

CoRR abs/2004.10934 (2020), https://arxiv.org/abs/2004.10934

Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. CoRR abs/2004.10934 (2020), https://arxiv.org/abs/2004.10934

arXiv 2004

[3] [3]

Kay,A.:Tesseract:anopen-sourceopticalcharacterrecognitionengine.LinuxJour- nal 2007(159), 2 (2007)

work page 2007

[4] [4]

In: Intl

Leibetseder, A., Kletz, S., Schoeffmann, K.: Sketch-based similarity search for col- laborative feature maps. In: Intl. Conf. on Multimedia Modeling. pp. 425–430. Springer (2018) 6 A. Leibetseder et al

work page 2018

[5] [5]

In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W

Leibetseder, A., Münzer, B., Primus, J., Kletz, S., Schoeffmann, K.: divexplore 4.0: The itec deep interactive video exploration system at vbs2020. In: Ro, Y.M., Cheng, W.H., Kim, J., Chu, W.T., Cui, P., Choi, J.W., Hu, M.C., De Neve, W. (eds.) MultiMedia Modeling. pp. 753–759. Springer Intl. Publishing, Cham (2020)

work page 2020

[6] [6]

In: Intl

Leibetseder, A., Schoeffmann, K.: Less is more-divexplore 5.0 at vbs 2021. In: Intl. Conf. on Multimedia Modeling. pp. 455–460. Springer (2021)

work page 2021

[7] [7]

IEEE Transactions on Multimedia pp

Lokoc, J., Bailer, W., Schoeffmann, K., Muenzer, B., Awad, G.: On influential trends in interactive video retrieval: Video browser show- down 2015-2017. IEEE Transactions on Multimedia pp. 1–1 (2018). https://doi.org/10.1109/TMM.2018.2830110

work page arXiv 2015

[8] [8]

ACM Trans

Lokoč, J., Kovalčík, G., Münzer, B., Schöffmann, K., Bailer, W., Gasser, R., Vrochidis, S., Nguyen, P.A., Rujikietgumjorn, S., Barthel, K.U.: Interactive search or sequential browsing? a detailed analysis of the video browser showdown 2018. ACM Trans. Multimedia Comput. Commun. Appl.15(1), 29:1–29:18 (Feb 2019). https://doi.org/10.1145/3295663, http://doi...

work page doi:10.1145/3295663 2018