Pith. sign in

REVIEW 4 major objections 5 minor 29 references

An Empirical study on LLM-based Log Retrieval for Software Engineering Metadata Management

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pipeline of LLM-written video and signal descriptions lets engineers search test logs in plain English and judge result reliability from score gaps.

desk verdict A useful engineering extension with informative prompt/model comparisons, but the reliability claims rest on an untested assumption about distance-distribution shape. read the letter →

arxiv 2506.11659 v1 pith:BP6BD372 submitted 2025-06-13 cs.SE

Simin Sun , Yuchuan Jin , Miroslaw Staron This is my paper
classification cs.SE
keywords logretrievalautonomousdrivingLLMnaturallanguagequeryvideodescriptionsignaldatacosinesimilarityreliability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that LLM-based retrieval can open up autonomous-driving test logs to natural-language queries, removing the need to know long signal names and SQL. It builds a two-modality pipeline: signal data is converted to text by fixed interpreters, video frames are converted to text by a video-language model, and a sentence-embedding model matches an engineer's query to the stored descriptions. Because log data has no ground-truth labels for what counts as a correct match, the paper proposes that query reliability can be read from the distribution of similarity scores across all records, gaps, range, and relative largest gap, and it validates this idea empirically on an open dataset of 1,473 driving sequences. If the approach holds, engineers can locate specific driving scenarios in plain English and gauge whether the results can be trusted without manual verification of every row. The paper also claims that concise prompts and video input are important for achieving discriminative descriptions.

What carries the argument

The scenario distance graph: for each query, the pipeline computes the cosine similarity between the query embedding and every record's combined (signal-plus-video) text description, sorts those distances $D_1 \le D_2 \le \dots \le D_{1473}$, and examines the consecutive gaps $G_n = D_{n+1} - D_n$. The six named metrics, LGap, MinD, MaxD, Range, StdDev, and RLGap, are all derived from this sorted distance distribution, and the paper defines a reliable result as one where a few records score above 0.9, some score below 0.4, and the rest spread out without forming clusters. This distribution-shape criterion is the mechanism that lets the paper evaluate retrieval quality in the absence of labels.

What would settle it

Choose a set of queries from the open dataset with human-annotated correct matches, compute the six metrics for each query, and check whether queries with large LGap, high RLGap, and wide range retrieve the correct records at the top; the reliability claim fails if any high-gap query returns unrelated records or if low-variance queries still retrieve correct ones.

Watch

Extended reading notes

Core claim

The paper claims that natural-language scenario search over autonomous-driving logs is feasible without SQL by converting both signal logs and video frames into text descriptions, signals via automated interpreters and video via a video-language LLM, then ranking records by cosine similarity of a sentence-embedding model. Its central empirical conclusion is that this combined, text-based pipeline retrieves relevant scenarios, that video descriptions contribute motion information unavailable from a single image, and that the reliability of a query can be judged without ground truth by the shape of the sorted similarity-distance distribution: large gaps, wide range, high standard deviation, and high relative largest gap signal a successful, discriminative search, whereas low variance signals a failed search.

Load-bearing premise

The paper assumes, without ground-truth validation, that the shape of the cosine-similarity distribution over all records, a few scores above 0.9, some below 0.4, and a spread in between, indicates that the query actually found the right records.

Editorial extensions

If this is right

  • Engineers can replace SQL and detailed schema knowledge with natural-language scenario descriptions, because the pipeline converts both signal data and video into text before matching.
  • Query reliability can be assessed without labeled ground truth: a wide range, a large absolute and relative largest gap, and an S-shaped distribution of similarity scores are treated as indicators of a successful match, while low variance indicates failure.
  • Combining video with signal data adds motion-related information, such as slowing down or approaching intersections, that a single still frame cannot provide, making descriptions more discriminative.
  • Prompt formatting matters: a concise, structured prompt produced more reliable retrieval than longer multi-run prompts, which generated over-detailed descriptions that inflated similarity across all records.
  • Among the tested configurations, a video-capable vision-language model with the concise prompt gave the most reliable descriptions, supporting that configuration for practical use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the distribution-shape reliability criterion is accepted, the same metrics could transfer to other unlabeled retrieval settings, such as code search, incident reports, or regression-test selection, where precision cannot be computed directly, though they would first need validation against ground truth.
  • A direct test is available: run the pipeline on a subset of the open dataset with human-annotated query-relevance labels and check whether high LGap, high RLGap, and wide range actually predict higher precision at the top of the ranking.
  • The paper's finding that queries whose keywords appear in descriptions with the wrong semantics, for example 'no pedestrians' matching a query for 'pedestrians', produce false positives points to a retrievable-but-not-semantically-aligned failure mode; a negation-aware check could be added to the pipeline.
  • Because the current pipeline stores all descriptions and does brute-force comparison, applying it at industrial scale would require an embedding index and keyword pre-filtering; the proposed metrics could then double as a query-level confidence signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript presents an LLM-based retrieval pipeline for searching autonomous driving test logs. For each of the 1,473 Zenseact Open Dataset sequences, it generates textual descriptions from video (via LLaVA, InternVideo2, or CogVLM2) and from numeric signal tables via automated interpreters; a user's natural-language scenario query is embedded with all-MiniLM-L6-v2 and compared by cosine similarity against each record's combined description, with the video and signal scores summed. The paper evaluates six prompts, image versus video modality, and three video models across nine scenarios, using violin plots and six distribution-shape metrics (LGap, MinD, MaxD, Range, StdDev, RLGap) to judge retrieval reliability. It reports that Prompt 4 and LLaVA give the most reliable results and that these metrics are effective for assessing query outcomes.

Significance. If the central claims were supported, the work would be a practical contribution to natural-language log retrieval for autonomous driving: it uses a public dataset, a reproducible pipeline, and a concrete API for querying and visualization, and the comparison of prompts and models is informative. However, the evaluation never measures retrieval correctness against ground truth, the reliability construct is defined by the same distribution statistics used to select the winning configuration, and no baseline or efficiency measurement is reported. The current evidence does not establish the headline efficiency and reliability claims, although the gap is addressable with additional evaluation.

major comments (4)
  1. [Section 4.3, Section 5.3, Section 8] The claim that LGap, MinD, MaxD, Range, StdDev, and RLGap are reliable indicators of query correctness is not validated against any ground truth. The paper explicitly states that the data lack labels (Section 1) and substitutes distribution-shape criteria for direct evaluation, but it never demonstrates that a large gap, wide range, or S-shaped distance curve actually corresponds to retrieved records that match the query. No precision/recall, NDCG, baseline (SQL, BM25), or inter-rater agreement is reported, and the qualitative checks in Figures 6 and 7 cover selected examples only. Consequently, the conclusion in Section 8 that these metrics 'proved to be effective' is unsupported. A labeled subset with manual relevance judgments and a baseline comparison is needed.
  2. [Section 5.1.1, Section 5.1.3, Section 5.3] The evaluation is circular. Prompt 4 and LLaVA are selected because their cosine-similarity violin plots show long tails, large gaps, and well-distributed middle ranges; Section 5.3 then uses exactly those same distribution shapes and the Section 4.3 thresholds (scores above 0.9 and below 0.4) as evidence of reliable retrieval. Because the selection criterion and the success criterion are the same unlabeled distance distributions, the comparison cannot independently support the reliability claim. A concrete test would be to pre-register the distribution-shape criteria and validate them on held-out queries with known relevant records.
  3. [Table 3, Figure 4, Section 4.3] The quantity D_n is not defined consistently. Table 3 reports MaxD values above 1 (e.g., IV-4 MaxD = 1.0317 and IV-6 MaxD = 1.0056), while Figure 4 labels the y-axis as cosine similarity and Section 4.3 defines reliable results using similarity thresholds of 0.9 and 0.4. If D_n is cosine distance (1 - similarity), then a MinD of 0.2345 (V-4) corresponds to similarity 0.7655, which contradicts the requirement of >0.9 in Section 4.3; if D_n is cosine similarity, values exceeding 1.0 are impossible. The definition of G_n in Section 4.3 also requires distances to be sorted, which is not stated. These issues affect the interpretation of every metric in Table 3.
  4. [Abstract, Section 6] The claim of 'improved efficiency and reliability' is not measured. No SQL baseline, no timing data, no query-composition cost, and no user-effort measurement are reported anywhere in Sections 4 or 5, and the only validation is the qualitative manual inspection of selected frames in Section 5.2. If the paper keeps the efficiency claim, it needs an empirical comparison (e.g., time or number of interactions to construct a query in SQL versus natural language); if only retrieval quality is claimed, the abstract and conclusions should be revised accordingly.
minor comments (5)
  1. [Section 3.1] The text says 'Driver consists of 29 multi-minute sequences'; this should read 'Drives'.
  2. [Section 4.1.3] The model listed as 'Cogvlm2-llama3-caption' is cited to reference [19], which is CogVideoX; please verify the citation and model name.
  3. [Section 4.3] The definitions of LGap and RLGap depend on consecutive differences G_n, which are only meaningful if the D_n values are sorted; please state this explicitly.
  4. [Section 5.1.1] The term 'cost-efficient' is asserted for Prompt 4 without reporting cost or runtime data; please add the measurement or soften the claim.
  5. [Figure 4, Table 3] The relationship between the violin plots (described in terms of similarity) and the table (which reports distances) should be stated explicitly, and the y-axis label should be aligned with the definition of D_n.

Circularity Check

3 steps flagged · score 6.0 of 10

Reliability evaluation is circular: the distribution-shape criteria defined in Sec. 4.3 are used to select Prompt 4 and LLaVa, and the same cosine-distance shapes are then reported as evidence of retrieval reliability, with no labeled relevance validation.

  1. self definitional [Section 4.3, Validation Metrics, reliability criteria]
    "When querying a specific scenario, the criteria for a reliable result includes: A few highly relevant records with similarity scores above 0.9, demonstrating the pipeline's ability to retrieve desired scenarios. Some irrelevant records should have similarity scores below 0.4, indicating the pipeline's effectiveness in filtering out unrelated responses. The remaining records, with similarity scores between 0.4 and 0.9, should be distributed without forming distinct clusters."

    The paper defines 'reliable result' entirely in terms of the cosine-similarity distribution between the query and the generated descriptions, i.e., the very scores used for retrieval and ranking. No external relevance labels, precision/recall, NDCG, human rating, or benchmark are used to test whether a >0.9 top, <0.4 tail, and unclustered middle actually corresponds to true query-relevant records. The paper itself states in Section 4.3 that 'the data lacks labels' and in Section 5.2 that 'there are no established benchmarks for evaluating the results.' Consequently, the reliability criteria are an asserted assumption, and any evaluation using them is self-referential by construction.

  2. self definitional [Section 5.1.1, RQ1 Prompt selection]
    "From the perspective of prompts: Prompt 4 generates the most reliable results. Its longer whiskers indicate a greater spread of data. Compared to other prompts, it has long needle-like shapes at both ends, suggesting some highly relevant (> 0.9) and less relevant (< 0.4) candidates."

    Prompt 4 is declared 'most reliable' because its violin plot has exactly the shape defined as reliable in Section 4.3: high spread, a needle above 0.9, and candidates below 0.4. Since the operational definition of reliability is that distribution shape, selecting the prompt that maximizes it and then concluding that it is most reliable is a tautology: the evaluation input (cosine-distance shape) is identical to the claimed output (reliability). The same pattern repeats for LLaVa in Section 5.1.3, where 'long needle-shaped extension... score > 0.9' is offered as evidence of retrieval ability, without any ground-truth relevance check.

1 more flagged steps
  1. self definitional [Section 5.3, RQ3 conclusion]
    "RQ3: Although there are no direct benchmarks for evaluating query results, reliable outcomes typically display consistent patterns. First, when plotting the sorted distances for all records, an 'S' shape typically emerges, helping to distinguish three categories: highly relevant, moderately relevant, and non-relevant items. Additionally, various metrics can be used to assess the quality of the results. A wide range of values and significant gaps between them generally indicate a successful match."

    The RQ3 answer restates the Section 4.3 criteria as a finding: wide range and significant gaps 'indicate a successful match' is exactly the definition used to construct LGap, RLGap, Range, and StdDev from the same D_n distances. With no labels or benchmarks (the paper acknowledges this in the same sentence), the 'S shape' is simply the assumed separation into >0.9, 0.4-0.9, and <0.4 bands, not an independently verified predictor of query correctness. The metric values in Table 3 are therefore descriptive statistics of the retrieval scores, not validated evidence of reliability.

full rationale

The paper's pipeline construction, API, and qualitative manual inspection of retrieved video frames are genuine contributions, and there is no load-bearing circular self-citation: reference [7] (authors' prior work) is used only to position the dataset and modality choice, not to justify the reliability claim. However, the central evaluation of retrieval reliability is circular. Section 4.3 defines a reliable result as a specific cosine-similarity distribution (top cluster >0.9, tail <0.4, unclustered middle) using the same embedding distances that perform ranking. Sections 5.1.1 and 5.1.3 then select Prompt 4 and LLaVa because their plots match that definition, and RQ3 concludes that wide range and large gaps indicate successful matches. Since no labeled relevance set, inter-rater agreement, or external benchmark is used, the 'predictions' that Prompt 4/LLaVa are more reliable reduce by construction to the paper's own criteria. The paper explicitly admits the missing ground truth ('the data lacks labels' in Section 4.3; 'there are no established benchmarks for evaluating the results' in Section 5.2), which confirms that the reliability metrics were never independently validated. The MaxD values above 1.0 in Table 3 (e.g., IV-4 MaxD = 1.0317) alongside text describing cosine similarity are an additional quality concern, but the circularity finding rests on the definitional reduction above. Overall score 6: some central 'reliability' conclusions are forced by definition, while the engineering contributions remain partially independent.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central evaluation rests on hand-chosen thresholds and an unvalidated assumption that distribution shape indicates correctness. No external ground truth or baseline is used, so the metrics cannot independently confirm retrieval quality.

free parameters (4)
  • Relevance thresholds (0.9 and 0.4) = 0.9 and 0.4
    Hand-chosen in Section 4.3 to classify records as highly relevant, moderately relevant, or non-relevant; no empirical or theoretical justification is provided.
  • Number of video frames sampled (32) = 32
    Chosen in Section 3.1 to fit the maximum input of LLaVA-NeXT-Video-7B-hf; this affects the information available for description generation.
  • Signal columns used (14 of 49) = 14 columns
    Filtered by interpretability and to remove constant signals in Section 3.1; the selection is post hoc and not reported in detail.
  • Downsampling target = satellite data frequency (lowest)
    All signal tables downsampled to match satellite data in Section 3.1; this choice discards temporal resolution of higher-frequency vehicle data.
assumptions (4)
  • domain assumption Cosine similarity between sentence embeddings measures semantic relevance between query and description.
    The entire retrieval ranking in Section 3.2.3 relies on all-MiniLM-L6-v2 cosine similarity as a proxy for relevance; this is not validated against human judgments.
  • domain assumption LLM-generated video and signal descriptions faithfully and sufficiently represent each record's content.
    Section 3.2.1 and 3.2.2 assume the generated text captures the features needed for scenario matching; the paper does not measure description accuracy.
  • ad hoc to paper The distribution shape of similarity scores is a valid indicator of retrieval reliability.
    Section 4.3 introduces this without external validation; it is the load-bearing evaluation assumption.
  • domain assumption The nine developer-selected scenarios form a representative evaluation set.
    Section 4.2 says scenarios were chosen with developers and to ensure keywords are searchable, which biases toward feasible queries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Empirical study on LLM-based Log Retrieval for Software Engineering Metadata Management." pith.science (2026). https://pith.science/paper/BP6BD372

@misc{pith2026250611659,
  author       = {Pith},
  title        = {Pith review of: An Empirical study on LLM-based Log Retrieval for Software Engineering Metadata Management},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BP6BD372}},
  note         = {Machine review of arXiv:2506.11659}
}
read the original abstract

Developing autonomous driving systems (ADSs) involves generating and storing extensive log data from test drives, which is essential for verification, research, and simulation. However, these high-frequency logs, recorded over varying durations, pose challenges for developers attempting to locate specific driving scenarios. This difficulty arises due to the wide range of signals representing various vehicle components and driving conditions, as well as unfamiliarity of some developers' with the detailed meaning of these signals. Traditional SQL-based querying exacerbates this challenge by demanding both domain expertise and database knowledge, often yielding results that are difficult to verify for accuracy. This paper introduces a Large Language Model (LLM)-supported approach that combines signal log data with video recordings from test drives, enabling natural language based scenario searches while reducing the need for specialized knowledge. By leveraging scenario distance graphs and relative gap indicators, it provides quantifiable metrics to evaluate the reliability of query results. The method is implemented as an API for efficient database querying and retrieval of relevant records, paired with video frames for intuitive visualization. Evaluation on an open industrial dataset demonstrates improved efficiency and reliability in scenario retrieval, eliminating dependency on a single data source and conventional SQL.

Figures

Figures reproduced from arXiv: 2506.11659 by the authors.

Figure 1
Figure 1. A video frame from record #000005 table lengths. To standardize the data, we downsized all tables based on the least frequent data, the satellite data. After reshaping, we have a table with approximately 1,700 rows. The signals (columns) that are difficult to interpret or remain constant are considered noise and, therefore, removed from the study. 3.2 Scenario Query The scenario query processing pipeline ( [PITH_FU… view at source ↗
Figure 2
Figure 2. The workflow contains two procedures: data processing and scenario query processing. When processing data, video [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. From the perspective of prompts: • Prompt 4 generates the most reliable results. Its longer whiskers indicate a greater spread of data. Compared to other prompts, it has long needle-like shapes at both ends, suggesting some highly relevant (> 0.9) and less relevant (< 0.4) candidates. The lowest score is around 0.2 (Scenario VI), and the highest is close to 1 (Scenario IV). In the middle range (0.4–0.9), it demonstr… view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Query results using Prompt 4 across scenario I to IX (from left to right, top to bottom) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 4
Figure 4. Figure 4: Records’ distances to nine scenarios’ queries using six prompts. The Y-axis represents the cosine similarity score [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: All records’ distances across three models. The Y [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Query results for scenario I using prompts 1 to 6 (from left to right) [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 15 canonical work pages

  1. [1]

    Meta AI. 2024. Llama 3.1: A Collection of Multilingual Large Language Models. https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct

    work page 2024

  2. [2]

    Miguel Alcon, Hamid Tabani, Leonidas Kosmidis, Enrico Mezzetti, Jaume Abella, and Francisco J Cazorla. 2020. Timing of autonomous driving software: Problem analysis and prospects for future solutions. In 2020 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). IEEE, 267–280

    work page 2020

  3. [3]

    Mina Alibeigi, William Ljungbergh, Adam Tonderski, Georg Hess, Adam Lilja, Carl Lindstrom, Daria Motorniuk, Junsheng Fu, Jenny Widahl, and Christoffer Petersson. 2023. Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving. arXiv:2305.02008 [cs.CV]

    arXiv 2023

  4. [4]

    Stefan Andreas Baur, David Josef Emmerichs, Frank Moosmann, Peter Pinggera, Björn Ommer, and Andreas Geiger. 2021. Slim: Self-supervised lidar scene flow and motion segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13126–13136

    work page 2021

  5. [5]

    Xieyuanli Chen, Benedikt Mersch, Lucas Nunes, Rodrigo Marcuzzi, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. 2022. Automatic labeling to generate training data for online LiDAR-based moving object segmentation. IEEE Robotics and Automation Letters 7, 3 (2022), 6107–6114

    work page 2022

  6. [6]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. arXiv preprint...

    arXiv 2023

  7. [7]

    Jesper Knapp, Klas Moberg, Yuchuan Jin, Simin Sun, and Miroslaw Staron. 2024. A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Devel- opment. In International Conference on Product-Focused Software Process Improve- ment. Springer, 35–49

    work page 2024

  8. [8]

    Keti Korini and Christian Bizer. 2023. Column type annotation using chatgpt. arXiv preprint arXiv:2306.00745 (2023)

    arXiv 2023

Show all 29 references
  1. [9]

    Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles

  2. [10]

    Jie Lei, Xinyu Liang, and Mohit Bansal. 2021. Less is more: ClipBERT for video- and-language learning via sparse sampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 7331–7341

    2021

  3. [11]

    Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. 2023. Table- gpt: Table-tuned gpt for diverse table tasks. arXiv preprint arXiv:2310.09263 (2023)

    2023 arXiv

  4. [12]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning. arXiv preprint arXiv:2304.08485 (2023)

    2023 arXiv

  5. [13]

    Guannan Lou, Yao Deng, Xi Zheng, Mengshi Zhang, and Tianyi Zhang. 2022. Testing of autonomous driving systems: where are we and where should we go?. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engin...

    2022

  6. [14]

    Vasilii Mosin, Darko Durisic, and Miroslaw Staron. 2021. Applicability of Machine Learning Architectural Patterns in Vehicle Architecture: A Case Study.. In ECSA (Companion)

    2021

  7. [15]

    Avanika Narayan, Ines Chami, Laurel Orr, Simran Arora, and Christopher Ré

  8. [16]

    Sushant Kumar Pandey, Vasilii Mosin, Darko Durisic, Ashok Chaitanya Kop- pisetty, and Miroslaw Staron. 2023. Data Handling for Assuring Production Quality of Image Intensive Autonomous Drive Systems: An Industrial Case Study. Journal of Software Engineering for Autonomous Syst...

    2023

  9. [17]

    Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid

  10. [18]

    Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Chenting Wang, Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, et al. 2024. Internvideo2: Scaling video foundation models for multimodal video understanding. arXiv preprint arXiv:2403.15377 (2024)

    2024 arXiv

  11. [19]

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al . 2024. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. arXiv preprint arXiv:2408.06072 (2024)

    2024 arXiv

  12. [20]

    Liang Yao, Atousa Torabi, Kyunghyun Cho, Nicolas Ballas, Christopher Pal, Hugo Larochelle, and Aaron Courville. 2015. Describing videos by exploiting temporal structure. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 4507–4515

    2015

  13. [21]

    Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. 2020. TaBERT: Pretraining for joint understanding of textual and tabular data. arXiv preprint arXiv:2005.08314 (2020)

    2020 arXiv

  14. [22]

    Ke Zhang, Wei-Lun Chao, Fei Sha, and Kristen Grauman. 2016. Video summa- rization with long short-term memory. In Proceedings of the European Conference on Computer Vision (ECCV) . 766–782

    2016

  15. [23]

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-NeXT: A Strong Zero-shot Video Understanding Model. https://llava-vl.github.io/blog/2024-04-30-llava-next- video/

    2024

  16. [24]

    Corso, Richard Socher, and Caiming Xiong

    Luowei Zhou, Yingbo Zhou, Jason J. Corso, Richard Socher, and Caiming Xiong

  17. [25]

    Linchao Zhu, Zijie Xu, and Yi Yang. 2020. ActBERT: Learning global-local video- text representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 8746–8755

    2020

  18. [2017]

    In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    Dense-captioning events in videos. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) . 706–715

  19. [2018]

    InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    End-to-end dense video captioning with masked transformer. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 8739– 8748

  20. [2019]

    In Proceedings of the IEEE International Conference on Computer Vision (ICCV)

    Videobert: A model for video-and-language pretraining. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) . 7464–7473

  21. [2022]

    Can foundation models wrangle your data? arXiv preprint arXiv:2205.09911 (2022)

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.