Pith. sign in

REVIEW 4 major objections 5 minor 39 references

Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that LoRA fine-tuning on a 12,000-video synthetic slide-animation dataset lets a 7-billion-parameter VLM beat GPT-4.1 and Gemini-2.5-Pro at describing slide animations.

desk verdict A useful new dataset and metric for slide-animation understanding, but the synthetic test split likely shares static slides with training, so the headline LoRA gains are not yet established. read the letter →

arxiv 2507.03916 v3 pith:SEDGJVDJ submitted 2025-07-05 cs.AI cs.CV

Yifan Jiang , Yibo Xue , Yukun Kang , Pin Zheng , Jian Peng , Feiran Wu , Changliang Xu This is my paper · ORCID
classification cs.AIcs.CV
keywords slideanimationvision-languagemodelLoRAsyntheticdatasettemporalreasoningvideocaptioningCODAmetricPowerPoint
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Slide animations are a blind spot for current vision-language models (VLMs): there is no public dataset that pairs videos of animated slides with natural-language descriptions of what moves, when, and how. This paper tries to close that gap by building the first public slide-animation dataset—12,000 text–JSON–video triplets covering every built-in PowerPoint effect—and showing that a comparatively small open VLM (Qwen-2.5-VL-7B) fine-tuned on it with LoRA outperforms the much larger closed models GPT-4.1 and Gemini-2.5-Pro on BLEU-4, ROUGE-L, SPICE, and the paper's own CODA metric. The claim is that the bottleneck for slide-animation understanding is not model scale but structured temporal training data, and that low-rank adaptation is enough to unlock temporal reasoning. On a manually created 50-slide test set the fine-tuned model still leads, though the margin narrows because human slides contain background motions, concurrent effects, and layouts absent from training. If the claim holds, slide-generation tools could add animation comprehension without waiting for bigger base models.

What carries the argument

The load-bearing resource is a synthetic dataset of 12,000 triplets, each consisting of a natural-language animation description, a structured animation JSON file, and a rendered video of the animated slide. The triplets are produced by an end-to-end pipeline in which GPT-4.1 generates static slide layouts and animation plans, python-pptx renders the static slides, and custom VBA scripts bind the JSON-specified effects (42 entrance–exit pairs and 10 emphasis effects, covering every built-in PowerPoint effect except custom motion paths) and export videos. The training mechanism is LoRA: low-rank update matrices $\Delta W = A \cdot B$ applied to the attention and MLP projections of frozen Qwen-2.5-VL-7B, with rank 8–32 and frame rates 1–4 FPS, trained for five epochs on 11,000 of the triplets. The evaluation mechanism is CODA, which splits a predicted and reference description into ordered action units, matches them left to right by nearest match, and scores action coverage ($|M|/n$), temporal order (longest common subsequence length divided by $n$), and detail fidelity (average over per-unit parameter matches, with perfect/partial/mismatch scored 1/0.5/0). Together these three pieces let the paper turn an unstructured video-description task into a structured, measurable recognition task.

What would settle it

Give the LoRA model, GPT-4.1, and Gemini-2.5-Pro a held-out set of several hundred human-authored PowerPoint animations with concurrent effects and background motion; if the LoRA model's CODA Coverage and Order scores do not beat both closed models by a margin comparable to the synthetic-set gap, the paper's generalization claim is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a 7-billion-parameter VLM adapted with low-rank matrices on 12,000 synthetic slide-animation videos learns to describe slide animations more accurately than GPT-4.1 and Gemini-2.5-Pro, despite being far smaller. The best configuration (LoRA rank 32, 4 FPS sampling) reaches BLEU-4 0.305, ROUGE-L 0.632, SPICE 0.457, and CODA coverage/order/detail scores of 0.983/0.965/0.810 on the synthetic test set, roughly doubling or tripling the base model's scores. The ablations attribute most of the gain to denser temporal sampling (4 FPS beats 1 FPS monotonically) rather than to larger adapter rank, and the generalization test on 50 manually created slides shows the LoRA model remains on top on every metric, with BLEU-4 rising about 60% and ROUGE-L about 30% relative to the base Qwen-2.5-VL. The paper presents this as evidence that the data, not model capacity, is the limiting factor, and that the released dataset plus the CODA metric constitute a reusable benchmark for VLM-based slide-animation comprehension.

Load-bearing premise

The paper assumes that the 12,000 synthetic animations generated by an LLM plus PowerPoint VBA scripts are representative enough of real human-made slide animations that a model trained on them transfers; the paper's own manual-test results show this assumption holds only partially, since gains shrink when slides contain background motion, concurrent effects, and unseen layouts.

Editorial extensions

If this is right

  • A 7B open model fine-tuned with LoRA on this dataset can caption slide animations better than GPT-4.1 and Gemini-2.5-Pro, so task-specific structured data can compensate for model scale.
  • The highest gains come from denser frame sampling: 4 FPS consistently beats 1 and 2 FPS on all CODA facets, so video sampling rate matters more than adapter rank.
  • The CODA metric (coverage, order, detail) offers a reusable evaluation for animation and video description tasks where n-gram and scene-graph metrics miss temporal ordering.
  • Improvements transfer to manually created slides, though the margin narrows, indicating the synthetic training distribution only partially covers real-world animation variability.
  • A full LoRA fine-tuning cycle runs in about 17 hours on four A800 GPUs, making this a practical recipe for slide-generation products.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: because the training slides are photo-and-LLM-text heavy (images drawn from Unsplash, text drafted by an LLM), the same recipe should be stress-tested on dense text-heavy business slides, where OCR and layout loading may matter more than motion vocabulary.
  • Our inference: CODA's left-to-right nearest-match alignment assumes the predicted description lists actions in roughly the reference order; reordered or paraphrased but semantically correct descriptions would be penalized on Order, so an alignment-free order metric might change the ranking.
  • Our inference: the dataset's one-animation-scheme-per-static-slide format omits interactive or click-triggered animations and custom motion paths, so the method's ceiling on real presentation software is untested.
  • Our inference: if the central claim is right, a natural next experiment is to fine-tune the same 7B model on mixed synthetic and human-annotated slide animations to see whether the generalization gap (currently 0.574 versus 0.983 CODA coverage) closes, or whether the synthetic distribution itself needs to be diversified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a dataset, a fine-tuning method, and a metric for slide-animation comprehension. The dataset comprises 12,000 text–JSON–video triplets synthesized from 300 static slides expanded into 40 animation schemes each, using GPT-4.1 for description generation and PowerPoint/VBA for rendering. The authors LoRA-tune Qwen-2.5-VL-7B on 11,000 pairs and compare against GPT-4.1 and Gemini-2.5-Pro on a 1,000-video synthetic test set and a 50-video manually created test set. They report consistent improvements on BLEU-4, ROUGE-L, SPICE, and a new CODA metric, with the gains on the manual set being smaller than on the synthetic set.

Significance. If the results survive a cleaner evaluation, the dataset is the first public resource of its kind for slide-animation modeling, and the paper provides a useful baseline-plus-metric package for an under-served task. The choice of an open 7B model with LoRA is appropriate for the low-resource setting, and the authors are honest in §5.5 and §6.2 that distribution shift reduces gains on manually created slides. However, the current evaluation protocol has three load-bearing weaknesses: possible static-slide overlap between training and test, training and test reference descriptions generated by the same GPT-4.1 pipeline, and an unvalidated new metric. These issues must be addressed before the stronger claims in the abstract, §5.3, and §6.1 can be accepted.

major comments (4)
  1. [§5.1 and §3.2] The test split is not shown to be disjoint at the static-slide level. The dataset is created from 300 static slides, each expanded into 40 animation variants, yet §5.1 only says that 1,000 paired samples 'were not used in training.' If the 1,000 test videos are drawn from the same 300 static slides as the 11,000 training videos, then training and test share slide layout, text, and images, and the LoRA model can memorize static content rather than learn temporal structure. Please state whether the split is at the level of animation variants or static slides, and if it is the former, rerun the comparison on a slide-disjoint split or otherwise report results on layouts unseen during training.
  2. [§3.2 and §5.3] The synthetic evaluation is confounded by distribution alignment between training and test references. Both the training descriptions and the synthetic test references are produced by the same GPT-4.1 pipeline (the action list is paraphrased into a natural-language narrative), so the LoRA model is fine-tuned to imitate the exact distribution it is scored against, whereas GPT-4.1 is evaluated zero-shot. This can inflate BLEU-4, ROUGE-L, and SPICE even without genuine temporal understanding. A concrete test would be to give the baselines few-shot in-context examples from the training distribution, or to score against human-written references for the synthetic domain. The paper's own Table 2 versus Table 4 shows the LoRA n-gram advantage shrinks sharply on human references (BLEU-4 from 0.305 to 0.057), which is consistent with this concern.
  3. [§5.2] CODA is a new metric, but its validity is asserted rather than established. The left-to-right nearest-match matching rule, the treatment of paraphrases and synonyms, and the specific LLM judge are not described in enough detail for replication, and no human-correlation study, no inter-annotator agreement, and no error analysis are reported. Since CODA is used as headline evidence in Tables 2 and 4, please add a validation study: for example, compute CODA sub-scores against human ratings on a sample of outputs, report the judge model and prompt, and analyze failure cases.
  4. [§5.5 and Table 4] The 50-video manual test set is too small to support the strong generalization claim in the abstract. No error bars or significance tests are reported, and several differences are small in absolute terms (e.g., BLEU-4 0.055 for LoRA-Rank16@2FPS versus 0.051 for Gemini-2.5-Pro@2FPS; SPICE 0.188 versus 0.161). The paper's own §5.5 notes that the manually created slides contain background motions, concurrent effects, and layouts not seen in training, and the gains narrow accordingly. Please report bootstrap confidence intervals or per-video significance tests, provide a per-video breakdown, and specify how the multiple volunteer annotations were adjudicated and whether inter-annotator agreement was measured.
minor comments (5)
  1. [Tables 2–4] The header 'ROGUE-L' should be 'ROUGE-L' in all three tables.
  2. [§3.3] The heading '3.3 Data Generation Pipeline' duplicates the heading of §3.2; the content of §3.3 is statistical analysis and should be retitled accordingly.
  3. [References] Reference [13] is cited as CLIP but is actually the CLIPScore paper, and reference [15] is cited as LLaVA but is actually GeoLLaVA; the reference list should be checked for other mismatches between citations and entries.
  4. [§4.2.1] The text says both LoRA matrices A and B are initialized with Gaussian distributions; standard LoRA initializes A with a Gaussian and B with zeros so that the initial update is zero. Please clarify or correct this implementation detail.
  5. [Abstract] The phrase 'provide a rigorous benchmark and a preparation for future research' is unidiomatic; 'foundation' would be more natural.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the LoRA gains are supported by an independent 50-video manual test set, and the synthetic-set leakage concern is a data-split caveat rather than a demonstrated by-construction reduction.

full rationale

The paper's derivation chain is self-contained: the model is fine-tuned on 11,000 synthetic triplets and evaluated on 1,000 held-out triplets from the same pipeline plus a manually created 50-video test set. The manually created test set provides independent evidence that the LoRA improvement is not solely an artifact of matching GPT-4.1-written training references, since its annotations were produced by volunteers and LoRA still outperforms the closed-source baselines on it. CODA is defined operationally via action-unit decomposition, left-to-right matching, and LCS formulas, so it is a concrete metric rather than a self-referential definition. There are no load-bearing self-citations and no uniqueness theorem imported from the authors. Two caveats, while relevant to experimental rigor, are not circularity: Section 5.1 states that 1,000 paired samples were not used in training but does not state a slide-level disjoint split, and since Section 3.2 produces 40 animation variants per static slide, a sample-level split could let the fine-tuned model exploit static layouts seen in training, inflating synthetic-set gains; and the CODA evaluator LLM is unspecified, so if it is the same GPT-4.1 that wrote the training descriptions it could favor the fine-tuned output style. Neither caveat amounts to a demonstrated by-construction reduction of the central claim, and the independent manual test set keeps the main generalization result non-circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters were fitted to data to produce the central claim; the LoRA hyperparameters are standard tuning choices. The assumptions are domain assumptions about data representativeness and metric validity, listed above. No new entities (particles, forces, dimensions) are introduced.

assumptions (4)
  • domain assumption The synthetic 12,000-triplet dataset is representative enough of real slide-animation distributions for fine-tuning to transfer.
    The paper acknowledges distribution shift in Section 6.2: manual slides contain background motions, concurrent effects, and layouts absent from training data. The transfer assumption is therefore load-bearing and only partially validated.
  • domain assumption GPT-4.1-generated natural-language descriptions are accurate, canonical references for slide-animation comprehension.
    Training labels and synthetic test references are produced by GPT-4.1 in Stage 2 of the data pipeline (Section 3.2); if these are flawed, the whole benchmark is unreliable.
  • domain assumption Frame sampling at up to 4 FPS captures the temporal dynamics of slide animations.
    The paper selects 1, 2, and 4 FPS heuristically (Section 4.3); no analysis proves 4 FPS is sufficient, though the model improves with FPS.
  • ad hoc to paper CODA is a valid and unbiased measure of action coverage, order, and detail.
    CODA is introduced in this paper (Section 5.2); the judge LLM, prompts, and decomposition rules are not fully specified in the available text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models." pith.science (2026). https://pith.science/paper/SEDGJVDJ

@misc{pith2026250703916,
  author       = {Pith},
  title        = {Pith review of: Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEDGJVDJ}},
  note         = {Machine review of arXiv:2507.03916}
}
read the original abstract

Slide animations, such as fade-in, fly-in, and wipe, are critical for audience engagement, efficient information delivery, and vivid visual expression. However, most AI-driven slide-generation tools still lack native animation support, and existing vision-language models (VLMs) struggle with animation tasks due to the absence of public datasets and limited temporal-reasoning capabilities. To address this gap, we release the first public dataset for slide-animation modeling: 12,000 triplets of natural-language descriptions, animation JSON files, and rendered videos, collectively covering every built-in PowerPoint effect. Using this resource, we fine-tune Qwen-2.5-VL-7B with Low-Rank Adaptation (LoRA) and achieve consistent improvements over GPT-4.1 and Gemini-2.5-Pro in BLEU-4, ROUGE-L, SPICE, and our Coverage-Order-Detail Assessment (CODA) metric, which evaluates action coverage, temporal order, and detail fidelity. On a manually created test set of slides, the LoRA model increases BLEU-4 by around 60%, ROUGE-L by 30%, and shows significant improvements in CODA-detail. This demonstrates that low-rank adaptation enables reliable temporal reasoning and generalization beyond synthetic data. Overall, our dataset, LoRA-enhanced model, and CODA metric provide a rigorous benchmark and foundation for future research on VLM-based dynamic slide generation.

Figures

Figures reproduced from arXiv: 2507.03916 by the authors.

Figure 1
Figure 1. Phase 1 pipeline for static slide synthesis: [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Phase 2 pipeline with Stage 1: static page description generation, Stage 2: animation plan generation, and Stage 3: slides video generation. 5 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Animation Type Distribution: Pie chart and bar chart illustrating the overall proportions of entrance, emphasis, and exit animations. The bar chart also displays the top 10 animation effects (e.g., Pinwheel, FlashBulb, etc.) in terms of frequency and percentage for each category, with the y-axis representing animation types and the x-axis showing frequency/percentage. LLMs exhibit distinct preferences for animation … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Animation Type Word Cloud: A word cloud displaying the frequency of all 52 animation types, with font size reflecting the frequency of occurrence. (Paired Entrance/Exit effects are merged.) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: (a) Duration Distribution Chart: Displays the distribution of animation effect durations ranging from 0.5s to 3s. (b) Delay Distribution Chart: Shows the distribution of delays between animations, ranging from 0s to 4s. (c) Number of Animations Distribution: The number…
Figure 6
Figure 6. Figure 6: Performance of different models on the 1,000-video synthetic test set. The leftmost bars correspond to our [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Performance variations on the synthetic test set across different frame rates and LoRA ranks, from Rank 8 at 1 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Performance of different models on the 50-video manually created test set. Left two bars correspond to our [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 19 canonical work pages

  1. [1]

    Docvqa: A dataset for vqa on document images,

    M. Mathew, D. Karatzas, and C. Jawahar, “Docvqa: A dataset for vqa on document images,” inProceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 2200–2209

    work page 2021

  2. [2]

    Infographicvqa,

    M. Mathew, V . Bagal, R. Tito, D. Karatzas, E. Valveny, and C. Jawahar, “Infographicvqa,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 1697–1706

    work page 2022

  3. [3]

    Slidevqa: A dataset for document visual question answering on multiple images,

    R. Tanaka, K. Nishida, K. Nishida, T. Hasegawa, I. Saito, and K. Saito, “Slidevqa: A dataset for document visual question answering on multiple images,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 11, 2023, pp. 13 636–13 645

    work page 2023

  4. [4]

    Msr-vtt: A large video description dataset for bridging video and language,

    J. Xu, T. Mei, T. Yao, and Y . Rui, “Msr-vtt: A large video description dataset for bridging video and language,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 5288–5296

    work page 2016

  5. [5]

    Dense-captioning events in videos,

    R. Krishna, K. Hata, F. Ren, L. Fei-Fei, and J. Carlos Niebles, “Dense-captioning events in videos,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 706–715

    work page 2017

  6. [6]

    Towards vqa models that can read,

    A. Singh, V . Natarajan, M. Shah, Y . Jiang, X. Chen, D. Batra, D. Parikh, and M. Rohrbach, “Towards vqa models that can read,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8317–8326

    work page 2019

  7. [7]

    Qwen2. 5-vl technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang et al., “Qwen2. 5-vl technical report,” arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  8. [8]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022. 16 Animation Needs Attention A PREPRINT

    work page 2022

Show all 39 references
  1. [9]

    python-pptx: Create open xml powerpoint documents in python,

    S. Canny, “python-pptx: Create open xml powerpoint documents in python,” 2019/05 2019, version 0.6.18. [Online]. Available: https://python-pptx.readthedocs.io/en/latest/

    2019

  2. [10]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318

    2002

  3. [11]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” inText summarization branches out, 2004, pp. 74–81

    2004

  4. [12]

    Spice: Semantic propositional image caption evaluation,

    P. Anderson, B. Fernando, M. Johnson, and S. Gould, “Spice: Semantic propositional image caption evaluation,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14. Springer, 2016, pp. 382–398

    2016

  5. [13]

    Clipscore: A reference-free evaluation metric for image captioning,

    J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y . Choi, “Clipscore: A reference-free evaluation metric for image captioning,” arXiv preprint arXiv:2104.08718, 2021

    2021 arXiv

  6. [14]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” in International conference on machine learning . PMLR, 2023, pp. 19 730–19 742

    2023

  7. [15]

    Geollava: Efficient fine-tuned vision-language models for temporal change detection in remote sensing,

    H. Elgendy, A. Sharshar, A. Aboeitta, Y . Ashraf, and M. Guizani, “Geollava: Efficient fine-tuned vision-language models for temporal change detection in remote sensing,” arXiv preprint arXiv:2410.19552, 2024

    2024 arXiv

  8. [16]

    Instructblip: Towards general-purpose vision-language models with instruction tuning,

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. Hoi, “Instructblip: Towards general-purpose vision-language models with instruction tuning,” 2023. [Online]. Available: https://arxiv.org/abs/2305.06500

    2023 arXiv

  9. [17]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al., “Flamingo: a visual language model for few-shot learning,” Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  10. [18]

    Layoutlm: Pre-training of text and layout for document image understanding,

    Y . Xu, M. Li, L. Cui, S. Huang, F. Wei, and M. Zhou, “Layoutlm: Pre-training of text and layout for document image understanding,” in Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 1192–1200

    2020

  11. [19]

    Layoutlmv2: Multi-modal pre-training for visually-rich document understanding,

    Y . Xu, Y . Xu, T. Lv, L. Cui, F. Wei, G. Wang, Y . Lu, D. Florencio, C. Zhang, W. Che et al., “Layoutlmv2: Multi-modal pre-training for visually-rich document understanding,” arXiv preprint arXiv:2012.14740, 2020

    2012 arXiv

  12. [20]

    Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding,

    Y . Xu, T. Lv, L. Cui, G. Wang, Y . Lu, D. Florencio, C. Zhang, and F. Wei, “Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding,” arXiv preprint arXiv:2104.08836, 2021

    2021 arXiv

  13. [21]

    Layoutlmv3: Pre-training for document ai with unified text and image masking,

    Y . Huang, T. Lv, L. Cui, Y . Lu, and F. Wei, “Layoutlmv3: Pre-training for document ai with unified text and image masking,” in Proceedings of the 30th ACM international conference on multimedia, 2022, pp. 4083–4091

    2022

  14. [22]

    Unifying vision, text, and layout for universal document processing,

    Z. Tang, Z. Yang, G. Wang, Y . Fang, Y . Liu, C. Zhu, M. Zeng, C. Zhang, and M. Bansal, “Unifying vision, text, and layout for universal document processing,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 19 254–19 264

    2023

  15. [23]

    Towards automatic learning of procedures from web instructional videos,

    L. Zhou, C. Xu, and J. Corso, “Towards automatic learning of procedures from web instructional videos,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018

    2018

  16. [24]

    Video-chatgpt: Towards detailed video understanding via large vision and language models,

    M. Maaz, H. Rasheed, S. Khan, and F. S. Khan, “Video-chatgpt: Towards detailed video understanding via large vision and language models,” arXiv preprint arXiv:2306.05424, 2023

    2023 arXiv

  17. [25]

    Video-llama: An instruction-tuned audio-visual language model for video understanding,

    H. Zhang, X. Li, and L. Bing, “Video-llama: An instruction-tuned audio-visual language model for video understanding,” arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

  18. [26]

    X-clip: End-to-end multi-grained contrastive learning for video-text retrieval,

    Y . Ma, G. Xu, X. Sun, M. Yan, J. Zhang, and R. Ji, “X-clip: End-to-end multi-grained contrastive learning for video-text retrieval,” inProceedings of the 30th ACM international conference on multimedia, 2022, pp. 638–647

    2022

  19. [27]

    Vcapsbench: A large-scale fine-grained benchmark for video caption quality evaluation,

    S.-X. Zhang, H. Wang, D. Huang, X. Li, X. Zhu, and X.-C. Yin, “Vcapsbench: A large-scale fine-grained benchmark for video caption quality evaluation,” arXiv preprint arXiv:2505.23484, 2025

    2025 arXiv

  20. [28]

    Llama-adapter v2: Parameter-efficient visual instruction model,

    P. Gao, J. Han, R. Zhang, Z. Lin, S. Geng, A. Zhou, W. Zhang, P. Lu, C. He, X. Yueet al., “Llama-adapter v2: Parameter-efficient visual instruction model,”arXiv preprint arXiv:2304.15010, 2023

    2023 arXiv

  21. [29]

    Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,

    Q. Zhang, M. Chen, A. Bukharin, N. Karampatziakis, P. He, Y . Cheng, W. Chen, and T. Zhao, “Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,”arXiv preprint arXiv:2303.10512, 2023

    2023 arXiv

  22. [30]

    Qlora: Efficient finetuning of quantized llms,

    T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “Qlora: Efficient finetuning of quantized llms,” Advances in neural information processing systems, vol. 36, pp. 10 088–10 115, 2023

    2023

  23. [31]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190, 2021. 17 Animation Needs Attention A PREPRINT

    2021 arXiv

  24. [32]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,

    X. Liu, K. Ji, Y . Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv preprint arXiv:2110.07602, 2021

    2021 arXiv

  25. [33]

    Evaluation metrics for video captioning: A survey,

    A. de Souza Inácio and H. S. Lopes, “Evaluation metrics for video captioning: A survey,” Machine Learning with Applications, vol. 13, p. 100488, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S2666827023000415

    2023

  26. [34]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis,

    C. Fu, Y . Dai, Y . Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y . Shen, M. Zhanget al., “Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, p...

    2025

  27. [35]

    Mvbench: A comprehensive multi-modal video understanding benchmark,

    K. Li, Y . Wang, Y . He, Y . Li, Y . Wang, Y . Liu, Z. Wang, J. Xu, G. Chen, P. Luoet al., “Mvbench: A comprehensive multi-modal video understanding benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 22 195–22 206

    2024

  28. [36]

    Mmbench-video: A long-form multi-shot benchmark for holistic video understanding,

    X. Fang, K. Mao, H. Duan, X. Zhao, Y . Li, D. Lin, and K. Chen, “Mmbench-video: A long-form multi-shot benchmark for holistic video understanding,” Advances in Neural Information Processing Systems, vol. 37, pp. 89 098–89 124, 2024

    2024

  29. [37]

    Tempcompass: Do video llms really understand videos?

    Y . Liu, S. Li, Y . Liu, Y . Wang, S. Ren, L. Li, S. Chen, X. Sun, and L. Hou, “Tempcompass: Do video llms really understand videos?” arXiv preprint arXiv:2403.00476, 2024

    2024 arXiv

  30. [38]

    Llamafactory: Unified efficient fine-tuning of 100+ language models,

    Y . Zheng, R. Zhang, J. Zhang, Y . Ye, Z. Luo, Z. Feng, and Y . Ma, “Llamafactory: Unified efficient fine-tuning of 100+ language models,” arXiv preprint arXiv:2403.13372, 2024

    2024 arXiv

  31. [39]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu et al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,” arXiv preprint arXiv:2402.03300, 2024. 18

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.