Pith. sign in

REVIEW 4 major objections 3 minor 64 references

AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single model can score AI-generated videos on nine aspects and explain each score.

desk verdict Solid, useful benchmark and plausible finetuning recipe, but comment-quality claims rest on LLM-revised gold with no human validation and the refinement loop is circular. read the letter →

arxiv 2507.01255 v1 pith:2RVTLUFZ submitted 2025-07-02 cs.CV

Xiao Liu , Jiawei Zhang This is my paper
classification cs.CV
keywords AI-generatedvideoevaluationvision-languagemodelmulti-aspectscoringexplanatorycommentsdynamicframesamplingtoken-wiseweightedlosshumanalignmentAIGVE-BENCH2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single vision-language model can replace collections of ad hoc metrics for judging AI-generated video: given a prompt and a generated clip, the model returns scores on nine quality aspects plus a written comment justifying each aspect. To train it, the authors assembled a benchmark of 2,500 videos from five generators with 22,500 human-annotated score-comment pairs, then fine-tuned a 7-billion-parameter vision-language model with a token-wise weighted loss and a dynamic frame sampler. On their own test set the model's average rank correlation with human scores is 52.20, close to twice the strongest baseline's 26.72, and it also transfers to three benchmarks it was never trained on, beating a prior specialist model while using one-tenth of its training data. The practical payoff is interpretable evaluation: scores and comments together, so developers can see not only that a generated video is weak but why, and feed that feedback back into generation.

What carries the argument

Two training mechanisms carry the result. The first is a token-wise weighted loss: during fine-tuning, the cross-entropy loss up-weights tokens that form the score and comment outputs by a factor of 50, so the model cannot learn to ignore the sparse task-bearing tokens in favor of abundant system, visual, and JSON-format tokens; the training data also places comments before scores to tie them together. The second is dynamic frame sampling: instead of taking equally spaced frames, the model selects frames whose grayscale pixel changes exceed a threshold, subject to a minimum gap, then uniformly subsamples if too many qualify, with uniform sampling as fallback. The benchmark AIGVE-BENCH 2 supplies the paired score-comment supervision, after an LLM-based revisor and human reviewer normalize the raw comments.

What would settle it

Take a random sample of videos from the benchmark and have fresh human raters independently score and comment on them using the same nine aspects; if their scores and comments diverge from the published annotations as much as the model does, or if the revised comments differ from the original human comments as much as they differ from the model's, then the alignment claims lack support.

Watch

Extended reading notes

Core claim

The central claim is that jointly predicting numeric scores and free-form comments for the same nine aspects—rather than attaching a scoring head or asking yes/no questions—makes a vision-language model align with human evaluators far better than prompting or scoring-only fine-tuning. Trained on AIGVE-BENCH 2, the model reports average Spearman $\rho = 52.20$ on the benchmark's test set, roughly double the best prompting baseline ($\rho = 26.72$), and it generates comments judged closer to human-written ones by ROUGE, BERTScore, UniEval-Fact, and G-Eval. The same model, evaluated without fine-tuning on three other video-evaluation benchmarks, beats prior zero-shot evaluators, including a 4.24% gain over a specialist scoring model on VBench while trained on one-tenth of that model's data. The paper further claims that when its scores and comments drive a multi-agent loop that rewrites the prompt and regenerates the video, instruction-aligned quality rises by 53.5% over at most four iterations, while purely visual aspects stay bounded by the generator.

Load-bearing premise

The load-bearing premise is that the 22,500 human score-comment annotations in the benchmark are trustworthy, and in particular that the LLM revision step preserves what human raters meant without injecting systematic bias; the paper reports no inter-annotator agreement or fidelity check for that step.

Editorial extensions

If this is right

  • Per-aspect scores plus justifications make a generated video diagnosable: a low score can be traced to physics, missing elements, or motion, not just an overall number.
  • Zero-shot transfer to three unseen benchmarks indicates the evaluation skill is general rather than memorized from one dataset.
  • The 53.5% improvement in the feedback loop suggests score-and-comment signals can guide prompt revision for instruction-driven aspects.
  • Training on roughly one day with two GPUs shows that a small, high-quality annotation set can turn a general vision-language model into a competitive video judge.
  • A unified evaluator that outputs both scores and comments can serve as a shared substrate for comparing video generators, since the same nine aspects apply across models and prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark annotations are as reliable as claimed, the generated comments could serve as weak supervision for smaller, faster evaluators or as filters for video-generation training data; the paper does not explore this.
  • The dynamic-sampling principle (favor frames with content change) should carry over to long-form video, where uniform sampling wastes context, but the paper only tests 5-second clips.
  • The refinement loop's 53.5% gain is measured on the evaluator's own scores; an independent human rating of the refined videos would clarify how much of the gain is perceived quality rather than conformity to the model's preferences.
  • Because the benchmark comments were rewritten by an LLM before training, part of the comment-quality advantage may reflect stylistic similarity to LLM text; a human preference study over matched comments would settle whether the explanations are genuinely more useful.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. AIGVE-MACS is a finetuned Qwen2.5-VL-7B model that produces nine aspect-wise numerical scores and natural-language comments for AI-generated videos. It is trained on AIGVE-BENCH 2, a benchmark of 2,500 videos with 22,500 score/comment annotations, using a token-wise weighted loss and a dynamic frame sampling strategy. The paper reports an average Spearman correlation of 52.20% on AIGVE-BENCH 2-TEST, zero-shot transfer to VideoFeedback, GenAI-Bench, and VBench, and a multi-agent iterative refinement framework that reportedly improves generated-video quality by 53.5%.

Significance. If the results hold, the paper delivers a useful artifact: a single model that outputs both interpretable comments and aspect-wise scores, with demonstrated zero-shot transfer to several benchmarks and an open release of the dataset and model. The design choices of token-wise weighted loss and dynamic frame sampling are sensible and are supported by ablation results. However, the comment-quality claim rests on a gold standard that has been revised by an LLM, and the refinement improvement is measured by the same model that generates the feedback; both of these are load-bearing for the paper's strongest claims and need independent validation before the conclusions can be accepted.

major comments (4)
  1. [Section 3.2, Table 2] The comment-quality evaluation is not anchored to original human judgments. The gold comments in AIGVE-BENCH 2 are produced by an LLM revisor, checked by an LLM evaluator, and then reviewed by a human, but the paper reports no inter-annotator agreement, no statistics on how often the human evaluator accepted or rewrote the LLM output, and no before/after comparison between original and revised comments. Since AIGVE-MACS is trained on these revised comments and evaluated against them, the high ROUGE-1 (49.50 vs. 18.30 for GPT-4o), BERTScore, and G-Eval values could reflect stylistic convergence to the revisor's generation habits rather than genuine alignment with human judgments. The authors should report such statistics or add a human evaluation of comment quality, and should compare against the original, unrevised comments as a control.
  2. [Section 6, Figure 6] The 53.5% quality-enhancement claim is circular. AIGVE-MACS both generates the feedback that drives the refinement loop and provides the scores used to measure the improvement; no human evaluation or independent metric validates the increase. Because the refinement pipeline is presented as a demonstration of real-world applicability, the authors should verify the reported gains with human raters or with an external, non-AIGVE-MACS evaluation protocol.
  3. [Table 1] The headline comparison in Table 1 is between a model finetuned on the benchmark (AIGVE-MACS) and zero-shot baselines such as GPT-4o and Qwen2.5-VL. The resulting 'nearly doubling' of average Spearman correlation is therefore not an apples-to-apples comparison and should not be described as state-of-the-art alignment without qualification. The authors should clarify that these baselines are zero-shot, or report results for baselines finetuned on the same training data, to make the advantage attributable to the method rather than to supervised adaptation.
  4. [Section 4.2, Appendix A, Eq. (5)] The dynamic frame sampling method depends on hyperparameters theta and gamma, and on the target number N of sampled frames, but none of these values are reported in Appendix A or elsewhere. The ablation in Table 5 shows that dynamic sampling matters, yet the paper does not indicate how robust the results are to these choices. Please report the exact values used and provide a small sensitivity analysis over theta, gamma, and N.
minor comments (3)
  1. [Abstract] The abstract calls the benchmark '22,500 human-annotated detailed comments,' but Section 3.2 explains that these comments were revised by an LLM and only reviewed by humans; the wording should be adjusted for precision.
  2. [Throughout] There are several typos and formatting issues, including 'viusal' in Section 4.1, 'accuratly' in Section 4.1, 'Tencet' in Section 2.1, 'trafic hum' and 'imp act' in Appendix D, and inconsistent spacing in the title and in Equations (2) and (3).
  3. [Figure 6] The figure shows average scores across iterations but does not include error bars, the number of videos that reached each iteration, or statistical significance tests for the claimed improvements.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: the 53.5% refinement gain is measured by AIGVE-MACS's own scores, and comment quality is judged against LLM-revised gold comments that the model was trained to imitate.

  1. self definitional [Section 6 'Multi-Agent Iterative Refinement Framework', Figure 6]
    "In each iteration, the Video Generator produces a video based on the current user instruction, and then AIGVE-MACS evaluates the generated video and provides feedback in the form of scores and comments. The Instruction Revisor refines the instruction based on the feedback, aiming to clarify or adjust the requirements for the next iteration. ... The iteration continues until the overall score exceeds 4 or the iteration limit of 4 is reached."

    The claimed 53.5% quality enhancement is computed as an increase in AIGVE-MACS's own overall scores, and the feedback that drives the revision is also generated by AIGVE-MACS. The same fitted model acts as both the optimization signal and the measurement instrument, so the result only shows that the Instruction Revisor can raise AIGVE-MACS's learned score. Without an independent human or external metric, the improvement is defined by the model being validated and therefore reduces to the model's own scoring function by construction.

  2. fitted input called prediction [Section 3.2 'Comment Processing Pipeline' and Table 2]
    "we first employ a Large Language Model (LLM) as a revisor to refine and extend the original comments, conditioned on the video instructions and associated scores. ... Finally, a human evaluator manually reviews and, if necessary, rewrites the revised comments to ensure clarity, factual accuracy, and faithful reflection of the corresponding scores."

    Table 2 evaluates AIGVE-MACS's generated comments against these LLM-revised comments as the gold standard. Since AIGVE-MACS is finetuned on the same revised-comment distribution, the high ROUGE, BERTScore, UniEval-Fact, and G-Eval values largely measure stylistic convergence to the revision pipeline rather than alignment with the original human annotations. The abstract's characterization of the benchmark as containing 'human-annotated detailed comments' is not reflected in the actual targets used for training and evaluation, so the comment-quality pillar is not independently established.

full rationale

The paper's numerical scoring claims are largely self-contained against external benchmarks: Table 3 (VideoFeedback), Table 4 (GenAI-Bench, VBench), and the zero-shot transfer results use independent human-preference or established benchmark labels. Those parts are not circular. The circularity is partial and concentrated in two places. First, the multi-agent refinement result is self-referential: AIGVE-MACS both supplies the feedback and measures the resulting quality gain, so the 53.5% improvement is an internal consistency result, not independent evidence of human-aligned quality improvement. Second, the comment-generation evaluation compares the model against LLM-revised gold comments produced by the same kind of pipeline on which the model was trained. The presence of a human review step does not repair the absence of reported inter-annotator agreement or any quantitative check that the LLM revision preserved the original human judgments. The central scoring results retain independent content, but the comment-quality claim and the refinement claim reduce in part to properties of the authors' own benchmark and model. The Table 1 supervised-versus-zero-shot comparison is also a fairness concern, but that is not a circularity issue.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper's contributions are an empirical system and a benchmark; it does not introduce new theoretical entities. The central claims rest on the quality of human annotations and the LLM revision pipeline, on the choice of sampling hyperparameters, and on the representational adequacy of the Qwen2.5-VL backbone. The refinement claim rests on accepting AIGVE-MACS's own judgments as ground truth.

free parameters (4)
  • alpha (token weight for score/comment tokens) = 50
    Set to 50 in Appendix A; controls the relative emphasis on score and comment tokens in the weighted loss. No sensitivity analysis is reported.
  • theta (content change threshold) = not reported
    Threshold in Eq. 5 that determines whether a frame is selected based on pixel change ratio. The value is not given in the paper.
  • gamma (minimum frame gap) = not reported
    Minimum gap between selected frames in Eq. 5. The value is not given.
  • N (target number of sampled frames) = not reported
    Number of frames fed to the VLM after dynamic sampling. The value is not given.
assumptions (5)
  • domain assumption Human annotations in AIGVE-BENCH 2 are accurate and unbiased.
    Section 3 describes expert raters but does not report inter-annotator agreement or any quality control statistics for the scores.
  • domain assumption LLM-revised comments preserve the content and intent of original human comments without hallucination.
    Section 3.2 relies on an LLM evaluator and human review, but no quantitative check of faithfulness is reported.
  • domain assumption The nine evaluation aspects are comprehensive and well-defined.
    Table 6 gives definitions, but there is no validation that these aspects cover all relevant quality dimensions of AI-generated video.
  • domain assumption Qwen2.5-VL-7B provides a sufficient visual representation for video evaluation.
    The model is built on this backbone with a frozen vision encoder; no comparison with other backbones is provided.
  • ad hoc to paper Pixel-level grayscale differences identify the most informative frames for evaluation.
    Equation 4 defines a simple change-counting measure without evidence that it captures semantically important moments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation." pith.science (2026). https://pith.science/paper/2RVTLUFZ

@misc{pith2026250701255,
  author       = {Pith},
  title        = {Pith review of: AIGVE-MACS: Unified Multi-Aspect Commenting and Scoring Model for AI-Generated Video Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2RVTLUFZ}},
  note         = {Machine review of arXiv:2507.01255}
}
read the original abstract

The rapid advancement of AI-generated video models has created a pressing need for robust and interpretable evaluation frameworks. Existing metrics are limited to producing numerical scores without explanatory comments, resulting in low interpretability and human evaluation alignment. To address those challenges, we introduce AIGVE-MACS, a unified model for AI-Generated Video Evaluation(AIGVE), which can provide not only numerical scores but also multi-aspect language comment feedback in evaluating these generated videos. Central to our approach is AIGVE-BENCH 2, a large-scale benchmark comprising 2,500 AI-generated videos and 22,500 human-annotated detailed comments and numerical scores across nine critical evaluation aspects. Leveraging AIGVE-BENCH 2, AIGVE-MACS incorporates recent Vision-Language Models with a novel token-wise weighted loss and a dynamic frame sampling strategy to better align with human evaluators. Comprehensive experiments across supervised and zero-shot benchmarks demonstrate that AIGVE-MACS achieves state-of-the-art performance in both scoring correlation and comment quality, significantly outperforming prior baselines including GPT-4o and VideoScore. In addition, we further showcase a multi-agent refinement framework where feedback from AIGVE-MACS drives iterative improvements in video generation, leading to 53.5% quality enhancement. This work establishes a new paradigm for comprehensive, human-aligned evaluation of AI-generated videos. We release the AIGVE-BENCH 2 and AIGVE-MACS at https://huggingface.co/xiaoliux/AIGVE-MACS.

Figures

Figures reproduced from arXiv: 2507.01255 by the authors.

Figure 1
Figure 1. AIGVE-MACS Pipeline. The left side of the diagram illustrates our Dynamic Frame Sampling strategy, which selects key moments based on content variation to better capture temporal dynamics. The right side highlights the Token-Wise Weighted Loss, which emphasizes score and comment tokens to improve alignment with human evaluations. TQ and OR refer to Technical Quality and Overall, respectively. Tokens in the yellow bo… view at source ↗
Figure 2
Figure 2. Overview of the comment processing pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Case Study of AIGVE-MACS. This example showcases AIGVE-MACS’s ability to produce human-aligned evaluations and distinguish fine-grained quality differences in AI-generated videos. The model not only accurately assesses natural object fidelity and realism but also captures temporal dynamics, such as smooth versus abrupt scene transitions, leading to coherent and interpretable multi-aspect judgments. to 61% improvemen… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The pipeline of the multi-agent iterative re [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 7
Figure 7. Figure 7: Comment Word Number Distribution of AIGVE-BENCH 2. The distribution of combined comment word counts indicates that the revised comments are notably rich and thorough, averaging around 267 words per entry. This high word count reflects a strong emphasis on providing con…
Figure 8
Figure 8. Figure 8: Case Study of AIGVE-MACS. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 16 canonical work pages

  1. [1]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. http://arxiv.org/abs/2308.12966 Qwen- VL : A Versatile Vision - Language Model for Understanding , Localization , Text Reading , and Beyond . arXiv preprint. ArXiv:2308.12966 [cs]

    arXiv 2023

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    arXiv 2025

  3. [3]

    Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. 2024. Videophy: Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.03520

    arXiv 2024

  4. [4]

    Shane Barratt and Rishi Sharma. 2018. A note on the inception score. arXiv preprint arXiv:1801.01973

    arXiv 2018

  5. [5]

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, and 1 others. 2023. Improving image generation with better captions (2023). URL https://cdn. openai. com/papers/dall-e-3. pdf

    work page 2023

  6. [6]

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \'e J \'e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. 2021. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650--9660

    2021

  7. [7]

    Baoliang Chen, Lingyu Zhu, Guo Li, Fangbo Lu, Hongfei Fan, and Shiqi Wang. 2022. https://doi.org/10.1109/TCSVT.2021.3088505 Learning generalized spatial-temporal deep feature representation for no-reference video quality assessment . IEEE Transactions on Circuits and Systems for Video Technology, 32(4):1903--1916

    work page arXiv 2022

  8. [8]

    Rui Chen, Lei Sun, Jing Tang, Geng Li, and Xiangxiang Chu. 2025. Finger: Content aware fine-grained evaluation with reasoning for ai-generated videos. arXiv preprint arXiv:2504.10358

    arXiv 2025

Show all 64 references
  1. [9]

    Zijian Chen, Wei Sun, Yuan Tian, Jun Jia, Zicheng Zhang, Jiarui Wang, Ru Huang, Xiongkuo Min, Guangtao Zhai, and Wenjun Zhang. 2024. https://arxiv.org/abs/2406.06087 Gaia: Rethinking action quality assessment for ai-generated videos . Preprint, arXiv:2406.06087

    2024 arXiv

  2. [10]

    Xianfu Cheng, Wei Zhang, Shiwei Zhang, Jian Yang, Xiangyuan Guan, Xianjie Wu, Xiang Li, Ge Zhang, Jiaheng Liu, Yuying Mai, Yutao Zeng, Zhoufutu Wen, Ke Jin, Baorui Wang, Weixiao Zhou, Yunhong Lu, Tongliang Li, Wenhao Huang, and Zhoujun Li. 2025. https://arxiv.org/abs/2502.1305...

    2025 arXiv

  3. [11]

    Jaemin Cho, Yushi Hu, Jason Baldridge, Roopal Garg, Peter Anderson, Ranjay Krishna, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2024. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. In ICLR

    2024

  4. [12]

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, and 1 others. 2021. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822--19835

    2021

  5. [13]

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bohan Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Bill Yuchen Lin, and Wenhu Chen. 2024 a . https://doi.org/...

    2024 doi

  6. [14]

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bohan Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Yuchen Lin, and Wenhu Chen. 2024 b . https://arxiv.org/abs...

    2024 arXiv

  7. [15]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.595 CLIPS core: A reference-free evaluation metric for image captioning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language ...

    2021 doi

  8. [16]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. 2022. https://arxiv.org/abs/2204.03458 Video diffusion models . Preprint, arXiv:2204.03458

    2022 arXiv

  9. [17]

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. 2023. https://arxiv.org/abs/2303.11897 Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering . Preprint, arXiv:2303.11897

    2023 arXiv

  10. [18]

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. 2023. https://doi.org/10.48550/arXiv.2311.17982 VBench : Comprehensi...

    2023 doi arXiv

  11. [19]

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. 2025. Pyramidal flow matching for efficient video generative modeling. In Proceedings of the International Conference on Learning Representations (ICLR)

    2025

  12. [20]

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. 2023. Pick-a-pic: An open dataset of user preferences for text-to-image generation

    2023

  13. [21]

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, and 33 others. 2024. https://arxiv.org/abs/2412.0...

    2024 arXiv

  14. [22]

    Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, and Ning Liu. 2024 a . https://doi.org/10.48550/arXiv.2403.11956 Subjective- Aligned Dataset and Metric for Text -to- Video Quality Assessment . arXiv preprint. ArXiv:2403.11956 [cs]

    2024 doi arXiv

  15. [23]

    Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, and Ning Liu. 2024 b . Subjective-aligned dataset and metric for text-to-video quality assessment. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 7793--7802

    2024

  16. [24]

    Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. 2023. https://doi.org/10.48550/arXiv.2312.14867 VIEScore : Towards Explainable Metrics for Conditional Image Synthesis Evaluation . arXiv preprint. ArXiv:2312.14867 [cs]

    2023 doi arXiv

  17. [25]

    Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh

    Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander M. Rush, Douwe Kiela, Matthieu Cord, and Victor Sanh. 2023. https://arxiv.org/abs/2306.16527 Obelics: An open web-scale filtered dataset of ...

    2023 arXiv

  18. [26]

    Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2024. https://arxiv.org/abs/2405.02246 What matters when building vision-language models? Preprint, arXiv:2405.02246

    2024 arXiv

  19. [27]

    Daeun Lee, Jaehong Yoon, Jaemin Cho, and Mohit Bansal. 2024. Videorepair: Improving text-to-video generation via misalignment evaluation and localized refinement. arXiv preprint arXiv:2411.15115

    2024 arXiv

  20. [28]

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, and Deva Ramanan. 2024. https://arxiv.org/abs/2406.13743 Genai-bench: Evaluating and improving compositional text-to-visual generation . Preprint, arXiv...

    2024 arXiv

  21. [29]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. https://arxiv.org/abs/2201.12086 Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation . Preprint, arXiv:2201.12086

    2022 arXiv

  22. [30]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  23. [31]

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. 2024. https://arxiv.org/abs/2404.09967 Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model . Preprint, arXiv:2404.09967

    2024 arXiv

  24. [32]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 a . https://arxiv.org/abs/2310.03744 Improved baselines with visual instruction tuning . Preprint, arXiv:2310.03744

    2024 arXiv

  25. [33]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 b . https://llava-vl.github.io/blog/2024-01-30-llava-next/ Llava-next: Improved reasoning, ocr, and world knowledge

    2024

  26. [34]

    Xiao Liu, Xinhao Xiang, Zizhong Li, Yongheng Wang, Zhuoheng Li, Zhuosheng Liu, Weidi Zhang, Weiqi Ye, and Jiawei Zhang. 2024 c . https://arxiv.org/abs/2410.19884 A survey of ai-generated video evaluation . arXiv preprint arXiv:2410.19884

    2024

  27. [35]

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023 a . https://arxiv.org/abs/2303.16634 G-eval: Nlg evaluation using gpt-4 with better human alignment . Preprint, arXiv:2303.16634

    2023 arXiv

  28. [36]

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. 2023 b . https://doi.org/10.48550/arXiv.2310.11440 EvalCrafter : Benchmarking and Evaluating Large Video Generation Models . arXiv preprint. ArXiv:2...

    2023 doi arXiv

  29. [37]

    Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, and Rongrong Ji. 2022. X-clip: End-to-end multi-grained contrastive learning for video-text retrieval. In Proceedings of the 30th ACM international conference on multimedia, pages 638--647

    2022

  30. [38]

    Yibo Miao, Yifan Zhu, Yinpeng Dong, Lijia Yu, Jun Zhu, and Xiao-Shan Gao. 2024. https://arxiv.org/abs/2407.05965 T2vsafetybench: Evaluating the safety of text-to-video generative models . Preprint, arXiv:2407.05965

    2024 arXiv

  31. [39]

    Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012. https://doi.org/10.1109/TIP.2012.2214050 No-reference image quality assessment in the spatial domain . IEEE Transactions on Image Processing, 21(12):4695--4708

    2012

  32. [40]

    OpenAI. 2024 a . Gpt-4o. https://openai.com/index/hello-gpt-4o/

    2024

  33. [41]

    OpenAI. 2024 b . Sora. https://openai.com/index/sora/

    2024

  34. [42]

    OpenAI. 2025. Gpt-4.1. https://openai.com/index/gpt-4-1/

    2025

  35. [43]

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In International conference on machine learning, pages 8821--8831. Pmlr

    2021

  36. [44]

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. 2022. https://arxiv.org/abs/2209.14792 Make-a-video: Text-to-video generation without text-video data . Preprin...

    2022 arXiv

  37. [45]

    Gemini Team. 2024 a . https://arxiv.org/abs/2403.05530 Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context . Preprint, arXiv:2403.05530

    2024 arXiv

  38. [46]

    Genmo Team. 2024 b . Mochi 1. https://github.com/genmoai/models

    2024

  39. [47]

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. 2019. https://arxiv.org/abs/1812.01717 Towards accurate generative models of video: A new metric and challenges . Preprint, arXiv:1812.01717

    2019 arXiv

  40. [48]

    Narasimhan Venkatanath, D Praneeth, Maruthi Chandrasekhar Bh, Sumohana S Channappayya, and Swarup S Medasani. 2015. Blind image quality evaluation using perception based features. In 2015 twenty first national conference on communications (NCC), pages 1--6. IEEE

    2015

  41. [49]

    Comparing the effectiveness between human-generated videos and ai-generated videos on learning

    Shreyas Verma and John Leddo. Comparing the effectiveness between human-generated videos and ai-generated videos on learning

  42. [50]

    Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. 2025. Swap attention in spatiotemporal diffusions for text-to-video generation. International Journal of Computer Vision, pages 1--19

    2025

  43. [51]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. https://doi.org/10.1109/TIP.2003.819861 Image quality assessment: from error visibility to structural similarity . IEEE Transactions on Image Processing, 13(4):600--612

    2004

  44. [52]

    Haoning Wu, Liang Liao, Annan Wang, Chaofeng Chen, Jingwen Hou, Wenxiu Sun, Qiong Yan, and Weisi Lin. 2023 a . https://arxiv.org/abs/2304.14672 Towards robust text-prompted semantic criterion for in-the-wild video quality assessment . Preprint, arXiv:2304.14672

    2023 arXiv

  45. [53]

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. 2023 b . http://arxiv.org/abs/2211.04894 Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives . arXiv preprint. ...

    2023 arXiv

  46. [54]

    Xinhao Xiang, Xiao Liu, Zizhong Li, Zhuosheng Liu, and Jiawei Zhang. 2025. https://arxiv.org/abs/2503.14064 Aigve-tool: Ai-generated video evaluation toolkit with multifaceted benchmark . arXiv preprint arXiv:2503.14064

    2025 arXiv

  47. [55]

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, and 1 others. 2024. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072

    2024 arXiv

  48. [56]

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. 2025. https://arxiv.org/abs/2501.13106 Videollama 3: Frontier multimodal foundation model...

    2025 arXiv

  49. [57]

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. 2023. https://arxiv.org/abs/2309.15818 Show-1: Marrying pixel and latent diffusion models for text-to-video generation . Preprint, arXiv:2309.15818

    2023 arXiv

  50. [58]

    Fan Zhang, Shulin Tian, Ziqi Huang, Yu Qiao, and Ziwei Liu. 2024 a . Evaluation agent: Efficient and promptable evaluation framework for visual generative models. arXiv preprint arXiv:2412.09645

    2024 arXiv

  51. [59]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://arxiv.org/abs/1904.09675 Bertscore: Evaluating text generation with bert . Preprint, arXiv:1904.09675

    2020 arXiv

  52. [60]

    Goh, and Chenshuo Sun

    Xinyi Zhang, Renyu Zhang, K. Goh, and Chenshuo Sun. 2024 b . https://doi.org/10.48550/arXiv.2412.18337 The value of ai-generated metadata for ugc platforms: Evidence from a large-scale field experiment . ArXiv, abs/2412.18337

    2024 doi arXiv

  53. [61]

    Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022. https://arxiv.org/abs/2210.07197 Towards a unified multi-dimensional evaluator for text generation . Preprint, arXiv:2210.07197

    2022 arXiv

  54. [62]

    Xunchu Zhou, Xiaohong Liu, Yunlong Dong, Tengchuan Kou, Yixuan Gao, Zicheng Zhang, Chunyi Li, Haoning Wu, and Guangtao Zhai. 2024. https://arxiv.org/abs/2405.03333 Light-vqa+: A video quality assessment model for exposure correction with vision-language guidance . Preprint, ar...

    2024 arXiv

  55. [63]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  56. [64]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.