Pith. sign in

REVIEW 4 major objections 6 minor 41 references

Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a two-stage closed-loop optimizer—prompt refinement through an mLLM rubric plus Bayesian search over seed and guidance scale—beats unguided trial-and-error for image-to-video generation in human preference tests.

desk verdict Solid two-stage optimization for I2V with a real human-preference win, but the search objective is never shown to be moving—worth peer review with a request for objective traces. read the letter →

arxiv 2608.12290 v1 pith:D7QWVZ7H submitted 2026-08-12 cs.CV cs.AIcs.MM

Aman Tyagi , Hemanth Boinpally , Jonathan Chen , Douglas Gebert , Steven Hickson This is my paper · ORCID
classification cs.CVcs.AIcs.MM
keywords agenticoptimizationimage-to-videogenerationpromptBayesianhumanpreferenceevaluationvideo-textadherenceDavidsonianSceneGraphvideocontrollability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a closed-loop optimization framework can turn black-box image-to-video generation from a trial-and-error lottery into a predictable, goal-directed process. It does so with two coupled stages: an iterative prompt-refinement loop that uses a multimodal large language model (mLLM) to generate a yes/no rubric (Davidsonian Scene Graph questions for semantic content, Common Mistake Questions for artifacts), and a Bayesian search that co-optimizes random seeds and classifier-free guidance scales. A new Video-Text Adherence (VTA) score, derived from the same rubric, is part of the multi-objective reward guiding both stages. If the paper is right, professional users can get consistently better-aligned videos under the same compute budget, which is what the human-preference win rates up to 69% are meant to show.

What carries the argument

The load-bearing mechanism is the generator-critic loop. An mLLM constructs a per-prompt tree of yes/no questions; a video is scored by the fraction answered yes, with each child question zeroed out whenever its parent is answered no. That recursive masking makes foundational elements (the presence of an object, say) carry disproportionate weight, because failing them cascades to all descendants. The resulting VTA score is combined with perceptual-quality and motion-consistency measures into a multi-objective reward, which drives both prompt rewriting and a Bayesian co-optimization of seeds (treated as a categorical variable) and classifier-free guidance scales.

What would settle it

Re-run the 100-generation comparison, logging the raw reward values for every candidate, and compare the Bayesian search's top-1 against a best-of-random top-1 selected by the same metric ranking. If the reward values show no separation between conditions or if the best-of-random win rate matches the Bayesian win rate, the central claim that the search itself drives the improvement would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that black-box image-to-video generation can be steered reliably by a closed-loop, two-stage optimizer rather than brute-force sampling. Stage one repeatedly rewrites the prompt, scoring each output against an mLLM-generated tree of yes/no questions—DSG questions for semantic content and CMQ questions for artifacts—and keeps the prompt with the highest alignment. Stage two keeps that prompt fixed and runs Bayesian optimization over random seeds and classifier-free guidance scales, guided by a multi-objective score that includes a new Video-Text Adherence (VTA) metric built from the same question trees. In head-to-head human preference tests, the framework's top output beats a randomly selected baseline output in 60–69% of comparisons at a 100-generation budget, and beats a Best-of-Random baseline (random search plus identical metric-based selection) 42% to 8% under RAHF ranking. The paper reads these results as evidence that the search itself, and not merely post-hoc metric selection, is doing the work.

Load-bearing premise

The premise the argument stands on is that the automated scores steering the search (the quality models plus the VTA rubric) capture the same quality differences human raters see; if those scores are flat or noise-dominated across seeds and guidance scales, then the Bayesian search has no signal to exploit and the reported human-preference advantage is unexplained.

Editorial extensions

If this is right

  • At a 100-generation budget, the top video from the agentic pipeline is preferred by human raters over a randomly chosen baseline video in roughly 60–69% of comparisons; at 10 generations the win rate is 42–47%.
  • Against the stronger Best-of-Random baseline—random search followed by the same artifact filters and metric-based selection—the framework still wins 42% to 8% (RAHF ranking), indicating the value comes from the search trajectory and not just from final selection.
  • Prompt optimization alone, with hyperparameters held fixed, makes human raters prefer the optimized-prompt video 27% of the time versus 5% for the original prompt, with most pairs tied.
  • Because the framework is a model-agnostic wrapper with modular evaluators, the same closed-loop recipe transfers to other black-box video generators and can incorporate alternative quality objectives without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the VTA hierarchy is doing the work, an ablation that replaces recursive masking with a flat average of question scores should reduce the win rate; the paper does not report this comparison.
  • The same generator-critic loop could optimize for objectives the current rubric does not cover—narrative coherence, emotional tone, physical plausibility—by swapping in new question categories; the paper points to this direction but does not demonstrate it.
  • Because the paper only reports human preference for the top-1 selection, it leaves open whether the full ranking produced by the reward suite is reliable; a rank-correlation study between VTA/quality scores and human ratings would clarify how much of the gain is attributable to each stage.
  • The near-flat automated benchmark scores reported by the paper suggest that standard public metrics may systematically under-report improvements that human raters detect, which would make the framework look weaker on automated leaderboards than it is.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces 'Agentic Self-Improvement,' a two-stage framework for controlling black-box image-to-video (I2V) generation. Stage one iteratively refines the text prompt using an mLLM that generates Davidsonian Scene Graph (DSG) questions and Common Mistake Questions (CMQ), then uses its own yes/no answers as an alignment score to drive prompt rewriting. Stage two uses Bayesian optimization (via Vizier) over CFG scales and random seeds, guided by a multi-objective reward combining RAHF, UVQ, and a novel Video-Text Adherence (VTA) score derived from the DSG/CMQ tree. The paper reports a human preference study on 100 V-Bench image-prompt pairs, claiming win rates up to 69% against random search and 42% against a Best-of-Random baseline, and concludes that the closed-loop optimizer actively finds higher-quality parameter regions. The central claim is that the complete pipeline yields perceptibly better videos than unguided search under the same compute budget.

Significance. If the central claim holds, the framework is a useful practical contribution: it offers a modular, model-agnostic wrapper around black-box I2V APIs, and the human-preference evaluation is a step beyond purely automated benchmarks. The paper also makes a methodological point that standard automated metrics (V-Bench++) are insensitive to the quality differences humans perceive, which is worth taking seriously. However, the manuscript does not currently demonstrate the mechanism by which Bayesian search outperforms random search: the reward values observed during optimization are never reported, and the VTA objective is generated and scored by the same mLLM that the prompt optimizer is trying to satisfy. The significance of the claimed gains therefore rests on unverified assumptions about the informativeness of the reward signal.

major comments (4)
  1. [Section 4.2, Table 2] The paper's strongest claim is that the Bayesian optimizer 'actively and effectively guides the generation process into higher-quality parameter regions,' evidenced by 42% vs. 8% win rates against Best-of-Random. This inference is not supported because the objective values (RAHF, UVQ, VTA) observed during the 10- and 100-run searches are never reported. If these objectives are nearly flat or dominated by noise—as the paper itself argues for V-Bench++ metrics in Appendix 7.1—then the reported advantage could come almost entirely from the hard artifact filter and the final Top-1 selection, not from the search trajectory. The authors should report the distribution of each reward component across BO iterations and across random candidates, and ideally run a null-objective control where BO optimizes a random or shuffled reward, to demonstrate that the search itself is responsible for the win-rate gap.
  2. [Section 3.2.1, VTA definition] The VTA score is self-referential in a way that is load-bearing for the optimization claim: the same mLLM (Gemini 2.5 Pro) generates the DSG/CMQ question trees, answers them, and the prompt optimizer then rewrites prompts to maximize those answers. The human validation in Section 3.1.3 reports 87% overall agreement on 100 video-question pairs, but this does not validate that VTA rankings correlate with human preferences at the candidate-selection level, nor does it rule out the optimizer exploiting mLLM answer biases (e.g., rewording prompts to make questions easier to answer affirmatively). The authors should report per-question-type agreement, inter-annotator agreement if both annotators scored the same items, and a direct correlation between VTA scores and human preference judgments on the candidate pools used in Table 2.
  3. [Section 3.1.2 and Section 4.1] The prompt optimization loop's effectiveness is not quantitatively documented. The paper states that the loop runs for a fixed number of iterations and keeps the prompt with the highest alignment score, but it does not report alignment-score trajectories, the number of iterations used, or any example of the original versus optimized prompt. Without this evidence, it is unclear whether iterative rewriting actually improves semantic adherence or merely changes surface phrasing that happens to score higher on the mLLM's own questions. Providing before/after prompts and score trajectories is necessary to support the claim that prompt optimization is a meaningful component of the framework.
  4. [Section 4, Human preference study] The human preference study uses only two expert annotators, each evaluating 50 disjoint prompts, with no inter-annotator agreement computed. This makes it impossible to assess the reliability of the preference labels. The paper should either have both annotators score an overlapping subset, report Cohen's kappa or a similar agreement measure, or otherwise justify that a single annotator per prompt is sufficient for the reported binomial test to be meaningful. This is not fatal to the paper's central claim, but it weakens the strength of the headline 69% win rate.
minor comments (6)
  1. [Title page] The author line contains apparent formatting artifacts: 'AMAN TY AGI, HEMANTH BOINPALL Y ET. AL.' and 'Aman T yagi' should be cleaned up before submission.
  2. [Figure 1 caption] The figure caption includes a local file path 'file:///Users/brhemanth/Downloads/...' that should be removed.
  3. [Table 2] The table formatting is garbled; the numbers appear to run together (e.g., '144739'), making raw counts and percentages unreadable. Please reformat the table with clear column separators.
  4. [Section 4.2] The sentence on statistical significance uses superscripts such as 'p<10−6' without rendered superscript formatting; clarify whether the reported p-values are two-tailed binomial tests on the pooled 100 prompts or on a per-annotator basis.
  5. [Section 3.2.1] The VTA equation uses 'S_p(i)' without defining p(i) explicitly in the text; please define the parent mapping and state whether the root mask S_p(root)=1 applies to all root nodes.
  6. [References] Reference [41] lists 'Kai Zo' as an author; this appears to be a truncation and should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central human-preference claim is independently grounded and the Bayesian search is isolated against a Best-of-Random baseline.

full rationale

The paper's derivation chain is self-contained. The two-stage pipeline is evaluated by head-to-head human preference, which is independent of the automated metrics used for optimization. The VTA metric is defined as an aggregate of mLLM yes/no answers to DSG/CMQ questions, and while the same mLLM generates those questions, the paper validates the VQA judgments against human labels (Table 1: 87% overall agreement), so the optimization signal is not purely self-referential. The end-to-end comparison in Table 2 includes a Best-of-Random baseline that uses the same artifact filters and the same ranking metrics, isolating the contribution of Bayesian search from final selection. No parameter is fitted to the human-preference test set, and no prediction is derived from its own input by construction. The reported win rates are direct human judgments, not automated-score extrapolations. The only notable weakness is that raw RAHF/UVQ/VTA trajectory values are not reported, so the mechanism of the Bayesian advantage is not directly evidenced, but that is a reporting gap rather than circularity. No self-citation is load-bearing; the DSG reference [11] is external prior work, and the Bayesian optimizer (Vizier [34]) is likewise an external system.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The VTA metric is a design, not a postulated entity. The ledger captures the hand-set parameters and domain assumptions that the framework's success depends on.

free parameters (6)
  • VTA uniform weights (w_i=1) = 1 (uniform)
    Section 3.2.1 defines VTA as a weighted average with uniform weights; a different weighting scheme would change the optimization target and resulting rankings.
  • Prompt optimization iterations = 10
    Section 4.1 states the prompt optimizer runs for 10 iterations; this is a hand-set stopping criterion that affects the final prompt quality.
  • BO UCB exploration constant sqrt(beta) = sqrt(1.8)
    Section 3.2 specifies the Vizier UCB acquisition with an exploration constant of sqrt(beta=1.8); this balances exploration and exploitation in the search.
  • CFG search range = [1, 15]
    Section 3.2 states CFG scales are explored within [1, 15]; this range is chosen by hand and bounds the search space.
  • RAHF frame sampling = 10 frames, min aggregation
    Section 3.2.1 samples 10 equidistant frames per video and takes the minimum RAHF score as the video score; this is a design choice.
  • Hard artifact filter thresholds = 2-second loop or freeze
    Section 3.2.1 and 4.2 define loop detection as a two-second repetition or static freeze; these thresholds determine which candidates are discarded before ranking.
assumptions (5)
  • domain assumption Gemini 2.5 Pro's yes/no answers on DSG and CMQ questions are an acceptable proxy for human judgment.
    Section 3.1.3 validates against 100 pairs (87% overall, 82% CMQ), but the sample is small and the questions were themselves generated by the same model.
  • domain assumption Veo 2.0 is a representative image-to-video model and findings generalize to other black-box I2V models.
    Section 4 uses Veo 2.0 as the only experimental vehicle; the framework is claimed to be model-agnostic but no other model is tested.
  • domain assumption The 100 image-prompt pairs sampled from V-Bench's I2V tasks represent professional creative briefs.
    Section 4.1 uses a random sample from V-Bench; the sample may not capture the full diversity of production video-generation requests.
  • domain assumption Human preference for 'realism and overall video quality' is the correct ground truth for prompt adherence.
    Section 4 defines the evaluation criterion; other criteria such as stylistic fidelity or exact motion semantics could yield different rankings.
  • standard math Bayesian optimization with UCB acquisition is valid for the mixed continuous/categorical search space of CFG and seeds.
    The method follows Snoek et al. and the Vizier algorithm [28, 34]; this is a standard optimization assumption, not specific to the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence." pith.science (2026). https://pith.science/paper/D7QWVZ7H

@misc{pith2026260812290,
  author       = {Pith},
  title        = {Pith review of: Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D7QWVZ7H}},
  note         = {Machine review of arXiv:2608.12290}
}
read the original abstract

Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes. To address these limitations, we introduce the ``Agentic Self-Improvement" framework, which reframes video synthesis into a closed-loop, goal-directed optimization. Our framework systematically navigates the generation parameter space using a novel two-stage approach. In the first stage, an iterative prompt optimization loop uses a multimodal Large Language Model (mLLM) to refine the input prompt. This refinement implements two automated evaluations: Davidsonian Scene Graph (DSG) queries ensure semantic adherence, and Common Mistake Questions (CMQ) for artifact detection. At the second stage, we use Bayesian optimization to efficiently co-optimize stochastic seeds and CFG scales. This search is guided by a suite of quality metrics, including the novel Video-Text Adherence (VTA) score derived from the DSG and CMQ evaluations. Our framework significantly outperforms unguided search methods: in human preference studies, videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69\%. This work provides a practical and extensible methodology for enhancing the predictability and control of state-of-the-art video generation models, moving the field beyond speculative curiosities toward reliable, production-ready tools.

Figures

Figures reproduced from arXiv: 2608.12290 by the authors.

Figure 1
Figure 1. The two-stage architecture of our Agentic Self-Improvement framework. Top: [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Example of our automated question generation process for a given input image [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 17 canonical work pages

  1. [1]

    Sora as an agi world model? a complete survey on text-to-video generation.arXiv preprint arXiv:2403.05131, 2024

    Joseph Cho, Fachrina Dewi Puspitasari, Sheng Zheng, Jingyao Zheng, Lik-Hang Lee, Tae-Ho Kim, Choong Seon Hong, and Chaoning Zhang. Sora as an agi world model? a complete survey on text-to-video generation.arXiv preprint arXiv:2403.05131, 2024

    arXiv 2024

  2. [2]

    From sora what we can see: A survey of text-to-video generation.arXiv preprint arXiv:2405.10674, 2024

    Rui Sun, Yumin Zhang, Tejal Shah, Jiahao Sun, Shuoying Zhang, Wenqi Li, Haoran Duan, Bo Wei, and Rajiv Ranjan. From sora what we can see: A survey of text-to-video generation.arXiv preprint arXiv:2405.10674, 2024. 12: AMAN TY AGI, HEMANTH BOINPALL Y ET. AL

    arXiv 2024

  3. [3]

    Video-to-video synthesis.arXiv preprint arXiv:1808.06601, 2018

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Video-to-video synthesis.arXiv preprint arXiv:1808.06601, 2018

    arXiv 2018

  4. [4]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7310–7320, 2024

    2024

  5. [5]

    Vpo: Aligning text-to-video generation models with prompt optimization

    Jiale Cheng, Ruiliang Lyu, Xiaotao Gu, Xiao Liu, Jiazheng Xu, Yida Lu, Jiayan Teng, Zhuoyi Yang, Yuxiao Dong, Jie Tang, et al. Vpo: Aligning text-to-video generation models with prompt optimization. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 15636–15645, 2025

    work page 2025

  6. [6]

    Open-sora 2.0: Training a commercial-level video genera- tion model in $200k.arXiv preprint arXiv:2503.09642, 2025

    Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, Yuhui Wang, Anbang Ye, Gang Ren, Qianran Ma, Wanying Liang, Xiang Lian, Xiwen Wu, Yuting Zhong, Zhuangyan Li, Chaoyu Gong, Guojun Lei, Leijun Cheng, Limin Zhang, Minghao Li, Ruijie Zhang, Silan Hu, Shijie Huang, Xiaokang Wang, Yu...

    arXiv 2025

  7. [7]

    Make it move: controllable image-to- video generation with text descriptions

    Yaosi Hu, Chong Luo, and Zhenzhong Chen. Make it move: controllable image-to- video generation with text descriptions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18219–18228, 2022

    work page 2022

  8. [8]

    Prompt-a-video: Prompt your video diffusion model via preference-aligned llm

    Yatai Ji, Jiacheng Zhang, Jie Wu, Shilong Zhang, Shoufa Chen, Chongjian Ge, Peize Sun, Weifeng Chen, Wenqi Shao, Xuefeng Xiao, et al. Prompt-a-video: Prompt your video diffusion model via preference-aligned llm. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 18725–18735, 2025

    work page 2025

Show all 41 references
  1. [9]

    We need to talk about random seeds.arXiv preprint arXiv:2210.13393, 2022

    Steven Bethard. We need to talk about random seeds.arXiv preprint arXiv:2210.13393, 2022

    2022 arXiv

  2. [10]

    Rethink- ing the spatial inconsistency in classifier-free diffusion guidance

    Dazhong Shen, Guanglu Song, Zeyue Xue, Fu-Yun Wang, and Yu Liu. Rethink- ing the spatial inconsistency in classifier-free diffusion guidance. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9370– 9379, 2024

    2024

  3. [11]

    Davidsonian scene graph: Im- proving reliability in fine-grained evaluation for text-to-image generation

    Jaemin Cho, Yushi Hu, Jason Baldridge, Roopal Garg, Peter Anderson, Ranjay Kr- ishna, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Im- proving reliability in fine-grained evaluation for text-to-image generation. InInter- national conference on learning...

    2024

  4. [12]

    Generative adversarial nets.Ad- vances in neural information processing systems, 27, 2014

    Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets.Ad- vances in neural information processing systems, 27, 2014

    2014

  5. [13]

    An introduction to variational autoencoders

    Diederik P Kingma, Max Welling, et al. An introduction to variational autoencoders. F oundations and Trends® in Machine Learning, 12(4):307–392, 2019. : AMAN TY AGI, HEMANTH BOINPALL Y ET. AL.13

    2019

  6. [14]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  7. [15]

    Generating videos with scene dynamics.Advances in neural information processing systems, 29, 2016

    Carl V ondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics.Advances in neural information processing systems, 29, 2016

    2016

  8. [16]

    Video generation models as world simulators.OpenAI Blog, 1:8, 2024

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators.OpenAI Blog, 1:8, 2024

    2024

  9. [17]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InPro- ceedings of the IEEE/CVF international conference on computer vision, pages 4195– 4205, 2023

    2023

  10. [18]

    Videomar: Autoregressive video generation with continuous tokens.Advances in neural information processing systems, 38:56928– 56958, 2026

    Hu Yu, Biao Gong, Hangjie Yuan, DanDan Zheng, Weilong Chai, Jingdong Chen, Kecheng Zheng, and Feng Zhao. Videomar: Autoregressive video generation with continuous tokens.Advances in neural information processing systems, 38:56928– 56958, 2026

    2026

  11. [19]

    Adding conditional control to text- to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text- to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  12. [20]

    Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    2022 arXiv

  13. [21]

    Control-a-video: Controllable text-to-video diffusion models with motion prior and reward feedback learning.arXiv preprint arXiv:2305.13840, 2023

    Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video diffusion models with motion prior and reward feedback learning.arXiv preprint arXiv:2305.13840, 2023

    2023 arXiv

  14. [22]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pa...

    2023

  15. [23]

    Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery.Advances in Neural Information Processing Systems, 36:51008– 51025, 2023

    Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery.Advances in Neural Information Processing Systems, 36:51008– 51025, 2023

    2023

  16. [24]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. InInternational Conference on Learning Representations, volume 2024, pages 4965–4987, 2024

    2024

  17. [25]

    Automated black-box prompt engineering for personalized text-to-image generation

    Yutong He, Alexander Robey, Naoki Murata, Yiding Jiang, Joshua Williams, George J Pappas, Hamed Hassani, Yuki Mitsufuji, Ruslan Salakhutdinov, and J Zico Kolter. Automated black-box prompt engineering for personalized text-to-image generation. arXiv preprint arXiv:2403.19103, ...

    2024 arXiv

  18. [26]

    Optimizing prompts for text-to-image generation.Advances in Neural Information Processing Systems, 36:66923–66939, 2023

    Yaru Hao, Zewen Chi, Li Dong, and Furu Wei. Optimizing prompts for text-to-image generation.Advances in Neural Information Processing Systems, 36:66923–66939, 2023

    2023

  19. [27]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  20. [28]

    Practical bayesian optimization of machine learning algorithms.Advances in neural information processing systems, 25, 2012

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms.Advances in neural information processing systems, 25, 2012

    2012

  21. [29]

    Mm-react: Prompting chatgpt for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    2023 arXiv

  22. [30]

    Dialoggen: Multi-modal in- teractive dialogue system for multi-turn text-to-image generation.arXiv preprint arXiv:2403.08857, 2024

    Minbin Huang, Yanxin Long, Xinchi Deng, Ruihang Chu, Jiangfeng Xiong, Xiao- dan Liang, Hong Cheng, Qinglin Lu, and Wei Liu. Dialoggen: Multi-modal in- teractive dialogue system for multi-turn text-to-image generation.arXiv preprint arXiv:2403.08857, 2024

    2024 arXiv

  23. [31]

    The rise and poten- tial of large language model based agents: A survey: arxiv preprint.arXiv preprint arXiv:2309.07864, 2023

    Z Xi, W Chen, X Guo, W He, Y Ding, B Hong, and T Gui. The rise and poten- tial of large language model based agents: A survey: arxiv preprint.arXiv preprint arXiv:2309.07864, 2023

    2023 arXiv

  24. [32]

    A survey on large language model based autonomous agents.Frontiers of Computer Science, 18(6):186345, 2024

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents.Frontiers of Computer Science, 18(6):186345, 2024

    2024

  25. [33]

    Gemini 2.5 pro (gemini-2.5-pro-preview-06-05): Our most advanced intel- ligent ai model, 2025

    Google. Gemini 2.5 pro (gemini-2.5-pro-preview-06-05): Our most advanced intel- ligent ai model, 2025. URLhttps://developers.googleblog.com/en/ gemini-2-5-video-understanding/. Accessed on July 7th, 2025

    2025

  26. [34]

    The vizier gaussian process bandit algorithm.arXiv preprint arXiv:2408.11527, 2024

    Xingyou Song, Qiuyi Zhang, Chansoo Lee, Emily Fertig, Tzu-Kuo Huang, Lior Be- lenki, Greg Kochanski, Setareh Ariafar, Srinivas Vasudevan, Sagi Perel, et al. The vizier gaussian process bandit algorithm.arXiv preprint arXiv:2408.11527, 2024

    2024 arXiv

  27. [35]

    Rich human feedback for text-to-image generation

    Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Car- olan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image generation. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recogni...

    2024

  28. [36]

    Rich features for perceptual quality assessment of ugc videos

    Yilin Wang, Junjie Ke, Hossein Talebi, Joong Gon Yim, Neil Birkbeck, Balu Adsumilli, Peyman Milanfar, and Feng Yang. Rich features for perceptual quality assessment of ugc videos. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13435–...

    2021

  29. [37]

    Vbench++: Com- prehensive and versatile benchmark suite for video generative models.arXiv preprint arXiv:2411.13503, 2024

    Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, Yaohui Wang, Xinyuan Chen, Ying- Cong Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench++: Com- prehensive and versatile benchmark suite fo...

    2024 arXiv

  30. [38]

    The devil is in the prompts: Retrieval-augmented prompt optimiza- tion for text-to-video generation

    Bingjie Gao, Xinyu Gao, Xiaoxue Wu, Yujie Zhou, Yu Qiao, Li Niu, Xinyuan Chen, and Yaohui Wang. The devil is in the prompts: Retrieval-augmented prompt optimiza- tion for text-to-video generation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 3...

    2025

  31. [39]

    Q-bench-video: Benchmark the video quality understanding of lmms

    Zicheng Zhang, Ziheng Jia, Haoning Wu, Chunyi Li, Zijian Chen, Yingjie Zhou, Wei Sun, Xiaohong Liu, Xiongkuo Min, Weisi Lin, et al. Q-bench-video: Benchmark the video quality understanding of lmms. InProceedings of the Computer Vision and Pattern Recognition Conference, pages ...

    2025

  32. [40]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503...

    2025 arXiv

  33. [41]

    scene_graph

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Nattapol Chanpaisit, Xiaojie Xu, Qianli Ma, Ziyue Dong, Dian Zheng, Hongbo Liu, and Kai Zo. Vbench leaderboard.https://huggingface.co/spaces/Vchitect/VBench_ Leaderboard, 2023. Accessed: 2025-08-05. 7 Appendix 7.1 Automated Metric R...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.