Pith. sign in

REVIEW 5 major objections 7 minor 1 cited by

AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AlignHuman claims that separating preference optimization into early motion steps and late fidelity steps, each handled by a dedicated LoRA, improves audio-driven human animation and enables a 3.3× inference speedup with minimal quality…

desk verdict A genuinely new post-training recipe for human animation with a real timestep-segment idea, but the evidence base is thinner than the speedup and SOTA claims. read the letter →

arxiv 2506.11144 v1 pith:5S6YUSRS submitted 2025-06-11 cs.CV

Chao Liang , Jianwen Jiang , Wang Liao , Jiaqi Yang , Zerong zheng , Weihong Zeng , Han Liang This is my paper
classification cs.CV
keywords audio-drivenhumananimationtimestep-segmentpreferenceoptimizationDirectLoRAdiffusionmodelmotionnaturalnessvisualfidelityinferenceacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the two hard-to-reconcile goals in audio-driven human animation—natural movement and detailed visual quality—can be optimized separately rather than traded off. The authors report that in a diffusion model's denoising process, early steps largely decide the motion while later steps refine fidelity, and that this split can be exploited during preference learning from human-ranked videos. They propose timestep-segment preference optimization (TPO), which trains one lightweight adapter for motion, active in the early segment, and a second for fidelity, active in the late segment. If the claim holds, AlignHuman improves quality over strong baselines and also generates at 30 instead of 100 function evaluations, a 3.3× speedup with little quality change.

What carries the argument

The central mechanism is timestep-segment preference optimization (TPO), which splits the denoising timeline at a switch timestep $t_{\mathrm{mid}} = 0.2T$: the early interval, from $T$ down to $0.8T$, is assigned to motion alignment, and the later interval, from $0.8T$ down to $0$, to fidelity alignment. Within each interval a separate expert LoRA—a low-rank adapter that adds a small set of trainable weights to the frozen transformer layers—is trained on that dimension's human-preference pairs and is the only active adapter during those steps. The training loss is a DPO-style flow-matching objective that compares the flow-matching losses of win and lose samples through the policy and a frozen reference model.

What would settle it

Repeat the step-skipping measurement on a diverse prompt set with several solvers and guidance strengths: decode the generated video at each intermediate step and check whether a motion metric is already stable after ten steps and whether skipping the first ten steps leaves a hand-confidence metric unchanged. Finding any common configuration where motion shifts after late steps or early skipping changes fidelity would break the timestep split that TPO relies on.

Watch

Extended reading notes

Core claim

The paper's central claim is that directly applying preference optimization to audio-driven human animation mostly repairs low-level artifacts while leaving motion unnatural, because fidelity is easier to learn and dominates the objective. Through a timestep analysis of the base model, AlignHuman finds that the body-motion metric stabilizes within the first ten denoising steps, while hand and fidelity metrics improve throughout, and that skipping those early steps changes motion but leaves fidelity nearly intact. On that basis it introduces timestep-segment preference optimization (TPO): the denoising range is split at $0.2T$, with a Motion LoRA trained and activated in the early interval and a Fidelity LoRA in the later interval. In the reported experiments the full system improves every benchmark metric over the base model and state-of-the-art baselines, and at 30 NFEs it still beats the base model at 100 NFEs.

Load-bearing premise

The method rests on the observation that movements are settled early in generation while visual details are settled late, an observation the authors make on a small test batch; if that split changes with different generation settings or video content, the fixed switch point and the two adapters would not transfer.

Editorial extensions

If this is right

  • Removing either expert LoRA hurts a specific dimension: dropping the fidelity LoRA lowers hand quality (HKC) and FID, while dropping the motion LoRA lowers motion richness (HKV) and FVD.
  • The switch point $0.2T$ is the best balance; moving toward either end degrades performance, with too-early switching compressing motion learning and lowering HKV.
  • At 30 NFEs the TPO model still outperforms the base model at 100 NFEs and stays comparable to OmniHuman-1 at 100 NFEs, so the quality-speed trade-off is favorable.
  • TPO beats naive DPO, IPO, and SimPO on the benchmark, indicating that the timestep segmentation, not preference optimization alone, drives the improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference beyond the paper: the claimed early-motion/late-fidelity split is measured on a small validation batch with one solver and one CFG setting, so the fixed $0.2T$ switch is the least tested component and the natural thing to probe with other solvers and scales.
  • Inference beyond the paper: because the preference videos come from nine candidate models with different training recipes, the LoRAs may absorb a particular style bias; whether the gains survive a change of base model or out-of-distribution prompts is left open.
  • Inference beyond the paper: the same divide-and-conquer idea could be applied to other generation tasks where semantic content is decided early and texture or identity details are decided late, for example lip-sync versus identity preservation or pose versus clothing texture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. This paper proposes AlignHuman, a post-training alignment method for audio-driven human animation built on a diffusion transformer base model. The authors first analyze the denoising process and claim that early timesteps primarily determine motion dynamics while later timesteps mainly refine visual fidelity. Based on this premise, they introduce timestep-segment preference optimization (TPO): motion-related preference pairs train a motion LoRA applied only in the early denoising interval, and fidelity-related pairs train a fidelity LoRA applied only in the later interval, with a switch timestep t_mid=0.2T. They evaluate on a 269-sample benchmark against several recent baselines (CyberHost, OmniHuman-1, FantasyTalking, DiffTED, DiffGest+MimicMotion) and report consistent improvements in IQA, AES, Sync-C, FID, FVD, HKV, and HKC. They further claim that TPO enables inference at 30 NFEs instead of 100 NFEs with minimal quality loss, corresponding to a 3.3x speedup.

Significance. If the timestep decomposition is robust, AlignHuman offers a practical divide-and-conquer recipe for multi-objective preference learning in generative video models, with lightweight LoRA adapters and a potential 3.3x reduction in inference cost. The paper is clearly written and the ablations cover the main design choices: switch timestep, LoRA rank, component removal, and alternatives such as Naive-DPO, IPO, and SimPO. The preference dataset construction with separate human rankings for motion and fidelity is a strength, as is the inclusion of strong recent baselines. The main weakness is that the load-bearing timestep-separation claim is supported by an under-specified analysis on a small validation batch, and it is not verified after TPO training or at the reduced NFE counts used for the speedup claim. If that premise fails, the method’s advantage and the 30-NFE claim are not established.

major comments (5)
  1. [Sec. 3.2 / Fig. 1] The central premise of the paper—that motion dynamics are set in early denoising steps and fidelity in later steps—is supported only by an analysis on “a small batch of the validation set” using two metrics, BM and HKC. BM (Body Motion) is never defined anywhere in the paper, and Fig. 1 reports no numerical values, error bars, or repeated runs. Because the entire TPO design and the choice of t_mid=0.2T rest on this analysis, the paper should specify the BM computation, the validation batch size, how intermediate latents are decoded, and the variance of these measurements.
  2. [Sec. 3.3.2 / Sec. 4.4] The switch timestep t_mid=0.2T and the fixed early/late interval split are carried over from the base model’s 50-step schedule to two new settings: (i) the TPO model, whose motion LoRA actively modifies early-step latents, and (ii) the 30-NFE (15-step) schedule used for the speedup claim. No experiment shows that the early/late separation of motion and fidelity persists after LoRA training or when each denoising step covers a much larger time interval. Without this transfer evidence, the segment-specific LoRA activation and the “minimal impact on generation quality at 30 NFEs” claim are not supported. The authors should re-run the timestep analysis on the TPO model and at 30 NFEs.
  3. [Sec. 4.3 / Table 2] The t_mid ablation is used to justify the choice of 0.2T, but the differences among 0.18T, 0.20T, and 0.22T are small (e.g., FVD 28.441 vs 28.002 vs 29.230; HKV 47.757 vs 48.604 vs 48.428), and no error bars, multiple seeds, or a held-out split are reported. Since the same 269-sample benchmark is used both to select t_mid and to report final numbers, the apparent plateau near 0.2T could reflect selection on the evaluation set rather than a stable property of the denoising process. The paper should report variance or use a separate validation split for this hyperparameter choice.
  4. [Sec. 4.2 / Table 1] The evaluation uses a single 269-sample benchmark with no confidence intervals or significance tests. Several headline gains are small (IQA 4.018 to 4.086; Sync-C 7.220 to 7.525), so the claim of “significant improvements” is not statistically supported. The authors should provide per-sample paired tests or bootstrap confidence intervals for the main metrics, especially for the AlignHuman-Base versus AlignHuman-TPO comparison, to show that the reported differences are not within the noise range.
  5. [Sec. 4.4] The 3.3x speedup is computed solely as the ratio of NFE counts (100/30), but no wall-clock inference time or measured latency is reported, and Fig. 4 shows only curves for FID, FVD, and HKC without numerical values or error bars. The speedup claim should be accompanied by actual inference-time measurements, and the quality-versus-NFE trade-off should be tabulated for at least 100, 50, and 30 NFEs to substantiate the claim of minimal quality loss.
minor comments (7)
  1. [Tables 2 and 3] The column header “ASE” in Tables 2 and 3 should be “AES” to match Table 1 and the text.
  2. [Sec. 3.3.1] The sentence “10,000 video pairs, each with four videos, totaling 40,000 samples” is ambiguous: these are 40,000 generated videos, not 40,000 preference pairs. Please clarify the dataset unit.
  3. [Sec. 3.3.2] There is a typo “dimesnsion d” in the description of the win/lose pair; it should be “dimension d.”
  4. [Sec. 1] In the contributions list, “timestep-segment preference omtimization” should be “optimization.”
  5. [Sec. 4.4] The section title “Discussions on Accerating” contains a typo; it should be “Acceleration.”
  6. [Table 3] The statement that removing the timestep segment “remains inferior to the full TPO solution” is not true for Sync-C: the w/o Timestep Segment row reports 7.550, higher than the full TPO’s 7.525. The text should acknowledge this exception or explain why the overall conclusion is unaffected.
  7. [Sec. 3.2] The use of HKC as the sole fidelity proxy in the timestep analysis should be justified, since fidelity in Sec. 4.1 is measured with FID, IQA, and AES; including a global fidelity metric in Fig. 1 would make the timestep decomposition more convincing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: TPO gains and the 30-NFE speedup are empirical measurements; the timestep split is an ablative design choice, not a fitted prediction.

full rationale

The paper's central claims—that TPO with two timestep-segmented LoRAs improves motion and fidelity and permits 30-NFE inference—are established by direct training and measurement on a fixed benchmark, not derived from the timestep analysis by construction. The Sec. 3.2 observation that early steps set motion and late steps set fidelity is an empirical measurement on decoded latents; it is used to motivate the t_mid=0.2T design choice, and the choice is then validated by the Tab. 2 ablation, which compares the same model under different t_mid values. No claim is a renamed fit: the speedup curve in Fig. 4 is measured by reducing NFEs, and the preference-data losses in Eq. 2 are not the evaluation metrics. The self-citations to CyberHost/OmniHuman-1 appear as architectural baselines and comparison points, not as evidence for the timestep decomposition or the TPO result. Concerns about the small validation batch, undefined BM, and transfer of t_mid to 30 NFEs are external-validity risks, not circular reductions.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on an empirical timestep decomposition and a proprietary base model. The method's hyperparameters (t_mid, beta, LoRA rank, sampling ratio) are chosen by validation ablation, so they are fitted parameters. No new theoretical entities are introduced.

free parameters (5)
  • Switch timestep t_mid = 0.2T
    Motivated by the timestep analysis in Sec 3.2, then swept in Table 2 with best performance at 0.2T; the paper's ablation shows performance varies with this value.
  • DPO strength beta = 50
    Set as a hyperparameter in Eq. 2 without sensitivity analysis; it controls preference optimization strength.
  • LoRA rank = 256
    Chosen by monotonic ablation in Table 4; ranks up to 512 show saturation, so 256 is the final selection.
  • Timestep sampling ratio = 50% per interval
    The paper states timesteps are sampled with equal probability from both intervals, and the two dimensions are optimized with 50% probability each; this balance is chosen by hand.
  • Inference NFE cutoff = 30 NFE
    The speedup claim uses 30 NFE as the reduced budget; the quality trade-off is reported for this specific value.
assumptions (5)
  • standard math Flow matching objective with the loss in Eq. 1
    The base model uses the standard flow matching training objective from [17]; accepted as a mathematical framework.
  • standard math DPO-style loss in Eq. 2 is a valid preference optimization objective
    Borrowed from DPO [27] and diffusion DPO [36]; the authors do not derive it, they apply it.
  • domain assumption Early denoising timesteps control motion, late timesteps control fidelity
    This is the key empirical claim from Fig 1 and Sec 3.2, based on BM and HKC metrics on a small validation batch; if false, TPO and the 0.2T split lose their foundation.
  • domain assumption Preference data can be cleanly separated into motion and fidelity dimensions
    The dataset construction in Sec 3.3.1 assumes human annotators can rank the four videos per pair independently on motion and fidelity, and that the two sets of pairs are distinct enough to train separate LoRAs.
  • domain assumption The base model architecture and training follow OmniHuman-1 [16]
    The method is validated on a proprietary base model that follows [16]; the paper's claims only apply to that specific model and may not transfer to other architectures.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation." pith.science (2026). https://pith.science/paper/5S6YUSRS

@misc{pith2026250611144,
  author       = {Pith},
  title        = {Pith review of: AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5S6YUSRS}},
  note         = {Machine review of arXiv:2506.11144}
}
abstract

Recent advancements in human video generation and animation tasks, driven by diffusion models, have achieved significant progress. However, expressive and realistic human animation remains challenging due to the trade-off between motion naturalness and visual fidelity. To address this, we propose \textbf{AlignHuman}, a framework that combines Preference Optimization as a post-training technique with a divide-and-conquer training strategy to jointly optimize these competing objectives. Our key insight stems from an analysis of the denoising process across timesteps: (1) early denoising timesteps primarily control motion dynamics, while (2) fidelity and human structure can be effectively managed by later timesteps, even if early steps are skipped. Building on this observation, we propose timestep-segment preference optimization (TPO) and introduce two specialized LoRAs as expert alignment modules, each targeting a specific dimension in its corresponding timestep interval. The LoRAs are trained using their respective preference data and activated in the corresponding intervals during inference to enhance motion naturalness and fidelity. Extensive experiments demonstrate that AlignHuman improves strong baselines and reduces NFEs during inference, achieving a 3.3$\times$ speedup (from 100 NFEs to 30 NFEs) with minimal impact on generation quality. Homepage: \href{https://alignhuman.github.io/}{https://alignhuman.github.io/}

Figures

Figures reproduced from arXiv: 2506.11144 by the authors.

Figure 1
Figure 1. The analysis of intermediate performance at different denoising timesteps. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of our timestep-segment preference optimization. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization results. Compared with baselines and the base model, our TPO model can [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance comparison of the TPO model and the base model under different NFEs. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Visualization results of our TPO model at 30NFEs. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    A three-part system, Talking-Critic, Talking-NSQ, and TLPO, aligns diffusion portrait animation models to human preferences and improves lip-sync, motion naturalness, and visual quality.

Reference graph

Works this paper leans on

51 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    A general theoretical paradigm to understand learning from human preferences

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, pages 4447–4455. PMLR, 2024

    2024

  2. [2]

    wav2vec 2.0: A framework for self-supervised learning of speech representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460, 2020

    2020

  3. [3]

    Skyreels-v2: Infinite-length film generative model

    Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Juncheng Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengchen Ma, et al. Skyreels-v2: Infinite-length film generative model. arXiv preprint arXiv:2504.13074, 2025

    arXiv 2025

  4. [4]

    Echomimic: Lifelike audio- driven portrait animations through editable landmark conditions

    Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, Yuming Li, and Chenguang Ma. Echomimic: Lifelike audio- driven portrait animations through editable landmark conditions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 2403–2410, 2025

    work page 2025

  5. [5]

    Out of time: automated lip sync in the wild

    Joon Son Chung and Andrew Zisserman. Out of time: automated lip sync in the wild. In Computer Vision–ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13, pages 251–263. Springer, 2017

    2016

  6. [6]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  7. [7]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. arXiv preprint arXiv:2406.15252, 2024

    arXiv 2024

  8. [8]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

Show all 51 references
  1. [9]

    Diffted: One-shot audio- driven ted talk video generation with diffusion-based co-speech gestures

    Steven Hogue, Chenxu Zhang, Hamza Daruger, Yapeng Tian, and Xiaohu Guo. Diffted: One-shot audio- driven ted talk video generation with diffusion-based co-speech gestures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1922–1931, 2024

    1922

  2. [10]

    Loopy: Taming audio-driven portrait avatar with long-term motion dependency

    Jianwen Jiang, Chao Liang, Jiaqi Yang, Gaojie Lin, Tianyun Zhong, and Yanbo Zheng. Loopy: Taming audio-driven portrait avatar with long-term motion dependency

  3. [11]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  4. [12]

    Auto-encoding variational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013

    2013

  5. [13]

    Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation

    Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation. arXiv preprint arXiv:2412.00115, 2024

    2024 arXiv

  6. [14]

    T2v-turbo: Breaking the quality bottleneck of video consistency model with mixed reward feedback

    Jiachen Li, Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sugato Basu, Wenhu Chen, and William Yang Wang. T2v-turbo: Breaking the quality bottleneck of video consistency model with mixed reward feedback. arXiv preprint arXiv:2405.18750, 2024

    2024 arXiv

  7. [15]

    Cyberhost: A one-stage diffusion framework for audio-driven talking body generation

    Gaojie Lin, Jianwen Jiang, Chao Liang, Tianyun Zhong, Jiaqi Yang, Zerong Zheng, and Yanbo Zheng. Cyberhost: A one-stage diffusion framework for audio-driven talking body generation. In The Thirteenth International Conference on Learning Representations

  8. [16]

    Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models

    Gaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng, and Chao Liang. Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models. arXiv preprint arXiv:2502.01061, 2025

    2025 arXiv

  9. [17]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  10. [18]

    Improving video generation with human feedback

    Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Wenyu Qin, Menghan Xia, et al. Improving video generation with human feedback. arXiv preprint arXiv:2501.13918, 2025. 10

    2025 arXiv

  11. [19]

    Videodpo: Omni-preference alignment for video diffusion generation

    Runtao Liu, Haoyu Wu, Zheng Ziqiang, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. Videodpo: Omni-preference alignment for video diffusion generation. arXiv preprint arXiv:2412.14167, 2024

    2024 arXiv

  12. [20]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  13. [21]

    Openelm: An efficient language model family with open training and inference framework

    Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, Maxwell Horton, Yanzi Jin, Chenfan Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal, et al. Openelm: An efficient language model family with open training and inference framework. arXiv preprint arXiv:240...

    2024 arXiv

  14. [22]

    Echomimicv2: Towards striking, simplified, and semi-body human animation

    Rang Meng, Xingyu Zhang, Yuming Li, and Chenguang Ma. Echomimicv2: Towards striking, simplified, and semi-body human animation. arXiv preprint arXiv:2411.10061, 2024

    2024

  15. [23]

    Simpo: Simple preference optimization with a reference-free reward

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    2024

  16. [24]

    Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms

    Yassine Ouali, Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos. Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms. In European Conference on Computer Vision, pages 395–413. Springer, 2024

    2024

  17. [25]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  18. [26]

    Skyreels-a1: Expressive portrait animation in video diffusion transformers

    Di Qiu, Zhengcong Fei, Rui Wang, Jialin Bai, Changqian Yu, Mingyuan Fan, Guibin Chen, and Xi- ang Wen. Skyreels-a1: Expressive portrait animation in video diffusion transformers. arXiv preprint arXiv:2502.10841, 2025

    2025 arXiv

  19. [27]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  20. [28]

    Seaweed-7b: Cost-effective training of video generation foundation model

    Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7b: Cost-effective training of video generation foundation model. arXiv preprint arXiv:2504.08685, 2025

    2025 arXiv

  21. [29]

    First order motion model for image animation

    Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019

    2019

  22. [30]

    Motion repre- sentations for articulated animation

    Aliaksandr Siarohin, Oliver J Woodford, Jian Ren, Menglei Chai, and Sergey Tulyakov. Motion repre- sentations for articulated animation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13653–13662, 2021

    2021

  23. [31]

    Learning to summarize with human feedback

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008–3021, 2020

    2020

  24. [32]

    Emo2: End-effector guided audio-driven avatar video generation

    Linrui Tian, Siqi Hu, Qi Wang, Bang Zhang, and Liefeng Bo. Emo2: End-effector guided audio-driven avatar video generation. arXiv preprint arXiv:2501.10687, 2025

    2025 arXiv

  25. [33]

    Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions. In European Conference on Computer Vision, pages 244–260. Springer, 2024

    2024

  26. [34]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processi...

    2024

  27. [35]

    Fvd: A new metric for video generation

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. 2019

    2019

  28. [36]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  29. [37]

    Fantasytalking: Realistic talking portrait generation via coherent motion synthesis

    Mengchao Wang, Qiang Wang, Fan Jiang, Yaqi Fan, Yunpeng Zhang, Yonggang Qi, Kun Zhao, and Mu Xu. Fantasytalking: Realistic talking portrait generation via coherent motion synthesis. arXiv preprint arXiv:2504.04842, 2025. 11

    2025 arXiv

  30. [38]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  31. [39]

    Video-to-video synthesis

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Video-to-video synthesis. arXiv preprint arXiv:1808.06601, 2018

    2018 arXiv

  32. [40]

    Mocha: Towards movie-grade talking character synthesis

    Cong Wei, Bo Sun, Haoyu Ma, Ji Hou, Felix Juefei-Xu, Zecheng He, Xiaoliang Dai, Luxin Zhang, Kunpeng Li, Tingbo Hou, et al. Mocha: Towards movie-grade talking character synthesis. arXiv preprint arXiv:2503.23307, 2025

    2025 arXiv

  33. [41]

    Q-align: Teaching lmms for visual scoring via discrete text-defined levels

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023

    2023 arXiv

  34. [42]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341, 2023

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341, 2023

    2023 arXiv

  35. [43]

    Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation

    Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, et al. Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint arXiv:2412.21059, 2024

    2024 arXiv

  36. [44]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903–15935, 2023

    2023

  37. [45]

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, et al. Qwen2. 5-1m technical report. arXiv preprint arXiv:2501.15383, 2025

    2025 arXiv

  38. [46]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  39. [47]

    Magicinfinite: Generating infinite talking videos with your words and voice

    Hongwei Yi, Tian Ye, Shitong Shao, Xuancheng Yang, Jiantong Zhao, Hanzhong Guo, Terrance Wang, Qingyu Yin, Zeke Xie, Lei Zhu, et al. Magicinfinite: Generating infinite talking videos with your words and voice. arXiv preprint arXiv:2503.05978, 2025

    2025 arXiv

  40. [48]

    Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024

    2024 arXiv

  41. [49]

    Learning to compare for better training and evaluation of open domain natural language generation models

    Wangchunshu Zhou and Ke Xu. Learning to compare for better training and evaluation of open domain natural language generation models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9717–9724, 2020

    2020

  42. [50]

    Taming diffusion models for audio-driven co-speech gesture generation

    Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, and Lequan Yu. Taming diffusion models for audio-driven co-speech gesture generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10544–10553, 2023

    2023

  43. [51]

    Vlogger: Make your dream a vlog

    Shaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, and Yali Wang. Vlogger: Make your dream a vlog. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8806–8817, 2024. 12

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.