Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Vidu S1 turns spoken instructions into a continuous, drift-free video stream at 42 FPS on a single consumer GPU.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 08:51 UTC pith:Y3SK2B4S

load-bearing objection Genuinely useful system integration, but the paper's central claim of indefinite stable streaming isn't backed by the experiments—the evaluation never measures drift as a function of time, and TwinCache gets no ablation. the 4 major comments →

arxiv 2607.03118 v2 pith:Y3SK2B4S submitted 2026-07-03 cs.CV cs.LG

Vidu S1: A Real-Time Interactive Video Generation Model

Jintao Zhang , Kai Jiang , Jintao Chen , Xu Wang , Yang Luo , Yuji Wang , Dechuang Chen , Jungang Li
show 19 more authors
Chengyang Ye Marco Chen Hongzhou Zhu Min Zhao Yuxuan Jiang Zhengkun Huang Chendong Xiang Kaiwen Zheng Haoxu Wang Xiaohang Wang Qi Jia Xin Chen Yimin Chen Youhe Jiang Fangcheng Fu Zhijie Deng Fan Bao Jianfei Chen Jun Zhu
This is my paper
classification cs.CV cs.LG
keywords real-time video generationstreaming diffusionspeech-guided controlavatar generationinfinite-length videoautoregressive video diffusioninference accelerationvoice-controlled characters
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Vidu S1, a video generation model designed to be interactive: a user speaks, and the on-screen character reacts, while the video continues beyond any fixed length. The authors argue that the usual offline 'type a prompt, wait, download' paradigm misses a large demand for live, responsive visual content, and that real-time generation requires solving modeling and serving together. They claim that a three-stage training recipe—bidirectional teacher, causal streaming adaptation, and few-step distillation—plus a cache design that keeps both noisy and clean history prevents drift and blur in open-ended streams. On their in-house benchmark and a public talking-head benchmark, they report the best scores on identity, lip-sync, and visual quality while running at 540p at up to 42 FPS on a single RTX 5090 GPU. If correct, the paper shows that interactive, indefinitely long video generation is feasible on ordinary consumer hardware.

Core claim

The central claim is that stable infinite-length video generation can be achieved by streaming autoregressive diffusion: at each step only the current frame is denoised, conditioned on a small sliding window that includes a persistent reference built from the first frame, while a stage-aware cache (TwinCache) feeds the network with a noisy historical state during intermediate denoising steps and a clean state at the final step. This separation is claimed to suppress error accumulation and high-frequency artifacts over arbitrarily long runs. Voice instructions act as an explicit conditioning signal, letting users change what happens next at any moment. The authors report that the resulting sy

What carries the argument

TwinCache, the paper's named mechanism, is the load-bearing piece: each generated video-audio state keeps two cached representations—a noisy cache taken from an intermediate denoising step and a clean cache from the final step. During intermediate denoising, attention uses the noisy cache as history, which acts as an implicit low-pass filter and prevents error accumulation; the final step swaps in the clean cache to restore detail, together with a persistent reference context constructed once from the first frame. Around this sits a streaming inference stack: sliding-window attention with repositioned rotary position embeddings, per-block W8A8 quantization, fused kernels, CUDA graphs, and mu

Load-bearing premise

The open-ended 'no drift, no blur' claim rests on the assumption that the curated training data and the in-house benchmark represent the full variety of real user inputs well enough that the few-step streaming recipe will keep errors from accumulating in any long session.

What would settle it

Take a public audio-visual benchmark and run streams lasting 60 to 120 minutes with varied, sometimes repetitive action instructions, then track identity (CSIM) and quality (DOVER) over time; if either score declines steadily or the character visibly drifts or collapses, the infinite-length claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claims hold, real-time interactive avatars—virtual hosts, live conversation partners, and game characters—become practical on hardware many users already own.
  • Speech can serve as a first-class control signal for video generation, not just an audio track to lip-sync, enabling mid-stream user direction.
  • Open-ended video streams of hours or more become possible without the drift and collapse that earlier autoregressive models exhibited.
  • The efficiency-quality tradeoff is not necessarily zero-sum: few-step distillation combined with cache scheduling is reported to keep quality metrics at or above the best offline systems.
  • Serving architecture matters as much as modeling; the same model may fail real-time without kernel-level co-design such as quantization, fusion, and graph replay.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the models and training data are not released, the infinite-length claim cannot yet be checked by outsiders; a strong test would be to run continuous streams for many hours on public long-tail inputs and measure drift.
  • The noisy-cache low-pass story suggests a general principle—deliberately retaining residual noise in streaming diffusion history as a regularizer—that could transfer to other autoregressive generative models such as audio or world models.
  • The paper's demand arithmetic implies a shift toward live generation; one could test whether real-time interactive use actually dominates offline viewing in consumer platforms.
  • The 42-FPS figure is tied to a specific 3-step, quantized configuration; a straightforward extension is to benchmark quality-latency tradeoffs on cheaper GPUs or with more denoising steps.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Vidu S1, a real-time interactive video generation model with speech-based control, targeting indefinitely long, stable streams at 540p resolution and up to 42 FPS on a single RTX 5090 GPU. The method combines a three-stage training pipeline (bidirectional teacher, causal teacher with teacher forcing / diffusion forcing, and DMD distillation with PCM regularization) with streaming inference techniques including a persistent reference context, RoPE repositioning, and a new TwinCache mechanism. Evaluation consists of an in-house preference benchmark (Vidu-StreamBench) against commercial avatar systems, quantitative metrics (CSIM, Sync-D, DOVER) on HDTF, and an FPS measurement.

Significance. If the central claims are substantiated, Vidu S1 would be a notable systems contribution, showing that streaming video generation with speech control can be made real-time and long-horizon stable on commodity hardware. The paper is strongest in its clear articulation of an integrated model-and-systems stack, and it usefully includes public HDTF metrics alongside the proprietary benchmark. However, the headline claim of 'infinite-length generation without blurring, drift, or visual distortion' is not directly tested, the TwinCache mechanism is not ablated, and no code, data, or error bars are provided. The work is therefore a plausible system description whose scientific claims are currently under-validated; the significance is real but conditional on the missing evidence being supplied.

major comments (4)
  1. [Abstract and §1; §3.2] The central claim is that Vidu S1 supports 'infinite-length real-time video generation without blurring, drift, or visual distortion.' Nowhere in Section 3 is this tested as a function of stream duration. HDTF results (Table 1) and Vidu-StreamBench preferences are aggregates over short clips; Figure 1's 15/30/60/90-minute timeline is illustrative, not measured. The paper itself notes (§1) that long-horizon generation is hard because 'small errors accumulate over time, causing drift, instability, and eventually visual collapse,' so this burden is not optional. I request a drift-vs-time curve (e.g., CSIM/DOVER/FID or a collapse metric at 1, 5, 15, 30, 60, 90 minutes), plus an explicit statement of the maximum tested duration and failure rate. Without this, the abstract's infinitude claim is unsupported.
  2. [§2.3.1 (TwinCache)] TwinCache is presented as the key mechanism for long-term temporal consistency, but no ablation isolates it. There is no comparison against a clean-cache-only variant, a no-persistent-reference variant, or different choices of the noisy-cache extraction timestep. Moreover, the 'predefined denoising step' from which the noisy cache is extracted is never specified, and the claim that residual noise 'acts as an implicit low-pass prior' is not verified experimentally. Given that the infinite-length stability claim rests on this mechanism, a TwinCache ablation with quantitative long-horizon results is necessary.
  3. [§3.1, Table 1, Figure 3] All quantitative claims rest on point estimates without error bars or significance tests. In Table 1, Vidu S1's CSIM (0.9192) is numerically indistinguishable from HeyGen's (0.9191), yet the paper concludes it 'achieves the best performance across all test metrics.' Similarly, Vidu-StreamBench is an in-house, unreleased benchmark; the preference methodology (number of raters, instructions, statistical aggregation) is not reported. Please provide error bars and significance tests for Table 1, release Vidu-StreamBench (or a public subset), and document the preference-test protocol.
  4. [§2.2, §2.3.2, §3.2.2] Key training and inference hyperparameters are missing, preventing replication and assessment of the 42 FPS claim. These include: the teacher-forcing probability p in Eq. (5), the DMD/PCM weighting and number of PCM steps, the noisy-cache timestep, model architecture and parameter count, and the exact RTX 5090 configuration. The 42 FPS figure is reported as an 'average throughput' in §3.2.2 and 'up to' in the abstract; end-to-end interaction latency and memory footprint are not given. Please provide a detailed configuration table and a latency/power/memory breakdown so that 'real-time' can be evaluated independently.
minor comments (5)
  1. [Abstract and §3.2.2] The abstract says 'up to 42 FPS,' while §3.2.2 says 'average throughput of 42 FPS.' Please make these consistent and report the distribution/variance across video segments.
  2. [Table 1] Several baselines have '–' for FPS/throughput and resolution, which makes the 'Real-Time' comparison incomplete. As a presentation matter, it would help to state explicitly which numbers are vendor-reported and which are measured in the current test environment, and to include a per-model note on whether instruction following is in the input prompt or conversational.
  3. [Figure 1] The timeline labels (0:00, 15:00, 30:00, 60:00, 90:00) imply an evaluated duration claim, but the figure is described as an overview. Please either add an actual measured example of that length or change the caption to avoid implying experimental verification.
  4. [Throughout] The author list and some headings contain typographical artifacts (e.g., 'Y ang', 'T able 1', 'V ideo'). Please run a proofreading pass.
  5. [References] The manuscript cites many very recent arXiv preprints, many of which are by the same group (e.g., [15], [16], [35], [38], [47]). Please ensure the related-work discussion distinguishes established methods from self-citations still under review, so readers can judge novelty appropriately.

Circularity Check

0 steps flagged

No significant circularity: measured metrics and external/public benchmarks carry the claims; self-citations are contextual, not load-bearing.

full rationale

Vidu S1's central quantitative claims are empirical measurements rather than derivations that reduce to their inputs. The 42 FPS figure is presented as a measured average throughput ('reaches an average throughput of 42 FPS on RTX 5090 GPUs'), and the quality results are reported on HDTF with external metric models (CSIM, Sync-D, DOVER), with HDTF explicitly described as 'a standard public reference.' The self-citations to TurboDiffusion [22] and TurboServe [23] describe the engineering stack ('aligned with the technical route of TurboDiffusion ... with TurboServe as a reference'), but they do not substitute for the runtime measurement; the throughput is independently measured in this paper. Vidu-StreamBench is an in-house, unreleased benchmark, and preference results on it are a validation limitation rather than a constructional circularity: no parameter is fitted to that benchmark and then renamed as a prediction. The 'infinite-length without drift' claim is asserted in Section 1 and the Conclusion without a duration-resolved drift measurement or a TwinCache ablation, but this is an evidence gap, not an equivalence-by-construction. No equation makes a predicted quantity equal to a fitted input, and no uniqueness theorem or ansatz is imported from the authors' prior work as the sole justification of the central result. The derivation chain is therefore self-contained with respect to circularity, though the empirical support for long-horizon stability is incomplete.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claims of real-time infinite video depend on the correctness of the training/distillation objectives (standard), and on the validity and representativeness of the proprietary data and benchmarks (domain assumptions). Hyperparameters like the 3-step setting and the teacher-forcing probability are not disclosed, but they are not fitted to the evaluation metrics.

free parameters (3)
  • Teacher forcing probability p = not specified
    Used to sample between teacher forcing and diffusion forcing in Eq. 4/5; value chosen without reporting, affecting training dynamics.
  • Number of denoising steps (3-step) = 3
    The 42 FPS result is reported with a 3-step configuration; this is a speed/quality trade-off, not a fundamental constant.
  • TwinCache noisy-cache extraction timestep = not specified
    The noisy cache is 'extracted from a predefined denoising step'; the step is not disclosed, yet affects temporal consistency.
axioms (5)
  • standard math Diffusion and diffusion forcing theory as per refs [5,13]
    The training objectives (Eq. 3, 5) rely on established diffusion loss derivation.
  • standard math DMD and PCM objectives as defined in refs [28,29]
    The distillation objectives (Eq. 6, 7) are taken directly from published methods.
  • domain assumption Training data (livestreams, films, TV) is sufficiently diverse and clean
    The pipeline filters heavily, but the raw data mix is proprietary and unquantified, so generalization cannot be assessed.
  • domain assumption CSIM, Sync-D, DOVER are valid proxies for generation quality
    These metrics are used to declare 'best performance' but are not shown to correlate with user experience for interactive long-form generation.
  • domain assumption RTX 5090 represents 'regular consumer GPUs'
    The abstract claims consumer GPUs, but the benchmark uses a single high-end RTX 5090; no other GPUs are tested.

pith-pipeline@v1.3.0-alltime-deepseek · 157 in / 9804 out tokens · 129857 ms · 2026-08-02T08:51:31.510653+00:00 · methodology

0 comments
read the original abstract

We introduce Vidu S1, a real-time interactive video generation model supporting voice control of digital characters. Users can control video generation content at any moment through voice instructions. Vidu S1 supports infinite-length real-time video generation without blurring, drift, or visual distortion. Built with TurboDiffusion and TurboServe, Vidu S1 outputs 540p real-time videos at up to 42 FPS on regular consumer GPUs. Users can upload custom images of real people, anime, and pets, and choose different voice tones for personalized experiences. Experiments show that Vidu S1 achieves the best performance across all test metrics while fully meeting real-time inference requirements. A playable online demo is available at https://vidu.com/vidu-stream.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation

    cs.CV 2026-07 conditional novelty 5.0

    A chunk-wise streaming video model with bounded multi-scale memory and streaming 4K upscaling reports real-time interactive long-form generation and top Arena preference/stability scores.

Reference graph

Works this paper leans on

61 extracted references · 22 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024

    2024

  2. [2]

    Veo: A text-to-video generation system

    Google DeepMind. Veo: A text-to-video generation system. Technical report, Google DeepMind, 2025. Veo 3 Tech Report

    2025

  3. [3]

    Wan: Open and advanced large-scale video generative models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Seedance 2.0: Advancing video generation for world complexity

    Team Seedance, De Chen, Liyang Chen, Xin Chen, Ying Chen, Zhuo Chen, Zhuowei Chen, Feng Cheng, Tianheng Cheng, Yufeng Cheng, et al. Seedance 2.0: Advancing video generation for world complexity. arXiv preprint arXiv:2604.14148, 2026

    Pith/arXiv arXiv 2026

  5. [5]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion

    Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081–24125, 2024

    2024

  6. [6]

    Fifo-diffusion: Generating infinite videos from text without training

    Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. Advances in Neural Information Processing Systems, 37:89834–89868, 2024

    2024

  7. [7]

    Streamingt2v: Consistent, dynamic, and extendable long video generation from text

    Roberto Henschel, Levon Khachatryan, Hayk Poghosyan, Daniil Hayrapetyan, Vahram Tadevosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 2568–2577, 2025

    2025

  8. [8]

    History-guided video diffusion

    Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion. arXiv preprint arXiv:2502.06764, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Rolling forcing: Autoregressive long video diffusion in real time

    Kunhao Liu, Wenbo Hu, Jiale Xu, Ying Shan, and Shijian Lu. Rolling forcing: Autoregressive long video diffusion in real time. arXiv preprint arXiv:2509.25161, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Ar-diffusion: Asynchronous video generation with auto-regressive diffusion

    Mingzhen Sun, Weining Wang, Gen Li, Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, SiYu Zhou, Qian He, and Jing Liu. Ar-diffusion: Asynchronous video generation with auto-regressive diffusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 7364–7373, 2025

    2025

  11. [11]

    Progressive autoregressive video diffusion models

    Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou. Progressive autoregressive video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 6322–6332, 2025

    2025

  12. [12]

    Streamdiffusionv2: A streaming system for dynamic and interactive video generation

    Tianrui Feng, Zhi Li, Shuo Yang, Haocheng Xi, Muyang Li, Xiuyu Li, Lvmin Zhang, Keting Yang, Kelly Peng, Song Han, et al. Streamdiffusionv2: A streaming system for dynamic and interactive video generation. arXiv preprint arXiv:2511.07399, 2025. 10

    arXiv 2025

  13. [13]

    From slow bidirectional to fast autoregressive video diffusion models

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22963–22974, 2025

    2025

  14. [14]

    Self forcing: Bridging the train-test gap in autoregressive video diffusion

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. Advances in Neural Information Processing Systems, 38:167283–167308, 2026

    2026

  15. [15]

    Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation

    Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, and Jun Zhu. Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation. arXiv preprint arXiv:2602.02214, 2026

    Pith/arXiv arXiv 2026

  16. [16]

    Causal forcing++: Scalable few-step autoregressive diffusion distillation for real-time interactive video generation

    Min Zhao, Hongzhou Zhu, Kaiwen Zheng, Zihan Zhou, Bokai Yan, Xinyuan Li, Xiao Yang, Chongxuan Li, and Jun Zhu. Causal forcing++: Scalable few-step autoregressive diffusion distillation for real-time interactive video generation. arXiv preprint arXiv:2605.15141, 2026

    Pith/arXiv arXiv 2026

  17. [17]

    Longlive: Real-time interactive long video generation

    Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time interactive long video generation. arXiv preprint arXiv:2509.22622, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Magi-1: Autoregressive video generation at scale

    Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, WQ Zhang, Weifeng Luo, et al. Magi-1: Autoregressive video generation at scale. arXiv preprint arXiv:2505.13211, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Skyreels-v2: Infinite-length film generative model

    Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma, et al. Skyreels-v2: Infinite-length film generative model. arXiv preprint arXiv:2504.13074, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Packing input frame context in next-frame prediction models for video generation

    Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv e-prints, pages arXiv–2504, 2025

    2025

  21. [21]

    Wan-s2v: Audio-driven cinematic video generation

    Xin Gao, Li Hu, Siqi Hu, Mingyang Huang, Chaonan Ji, Dechao Meng, Jinwei Qi, Penchong Qiao, Zhen Shen, Yafei Song, et al. Wan-s2v: Audio-driven cinematic video generation. arXiv preprint arXiv:2508.18621, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Turbodiffusion: Accelerating video diffusion models by 100-200 times

    Jintao Zhang, Kaiwen Zheng, Kai Jiang, Haoxu Wang, Ion Stoica, Joseph E Gonzalez, Jianfei Chen, and Jun Zhu. Turbodiffusion: Accelerating video diffusion models by 100-200 times. arXiv preprint arXiv:2512.16093, 2025

    arXiv 2025

  23. [23]

    Turboserve: Serving streaming video generation efficiently and economically

    Youhe Jiang, Haoxu Wang, Haotong Bao, Kai Jiang, Jianfei Chen, Jun Zhu, Fangcheng Fu, and Jintao Zhang. Turboserve: Serving streaming video generation efficiently and economically. arXiv preprint arXiv:2606.19271, 2026

    Pith/arXiv arXiv 2026

  24. [24]

    Qwen3-omni technical report

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. Qwen3-omni technical report. arXiv preprint arXiv:2509.17765, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    Ca2-vdm: Efficient autoregres- sive video diffusion model with causal generation and cache sharing, 2025

    Kaifeng Gao, Jiaxin Shi, Hanwang Zhang, Chunping Wang, Jun Xiao, and Long Chen. Ca2-vdm: Efficient autoregres- sive video diffusion model with causal generation and cache sharing, 2025

    2025

  27. [27]

    Pyramidal flow matching for efficient video generative modeling

    Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong MU, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  28. [28]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6613–6623, 2024

    2024

  29. [29]

    Phased consistency models

    Fu-Yun Wang, Zhaoyang Huang, Alexander W Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consistency models. Advances in neural information processing systems, 37:83951–84009, 2024

    2024

  30. [30]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In International Conference on Learning Representations, volume 2024, pages 21875–21895, 2024

    2024

  31. [31]

    Infinity-rope: Action-controllable infinite video generation emerges from autoregressive self-rollout

    Hidir Yesiltepe, Tuna Meral, Adil Kaan Akan, Kaan Oktay, and Pinar Yanardag. Infinity-rope: Action-controllable infinite video generation emerges from autoregressive self-rollout. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 40256–40265, 2026. 11

    2026

  32. [32]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024

    2024

  33. [33]

    Deep forcing: Training-free long video generation with deep sink and participative compression

    Jung Yi, Wooseok Jang, Paul Hyunbin Cho, Jisu Nam, Heeji Yoon, and Seungryong Kim. Deep forcing: Training-free long video generation with deep sink and participative compression. arXiv preprint arXiv:2512.05081, 2025

    arXiv 2025

  34. [34]

    Rolling sink: Bridging limited-horizon training and open-ended testing in autoregressive video diffusion

    Haodong Li, Shaoteng Liu, Zhe Lin, and Manmohan Chandraker. Rolling sink: Bridging limited-horizon training and open-ended testing in autoregressive video diffusion. arXiv preprint arXiv:2602.07775, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Grounded forcing: Bridging time-independent semantics and proximal dynamics in autoregressive video synthesis

    Jintao Chen, Chengyu Bai, Junjun Hu, Xinda Xue, and Mu Xu. Grounded forcing: Bridging time-independent semantics and proximal dynamics in autoregressive video synthesis. arXiv preprint arXiv:2604.06939, 2026

    Pith/arXiv arXiv 2026

  36. [36]

    Memrope: Training-free infinite video generation via evolving memory tokens

    Youngrae Kim, Qixin Hu, C-C Jay Kuo, and Peter A Beerel. Memrope: Training-free infinite video generation via evolving memory tokens. arXiv preprint arXiv:2603.12513, 2026

    arXiv 2026

  37. [37]

    Live avatar: Streaming real-time audio-driven avatar generation with infinite length

    Yubo Huang, Hailong Guo, Fangtai Wu, Weiqiang Wang, Shifeng Zhang, Shijie Huang, Qijun Gan, Lin Liu, Sirui Zhao, Enhong Chen, et al. Live avatar: Streaming real-time audio-driven avatar generation with infinite length. arXiv preprint arXiv:2512.04677, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Causal-rcm: A unified teacher-forcing and self-forcing open recipe for autoregressive diffusion distillation in streaming video generation and interactive world models

    Kaiwen Zheng, Guande He, Min Zhao, Jintao Zhang, Huayu Chen, Jianfei Chen, Chen-Hsuan Lin, Ming-Yu Liu, Jun Zhu, and Qianli Ma. Causal-rcm: A unified teacher-forcing and self-forcing open recipe for autoregressive diffusion distillation in streaming video generation and interactive world models. arXiv preprint arXiv:2606.25473, 2026

    Pith/arXiv arXiv 2026

  39. [39]

    Lpm 1.0: Video-based character performance model

    Ailing Zeng, Casper Yang, Chauncey Ge, Eddie Zhang, Garvey Xu, Gavin Lin, Gilbert Gu, Jeremy Pi, Leo Li, Mingyi Shi, et al. Lpm 1.0: Video-based character performance model. arXiv preprint arXiv:2604.07823, 2026

    Pith/arXiv arXiv 2026

  40. [40]

    Efficient attention methods: Hardware-efficient, sparse, compact, and linear attention

    Jintao Zhang, Rundong Su, Chunyu Liu, Jia Wei, Ziteng Wang, Haoxu Wang, Pengle Zhang, Huiqiang Jiang, Haofeng Huang, Chendong Xiang, et al. Efficient attention methods: Hardware-efficient, sparse, compact, and linear attention

  41. [41]

    Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration

    Jintao Zhang, Jia Wei, Pengle Zhang, Jun Zhu, and Jianfei Chen. Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration. In International Conference on Learning Representations (ICLR), 2025

    2025

  42. [42]

    Sageattention2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization

    Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, and Jianfei Chen. Sageattention2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization. InInternational Conference on Machine Learning (ICML), 2025

    2025

  43. [43]

    Sageattention2++: A more efficient implementation of sageattention2

    Jintao Zhang, Xiaoming Xu, Jia Wei, Haofeng Huang, Pengle Zhang, Chendong Xiang, Jun Zhu, and Jianfei Chen. Sageattention2++: A more efficient implementation of sageattention2. arXiv preprint arXiv:2505.21136, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Sageattention3: Microscaling fp4 attention for inference and an exploration of 8-bit training

    Jintao Zhang, Jia Wei, Pengle Zhang, Xiaoming Xu, Haofeng Huang, Haoxu Wang, Kai Jiang, Jun Zhu, and Jianfei Chen. Sageattention3: Microscaling fp4 attention for inference and an exploration of 8-bit training. arXiv preprint arXiv:2505.11594, 2025

    arXiv 2025

  45. [45]

    Sagebwd: A trainable low-bit attention

    Jintao Zhang, Marco Chen, Haoxu Wang, Kai Jiang, Ion Stoica, Joseph E Gonzalez, Jianfei Chen, and Jun Zhu. Sagebwd: A trainable low-bit attention. arXiv preprint arXiv:2603.02170, 2026

    arXiv 2026

  46. [46]

    Spargeattention: Accurate and training-free sparse attention accelerating any model inference

    Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattention: Accurate and training-free sparse attention accelerating any model inference. arXiv preprint arXiv:2502.18137, 2025

    arXiv 2025

  47. [47]

    Spargeattention2: Trainable sparse attention via hybrid top-k+ top-p masking and distillation fine-tuning

    Jintao Zhang, Kai Jiang, Chendong Xiang, Weiqi Feng, Yuezhou Hu, Haocheng Xi, Jianfei Chen, and Jun Zhu. Spargeattention2: Trainable sparse attention via hybrid top-k+ top-p masking and distillation fine-tuning. arXiv preprint arXiv:2602.13515, 2026

    arXiv 2026

  48. [48]

    Gonzalez, Jun Zhu, and Jianfei Chen

    Jintao Zhang, Haoxu Wang, Kai Jiang, Shuo Yang, Kaiwen Zheng, Haocheng Xi, Ziteng Wang, Hongzhou Zhu, Min Zhao, Ion Stoica, Joseph E. Gonzalez, Jun Zhu, and Jianfei Chen. Sla: Beyond sparsity in diffusion transformers via fine-tunable sparse-linear attention. arXiv preprint arXiv:2509.24006, 2025

    arXiv 2025

  49. [49]

    Sla2: Sparse-linear attention with learnable routing and qat

    Jintao Zhang, Haoxu Wang, Kai Jiang, Kaiwen Zheng, Youhe Jiang, Ion Stoica, Jianfei Chen, Jun Zhu, and Joseph E Gonzalez. Sla2: Sparse-linear attention with learnable routing and qat. arXiv preprint arXiv:2602.12675, 2026

    arXiv 2026

  50. [50]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509, 2023. 12

    Pith/arXiv arXiv 2023

  51. [51]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3661–3670, 2021

    2021

  52. [52]

    Heygen ai video avatar

    HeyGen. Heygen ai video avatar. https://www.heygen.com/avatars/ai-video-avatar , 2026. Accessed: 2026- 07-02

    2026

  53. [53]

    Lemonslice studio: Create talking and singing ai avatar videos

    LemonSlice. Lemonslice studio: Create talking and singing ai avatar videos. https://lemonslice.com/studio,

  54. [54]

    Klingavatar 2.0 technical report, 2025

    Kling Team, Jialu Chen, Yikang Ding, Zhixue Fang, Kun Gai, Yuan Gao, Kang He, Jingyun Hua, Boyuan Jiang, Mingming Lao, Xiaohan Li, Hui Liu, Jiwen Liu, Xiaoqiang Liu, Yuan Liu, Shun Lu, Yongsen Mao, Yingchao Shao, Huafeng Shi, Xiaoyu Shi, Peiqin Sun, Songlin Tang, Pengfei Wan, Chao Wang, Xuebo Wang, Haoxian Zhang, Yuanxing Zhang, and Yan Zhou. Klingavatar ...

    2025

  55. [55]

    Omniavatar: Efficient audio-driven avatar video generation with adaptive body animation

    Qijun Gan, Ruizi Yang, Jianke Zhu, Shaofei Xue, and Steven Hoi. Omniavatar: Efficient audio-driven avatar video generation with adaptive body animation. arXiv preprint arXiv:2506.18866, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer

    Jiahao Cui, Hui Li, Yun Zhan, Hanlin Shang, Kaihui Cheng, Yuqi Ma, Shan Mu, Hang Zhou, Jingdong Wang, and Siyu Zhu. Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 21086–21095, 2025

    2025

  57. [57]

    Stableavatar: Infinite-length audio-driven avatar video generation

    Shuyuan Tu, Yueming Pan, Yinming Huang, Xintong Han, Zhen Xing, Qi Dai, Chong Luo, Zuxuan Wu, and Yu-Gang Jiang. Stableavatar: Infinite-length audio-driven avatar video generation. arXiv preprint arXiv:2508.08248, 2025

    Pith/arXiv arXiv 2025

  58. [58]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019

    2019

  59. [59]

    A lip sync expert is all you need for speech to lip generation in the wild

    KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Namboodiri, and CV Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM international conference on multimedia, pages 484–492, 2020

    2020

  60. [60]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF international conference on computer vision, pages 20144–20154, 2023. 13

    2023

  61. [2026]

    Accessed: 2026-07-02

    2026