Pith. sign in

REVIEW 34 references

Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction

T0 review · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that feeding separate text captions to the video and audio towers, connected by symmetric cross-attention, yields top-performing text-to-sounding-video generation.

arxiv 2510.03117 v2 pith:HADV5OV7 submitted 2025-10-03 cs.CV cs.SD

Kaisi Guan , Xihua Wang , Zhengfeng Lai , Xin Cheng , Peng Zhang , XiaoJiang Liu , Ruihua Song , Meng Cao This is my paper
classification cs.CVcs.SD
keywords text-to-sounding-videojointaudio-videogenerationdiffusiontransformerdual-towerarchitecturecross-modalattentioncaptiondisentanglementsynchronizationvisually-groundedcaptioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles text-to-sounding-video generation, the task of producing a video with synchronized audio from a text description. It argues that two design choices explain why previous dual-tower models underperform: sharing a single text caption across the video and audio towers injects irrelevant modal information, and the cross-modal interaction module has been designed one-way or too weakly. To fix these, it introduces HVGC, a three-stage captioning pipeline that produces separate, modality-pure video and audio captions, and BridgeDiT, a dual-tower diffusion transformer whose Dual CrossAttention mechanism lets the two towers exchange features symmetrically and bidirectionally. The paper reports the best scores on most quality, alignment, and synchronization metrics across three benchmarks, plus the highest human preference ratings, and presents ablations tracing these gains to the disentangled captions and to the two-way attention.

What carries the argument

The central object is the BridgeDiT block, a trainable fusion module placed at several layers of two parallel, largely frozen diffusion transformer backbones. Its Dual CrossAttention (DCA) mechanism runs two symmetric cross-attention streams: in the audio-to-video stream the video latent is the query and the audio latent supplies keys and values, and in the video-to-audio stream the roles are swapped, each with a residual connection. The second pillar is HVGC, a three-stage captioning pipeline (visual description, auditory concept extraction, audio caption synthesis) that generates the disentangled video and audio captions used for conditioning. The claimed work of the machinery is to keep e

What would settle it

Take a single user prompt, such as 'a blacksmith hammers a glowing iron bar,' and run BridgeDiT at inference without any separately supplied caption pair. If the model cannot produce a synchronized output because it requires both a video caption and an audio caption, or if its quality drops below the reported numbers, then the claimed end-to-end text-to-sounding-video capability is not established.

Watch

Extended reading notes

Core claim

The central claim is that the two obstacles—modal interference from a shared text caption and unclear cross-modal interaction—can be removed by conditioning each tower on its own modality-specific caption and fusing the two streams with a symmetric dual cross-attention block. HVGC produces those captions in three stages: a vision-language model writes a detailed video caption, an LLM distills audio event tags from it, and an LLM writes an audio-only caption grounded in those tags, which the paper argues avoids the hallucination typical of audio-only captioners. BridgeDiT keeps pretrained video and audio diffusion backbones largely frozen and inserts lightweight BridgeDiT blocks that run two

Load-bearing premise

The method's pipeline assumes that at generation time both a video caption and an audio caption are available as separate inputs, but the paper never shows how a single user text prompt is split into those two captions.

Editorial extensions

If this is right

  • Disentangled captions remove out-of-distribution conditioning, so both towers see only text that matches their pretraining, which the ablations trace to better audio-text alignment, lower audio distance, and stronger synchronization.
  • Symmetric bidirectional cross-attention outperforms full-attention, additive, and one-way fusion, indicating that equal two-way exchange is what drives temporal alignment.
  • Joint generation with a lightweight interaction module on top of frozen pretrained backbones can outperform pipelined approaches that chain separate text-to-video and video-to-audio models, because it avoids error accumulation.
  • Grounding audio captions in visual evidence reduces hallucinated sounds, and the comparison against audio-LLM captions associates this grounding with better alignment and synchronization.
  • The model generalizes across three datasets with different content distributions, suggesting the benefits are not dataset-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper leaves deployment underspecified—it never explains how a single user prompt becomes the two captions its pipeline needs. As written, the method is fully specified for captioning training data, not for end-to-end text-to-sounding-video from one prompt.
  • Editorial inference: if the HVGC benefit is general, then any dual-tower joint generator could likely improve by recaptioning its dataset with modality-specific captions, independent of which fusion module it uses.
  • Editorial inference: the finding that early-to-mid layer fusion works best hints that temporal synchronization is carried by lower-level spatio-temporal features; a testable consequence is that other multimodal joint generative models should place their interaction modules at similar depths.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's SOTA claim rests on empirical training and evaluation, not on a parameter fitted to the target metric or on a self-citation chain.

full rationale

The derivation chain in this paper is empirical rather than deductive. BridgeDiT is trained with a standard sum of audio and video tower losses (Eqs. 4-6), and the claimed results are measured on held-out automatic metrics and a user study. The HVGC captioning framework is used to create conditioning captions (TV, TA) for training and evaluation; these captions are inputs to the generator, not quantities predicted by the model or fitted to the evaluation metrics, so no 'prediction' reduces to its own input by construction. The ablations in Table 3 compare caption sources under shared and disentangled settings, and the superiority of HVGC is an empirical outcome rather than a definitional equivalence. The Dual CrossAttention equations (Eqs. 7-8) define the proposed mechanism; the comparison against Full-Attention, Additive, and unidirectional Cross-Attention baselines is an architecture ablation, not a result forced by definitions. The only author-overlap citations (e.g., ETVA, TIVA, Animate and Sound an Image) are used as related work or as baseline implementations; none is invoked as a uniqueness theorem or as the sole justification for the central SOTA claim. The paper's known limitation that HVGC is specified for video-to-caption generation rather than for splitting a single user prompt at inference is a deployment/annotation gap, not a circular derivation. Thus, under the stated standard requiring a quotable reduction by construction, no significant circularity is found.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are postulated; HVGC and DCA are algorithmic components built from existing models and attention mechanisms. The ledger instead captures the empirical choices, annotation assumptions, and evaluation assumptions that the SOTA claim depends on.

free parameters (4)
  • Number of BridgeDiT Blocks = 4
    Chosen by architecture search; fusion capacity and placement affect the reported synchronization gains.
  • Bridge block layer placement = Video [3,7,11,15]; Audio [2,5,8,11]
    Selected from five placement strategies in Appendix D; the 'uniform early bias' placement is the one used in the main SOTA table.
  • Video CFG guidance scale = 6.0
    Chosen by hand for sampling; affects the quality/alignment trade-off. The audio guidance scale is not reported in Table 5.
  • ImageBind filtering threshold = 0.3
    Videos with ImageBind score below 0.3 are discarded before training; this filters the training distribution and can affect all downstream metrics.
assumptions (5)
  • standard math Diffusion and flow-matching training objectives (Eqs. 1-2) are valid for their respective towers.
    Standard background assumed from Ho et al. and Lipman et al.; invoked in Section 3.1.
  • domain assumption Frozen pretrained Wan 2.1 and Stable Audio Open backbones provide sufficient representation for joint generation.
    The dual-tower design keeps most backbone layers frozen and only trains interaction blocks plus last five layers; Section 3.3 and Table 5.
  • domain assumption HVGC-generated captions are accurate and modality-pure.
    No human verification of caption quality is reported; the paper relies on the VLLM/LLM pipeline to avoid hallucination (Section 3.2).
  • domain assumption FVD, FAD, CLAP, ImageBind, and AV-Align are accepted proxies for generation quality and synchronization.
    Metrics are used as the main evidence for SOTA; the only external check is a 5-rater user study on 150 AVSync15 samples (Section 4.5).
  • domain assumption The evaluation datasets and the 150-sample test sets are representative of the T2SV task.
    VGGSound-SS test set is a random sample of 150 videos; AVSync15 has a fixed 150-test split; no power analysis is given (Section 4.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction." pith.science (2026). https://pith.science/paper/HADV5OV7

@misc{pith2026251003117,
  author       = {Pith},
  title        = {Pith review of: Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HADV5OV7}},
  note         = {Machine review of arXiv:2510.03117}
}
read the original abstract

This study focuses on a challenging yet promising task, Text-to-Sounding-Video (T2SV) generation, which aims to generate a video with synchronized audio from text conditions, meanwhile ensuring both modalities are aligned with text. Despite progress in joint audio-video training, two critical challenges still remain unaddressed: (1) a single, shared text caption where the text for video is equal to the text for audio often creates modal interference, confusing the pretrained backbones, and (2) the optimal mechanism for cross-modal feature interaction remains unclear. To address these challenges, we first propose the Hierarchical Visual-Grounded Captioning (HVGC) framework that generates pairs of disentangled captions, a video caption, and an audio caption, eliminating interference at the conditioning stage. Based on HVGC, we further introduce BridgeDiT, a novel dual-tower diffusion transformer, which employs a Dual CrossAttention (DCA) mechanism that acts as a robust ``bridge" to enable a symmetric, bidirectional exchange of information, achieving both semantic and temporal synchronization. Extensive experiments on three benchmark datasets, supported by human evaluations, demonstrate that our method achieves state-of-the-art results on most metrics. Comprehensive ablation studies further validate the effectiveness of our contributions, offering key insights for the future T2SV task. All the codes and checkpoints will be publicly released.

Figures

Figures reproduced from arXiv: 2510.03117 by the authors.

Figure 1
Figure 1. Examples of sounding videos generated by our BridgeDiT model, showcasing high quality, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our three-stage Hierarchical Visual-Grounded Captioning (HVGC) framework generates [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The BridgeDiT Architecture. (a): The overall dual-tower architecture. Parallel video and audio DiT streams are connected by our proposed BridgeDiT Block at specific layers. Right: Details of fusion strategies within the block, showcasing our proposed Dual Cross-Attention (b) alongside the Full-Attention (c) and Additive Fusion (d) baselines. 3.2 HIERARCHICAL VISUAL-GROUNDED CAPTIONING FRAMEWORK To address the condit… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparing different fusion mechanisms. Our DCA fusion mecha￾nism outperforms all other baselines in both AV-Align and VA-IB Score. To investigate the optimal architecture for cross-modal in￾teraction, we conduct an ablation study on different fu￾sion mechanisms, as ill…
Figure 5
Figure 5. Figure 5: Prompts for Stage1: Detailed Visual Scene Description [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Prompts for Stage2: Auditory Concept Abstraction [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Prompts for Stage3: Visually-Grounded Audio Caption Generation [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Detailed Command for Human Annotation 21 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 28 linked inside Pith

  1. [1]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923,

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report.ar...

    arXiv

  2. [5]

    Clap learning audio concepts from natural language supervision

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from natural language supervision. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE,

    2023

  3. [6]

    Stable audio open

    Zach Evans, Julian D Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. Stable audio open. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE,

    2025

  4. [7]

    Etva: Evaluation of text-to-video alignment via fine-grained question generation and answering.arXiv preprint arXiv:2503.16867,

    Kaisi Guan, Zhengfeng Lai, Yuchong Sun, Peng Zhang, Wei Liu, Kieran Liu, Meng Cao, and Rui- hua Song. Etva: Evaluation of text-to-video alignment via fine-grained question generation and answering.arXiv preprint arXiv:2503.16867,

    arXiv

  5. [10]

    A simple but strong baseline for sounding video generation: Effective adaptation of audio and video diffusion models for joint generation.arXiv preprint arXiv:2409.17550,

    Masato Ishii, Akio Hayakawa, Takashi Shibuya, and Yuki Mitsufuji. A simple but strong baseline for sounding video generation: Effective adaptation of audio and video diffusion models for joint generation.arXiv preprint arXiv:2409.17550,

    arXiv

  6. [12]

    Inc Kuaishou

    URLhttps: //arxiv.org/abs/1312.6114. Inc Kuaishou. Kling video generation,

    arXiv

  7. [13]

    Chun-Yi Kuan, Wei-Ping Huang, and Hung-yi Lee

    URLhttps://klingai.com/. Chun-Yi Kuan, Wei-Ping Huang, and Hung-yi Lee. Understanding sounds, missing the ques- tions: The challenge of object hallucination in large audio-language models.arXiv preprint arXiv:2406.08402,

    arXiv

  8. [14]

    Sound-guided semantic video generation

    11 Seung Hyun Lee, Gyeongrok Oh, Wonmin Byeon, Chanyoung Kim, Won Jeong Ryoo, Sang Ho Yoon, Hyunjun Cho, Jihyun Bae, Jinkyu Kim, and Sangpil Kim. Sound-guided semantic video generation. InComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, Octo- ber 23–27, 2022, Proceedings, Part XVII, pp. 34–50. Springer,

    2022

Show all 34 references
  1. [15]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131,

    arXiv

  2. [16]

    Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,

    arXiv

  3. [17]

    Audioldm: Text-to-audio generation with latent diffusion models.arXiv preprint arXiv:2301.12503,

    Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models.arXiv preprint arXiv:2301.12503,

    arXiv

  4. [18]

    Syncflow: Toward temporally aligned joint audio-video generation from text.arXiv preprint arXiv:2412.15220, 2024a

    Haohe Liu, Gael Le Lan, Xinhao Mei, Zhaoheng Ni, Anurag Kumar, Varun Nagaraja, Wenwu Wang, Mark D Plumbley, Yangyang Shi, and Vikas Chandra. Syncflow: Toward temporally aligned joint audio-video generation from text.arXiv preprint arXiv:2412.15220, 2024a. Haohe Liu, Yi Yuan, X...

    arXiv

  5. [19]

    On the audio hallucinations in large audio- video language models.arXiv preprint arXiv:2401.09774,

    Taichi Nishimura, Shota Nakada, and Masayoshi Kondo. On the audio hallucinations in large audio- video language models.arXiv preprint arXiv:2401.09774,

    arXiv

  6. [20]

    Scalable diffusion models with transformers.arXiv preprint arXiv:2212.09748,

    William Peebles and Saining Xie. Scalable diffusion models with transformers.arXiv preprint arXiv:2212.09748,

    arXiv

  7. [21]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512,

    arXiv

  8. [22]

    Avhbench: A cross-modal hallucination benchmark for audio-visual large language models.arXiv preprint arXiv:2410.18325,

    Kim Sung-Bin, Oh Hyun-Bin, JungMok Lee, Arda Senocak, Joon Son Chung, and Tae-Hyun Oh. Avhbench: A cross-modal hallucination benchmark for audio-visual large language models.arXiv preprint arXiv:2410.18325,

    arXiv

  9. [23]

    Atom of thoughts for markov llm test-time scaling.arXiv preprint arXiv:2502.12018,

    Fengwei Teng, Zhaoyang Yu, Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo. Atom of thoughts for markov llm test-time scaling.arXiv preprint arXiv:2502.12018,

  10. [24]

    Towards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717,

    arXiv

  11. [26]

    URLhttps://arxiv.org/abs/2308. 06571. Jun Wang, Xijuan Zeng, Chunyu Qiang, Ruilong Chen, Shiyao Wang, Le Wang, Wangjing Zhou, Pengfei Cai, Jiahui Zhao, Nan Li, et al. Kling-foley: Multimodal diffusion transformer for high- quality video-to-audio generation.arXiv preprint arXiv...

  12. [27]

    Shuchen Weng, Haojie Zheng, Zheng Chang, Si Li, Boxin Shi, and Xinlong Wang

    URLhttps://arxiv.org/abs/2412.03603. Shuchen Weng, Haojie Zheng, Zheng Chang, Si Li, Boxin Shi, and Xinlong Wang. Audio-sync video generation with multi-stream temporal control.arXiv preprint arXiv:2506.08003,

    arXiv

  13. [28]

    Yazhou Xing, Yingqing He, Zeyue Tian, Xintao Wang, and Qifeng Chen

    URLhttps://arxiv.org/abs/ 2508.02324. Yazhou Xing, Yingqing He, Zeyue Tian, Xintao Wang, and Qifeng Chen. Seeing and hearing: Open- domain visual-audio generation with diffusion latent aligners. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    arXiv

  14. [29]

    Qwen2.5-omni technical report.arXiv preprint arXiv:2503.20215,

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report.arXiv preprint arXiv:2503.20215,

    arXiv

  15. [30]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115,

    13 An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu,...

    arXiv

  16. [31]

    Uniform: A unified multi-task diffusion transformer for audio-video generation

    Lei Zhao, Linfeng Feng, Dongxu Ge, Rujin Chen, Fangqiu Yi, Chi Zhang, Xiao-Lei Zhang, and Xuelong Li. Uniform: A unified multi-task diffusion transformer for audio-video generation. arXiv preprint arXiv:2502.03897,

    arXiv

  17. [32]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404,

    arXiv

  18. [34]

    This technique steers the generation pro- cess towards a desired conditionc(e.g., a text prompt) without needing an external classifier

    Classifier-Free GuidanceConditional generation in these models is commonly achieved using Classifier-Free Guidance (CFG) (Ho & Salimans, 2022). This technique steers the generation pro- cess towards a desired conditionc(e.g., a text prompt) without needing an external classifi...

    2022

  19. [2016]

    14 A THEUSE OFLARGELANGUAGEMODELS In this work, Large Language Models (LLMs) are used solely for enhancing writing clarity and En- glish expression

    URLhttps: //arxiv.org/abs/1606.06650. 14 A THEUSE OFLARGELANGUAGEMODELS In this work, Large Language Models (LLMs) are used solely for enhancing writing clarity and En- glish expression. All core contributions, including model design, mathematical formulations, and experimenta...

    2020 arXiv

  20. [2018]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314,

    arXiv

  21. [2020]

    Jiawei Huang, Yi Ren, Rongjie Huang, Dongchao Yang, Zhenhui Ye, Chen Zhang, Jinglin Liu, Xiang Yin, Zejun Ma, and Zhou Zhao

    URL https://arxiv.org/abs/2006.11239. Jiawei Huang, Yi Ren, Rongjie Huang, Dongchao Yang, Zhenhui Ye, Chen Zhang, Jinglin Liu, Xiang Yin, Zejun Ma, and Zhou Zhao. Make-an-audio 2: Temporal-enhanced text-to-audio gen- eration,

    2006 arXiv

  22. [2021]

    Mmaudio: Taming multimodal joint training for high-quality video-to-audio synthesis

    10 Ho Kei Cheng, Masato Ishii, Akio Hayakawa, Takashi Shibuya, Alexander Schwing, and Yuki Mit- sufuji. Mmaudio: Taming multimodal joint training for high-quality video-to-audio synthesis. InProceedings of the Computer Vision and Pattern Recognition Conference, pp. 28901–28911...

    2025 arXiv

  23. [2022]

    Fr\’echet audio distance: A metric for evaluating music enhancement algorithms.arXiv preprint arXiv:1812.08466,

    Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fr\’echet audio distance: A metric for evaluating music enhancement algorithms.arXiv preprint arXiv:1812.08466,

    arXiv

  24. [2023]

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh

    URL https://arxiv.org/abs/2311.15127. Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators,

    arXiv

  25. [2024]

    Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu

    URLhttps://openai.com/research/ video-generation-models-as-world-simulators. Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting.arXiv preprint arXiv:2310.04948,

    arXiv

  26. [2025]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.