Pith. sign in

REVIEW 3 major objections 7 minor 37 references

Spotlighting Partially Visible Cinematic Language for Video-to-Audio Generation via Self-distillation

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that video-to-audio models fail on cinematic shots because the video encoder cannot cope with partial visibility, and proposes a self-distillation fix that aligns partially visible features with full-view features.

desk verdict A sensible, clearly motivated self-distillation method for video-to-audio under partial visibility, but the main evaluation is partly circular because the VGG-CU/VGG-CM test sets come from the same augmentation distribution used in training. read the letter →

arxiv 2507.02271 v1 pith:42UWX46X submitted 2025-07-03 cs.CV cs.AIcs.MM

Feizhen Huang , Yu Wu , Yutian Lin , Bo Du This is my paper · ORCID
classification cs.CVcs.AIcs.MM
keywords video-to-audiogenerationcinematiclanguagepartialvisibilityself-distillationteacher-studentframeworklatentdiffusionaudio-visualalignmentdataaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that state-of-the-art video-to-audio (V2A) models fail on cinematic shots—close-ups and camera movements—because the video encoder produces weak conditions when the sound source is only partially visible, not because the audio generator is broken. The proposed fix is a teacher–student self-distillation: the frozen original encoder serves as teacher, and a trainable student encoder is pushed to map features from artificially cropped or shifted clips onto the teacher's features from the same clip's full view. Because both clips share the same audio, aligning their features teaches the student to infer sound from partial visual clues. The paper reports that this alignment improves every metric on two simulated cinematic test sets, and also improves the original VGGSound benchmark.

What carries the argument

The load-bearing mechanism is a self-distillation alignment between two views created by Cinematic Language Variations f. A close-up variation fcu crops each frame to a random rectangle (ratio uniform in [0.4, 0.6]); a camera-movement variation fcm applies the same crop and shifts it left/right or up/down along the central axis. The teacher is the frozen pre-trained CA VP encoder, and the student is the same architecture initialized from it. The loss Lp = cos(ct, cs′) + MSE(ct, cs′) aligns the student's feature from either the varied or the original clip with the teacher's feature from the original clip. This pushes partially visible videos into the original feature space without touching the audio generator.

What would settle it

A reader could test this by taking real cinematic footage with known Foley ground truth, computing audio metrics with the student encoder versus the teacher encoder while freezing the same generator; if the teacher's full-view features do not yield better audio than the student's partial-view features, the encoder-side diagnosis fails. A second check: if a generator fine-tuned directly on real close-up clips outperforms the distilled encoder while the distilled encoder underperforms on real footage, then the synthetic crop simulation does not transfer.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that partial visibility in cinematic language is primarily an encoder-side failure: the frozen video encoder extracts degraded audio-visual conditions, so the generative model receives poor priors. Simulating close-ups with random crops and camera movements with random shifts creates paired videos (xv, xfv) with identical audio-visual content but different visibility. Training the student encoder to align features of the partially visible clip with the teacher's full-view features, using a cosine-plus-MSE loss, transfers the pre-trained audio-visual knowledge into the partial-view regime. The trained student then supplies better conditioning for the unchanged latent-diffusion audio generator, restoring synchronized Foley sounds under close-ups and camera movements.

Load-bearing premise

The whole approach assumes the audio-quality drop in cinematic shots comes from the video encoder's features and not from the audio generator, and that random crops and shifts faithfully recreate the partial visibility of real close-ups and camera moves.

Editorial extensions

If this is right

  • Under the close-up test set (VGG-CU), the method improves all reported metrics over the Diff-Foley baseline.
  • Under the camera-movement test set (VGG-CM), improvements appear even when training used only close-up variations, suggesting partial-visibility learning transfers across spatial and temporal occlusion.
  • The original VGGSound test set also improves slightly, indicating the alignment does not sacrifice ordinary-video performance.
  • The method requires no extra cinematic data and no modification to the generative model, so it can be layered onto existing V2A pipelines.
  • In a human study on 31 real YouTube cinematic videos, 74.51% of preferences favored the proposed method over the baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the encoder-conditioning diagnosis holds, other video-conditioned generation tasks that suffer under occlusion or cropping—such as text-to-audio with video context or visual question answering with partial views—could benefit from the same view-alignment self-distillation recipe.
  • The random-crop and random-shift simulation may not cover real cinematic phenomena like depth-of-field blur, rack focus, or complex dolly moves; a natural extension would train on real close-up footage with synthetic or transferred audio labels.
  • Because the student is initialized from the teacher and only features are aligned, the approach is architecture-agnostic and likely compatible with newer V2A backbones beyond Diff-Foley.
  • The k=75% sweet spot (versus 50% and 100%) suggests the pairing signal matters more than augmentation volume, pointing toward a general scheduling principle for self-distillation under partial visibility.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper addresses Video-to-Audio (V2A) generation in cinematic scenarios where Foley targets are only partially visible due to close-ups and camera movements. The authors propose a self-distillation method: cinematic language variations fcu (random spatial cropping) and fcm (crop plus directional shift) create paired videos (xv, xf_v), and a student video encoder is trained to align its features from the partially visible video with the frozen teacher's features from the fully visible video, leaving the downstream diffusion audio generator unchanged. Experiments on VGGSound and two synthetically modified test sets, VGG-CU and VGG-CM, report consistent improvements over the Diff-Foley baseline across all metrics, and a small human study on 31 real YouTube videos reports 74.5% preference for the proposed method.

Significance. If the central claim holds, this is a practical and general contribution: a simple, model-agnostic self-distillation procedure that improves V2A under spatial and temporal partial visibility without retraining the audio generator or collecting cinematic footage. The authors are the first to frame partial visibility as a cinematic-language problem for V2A, and the consistent gains on the synthetic benchmarks plus the positive human preference on real clips are encouraging. The main open question is whether the synthetic augmentations transfer to real cinematic footage; the current evidence for that claim is a small preference study, so the real-world significance is not yet fully established.

major comments (3)
  1. [Section 4 (Implementation Details; Tables 1 and 4)] The VGG-CU and VGG-CM test sets are produced by applying the same transformations fcu and fcm that are used to augment the training set, and Table 4 selects the augmentation proportion k directly on these test sets. The reported gains may therefore reflect matching the exact augmentation distribution rather than learning to handle genuine partial visibility, and the headline numbers are optimistically biased. Please introduce a held-out validation split for model selection and evaluate on test transformations that differ from the training augmentation, including objective metrics on real cinematic clips.
  2. [Section 4.1 (Real Cinematic Scene; Table 2)] The only real-world evidence is a preference study with 20 participants over 31 YouTube videos, reporting only the fraction of clips preferred and no objective audio quality or synchronization metrics. This is too small to establish that the synthetic fcu/fcm distortions transfer to real close-ups and camera movements, which involve occlusions, motion blur, and framing changes not captured by uniform cropping and shifting. Please add objective metrics on the real clips and report inter-participant agreement or a confidence interval.
  3. [Tables 1, 3, and 4] All experiments are single training runs without error bars or significance tests, while the evaluation itself samples 10 generated clips per video. Given the stochasticity of diffusion sampling and training, the reported differences may be within run-to-run or sample-to-sample noise; please provide multiple seeds or bootstrap confidence intervals for at least the main comparison and the k ablation.
minor comments (7)
  1. [Section 3.1] The phrase 'audio-visual corrections' should presumably be 'audio-visual correlations'.
  2. [Equation (2)] Equation (2) uses c_s′ before the notation is introduced; please define c_s, c_sf, and c_t explicitly in the text preceding the equation.
  3. [Section 4.3 (Table 4 discussion)] The statement that at k = 100% 'no data from the original dataset is used' is inaccurate: the teacher still receives original videos x_v to produce c_t, so original data are used as supervision. Please rephrase.
  4. [Section 4.1] The text says 'even when trained only on VGG-CU training set', but the method is trained on VGGSound with fcu augmentation; this wording is confusing and should be corrected.
  5. [Table 3] Table 3 compares fcu-only at k = 50%, fcm-only at k = 50%, and fcu&fcm at 66.7% total, so the proportion of augmented data is not matched; please report results at matched total augmentation proportions or justify the chosen values.
  6. [Figure 5] Figure 5 reports per-category Align Acc for the top 25 categories without error bars; please clarify how the improvement ranking is computed and whether the differences are significant.
  7. [General] No code or checkpoints are released; for a method whose main benefits are practical, releasing code would aid reproducibility.

Circularity Check

1 steps flagged · score 3.0 of 10

Simulated partial-visibility test sets reuse the training augmentation, so VGG-CU/VGG-CM gains are partly forced; independent fcm and human-study evidence keeps the central claim non-tautological.

  1. fitted input called prediction [Section 4, Implementation Details and Evaluation Metrics (Table 1)]
    "For training, we only apply cinematic language variation fcu on VGGSound training set with k = 75%, where a1 = 0.4 and a2 = 0.6. To evaluate performance under partial visibility, we create two modified test sets by applying cinematic language variations to the VGGSound test set. Specifically, fcu is used to create VGG-CU (close-ups) test set, and fcm is used to create VGG-CM (camera movements) test set."

    The student is trained on fcu-corrupted clips to minimize Lp = cos(ct, cs′) + MSE(ct, cs′), i.e., to map partial-view features onto full-view teacher features. VGG-CU is then built by applying the same fcu to the test set. Since the audio generator is frozen, the VGG-CU gain directly reflects this trained fcu-to-full-view mapping on the same augmentation distribution; the benchmark does not independently test real cinematic partial visibility. The fcm and human-study results mitigate this, but the headline 'impressive improvements under partial visibility across all evaluation metrics' rests substantially on this shared-distribution evaluation.

full rationale

The paper's core training loss explicitly aligns student features from partially visible (fcu-cropped) videos with teacher features from fully visible videos, and the main simulated test set VGG-CU is generated with the same fcu transformation and crop-ratio range used in training. Consequently, the reported gains on VGG-CU are substantially forced by the training objective and the shared augmentation distribution, not by an independent test of cinematic partial visibility. This is the sole notable circular element. It is not a self-citation chain: the teacher encoder and Diff-Foley baseline are external, open-source components, and no load-bearing argument reduces to the authors' own prior results. The paper also provides partially independent evidence: training only on fcu still improves the fcm-created test set, the original VGGSound test set improves, and a 31-video real YouTube human study favors the method. These independent checks keep the central claim from being purely tautological, so the overall circularity is moderate rather than severe.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three unproven premises: the encoder bottleneck, the adequacy of crop/shift as a cinematic proxy, and the quality of teacher features. The hyperparameters a1, a2, and k are hand-chosen or tuned on test sets, which adds to the free-parameter burden. No new entities are introduced.

free parameters (3)
  • crop ratio bounds a1, a2 = 0.4, 0.6
    Chosen by hand to define the simulated close-up magnitude. The paper states the range should keep Foley targets partially visible while retaining sufficient information, but no data-driven justification is given.
  • augmentation proportion k = 75%
    The fraction of training clips receiving cinematic variations. Table 4 selects 75% by comparing test-set metrics, meaning it is effectively tuned on the evaluation sets.
  • loss term weights for cosine vs MSE = unspecified (presumably equal)
    The composite loss in Eq. (2) adds cosine and MSE terms with no stated weighting, leaving a hidden free choice that could affect the feature alignment.
assumptions (3)
  • domain assumption Performance degradation under cinematic language is caused by the video encoder, not the audio generator.
    Stated as an inference in Section 3.1 and never directly tested, for example by feeding teacher features from full-visible frames to the generator on partial videos.
  • ad hoc to paper Random cropping and shifting adequately simulate real cinematic close-ups and camera movements.
    The paper motivates fcu and fcm as examples of cinematic language but provides no validation that these synthetic transformations capture the difficulty of real partial visibility beyond the small human study.
  • domain assumption The teacher's features from fully visible clips are accurate and sufficient audio-visual conditions.
    The paper relies on the pretrained CAVP encoder's quality and cites strong performance on non-cinematic videos as evidence, but does not directly measure feature quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spotlighting Partially Visible Cinematic Language for Video-to-Audio Generation via Self-distillation." pith.science (2026). https://pith.science/paper/42UWX46X

@misc{pith2026250702271,
  author       = {Pith},
  title        = {Pith review of: Spotlighting Partially Visible Cinematic Language for Video-to-Audio Generation via Self-distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/42UWX46X}},
  note         = {Machine review of arXiv:2507.02271}
}
read the original abstract

Video-to-Audio (V2A) Generation achieves significant progress and plays a crucial role in film and video post-production. However, current methods overlook the cinematic language, a critical component of artistic expression in filmmaking. As a result, their performance deteriorates in scenarios where Foley targets are only partially visible. To address this challenge, we propose a simple self-distillation approach to extend V2A models to cinematic language scenarios. By simulating the cinematic language variations, the student model learns to align the video features of training pairs with the same audio-visual correspondences, enabling it to effectively capture the associations between sounds and partial visual information. Our method not only achieves impressive improvements under partial visibility across all evaluation metrics, but also enhances performance on the large-scale V2A dataset, VGGSound.

Figures

Figures reproduced from arXiv: 2507.02271 by the authors.

Figure 1
Figure 1. Foley is the process of adding sound effects to silent videos, playing an essential role in film/video production due to fac￾tors illustrated in the pink boxes. Traditional Foley relies on skilled Foley artists to manually reproduce sounds, whereas V2A Genera￾tion can directly generate corresponding audio from silent videos, providing a more efficient and convenient solution. Wang et al., 2024a; Du et al., 2023] off… view at source ↗
Figure 2
Figure 2. Cinematic language is a fundamental element of artistic expression in film, such as close-ups and camera movements. These camera techniques often create scenarios where Foley targets are only partially visible spatially or temporally. We refer to them as partial visibility. ters or objects by zooming in, while camera movements dy￾namically introduce or remove them from the frame. These camera techniques aim to portr… view at source ↗
Figure 3
Figure 3. Our approach involves two key components: cinematic language variations [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The figures show the qualitative results for V2A genera [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: We evaluate the Align Acc metric for each video category on VGG-CU test set and present the top 25 video categories (309 in total) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: We use the Grad-CAM [Selvaraju et al., 2017] to visualize the model’s attention in close-up scenarios. Taking the machine gun shooting video as an example, the pink dashed boxes and text rep￾resent the partial visual information, while the corresponding audio ground tr…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 27 canonical work pages

  1. [1]

    The Foley grail: The art of performing sound for film, games, and animation

    [Ament, 2014] Vanessa Theme Ament. The Foley grail: The art of performing sound for film, games, and animation . Routledge,

    work page 2014

  2. [7]

    Deep residual learning for image recog- nition

    [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778,

    work page 2016

  3. [10]

    Denoising diffusion probabilistic models

    [Ho et al., 2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840–6851,

    2020

  4. [11]

    Densely connected convolutional networks

    [Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 4700–4708,

    work page 2017

  5. [13]

    Read, watch and scream! sound generation from text and video

    [Jeong et al., 2024] Yujin Jeong, Yunji Kim, Sanghyuk Chun, and Jiyoung Lee. Read, watch and scream! sound generation from text and video. arXiv preprint arXiv:2407.05551,

    arXiv 2024

  6. [15]

    Diverse part dis- covery: Occluded person re-identification with part-aware transformer

    [Li et al., 2021] Yulin Li, Jianfeng He, Tianzhu Zhang, Xi- ang Liu, Yongdong Zhang, and Feng Wu. Diverse part dis- covery: Occluded person re-identification with part-aware transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2898– 2907,

    work page 2021

  7. [16]

    Mitigating and evaluating static bias of ac- tion representations in the background and the foreground

    [Li et al., 2023] Haoxin Li, Yuan Liu, Hanwang Zhang, and Boyang Li. Mitigating and evaluating static bias of ac- tion representations in the background and the foreground. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19911–19923,

    work page 2023

  8. [17]

    AudioLDM: Text-to-audio generation with latent diffusion models

    [Liu et al., 2023] Haohe Liu, Zehua Chen, Yi Yuan, Xin- hao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. AudioLDM: Text-to-audio generation with latent diffusion models. Proceedings of the Inter- national Conference on Machine Learning, pages 21450– 21474,

    work page 2023

Show all 37 references
  1. [18]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic mod- els

    [Lu et al., 2022] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic mod- els. arXiv preprint arXiv:2211.01095,

    2022 arXiv

  2. [19]

    Diff-foley: Synchronized video-to-audio synthesis with latent diffusion models.Advances in Neural Information Processing Systems, 36,

    [Luo et al., 2024] Simian Luo, Chuanhao Yan, Chenxu Hu, and Hang Zhao. Diff-foley: Synchronized video-to-audio synthesis with latent diffusion models.Advances in Neural Information Processing Systems, 36,

    2024

  3. [20]

    The filmmaker’s eye: The language of the lens: The power of lenses and the expressive cinematic image

    [Mercado, 2019] Gustavo Mercado. The filmmaker’s eye: The language of the lens: The power of lenses and the expressive cinematic image. Routledge,

    2019

  4. [22]

    Masked generative video-to-audio transformers with enhanced synchronicity

    [Pascual et al., 2025] Santiago Pascual, Chunghsin Yeh, Ioannis Tsiamas, and Joan Serr `a. Masked generative video-to-audio transformers with enhanced synchronicity. In European Conference on Computer Vision, pages 247–

    2025

  5. [24]

    High-resolution image synthesis with latent diffusion models

    [Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695,

    2022

  6. [25]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    [Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on com- p...

    2017

  7. [26]

    Denoising diffusion implicit models

    [Song et al., 2021] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7,

    2021

  8. [27]

    Temporally aligned audio for video with autore- gression

    [Viertola et al., 2024] Ilpo Viertola, Vladimir Iashin, and Esa Rahtu. Temporally aligned audio for video with autore- gression. arXiv preprint arXiv:2409.13689,

    2024 arXiv

  9. [28]

    Removing the background by adding the background: Towards background robust self- supervised video representation learning

    [Wang et al., 2021] Jinpeng Wang, Yuting Gao, Ke Li, Yiqi Lin, Andy J Ma, Hao Cheng, Pai Peng, Feiyue Huang, Rongrong Ji, and Xing Sun. Removing the background by adding the background: Towards background robust self- supervised video representation learning. In Proceedings of...

    2021

  10. [29]

    Frieren: Efficient video-to-audio generation with rectified flow matching

    [Wang et al., 2024b] Yongqi Wang, Wenxiang Guo, Rongjie Huang, Jiawei Huang, Zehan Wang, Fuming You, Ruiqi Li, and Zhou Zhao. Frieren: Efficient video-to-audio generation with rectified flow matching. arXiv preprint arXiv:2406.00320,

    arXiv

  11. [30]

    Sonicvisionlm: Playing sound with vi- sion language models

    [Xie et al., 2024] Zhifeng Xie, Shengye Yu, Qile He, and Mengtian Li. Sonicvisionlm: Playing sound with vi- sion language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26866–26875,

    2024

  12. [31]

    Data- distortion guided self-distillation for deep neural networks

    [Xu and Liu, 2019] Ting-Bing Xu and Cheng-Lin Liu. Data- distortion guided self-distillation for deep neural networks. In Proceedings of the AAAI conference on artificial intel- ligence, pages 5565–5572,

    2019

  13. [32]

    Cutmix: Regularization strategy to train strong classifiers with localizable features

    [Yun et al., 2019] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages ...

    2019

  14. [33]

    Self- supervised scene de-occlusion

    [Zhan et al., 2020] Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, and Chen Change Loy. Self- supervised scene de-occlusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3784–3792,

    2020

  15. [34]

    Be your own teacher: Improve the performance of convolutional neural networks via self distillation

    [Zhang et al., 2019] Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF international conference on computer visi...

    2019

  16. [35]

    Foleycrafter: Bring silent videos to life with lifelike and synchronized sounds

    [Zhang et al., 2024a] Yiming Zhang, Yicheng Gu, Yanhong Zeng, Zhening Xing, Yuancheng Wang, Zhizheng Wu, and Kai Chen. Foleycrafter: Bring silent videos to life with lifelike and synchronized sounds. arXiv preprint arXiv:2407.01494,

    arXiv

  17. [36]

    Human orientation estimation un- der partial observation

    [Zhao et al., 2024] Jieting Zhao, Hanjing Ye, Yu Zhan, Hao Luan, and Hong Zhang. Human orientation estimation un- der partial observation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 11544–11551. IEEE,

    2024

  18. [37]

    Knowledge distillation by on-the-fly native ensemble

    [Zhu et al., 2018] Xiatian Zhu, Shaogang Gong, et al. Knowledge distillation by on-the-fly native ensemble. Ad- vances in neural information processing systems , 31, 2018

    2018

  19. [2014]

    Vggsound: A large-scale audio-visual dataset

    [Chen et al., 2020a] Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio-visual dataset. In ICASSP 2020-2020 IEEE Interna- tional Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pages 721–725. IEEE,

    2020

  20. [2015]

    Classifier-free diffusion guidance

    [Ho and Salimans, 2022] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598,

    2022 arXiv

  21. [2016]

    Distilling the knowledge in a neural network

    [Hinton, 2015] Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

    2015 arXiv

  22. [2017]

    Taming visually guided sound generation

    [Iashin and Rahtu, 2021] Vladimir Iashin and Esa Rahtu. Taming visually guided sound generation. In 32nd British Machine Vision Conference 2021, BMVC 2021, Online, November 22-25, 2021, page

    2021

  23. [2019]

    Pose-guided feature alignment for occluded person re-identification

    [Miao et al., 2019] Jiaxu Miao, Yu Wu, Ping Liu, Yuhang Ding, and Yi Yang. Pose-guided feature alignment for occluded person re-identification. In Proceedings of the IEEE/CVF international conference on computer vision , pages 542–551,

    2019

  24. [2020]

    Diffusion models beat gans on image synthe- sis

    [Dhariwal and Nichol, 2021] Prafulla Dhariwal and Alexan- der Nichol. Diffusion models beat gans on image synthe- sis. Advances in neural information processing systems , 34:8780–8794,

    2021

  25. [2021]

    Motion-aware contrastive video rep- resentation learning via foreground-background merging

    [Ding et al., 2022] Shuangrui Ding, Maomao Li, Tianyu Yang, Rui Qian, Haohang Xu, Qingyi Chen, Jue Wang, and Hongkai Xiong. Motion-aware contrastive video rep- resentation learning via foreground-background merging. In Proceedings of the IEEE/CVF conference on computer vision ...

    2022

  26. [2022]

    Conditional gener- ation of audio from video via foley analogies

    [Du et al., 2023] Yuexi Du, Ziyang Chen, Justin Salamon, Bryan Russell, and Andrew Owens. Conditional gener- ation of audio from video via foley analogies. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2426–2436,

    2023

  27. [2023]

    Learn2augment: learning to composite videos for data augmentation in ac- tion recognition

    [Gowda et al., 2022] Shreyank N Gowda, Marcus Rohrbach, Frank Keller, and Laura Sevilla-Lara. Learn2augment: learning to composite videos for data augmentation in ac- tion recognition. In European conference on computer vi- sion, pages 242–259. Springer,

    2022

  28. [2024]

    Instance-wise occlusion and depth orders in natural scenes

    [Lee and Park, 2022] Hyunmin Lee and Jaesik Park. Instance-wise occlusion and depth orders in natural scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 21210–21221,

    2022

  29. [2025]

    Sta-v2a: Video- to-audio generation with semantic and temporal align- ment

    [Ren et al., 2024] Yong Ren, Chenxing Li, Manjie Xu, Wei Liang, Yu Gu, Rilin Chen, and Dong Yu. Sta-v2a: Video- to-audio generation with semantic and temporal align- ment. arXiv preprint arXiv:2409.08601,

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.