REVIEW 74 references
Controllable and Expressive One-Shot Video Head Swapping
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this paper, we propose a novel diffusion-based multi-condition controllable framework for video head swapping, which seamlessly transplant a human head from a static image into a dynamic video, while preserving the original body and background of target video, and further allowing to tweak head expressions and movements during swapping as needed. Existing face-swapping methods mainly focus on localized facial replacement neglecting holistic head morphology, while head-swapping approaches struggling with hairstyle diversity and complex backgrounds, and none of these methods allow users to modify the transplanted head expressions after swapping. To tackle these challenges, our method incorporates several innovative strategies through a unified latent diffusion paradigm. 1) Identity-preserving context fusion: We propose a shape-agnostic mask strategy to explicitly disentangle foreground head identity features from background/body contexts, combining hair enhancement strategy to achieve robust holistic head identity preservation across diverse hair types and complex backgrounds. 2) Expression-aware landmark retargeting and editing: We propose a disentangled 3DMM-driven retargeting module that decouples identity, expression, and head poses, minimizing the impact of original expressions in input images and supporting expression editing. While a scale-aware retargeting strategy is further employed to minimize cross-identity expression distortion for higher transfer precision. Experimental results demonstrate that our method excels in seamless background integration while preserving the identity of the source portrait, as well as showcasing superior expression transfer capabilities applicable to both real and virtual characters.
Reference graph
Works this paper leans on
-
[1]
Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion
Di Chang, Yichun Shi, Quankai Gao, Hongyi Xu, Jessica Fu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mo- hammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. InForty-first International Conference on Machine Learn- ing, ICML 2024, Vienna, Austria, July 21-27, 2024. Open- Review.net, 2024. 3
2024
-
[2]
Topiq: A top-down approach from semantics to distortions for image quality assessment.IEEE Transactions on Image Processing, 33:2404–2418, 2024
Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics to distortions for image quality assessment.IEEE Transactions on Image Processing, 33:2404–2418, 2024. 6
2024
-
[3]
Videocrafter1: Open diffusion models for high-quality video generation.CoRR, abs/2310.19512, 2023
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation.CoRR, abs/2310.19512, 2023. 3
-
[4]
Maddox, Zhiyao Duan, and Chenliang Xu
Lele Chen, Zhiheng Li, Ross K. Maddox, Zhiyao Duan, and Chenliang Xu. Lip movements generation at a glance. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VII, pages 538–553. Springer, 2018. 3
2018
-
[5]
Simswap: An efficient framework for high fidelity face swapping
Renwang Chen, Xuanhong Chen, Bingbing Ni, and Yanhao Ge. Simswap: An efficient framework for high fidelity face swapping. InMM ’20: The 28th ACM International Confer- ence on Multimedia, Virtual Event / Seattle, WA, USA, Octo- ber 12-16, 2020, pages 2003–2011. ACM, 2020. 2, 3, 6
2020
-
[6]
Arcface: Additive angular margin loss for deep face recognition
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. InIEEE Conference on Computer Vi- sion and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 4690–4699. Computer Vision Foundation / IEEE, 2019. 4, 7
2019
-
[7]
Megaportraits: One-shot megapixel neural head avatars
Nikita Drobyshev, Jenya Chelishev, Taras Khakhulin, Alek- sei Ivakhnenko, Victor Lempitsky, and Egor Zakharov. Megaportraits: One-shot megapixel neural head avatars. In MM ’22: The 30th ACM International Conference on Multi- media, Lisboa, Portugal, October 10 - 14, 2022, pages 2663–
2022
-
[8]
Emoportraits: Emotion-enhanced multimodal one-shot head avatars
Nikita Drobyshev, Antoni Bigata Casademunt, Konstantinos V ougioukas, Zoe Landgraf, Stavros Petridis, and Maja Pan- tic. Emoportraits: Emotion-enhanced multimodal one-shot head avatars. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 8498–8507. IEEE, 2024. 3
2024
Show all 74 references
-
[9]
Dreambooth++: Boosting subject-driven gen- eration via region-level references packing
Zhongyi Fan, Zixin Yin, Gang Li, Yibing Zhan, and He- liang Zheng. Dreambooth++: Boosting subject-driven gen- eration via region-level references packing. InProceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024 ...
2024
-
[10]
Mitra, Peter Wonka, and Jingwan Lu
Anna Fr ¨uhst¨uck, Krishna Kumar Singh, Eli Shechtman, Niloy J. Mitra, Peter Wonka, and Jingwan Lu. Insetgan for full-body image generation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 7713–7722. IE...
2022
-
[11]
Stylesync: High-fidelity generalized and personalized lip sync in style- based generator
Jiazhi Guan, Zhanwang Zhang, Hang Zhou, Tianshu Hu, Kaisiyuan Wang, Dongliang He, Haocheng Feng, Jingtuo Liu, Errui Ding, Ziwei Liu, and Jingdong Wang. Stylesync: High-fidelity generalized and personalized lip sync in style- based generator. InIEEE/CVF Conference on Computer V...
2023
-
[12]
Livepor- trait: Efficient portrait animation with stitching and retarget- ing control.CoRR, abs/2407.03168, 2024
Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. Livepor- trait: Efficient portrait animation with stitching and retarget- ing control.CoRR, abs/2407.03168, 2024. 3
2024 arXiv
-
[13]
Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis
Yudong Guo, Keyu Chen, Sen Liang, Yong-Jin Liu, Hujun Bao, and Juyong Zhang. Ad-nerf: Audio driven neural ra- diance fields for talking head synthesis. In2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 5764–
2021
-
[14]
Animatediff: Animate your personalized text- to-image diffusion models without specific tuning
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. InThe Twelfth International Conference on Learning Representa- tion...
2024
-
[15]
Latent video diffusion models for high-fidelity long video generation, 2023
Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation, 2023. 3
2023
-
[16]
Animate anyone: Consistent and controllable image- to-video synthesis for character animation
Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 8153–8163. IEEE, 2024. 3
2024
-
[17]
EAMM: one-shot emotional talking face via audio-based emotion-aware motion model
Xinya Ji, Hang Zhou, Kaisiyuan Wang, Qianyi Wu, Wayne Wu, Feng Xu, and Xun Cao. EAMM: one-shot emotional talking face via audio-based emotion-aware motion model. InSIGGRAPH ’22: Special Interest Group on Computer Graphics and Interactive Techniques Conference, Vancouver, BC, C...
2022
-
[18]
Sapiens: Foundation for human vision mod- els.arXiv preprint arXiv:2408.12569, 2024
Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els.arXiv preprint arXiv:2408.12569, 2024. 4
2024 arXiv
-
[19]
FLOAT: generative motion latent flow matching for audio-driven talk- ing portrait.CoRR, abs/2412.01064, 2024
Taekyung Ki, Dongchan Min, and Gyeongsu Chae. FLOAT: generative motion latent flow matching for audio-driven talk- ing portrait.CoRR, abs/2412.01064, 2024. 3
2024
-
[20]
Smooth- swap: A simple enhancement for face-swapping with smoothness
Jiseob Kim, Jihoon Lee, and Byoung-Tak Zhang. Smooth- swap: A simple enhancement for face-swapping with smoothness. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10769–10778. IEEE, 2022. 2, 3
2022
-
[21]
Obamanet: Photo-realistic lip-sync from text.CoRR, abs/1801.01442, 2018
Rithesh Kumar, Jose Sotelo, Kundan Kumar, Alexandre de Br´ebisson, and Yoshua Bengio. Obamanet: Photo-realistic lip-sync from text.CoRR, abs/1801.01442, 2018. 3 9
2018 arXiv
-
[22]
Faceinpainter: High fidelity face adaptation to hetero- geneous domains
Jia Li, Zhaoyang Li, Jie Cao, Xingguang Song, and Ran He. Faceinpainter: High fidelity face adaptation to hetero- geneous domains. InIEEE Conference on Computer Vi- sion and Pattern Recognition, CVPR 2021, virtual, June 19- 25, 2021, pages 5089–5098. Computer Vision Foundation...
2021
-
[23]
Faceshifter: Towards high fidelity and occlusion aware face swapping.CoRR, abs/1912.13457, 2019
Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. Faceshifter: Towards high fidelity and occlusion aware face swapping.CoRR, abs/1912.13457, 2019. 2, 3
1912 arXiv
-
[24]
MAT: mask-aware transformer for large hole image in- painting
Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Jiaya Jia. MAT: mask-aware transformer for large hole image in- painting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10748–10758. IEEE, 2022. 6
2022
-
[25]
MAT: mask-aware transformer for large hole image in- painting
Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Jiaya Jia. MAT: mask-aware transformer for large hole image in- painting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10748–10758. IEEE, 2022. 3
2022
-
[26]
Expressive talking head generation with granular audio-visual control
Borong Liang, Yan Pan, Zhizhi Guo, Hang Zhou, Zhibin Hong, Xiaoguang Han, Junyu Han, Jingtuo Liu, Errui Ding, and Jingdong Wang. Expressive talking head generation with granular audio-visual control. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, ...
2022
-
[27]
Deepfacelab: Integrated, flexible and extensible face-swapping framework.Pattern Recognit., 141:109628, 2023
Kunlin Liu, Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Wenbo Zhou, and Weiming Zhang. Deepfacelab: Integrated, flexible and extensible face-swapping framework.Pattern Recognit., 141:109628, 2023. 3
2023
-
[28]
Anitalker: Animate vivid and di- verse talking faces through identity-decoupled facial motion encoding
Tao Liu, Feilong Chen, Shuai Fan, Chenpeng Du, Qi Chen, Xie Chen, and Kai Yu. Anitalker: Animate vivid and di- verse talking faces through identity-decoupled facial motion encoding. InProceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, ...
2024
-
[29]
Mediapipe: A framework for building perception pipelines,
Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris Mc- Clanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo- Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. Mediapipe: A framework for building perception pipelines,
-
[30]
Follow your pose: Pose- guided text-to-video generation using pose-free videos
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose- guided text-to-video generation using pose-free videos. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Ap...
2024
-
[31]
Follow-your-click: Open-domain regional image animation via short prompts
Yue Ma, Yingqing He, Hongfa Wang, Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung- Yeung Shum, Wei Liu, and Qifeng Chen. Follow-your-click: Open-domain regional image animation via short prompts. CoRR, abs/2403.08268, 2024
2024 arXiv
-
[32]
Follow-your-emoji: Fine- controllable and expressive freestyle portrait animation
Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, and Qifeng Chen. Follow-your-emoji: Fine- controllable and expressive freestyle portrait animation. In SIGGRAPH Asia 2024 Conference Papers, SA 2024, Toky...
2024
-
[33]
V oxceleb: A large-scale speaker identification dataset
Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. V oxceleb: A large-scale speaker identification dataset. In 18th Annual Conference of the International Speech Commu- nication Association, Interspeech 2017, Stockholm, Sweden, August 20-24, 2017, pages 2616–2620. ISCA, 2017. 6
2017
-
[34]
Fsganv2: Im- proved subject agnostic face swapping and reenactment
Yuval Nirkin, Yosi Keller, and Tal Hassner. Fsganv2: Im- proved subject agnostic face swapping and reenactment. IEEE Trans. Pattern Anal. Mach. Intell., 45(1):560–575,
-
[35]
DPE: disen- tanglement of pose and expression for general video portrait editing
Youxin Pang, Yong Zhang, Weize Quan, Yanbo Fan, Xi- aodong Cun, Ying Shan, and Dong-Ming Yan. DPE: disen- tanglement of pose and expression for general video portrait editing. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, J...
2023
-
[36]
Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory
Se Jin Park, Minsu Kim, Joanna Hong, Jeongsoo Choi, and Yong Man Ro. Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory. InThirty- Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications ...
2022
-
[37]
Nambood- iri, and C.V
K R Prajwal, Rudrabha Mukhopadhyay, Vinay P. Nambood- iri, and C.V . Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. InProceedings of the 28th ACM International Conference on Multimedia, page 484–492, New York, NY , USA, 2020. Association f...
2020
-
[38]
High-resolution image syn- thesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10674– ...
2022
-
[39]
Nataniel Ruiz, Eunji Chong, and James M. Rehg. Fine- grained head pose estimation without keypoints. In2018 IEEE Conference on Computer Vision and Pattern Recogni- tion Workshops, CVPR Workshops 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 2074–2083. Computer Vision ...
2018
-
[40]
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. InIEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR 2023, Vancouver, BC, Can...
2023
-
[41]
Style and pose control for image syn- 10 thesis of humans from a single monocular view.CoRR, abs/2102.11263, 2021
Kripasindhu Sarkar, Vladislav Golyanik, Lingjie Liu, and Christian Theobalt. Style and pose control for image syn- 10 thesis of humans from a single monocular view.CoRR, abs/2102.11263, 2021. 2, 3
2021 arXiv
-
[42]
Few-shot head swapping in the wild
Changyong Shu, Hemao Wu, Hang Zhou, Jiaming Liu, Zhibin Hong, Changxing Ding, Junyu Han, Jingtuo Liu, Er- rui Ding, and Jingdong Wang. Few-shot head swapping in the wild. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18...
2022
-
[43]
Make-a-video: Text-to-video generation without text-video data
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data. InThe Eleventh International Conference on Lear...
2023
-
[44]
Everybody’s talkin’: Let me talk as you want.IEEE Trans
Linsen Song, Wayne Wu, Chen Qian, Ran He, and Chen Change Loy. Everybody’s talkin’: Let me talk as you want.IEEE Trans. Inf. Forensics Secur., 17:585–598, 2022. 3
2022
-
[45]
Talking face generation by conditional recurrent adversarial network
Yang Song, Jingwen Zhu, Dawei Li, Andy Wang, and Hairong Qi. Talking face generation by conditional recurrent adversarial network. InProceedings of the Twenty-Eighth In- ternational Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, page...
2019
-
[46]
Speech2talking-face: Inferring and driving a face with syn- chronized audio-visual representation
Yasheng Sun, Hang Zhou, Ziwei Liu, and Hideki Koike. Speech2talking-face: Inferring and driving a face with syn- chronized audio-visual representation. InProceedings of the Thirtieth International Joint Conference on Artificial Intelli- gence, IJCAI 2021, Virtual Event / Montr...
2021
-
[47]
Resolution-robust large mask inpainting with fourier convolutions
Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InIEEE/CVF Winter Conference on Appl...
2022
-
[48]
Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization
Shuai Tan, Bin Ji, and Ye Pan. Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 26307–26317. IEEE, 2024. 3
2024
-
[49]
Neural voice puppetry: Audio-driven facial reenactment
Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, and Matthias Nießner. Neural voice puppetry: Audio-driven facial reenactment. InComputer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, Au- gust 23-28, 2020, Proceedings, Part XVI, pages 716–731. Sp...
2020
-
[50]
EMO: emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions
Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. EMO: emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proc...
2024
-
[51]
Faceverse: a fine-grained and detail- controllable 3d face morphable model from a hybrid dataset
Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang Ma, Liang Li, and Yebin Liu. Faceverse: a fine-grained and detail- controllable 3d face morphable model from a hybrid dataset. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 1...
2022
-
[52]
Stylea- vatar: Real-time photo-realistic portrait avatar from a sin- gle video
Lizhen Wang, Xiaochen Zhao, Jingxiang Sun, Yuxiang Zhang, Hongwen Zhang, Tao Yu, and Yebin Liu. Stylea- vatar: Real-time photo-realistic portrait avatar from a sin- gle video. InACM SIGGRAPH 2023 Conference Proceed- ings, SIGGRAPH 2023, Los Angeles, CA, USA, August 6-10, 2023,...
2023
-
[53]
Hs-diffusion: Semantic-mixing diffusion for head swapping, 2023
Qinghe Wang, Lijie Liu, Miao Hua, Pengfei Zhu, Wangmeng Zuo, Qinghua Hu, Huchuan Lu, and Bing Cao. Hs-diffusion: Semantic-mixing diffusion for head swapping, 2023. 2, 3
2023
-
[54]
One-shot free-view neural talking-head synthesis for video conferenc- ing
Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 10039–10049. Computer Vision Foundation / IEEE, 2021. 3
2021
-
[55]
Hififace: 3d shape and semantic prior guided high fidelity face swapping
Yuhan Wang, Xu Chen, Junwei Zhu, Wenqing Chu, Ying Tai, Chengjie Wang, Jilin Li, Yongjian Wu, Feiyue Huang, and Rongrong Ji. Hififace: 3d shape and semantic prior guided high fidelity face swapping. InProceedings of the Thirtieth International Joint Conference on Artificial In...
2021
-
[56]
LIA: latent image animator.IEEE Trans
Yaohui Wang, Di Yang, Franc ¸ois Br ´emond, and Antitza Dantcheva. LIA: latent image animator.IEEE Trans. Pat- tern Anal. Mach. Intell., 46(12):10829–10844, 2024. 3
2024
-
[57]
Aniportrait: Audio-driven synthesis of photorealistic portrait animation
Huawei Wei, Zejun Yang, and Zhisheng Wang. Aniportrait: Audio-driven synthesis of photorealistic portrait animation. CoRR, abs/2403.17694, 2024. 2, 3, 6, 7
2024 arXiv
-
[58]
Imitating arbitrary talking style for re- alistic audio-driven talking face synthesis
Haozhe Wu, Jia Jia, Haoyu Wang, Yishun Dou, Chao Duan, and Qingshan Deng. Imitating arbitrary talking style for re- alistic audio-driven talking face synthesis. InMM ’21: ACM Multimedia Conference, Virtual Event, China, October 20 - 24, 2021, pages 1478–1486. ACM, 2021. 3
2021
-
[59]
VFHQ: A high-quality dataset and bench- mark for video face super-resolution
Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. VFHQ: A high-quality dataset and bench- mark for video face super-resolution. InIEEE/CVF Con- ference on Computer Vision and Pattern Recognition Work- shops, CVPR Workshops 2022, New Orleans, LA, USA, June 19-...
2022
-
[60]
X-portrait: Expressive portrait anima- tion with hierarchical motion attention
You Xie, Hongyi Xu, Guoxian Song, Chao Wang, Yichun Shi, and Linjie Luo. X-portrait: Expressive portrait anima- tion with hierarchical motion attention. InACM SIGGRAPH 2024 Conference Papers, SIGGRAPH 2024, Denver, CO, USA, 27 July 2024- 1 August 2024, page 115. ACM, 2024. 3
2024
-
[61]
Make- your-video: Customized video generation using textual and structural guidance.IEEE Trans
Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Make- your-video: Customized video generation using textual and structural guidance.IEEE Trans. Vis. Comput. Graph., ...
2025
-
[62]
V ASA-1: lifelike audio-driven talking faces generated in real time
Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. V ASA-1: lifelike audio-driven talking faces generated in real time. InAdvances in Neural Information 11 Processing Systems 38: Annual Conference on Neural Inf...
2024
-
[63]
High-resolution face swapping via latent semantics disentanglement
Yangyang Xu, Bailin Deng, Junle Wang, Yanqing Jing, Jia Pan, and Shengfeng He. High-resolution face swapping via latent semantics disentanglement. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 7632–76...
2022
-
[64]
Magicanimate: Temporally consistent human im- age animation using diffusion model
Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. InIEEE/CVF Con- ference on Computer Vision and Pattern Recognition, CVPR 2024...
2024
-
[65]
Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.CoRR, abs/2308.06721, 2023
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.CoRR, abs/2308.06721, 2023. 2, 3
2023 arXiv
-
[66]
Metaportrait: Identity-preserving talking head gener- ation with fast personalized adaptation
Bowen Zhang, Chenyang Qi, Pan Zhang, Bo Zhang, Hsiang- Tao Wu, Dong Chen, Qifeng Chen, Yong Wang, and Fang Wen. Metaportrait: Identity-preserving talking head gener- ation with fast personalized adaptation. InIEEE/CVF Con- ference on Computer Vision and Pattern Recognition, CV...
2023
-
[67]
Adding conditional control to text-to-image diffusion models
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 3813–
2023
-
[68]
Personatalk: Bring attention to your persona in vi- sual dubbing
Longhao Zhang, Shuang Liang, Zhipeng Ge, and Tianshu Hu. Personatalk: Bring attention to your persona in vi- sual dubbing. InSIGGRAPH Asia 2024 Conference Papers, SA 2024, Tokyo, Japan, December 3-6, 2024, pages 108:1– 108:9. ACM, 2024. 3
2024
-
[69]
Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation
Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, ...
2023
-
[70]
Controlvideo: Training-free controllable text-to-video generation
Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. InThe Twelfth International Conference on Learning Representa- tions, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open- Revie...
2024
-
[71]
Flow-guided one-shot talking face generation with a high- resolution audio-visual dataset
Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. Flow-guided one-shot talking face generation with a high- resolution audio-visual dataset. InIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, vir- tual, June 19-25, 2021, pages 3661–3670. Computer Vis...
2021
-
[72]
Diffswap: High-fidelity and controllable face swapping via 3d-aware masked diffusion
Wenliang Zhao, Yongming Rao, Weikang Shi, Zuyan Liu, Jie Zhou, and Jiwen Lu. Diffswap: High-fidelity and controllable face swapping via 3d-aware masked diffusion. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17- 24, 2...
2023
-
[73]
Magicvideo: Efficient video generation with latent diffusion models.CoRR, abs/2211.11018, 2022
Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models.CoRR, abs/2211.11018, 2022. 3 12
2022 arXiv
-
[2023]
OpenReview.net, 2023. 3
2023
Discussion (0). Sign in to comment.