Awesome Talking Face

This is a repository for organizing papres, codes and other resources related to talking face/head. Most papers are linked to the pdf address provided by "arXiv" or "OpenAccess". However, some papers require an academic license to browse. For example, IEEE, springer, and elsevier journal, etc.

🔆 This project is still on-going, pull requests are welcomed!!

If you have any suggestions (missing papers, new papers, key researchers or typos), please feel free to edit and pull a request. Just letting me know the title of papers can also be a big contribution to us. You can do this by open issue or contact me directly via email.

⭐ If you find this repo useful, please star it!!

TO DO LIST

• Main paper list
• Add paper link
• Add codes if have
• Add project page if have
• Datasets introduction
• Add table menu
• Different category criteria

Papers

2D Video - Subject independent

• HeadGAN: Video-and-Audio-Driven Talking Head Synthesis [arXiv 2020] Paper
• Talking-head Generation with Rhythmic Head Motion [ECCV 2020] Paper
• Learning Individual Speaking Styles for Accurate Lip to Speech Synthesis [CVPR 2020] Paper
• Robust One Shot Audio to Video Generation [CVPRW 2020] Paper
• A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild [ACMMM 2020] Paper
• MakeItTalk: Speaker-Aware Talking Head Animation [SIGGRAPH Asia 2020] Paper
• FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesis. [AAAI 2020] Paper
• Realistic Face Reenactment via Self-Supervised Disentangling of Identity and Pose [AAAI 2020] Paper
• Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose [arXiv 2020] Paper
• Photorealistic Lip Sync with Adversarial Temporal Convolutional [arXiv 2020] Paper
• SPEECH-DRIVEN FACIAL ANIMATION USING POLYNOMIAL FUSION OF FEATURES [arXiv 2020] Paper
• Animating Face using Disentangled Audio Representations [WACV 2020] Paper
• Realistic Speech-Driven Facial Animation with GANs. [IJCV 2019] Paper PorjectPage
• Few-Shot Adversarial Learning of Realistic Neural Talking Head Models [ICCV 2019] Paper Code
• Hierarchical Cross-Modal Talking Face Generation with Dynamic Pixel-Wise Loss [CVPR 2019] Paper Code
• Talking Face Generation by Adversarially Disentangled Audio-Visual Representation [AAAI 2019] Paper Code ProjectPage
• Lip Movements Generation at a Glance [ECCV 2018] Paper
• X2Face: A network for controlling face generation using images, audio, and pose codes [ECCV 2018] Paper Code ProjectPage
• Talking Face Generation by Conditional Recurrent Adversarial Network [IJCAI 2019] Paper Code
• Speech-Driven Facial Reenactment Using Conditional Generative Adversarial Networks [arXiv 2018] Paper
• High-Resolution Talking Face Generation via Mutual Information Approximation [arXiv 2018] Paper
• Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network [arXiv 2018] Paper
• You said that? [BMVC 2017] Paper

2D Video - Subject dependent

• Synthesizing Obama: Learning Lip Sync from Audio [SIGGRAPH 2017] Paper Project Page
• PHOTOREALISTIC ADAPTATION AND INTERPOLATION OF FACIAL EXPRESSIONS USING HMMS AND AAMS FOR AUDIO-VISUAL SPEECH SYNTHESIS [ICIP 2017] Paper
• HMM-Based Photo-Realistic Talking Face Synthesis Using Facial Expression Parameter Mapping with Deep Neural Networks [Journal of Computer and Communications2017] Paper
• ObamaNet: Photo-realistic lip-sync from text [arXiv 2017] Paper
• A deep bidirectional LSTM approach for video-realistic talking head [Multimedia Tools Appl 2015] Paper
• Photo-Realistic Expressive Text to Talking Head Synthesis [Interspeech 2013] Paper
• PHOTO-REAL TALKING HEAD WITH DEEP BIDIRECTIONAL LSTM [ICASSP 2015] Paper
• Expressive Speech-Driven Facial Animation [TOG 2005] Paper

3D Animation

• Modality Dropout for Improved Performance-driven Talking Faces [ICMI 2020] Paper
• Audio- and Gaze-driven Facial Animation of Codec Avatars [arXiv 2020] Paper
• Capture, Learning, and Synthesis of 3D Speaking Styles [CVPR 2019] Paper
• VisemeNet: Audio-Driven Animator-Centric Speech Animation [TOG 2018] Paper
• Speech-Driven Expressive Talking Lips with Conditional Sequential Generative Adversarial Networks [TAC 2018] Paper
• End-to-end Learning for 3D Facial Animation from Speech [ICMI 2018] Paper
• Visual Speech Emotion Conversion using Deep Learning for 3D Talking Head [MMAC 2018]
• A Deep Learning Approach for Generalized Speech Animation [SIGGRAPH 2017] Paper
• Audio-Driven Facial Animation by Joint End-to-End Learning of Pose and Emotion [TOG 2017] Paper
• Speech-driven 3D Facial Animation with Implicit Emotional Awareness A Deep Learning Approach [CVPR 2017]
• Expressive Speech Driven Talking Avatar Synthesis with DBLSTM using Limited Amount of Emotional Bimodal Data [Interspeech 2016] Paper
• Real-Time Speech-Driven Face Animation With Expressions Using Neural Networks [TONN 2012] Paper
• Facial Expression Synthesis Based on Emotion Dimensions for Affective Talking Avatar [SIST 2010] Paper

Datasets

Coming soon...