关键词: 扩散模型, 隐式关键点模型, 音频-嘴型同步, 隐式空间, 映射网络, 面部动画, 身份与行为失真

Abstract: In recent years,diffusion model-based approaches for speech-driven facial animation generation have achieved breakthrough progress,which can efficiently produce high-resolution talking videos with long temporal sequences and precise audio-lip synchronization.However,the videos generated by current methods generally suffer from noticeable blurring and artifacts in the mouth region,which seriously impairs the realism and visual credibility of the synthesized videos.To address this issue,this paper proposes LiveEchoMimic,an improved facial animation generation algorithm based on EchoMimic,and further explores its stan-dardized application paradigm.From the technical implementation perspective,it constructs an end-to-end framework for natural talking video generation,with the EchoMimic diffusion model and implicit key point model serving as the dual-core architecture.Specifically,the EchoMimic diffusion model leverages a joint constraint mechanism of audio features and facial key points to accomplish the generation of coarse-grained talking videos.In contrast,the implicit key point model adopts a video-driven paradigm,which realizes the refined generation of high-quality facial animation videos by regulating the displacement features in the implicit key point space.Furthermore,it constructs an audio-lip mapping model to accurately model the intrinsic correlation between audio features and mouth motion states,and a dedicated mapping network is designed to enhance the audio-lip synchronization accuracy of the generated videos.Finally,extensive experimental evaluations are conducted on two public datasets (CelebV-HQ and MEAD) and one private dataset (Avatar).Both quantitative and qualitative results demonstrate that the proposed LiveEchoMimic method significantly outperforms state-of-the-art approaches in core metrics such as visual quality and audio-lip synchronization,achieving superior video generation performance.From the perspective of application norms,considering that highly realistic speech-driven facial animation technology may give rise to identity forgery and behavioral distortion issues,this paper puts forward operable recommendations from the dimensions of challenges,application principles,and implementation measures.These recommendations are intended to promote the sound development of speech-driven facial animation technology to better meet the demands of social development under controllable and secure premises.

Key words: Diffusion model, Implicit keypoint model, Audio-lip synchronization, Implicit space, Mapping network, Facial animation, Identity and behavior incongruence