Details

Autor(en) / Beteiligte

• Wang, Weilun
• Zhou, Wengang
• Bao, Jianmin
• Li, Houqiang

Titel

Coherent Image Animation Using Spatial-Temporal Correspondence

Ist Teil von

• IEEE transactions on multimedia, 2023, Vol.25, p.3397-3408

Ort / Verlag

Piscataway: IEEE

Erscheinungsjahr

2023

Quelle

IEEE Xplore

Beschreibungen/Notizen

• Recent studies have achieved remarkable success using deep generative models for the image animation of an arbitrary object.However, previous methods synthesize animated results in a frame-by-frame manner, which is prone to producing flickering and temporally inconsistent results. In this paper, we propose a novel self-supervised framework leveraging temporal information for image animation. Our framework processes a video clip directly instead of processing each frame independently. To achieve coherence in the animated video, we design a spatial-temporal correspondence network (STCN) to maintain the consistency of the keypoints. Specifically, the STCN takes full advantage of temporal information to propagate the keypoints between adjacent frames, and it can be trained with consistent keypoints during the forward and backward process. Furthermore, we apply a 3D-CNN-based generator and discriminator in our framework to ensure coherence in the final output video. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our method.