Sie befinden Sich nicht im Netzwerk der Universität Paderborn. Der Zugriff auf elektronische Ressourcen ist gegebenenfalls nur via VPN oder Shibboleth (DFN-AAI) möglich. mehr Informationen...
Coherent Image Animation Using Spatial-Temporal Correspondence
Ist Teil von
IEEE transactions on multimedia, 2023, Vol.25, p.3397-3408
Ort / Verlag
Piscataway: IEEE
Erscheinungsjahr
2023
Quelle
IEEE/IET Electronic Library (IEL)
Beschreibungen/Notizen
Recent studies have achieved remarkable success using deep generative models for the image animation of an arbitrary object.However, previous methods synthesize animated results in a frame-by-frame manner, which is prone to producing flickering and temporally inconsistent results. In this paper, we propose a novel self-supervised framework leveraging temporal information for image animation. Our framework processes a video clip directly instead of processing each frame independently. To achieve coherence in the animated video, we design a spatial-temporal correspondence network (STCN) to maintain the consistency of the keypoints. Specifically, the STCN takes full advantage of temporal information to propagate the keypoints between adjacent frames, and it can be trained with consistent keypoints during the forward and backward process. Furthermore, we apply a 3D-CNN-based generator and discriminator in our framework to ensure coherence in the final output video. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our method.