Details

Autor(en) / Beteiligte

• Baohan Xu
• Yanwei Fu
• Yu-Gang Jiang
• Boyang Li
• Sigal, Leonid

Titel

Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization

Ist Teil von

• IEEE transactions on affective computing, 2018-04, Vol.9 (2), p.255-270

Ort / Verlag

Piscataway: IEEE

Erscheinungsjahr

2018

Link zum Volltext

• IEEE_Xplore

Quelle

IEEE Explore

Beschreibungen/Notizen

• Emotion is a key element in user-generated video. However, it is difficult to understand emotions conveyed in such videos due to the complex and unstructured nature of user-generated content and the sparsity of video frames expressing emotion. In this paper, for the first time, we propose a technique for transferring knowledge from heterogeneous external sources, including image and textual data, to facilitate three related tasks in understanding video emotion: emotion recognition, emotion attribution and emotion-oriented summarization. Specifically, our framework (1) learns a video encoding from an auxiliary emotional image dataset in order to improve supervised video emotion recognition, and (2) transfers knowledge from an auxiliary textual corpora for zero-shot recognition of emotion classes unseen during training. The proposed technique for knowledge transfer facilitates novel applications of emotion attribution and emotion-oriented summarization. A comprehensive set of experiments on multiple datasets demonstrate the effectiveness of our framework.