Details

Autor(en) / Beteiligte

• Jiang, Boyuan
• Wang, Mengmeng
• Gan, Weihao
• Wu, Wei
• Yan, Junjie

Titel

STM: SpatioTemporal and Motion Encoding for Action Recognition

Ist Teil von

• 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, p.2000-2009

Ort / Verlag

IEEE

Erscheinungsjahr

2019

Quelle

IEEE Electronic Library Online

Beschreibungen/Notizen

• Spatiotemporal and motion features are two complementary and crucial information for video action recognition. Recent state-of-the-art methods adopt a 3D CNN stream to learn spatiotemporal features and another flow stream to learn motion features. In this work, we aim to efficiently encode these two features in a unified 2D framework. To this end, we first propose a STM block, which contains a Channel-wise SpatioTemporal Module (CSTM) to present the spatiotemporal features and a Channel-wise Motion Module (CMM) to efficiently encode motion features. We then replace original residual blocks in the ResNet architecture with STM blcoks to form a simple yet effective STM network by introducing very limited extra computation cost. Extensive experiments demonstrate that the proposed STM network outperforms the state-of-the-art methods on both temporal-related datasets (i.e., Something-Something v1 & v2 and Jester) and scene-related datasets (i.e., Kinetics-400, UCF-101, and HMDB-51) with the help of encoding spatiotemporal and motion features together.