Details

Autor(en) / Beteiligte

• Kang, Junhyuk
• Shin, Jieun
• Shin, Jaewon
• Lee, Daeho
• Choi, Ahyoung

Titel

Robust Human Activity Recognition by Integrating Image and Accelerometer Sensor Data Using Deep Fusion Network

Ist Teil von

• Sensors (Basel, Switzerland), 2021-12, Vol.22 (1), p.174

Ort / Verlag

Switzerland: MDPI AG

Erscheinungsjahr

2021

Link zum Volltext

Quelle

Free E-Journal (出版社公開部分のみ）

Beschreibungen/Notizen

• Studies on deep-learning-based behavioral pattern recognition have recently received considerable attention. However, if there are insufficient data and the activity to be identified is changed, a robust deep learning model cannot be created. This work contributes a generalized deep learning model that is robust to noise not dependent on input signals by extracting features through a deep learning model for each heterogeneous input signal that can maintain performance while minimizing preprocessing of the input signal. We propose a hybrid deep learning model that takes heterogeneous sensor data, an acceleration sensor, and an image as inputs. For accelerometer data, we use a convolutional neural network (CNN) and convolutional block attention module models (CBAM), and apply bidirectional long short-term memory and a residual neural network. The overall accuracy was 94.8% with a skeleton image and accelerometer data, and 93.1% with a skeleton image, coordinates, and accelerometer data after evaluating nine behaviors using the Berkeley Multimodal Human Action Database (MHAD). Furthermore, the accuracy of the investigation was revealed to be 93.4% with inverted images and 93.2% with white noise added to the accelerometer data. Testing with data that included inversion and noise data indicated that the suggested model was robust, with a performance deterioration of approximately 1%.