Details

Autor(en) / Beteiligte

• Perez, Mauricio
• Liu, Jun
• Kot, Alex C.

Titel

Skeleton-based relational reasoning for group activity analysis

Ist Teil von

• Pattern recognition, 2022-02, Vol.122, p.108360, Article 108360

Ort / Verlag

Elsevier Ltd

Erscheinungsjahr

2022

Link zum Volltext

Quelle

Elsevier ScienceDirect Journals Complete

Beschreibungen/Notizen

• •Group activity recognition using skeletons to reason about the actors interactions.•Novel method describes the interactions via learned relations between human joints.•Usage of previously unexplored interactions between humans and objects.•Leveraging of individual action information for higher quality feature learning.•Tailored attention mechanism over relations gives more importance to key individuals. Research on group activity recognition mostly leans on the standard two-stream approach (RGB and Optical Flow) as their input features. Few have explored explicit pose information, with none using it directly to reason about the persons interactions. In this paper, we leverage the skeleton information to learn the interactions between the individuals straight from it. With our proposed method GIRN, multiple relationship types are inferred from independent modules, that describe the relations between the body joints pair-by-pair. Additionally to the joints relations, we also experiment with the previously unexplored relationship between individuals and relevant objects (e.g. volleyball). The individuals distinct relations are then merged through an attention mechanism, that gives more importance to those individuals more relevant for distinguishing the group activity. We evaluate our method in the Volleyball dataset, obtaining competitive results to the state-of-the-art. Our experiments demonstrate the potential of skeleton-based approaches for modeling multi-person interactions.