Details

Autor(en) / Beteiligte

• Kim, Bumsoo
• Mun, Jonghwan
• On, Kyoung-Woon
• Shin, Minchul
• Lee, Junhyun
• Kim, Eun-Sol

Titel

MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detection

Ist Teil von

• 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, p.19556-19565

Ort / Verlag

IEEE

Erscheinungsjahr

2022

Quelle

IEL

Beschreibungen/Notizen

• Human-Object Interaction (HOI) detection is the task of identifying a set of (human, object, interaction) triplets from an image. Recent work proposed transformer encoder-decoder architectures that successfully eliminated the need for many hand-designed components in HOI detection through end-to-end training. However, they are limited to single-scale feature resolution, providing suboptimal performance in scenes containing humans, objects, and their interactions with vastly different scales and distances. To tackle this problem, we propose a Multi-Scale TRansformer (MSTR) for HOI detection powered by two novel HOI-aware deformable attention modules called Dual-Entity attention and Entity-conditioned Context attention. While existing deformable attention comes at a huge cost in HOI detection performance, our proposed attention modules of MSTR learn to effectively attend to sampling points that are essential to identify interactions. In experiments, we achieve the new state-of-the-art performance on two HOI detection benchmarks.