Details

Autor(en) / Beteiligte

• Tan, Xiaomeng
• Xi, Bobo
• Xu, Haitao
• Li, Yunsong
• Xue, Changbin
• Chanussot, Jocelyn

Titel

MFINet: A Novel Zero-Shot Remote Sensing Scene Classification Network Based on Multimodal Feature Interaction

Ist Teil von

• IEEE journal of selected topics in applied earth observations and remote sensing, 2024, Vol.17, p.11670-11684

Ort / Verlag

Piscataway: IEEE

Erscheinungsjahr

2024

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• Zero-shot classification models aim to recognize image categories that are not included in the training phase by learning seen scenes with semantic information. This approach is particularly useful in remote sensing (RS) since it can identify previously unseen classes. However, most zero-shot RS scene classification approaches focus on matching visual and semantic features, while disregarding the importance of visual feature extraction, especially regarding local-global joint information. Furthermore, the visual and semantic relationships have not been thoroughly investigated due to the separate analysis of these features. To address these issues, we propose a novel zero-shot RS scene classification network based on multimodal feature interaction (MFINet). Specifically, the MFINet deploys hybrid image feature extraction networks, combining convolutional neural networks and an improved Transformer, to capture local discriminant information and long-range contextual information, respectively. Notably, we design a cross-modal feature fusion module to facilitate the MFINet, thereby enhancing relevant information in both the visual and semantic domains. Extensive experiments are conducted on the public zero-shot RS scene dataset, and the results consistently demonstrate that our proposed MFINet outperforms the state-of-the-art methods across various seen/unseen category ratios.