Details

Autor(en) / Beteiligte

• Che, Liwei
• Wang, Jiaqi
• Zhou, Yao
• Ma, Fenglong

Titel

Multimodal Federated Learning: A Survey

Ist Teil von

• Sensors (Basel, Switzerland), 2023-08, Vol.23 (15), p.6986

Ort / Verlag

Switzerland: MDPI AG

Erscheinungsjahr

2023

Link zum Volltext

Quelle

EZB Electronic Journals Library

Beschreibungen/Notizen

• Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.