Details

Autor(en) / Beteiligte

• Nguyen, Hung T.
• Sehwag, Vikash
• Hosseinalipour, Seyyedali
• Brinton, Christopher G.
• Chiang, Mung
• Vincent Poor, H.

Titel

Fast-Convergent Federated Learning

Ist Teil von

• IEEE journal on selected areas in communications, 2021-01, Vol.39 (1), p.201-218

Ort / Verlag

New York: IEEE

Erscheinungsjahr

2021

Link zum Volltext

• IEEE_Xplore

Quelle

IEEE Xplore Digital Library

Beschreibungen/Notizen

• Federated learning has emerged recently as a promising solution for distributing machine learning tasks through modern networks of mobile devices. Recent studies have obtained lower bounds on the expected decrease in model loss that is achieved through each round of federated learning. However, convergence generally requires a large number of communication rounds, which induces delay in model training and is costly in terms of network resources. In this paper, we propose a fast-convergent federated learning algorithm, called <inline-formula> <tex-math notation="LaTeX">\mathsf {FOLB} </tex-math></inline-formula>, which performs intelligent sampling of devices in each round of model training to optimize the expected convergence speed. We first theoretically characterize a lower bound on improvement that can be obtained in each round if devices are selected according to the expected improvement their local models will provide to the current global model. Then, we show that <inline-formula> <tex-math notation="LaTeX">\mathsf {FOLB} </tex-math></inline-formula> obtains this bound through uniform sampling by weighting device updates according to their gradient information. <inline-formula> <tex-math notation="LaTeX">\mathsf {FOLB} </tex-math></inline-formula> is able to handle both communication and computation heterogeneity of devices by adapting the aggregations according to estimates of device's capabilities of contributing to the updates. We evaluate <inline-formula> <tex-math notation="LaTeX">\mathsf {FOLB} </tex-math></inline-formula> in comparison with existing federated learning algorithms and experimentally show its improvement in trained model accuracy, convergence speed, and/or model stability across various machine learning tasks and datasets.