User Tools

Site Tools


10a.008.UL - Decentralized and Distributed Deep Learning for Industrial IoT Devices

Project - Summary

power of mobile devices, lead to the extensive adoption of ML-based applications. Different from conventional model training that needs to collect all the user data in centralized cloud servers, federated learning (FL) has recently drawn increasing research attention as it enables privacy-preserving model training. With FL, decentralized edge devices in participation, train their model copies locally over their siloed datasets, and periodically synchronize the model parameters. However, model training is computationally extensive which easily drains the battery of mobile devices. In addition, due to the uneven distribution of siloed datasets, the shared model may become biased. To address the efficiency and fairness concerns in a resource-constrained federated learning setting, we have developed a strategy to judiciously select mobile devices to participate in the global model aggregation, and adaptively adjust the frequency of local and global model updates. Our strategy makes scheduling and coordination for the federated learning towards both resource efficiency and model fairness. We have conducted a theoretical analysis from the perspectives of fairness and convergence. Extensive experiments with a wide variety of real-world datasets and models, both on a networked prototype system and in a larger-scale simulated environment, have demonstrated that while maintaining similar accuracy performance, our strategy outperforms existing baselines with respect to reducing communication overhead by up to 6× for higher efficiency and improving the fairness metric by up to 57% compared to the state-of-the-art algorithms.

Project - Team

Team Member Role Email Phone Number Academic Site/IAB
Xiali (Sharon) Hei PI (337) 482-1037 UL Lafayette
Li Chen Co-PI N/A UL Lafayette
Funded by: CGI & CSL Behring

Project - Novelty of Approach

Under the resource-constrained federated learning environment, existing works have proposed a variety of approaches towards efficiency improvement, such as reducing the communication traffic volume with gradient compression or sparsification, reducing the communication frequency by adaptive model synchronization, reducing the number of communicating entities through dynamic participant selection, etc. Apart from the efficiency goal, another important concern is fairness with respect to how the collaboratively learned model performs (measured by loss value or ac- curacy level) across user devices. A common definition of fairness in traditional machine learning is with respect to the accuracy parity across protected groups, which cannot be trivially extended to federated learning since it makes no sense to ensure identical accuracy on each device given the significant variation among the data. In sharp contrast to the existing approaches, our developed strategy takes into account both resource efficiency and model fairness in resource-constrained federated learning involving heterogeneous devices. To the best of our knowledge, we are the first to incorporate both the adaptive update frequency and the selection of user devices per round in the synchronous federated learning setting, achieving the best utilization of limited resources while ensuring the fairness of the learned model with a theoretical guarantee.

Project - Deliverables

1 Memory/Computing cost evaluation
2 Energy/Communication cost evaluation
3 Design of system architecture
4 Design of techniques and unit test

Project - Benefits to IAB

Due to the privacy concern of raw data generated and stored at edge devices, federated learning, without exposing raw data, has been increasingly employed by large companies and organizations for machine learning tasks across thousands to millions of user devices. Unique challenges and open problems come long with its promising advantages to increasingly draw wide attention, including uneven data distribution (non-i.i.d.) across devices, constrained resources (power condition), network dynamics (bandwidth, latency) which impact the communication stage, etc. Our solution could be generally adopted by and integrated into the machine learning and federated learning platforms to improve the learning efficiency and model quality, which contributes to the profits.

The federated learning’s local models could be combined with Differential Privacy SGD to update the respective model to achieve better privacy. The EfficientNet with pre-trained models could be a better model for federated learning because it can achieve better accuracy and train much faster. Fixed-round few-shot federated learning could be a good model for distributed devices.

Project - Documents

projects/year10/10a.008.ul.txt · Last modified: 2022/10/17 08:46 by sally.johnson