Blockchain Technologies for Decentralized Machine Learning
Purchased from Istockphoto. Copyright.
Federated Learning (FL) operates in an iterative manner by selecting a subset of clients in each round, asking them to train using their local data. The clients then share their model with the server, which aggregates all client models into a single global model. Traditional FL approaches typically select random sets of clients for each round, which can negatively impact the quality of the trained model. Moreover, clients are often reluctant to participate in collaborative learning due to a lack of secure monetization mechanism. This article presents FL-MAB (FL-Multi-Auction using Blockchain), a client selection mechanism for FL operation in a smart contract, which rewards clients for their participation in using cryptocurrencies.
Introduction
Data generated by the Internet of Things (IoT) are often not a single entity but rather distributed among many parties. However, due to new data privacy laws and transmission overhead, it is impractical to gather all the data from IoT devices to the central cloud in conducting centralized Machine Learning model training.
Federated Learning (FL) emerged from Machine Learning and overcomes the privacy issue by eliminating the need for centralized storage and enabling the collaborative training of a shared model with distributed data. In traditional FL settings, client selection is random for training in each FL round. These randomly selected clients in FL might have heterogeneous resources, i.e. battery level and internet connectivity. Therefore, any client within a network may go offline during the model training process, which will affect the global model accuracy. Moreover, there are also some malicious clients who intentionally want to jeopardize the FL ecosystem by intimidating the global model through attacks.
The server can be biased by being involved in nepotism—endorsing its peers by selecting and rewarding them. Moreover, resource-rich clients are demotivated to participate in collaborative training due to a lack of secure monetization mechanism. Therefore, it is imperative to develop a secure, privacy-preserving FL framework that motivates resource-rich clients to participate, giving them rewards based on their contribution.
Blockchain Integration with FL
A blockchain system is a distributed tamper-proof database which provides auditability and accountability. Therefore, a blockchain-backed federated learning network promotes the federation of more devices by providing a secure FL System. The blockchain enables interaction between network entities—clients and server—and compels them to be fair by providing auditability, traceability, non-repudiation, and secure monetization as shown in Figure 1.
Figure 1: Blockchain integration with FL
A Proposed Federated Learning Protocol
The proposed client selection protocol FL-MAB consists of three phases—auction, federated learning, and reward phase. First, client selection will be performed by blockchain-based smart contracts through auctioning. Then, the clients will submit their bid with resources, i.e. total samples of local data, bandwidth, and internet connectivity. Finally, high-scoring clients will be nominated as winners by the blockchain.
In the FL phase, winners will start training their local model by using their private data. The local model hash will be saved on-chain to avoid tampering. Finally, the server will perform an aggregation to calculate the global model. The contribution of every client in the global model will be calculated in the rewarding phase. Finally, the clients will be incentivized based on their performance using cryptocurrency.
Validating the Proposed Approach
We compared our solution with Random FL, a traditional FL approach where clients are randomly selected by the server to participate in the FL rounds. Figure 2 shows that FL-MAB increases the convergence speed and accuracy of the model for the MNIST-O handwritten character recognition task after the 20th round of training compared to RandomFL.
Figure 3 also shows an increase in the model performance of the CIFAR-10 image classification task. In comparison with RandomFL, the accuracy of FL-MAB remains markedly higher over 20th FL round of training.
Figure 2 : MNIST—Accuracy comparison
Figure 3: CIFAR-10 Accuracy comparison
We implemented three different methods to calculate the marginal contribution of clients, i.e., Shapley value, contribution index, and influence function. Figure 4 illustrates that our heuristic method, the Shapley-based contribution index, is approximately 34% faster as it calculates the marginal contribution of clients by approximating the coalition models.
Figure 4: Comparison of reward distribution methods
Conclusion
We proposed a solution to promote a Federated Learning process while providing both client selection and fair monetization using smart contracts. Our experiments demonstrate that our solution reduced the training rounds by eliminating straggler clients and improved the accuracy of the model. Furthermore, our reward heuristic provides fair monetization while being significantly faster to execute than the baseline.
Additional Information
For more information on this research, please read the following paper: Batool, Z., Zhang, K., & Toews, M. (2022, April). FL-MAB: client selection and monetization for blockchain-based federated learning. In Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing (pp. 299-307).
