Decentralized AI training
In short:
The Decentralised AI training building block allows data sources of the data space to enable their users to grant access to their data in order to train ML algorithms from participants of the data space.
The data never leaves the source and the owner of the algorithm never has access to it.
Timeline
Start date: T0 (expected: Q1 2024)
End date : T0 + 12 months
Duration (in months): 12
Where we are right now
First version of such a building block implemented by CozyCloud
Want to join the effort? See the Working Groups.
Objectives and expected outcomes
AI providers need user data to train their models, while the data providers need AI models to provide innovative features to their users. This building block is an answer to this need by making the link between AI providers and data providers through secure and trusted decentralized learning, notably in the fields of education and learning.
Once a user gives consent to participate in an AI model, it becomes part of the contributors nodes. Once enough users give consent, an execution tree is computed, including contributors at and aggregators.
Each contributor securely receives the relevant user data identified by the data provider, as well as the AI model to train including the weights, and the related algorithms.
The computation is then made in a secure environment to guarantee the robustness and trustworthiness of the execution.
Once the contribution is computed, the result is splitted in shares and a noise is added to each share to ensure the confidentiality of the contribution. The noise is computed in such a way that at the end of the execution, the aggregation of all the contributions removes the overall noise and produces the final trained model, which can be retrieved by the AI provider.
During the process, no user data is exposed whatsoever, ensuring the security and the privacy of the users.
In short the development phases of this building block are:
Conception and implementation of a decentralized federated AI protocol, to orchestrate node computations and communications.
Conception and implementation of an efficient and secure data retrieval process from nodes, including AI model, algorithms and user data.
Conception and implementation of a training computation model, to cope with the decentralized federated AI protocol properties, in the fields of education and learning.
Trustworthiness and explainability aspects of the solution: the computed AI models should be both trustworthy and explainable.
The standards the building block will rely on:
The learning protocol will be based on the main federated learning principles:
Data between participants is not independent and identically distributed (i.i.d)
Participants do not have the same amount of data
Potentially many participants
Large models with limited communication
The nodes orchestration and computations roles will be based on the DISPERS concepts that describe how to securely build and execute a distributed computation tree
All network communications and encryption mechanisms will use state of the art standards
Roles between partners:
Cozy Cloud is conducting a thesis in collaboration with an Inria team about distributed machine learning in the personal cloud. Preliminary results have been academically published and a proof-of-concept has been developed at Cozy Cloud based on this work.
In this project, Cozy Cloud will bring its expertise to design and implement a decentralized federated learning protocol based on this preliminary work. This protocol should be able to create an aggregation tree of contributing nodes in a p2p fashion, and securely transmit data computations between nodes, to eventually produce trained AI models.
polypoly will provide and further develop a privacy-preserving data repository and execution environment (so called polyPod) that can gather data from different sources. Other types of data will be collected and will be made available, upon user consent, for model learning and validation purposes in order to generate new knowledge and insights that can benefit users.
The polyPod is currently built by the polypoly cooperative, ownership of which is open to all European citizens, and lets users physically store and process data on their own devices in a fully distributed edge-based approach. The polyPod is an open, standardised and non-proprietary platform.
Fraunhofer ISST / Uni Koblenz will establish the trustworthiness and explainability aspects of the solutions developed in this task.
Machine-learning (ML) models need data to be trained, tested, and validated. For most real-world applications, the data is generated among thousands, millions, or more clients (devices). A relatively recent method of creating ML models is called federated learning, where each federated device (a client) communicates only the local model parameters rather than the local training dataset. This feature addresses the need for data privacy and confidentiality that are highly required in education. The way parameters are communicated depends on the federated learning topology, which is either centralized (using a central server to aggregate all the parameters) or decentralized (e.g peer-to-peer by sharing the parameters with a subset of parties). Even though, federated learning may be a viable solution for developing ML models that require large and dispersed data, it presents some challenges that Loria proposes to address:
Data heterogeneity: Data from federated parties can be highly heterogeneous in terms of quantity, quality, and variety of data. It is difficult to predict and quantify the effects of the heterogeneity of training data on the trained ML model. To mitigate the adverse effect of parties with poor data, it is essential to develop indicators that quantify the contribution of the parties involved in the training phase and, as a result, do not consider their parameters when building the global model.
Device heterogeneity: new ML models are required. The federated devices' computing capabilities are frequently heterogeneous. It is always difficult to ensure that training tasks will work across a diverse set of devices. As a result, there is a need to propose new machine learning algorithms that are tailored to Edge infrastructures and consider the computational heterogeneity.
Explainability of decentralized IA and interpretation.
Last updated