AI glossary
Federated Learning
Federated learning is a distributed machine learning approach in which a shared model is trained across many devices or servers that each hold their own local data, without that raw data ever being sent to a central server — only model updates are shared and combined.
The approach was introduced by researchers at Google in a 2016 paper, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” by H. Brendan McMahan and colleagues. It was originally developed to improve Gboard, Google’s predictive keyboard for Android, by training a shared next-word-prediction model using typing data that stayed on users’ phones rather than being uploaded to Google’s servers. This origin story highlights the core motivation: enable powerful AI without compromising the Personally identifiable information or sensitive usage patterns of individual users.
How federated learning works
The process begins when a central server sends the current version of a shared model to a selected group of participating devices or organizations. These participants might be smartphones, edge devices, or distinct hospital networks. Once the model arrives, each participant trains it further using only its own local data. This step is crucial because it allows the system to learn from training data that remains in its native environment.
After local training, each participant produces an updated set of model weights or gradients. Instead of sending the raw data itself, each participant sends back only these model updates to the central server. The server then aggregates the updates from all participants into a new, improved version of the shared model. This updated model is sent back out for the next round of training. This cycle repeats, allowing the global model to learn from diverse data sources without ever seeing the underlying records.
Federated averaging
The most common aggregation method is Federated Averaging (FedAvg), introduced in the original 2016 paper. This technique combines the model updates from participants by averaging them. The aggregation is weighted by how much local data each participant used, ensuring that participants with larger datasets have a proportional influence on the final model.
This process repeats over many rounds. As the shared global model incorporates patterns learned across all participants’ local data, it gradually improves. The key distinction here is that no participant’s raw data leaves their device or organization during this aggregation. You can think of the model parameters as the vehicle carrying the learned knowledge, rather than the data itself. For more on how models are structured and updated, see the section on the model parameter.
Why it protects privacy
Because raw data never leaves the local device or organization, federated learning reduces the risk of exposing sensitive information during model training. This is particularly valuable for AI ethics considerations regarding user consent and data ownership. For instance, personal messages, health records, or financial data remain on the user’s phone or within a hospital’s secure server.
However, model updates can sometimes reveal traces of the underlying data. To further reduce this risk, federated learning is often combined with additional privacy techniques. These include adding statistical noise to updates, known as differential privacy, or encrypting updates before aggregation. These layers ensure that even if an update is intercepted or analyzed, it is much harder to reverse-engineer specific Personally identifiable information from the model changes.
Real-world uses
The primary application remains in mobile technology. Google’s Gboard uses federated learning to improve next-word prediction and emoji suggestions using on-device typing data. This allows the keyboard to adapt to individual writing styles while keeping the actual keystrokes private. The efficiency of this approach is often compared to other edge model strategies that process data locally.
In healthcare, hospitals can collaboratively train diagnostic models across patient records at multiple institutions. Without federated learning, a hospital would typically need to share raw patient data with a central research body, which involves significant logistical and privacy hurdles. With this method, each hospital trains the model locally and shares only the updates. This is ideal for any setting where data is naturally distributed across many devices or organizations and cannot be centralized for legal, privacy, or practical reasons.
Limitations
Despite its benefits, federated learning is not a silver bullet. One major constraint is communication cost. Sending model updates back and forth across many rounds, especially with large models, requires significant network bandwidth compared to training on centralized data. If participants have unreliable connectivity or limited compute, coordinating training rounds becomes complicated.
Another challenge is uneven or non-representative data. Participants’ local data can differ substantially from each other, a condition known as non-independent and identically distributed (non-IID) data. This heterogeneity can make the shared model harder to train effectively than it would be on a single centralized dataset. Additionally, if the local data drifts over time, the global model may experience model drift, requiring careful management to maintain accuracy.
FAQ
What is federated learning in simple terms?
Federated learning is a method where an AI model is trained across many devices or servers holding local data, without that raw data ever being sent to a central server. Only model updates are shared and combined to improve the global model.
What is the difference between federated learning and distributed learning?
While both involve multiple computing nodes, federated learning specifically emphasizes that raw data remains local to each device or organization. Distributed learning is a broader term that may involve centralizing data for processing across servers. For a deeper look at how models evolve, see the small language model glossary entry.
How does federated learning protect privacy?
It protects privacy by ensuring that sensitive raw data, such as health records or personal messages, never leaves the local device or institution. Only aggregated model updates are transmitted, which can be further secured with techniques like differential privacy or encryption.
What are the main challenges of federated learning?
The primary challenges include high communication costs due to frequent model updates, issues with non-IID data across participants, and the need for reliable connectivity and compute power on participating devices.