Awesome Federated Machine Learning

Federated Learning (FL) is a new machine learning framework, which enables multiple devices collaboratively to train a shared model without compromising data privacy and security.

This repository aims to keep tracking the latest research advancements of federated learning, including but not limited to research papers, books, codes, tutorials, and videos.

Table of Contents

• Top Machine Learning Conferences
  • ICML ICLR NeurIPS
• Top Computer Vision Conferences
  • CVPR ICCV ECCV
• Top Artificial Intelligence and Data Mining Conferences
  • AAAI AISTATS KDD
• Books
• Papers (Research directions)
  • Model Aggregation
  • Personalization  
  • Recommender system
  • Security  
  • Survey  
  • System Architecture
  • Efficiency  
  • Optimization  
  • Fairness
    
  • Application  
  • Boosting  
  • Incentive mechanism
  • Unsupervised Learning  
  • Heterogeneity  
  • Client Selection
  • Graph Neural Networks
• Google FL Workshops
• Videos and Lectures
• Tutorials and Blogs
• Open-Sources

 

Top Machine Learning Conferences

In this section, we will summarize Federated Learning papers accepted by top machine learning conference, Including NeurIPS, ICML, ICLR.

ICML

Years	Title	Affiliations	Materials
ICML 2021	Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix	Harvard University	video code
	FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Analysis	Peking University; Princeton University	video
	Personalized Federated Learning using Hypernetworks	Bar-Ilan University; NVIDIA	code HomePage video
	Federated Composite Optimization	Stanford University; Google	code video slides
	Exploiting Shared Representations for Personalized Federated Learning	University of Texas at Austin; University of Pennsylvania	code video
	Data-Free Knowledge Distillation for Heterogeneous Federated Learning	Michigan State University	code video
	Federated Continual Learning with Weighted Inter-client Transfer	KAIST	code video
	Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	The University of Iowa	video
	Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning	The University of Tokyo	video
	Federated Learning of User Verification Models Without Sharing Embeddings	Qualcomm	video
	Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning	Accenture	code video
	Ditto: Fair and Robust Federated Learning Through Personalization	CMU; Facebook AI	code video
	Heterogeneity for the Win: One-Shot Federated Clustering	CMU	video
	The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation	Google	video
	Debiasing Model Updates for Improving Personalized Federated Training	Boston University; Arm	video
	One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning	Toyota; Berkeley; Cornell University	code video
	CRFL: Certifiably Robust Federated Learning against Backdoor Attacks	UIUC; IBM	code video
	Federated Learning under Arbitrary Communication Patterns	Indiana University; Amazon	video
ICML 2020	FedBoost: A Communication-Efficient Algorithm for Federated Learning	Google	Video
	FetchSGD: Communication-Efficient Federated Learning with Sketching	UC Berkeley; Johns Hopkins University; Amazon	Video Code
	SCAFFOLD: Stochastic Controlled Averaging for Federated Learning	EPFL; Google	Video
	Federated Learning with Only Positive Labels	Google	Video
	From Local SGD to Local Fixed-Point Methods for Federated Learning	Moscow Institute of Physics and Technology; KAUST	Slide Video
	Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization	KAUST	Slide Video
ICML 2019	Bayesian Nonparametric Federated Learning of Neural Networks	IBM	Code
	Analyzing Federated Learning through an Adversarial Lens	Princeton University; IBM	Code
	Agnostic Federated Learning	Google

ICLR

Years	Title	Affiliation	Materials
ICLR 2022	Bayesian Framework for Gradient Leakage	ETH Zurich	Code
	Federated Learning from only unlabeled data with class-conditional-sharing clients	The University of Tokyo; The Chinese University of Hong Kong	Code
	FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning	Carnegie Mellon University; University of Illinois at Urbana-Champaign; University of Washington
	Acceleration of Federated Learning with Alleviated Forgetting in Local Training	Tsinghua University	Code
	FedPara: Low-rank Hadamard Product for Communication-Efficient Federated Learning	POSTECH	Code
	An Agnostic Approach to Federated Learning with Class Imbalance	University of Pennsylvania	Code
	Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization	Michigan State University; The University of Texas at Austin	code
	Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models	University of Maryland; New York University	code (Minimum) code (Comprehensive)
	ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity	University of Cambridge; University of Oxford
	Diverse Client Selection for Federated Learning via Submodular Maximization	Intel; Carnegie Mellon University	code
	Recycling Model Updates in Federated Learning: Are Gradient Subspaces Low-Rank?	Purdue University	code
	Diurnal or Nocturnal? Federated Learning of Multi-branch Networks from Periodically Shifting Distributions	University of Maryland; Google	code
	Towards Model Agnostic Federated Learning Using Knowledge Distillation	EPFL
	Divergence-aware Federated Self-Supervised Learning	Nanyang Technological University; SenseTime
	What Do We Mean by Generalization in Federated Learning?	Stanford University; Google	code
	FedBABU: Toward Enhanced Representation for Federated Image Classification	KAIST	code
	Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing	EPFL	code
	Improving Federated Learning Face Recognition via Privacy-Agnostic Clusters	Aibee	code
	Hybrid Local SGD for Federated Learning with Heterogeneous Communications	University of Texas; Pennsylvania State University
	On Bridging Generic and Personalized Federated Learning for Image Classification	The Ohio State University	code
	Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond	KAIST; MIT
ICLR 2021	Federated Learning Based on Dynamic Regularization	Boston University; ARM
	Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning	The Ohio State University
	HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients	Duke University	code
	FedMix: Approximation of Mixup under Mean Augmented Federated Learning	KAIST
	Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms	CMU; Google	code
	Adaptive Federated Optimization	Google	code
	Personalized Federated Learning with First Order Model Optimization	Stanford University; NVIDIA
	FedBN: Federated Learning on Non-IID Features via Local Batch Normalization	Princeton University	code
	FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning	The Ohio State University
	Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning	KAIST	code
ICLR 2020	Federated Adversarial Domain Adaptation	Boston University; Columbia University; Rutgers University
	DBA: Distributed Backdoor Attacks against Federated Learning	Zhejiang University; IBM Research	Code
	Fair Resource Allocation in Federated Learning	CMU; Facebook AI	Code
	Federated Learning with Matched Averaging	University of Wisconsin-Madison; IBM Research	Code
	Differentially Private Meta-Learning	CMU
	Generative Models for Effective ML on Private, Decentralized Datasets	Google	Code
	On the Convergence of FedAvg on Non-IID Data	Peking University	Code

NeurIPS

Years	Title	Affiliation	Materials
NeurIPS 2021	Sageflow: Robust Federated Learning against Both Stragglers and Adversaries	KAIST	HomePage
	CAFE: Catastrophic Data Leakage in Vertical Federated Learning	Rensselaer Polytechnic Institute; IBM Research	code HomePage
	Fault-Tolerant Federated Reinforcement Learning with Theoretical Guarantee	NUS	code HomePage
	Optimality and Stability in Federated Learning: A Game-theoretic Approach	Cornell University	code HomePage
	QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning	UCLA	HomePage
	The Skellam Mechanism for Differentially Private Federated Learning	Google Research; CMU	HomePage
	No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data	NUS; Huawei	HomePage
	STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning	University of Minnesota	HomePage
	Subgraph Federated Learning with Missing Neighbor Generation	Emory University; University of British Columbia; Lehigh University	HomePage
	Evaluating Gradient Inversion Attacks and Defenses in Federated Learning	Princeton University	Code HomePage
	Personalized Federated Learning With Gaussian Processes	Bar-Ilan University	code HomePage
	Differentially Private Federated Bayesian Optimization with Distributed Exploration	MIT; NUS	code HomePage
	Parameterized Knowledge Transfer for Personalized Federated Learning	Hong Kong Polytechnic University;	HomePage
	Federated Reconstruction: Partially Local Federated Learning	Google Research	HomePage
	Fast Federated Learning in the Presence of Arbitrary Device Unavailability	Tsinghua University; Princeton University; MIT	code HomePage
	FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective	Duke University; Accenture Labs	code HomePage
	FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout	KAUST; Samsung AI Center	HomePage
	Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients	University of Pennsylvania	HomePage
	Federated Multi-Task Learning under a Mixture of Distributions	INRIA; Accenture Labs	code HomePage
	Federated Graph Classification over Non-IID Graphs	Emory University	HomePage
	Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing	CMU; Hewlett Packard Enterprise	code HomePage
	On Large-Cohort Training for Federated Learning	Google; CMU	code HomePage
	DeepReduce: A Sparse-tensor Communication Framework for Federated Deep Learning	KAUST; Columbia University; University of Central Florida	code HomePage
	PartialFed: Cross-Domain Personalized Federated Learning via Partial Initialization	Huawei	HomePage
	Federated Split Task-Agnostic Vision Transformer for COVID-19 CXR Diagnosis	KAIST	HomePage
	Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning	Tsinghua University; Alibaba; Weill Cornell Medicine	code HomePage
	Federated Linear Contextual Bandits	The Pennsylvania State University; Facebook; University of Virginia	HomePage
	Few-Round Learning for Federated Learning	KAIST	HomePage
	Breaking the centralized barrier for cross-device federated learning	EPFL; Google Research	code HomePage
	Federated-EM with heterogeneity mitigation and variance reduction	Ecole Polytechnique; Google Research	HomePage
	Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning	MIT; Amazon; Google	HomePage
	FedDR – Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization	University of North Carolina at Chapel Hill; IBM Research	code HomePage
	Gradient Inversion with Generative Image Prior	Pohang University of Science and Technology; University of Wisconsin-Madison; University of Washington	code HomePage
NeurIPS 2020	Differentially-Private Federated Linear Bandits	MIT	code
	Federated Principal Component Analysis	University of Cambridge; Quine Technologies	code
	FedSplit: an algorithmic framework for fast federated optimization	UC Berkeley
	Federated Bayesian Optimization via Thompson Sampling	NUS; MIT
	Lower Bounds and Optimal Algorithms for Personalized Federated Learning	KAUST
	Robust Federated Learning: The Case of Affine Distribution Shifts	UC Santa Barbara; MIT
	An Efficient Framework for Clustered Federated Learning	UC Berkeley; DeepMind	Code
	Distributionally Robust Federated Averaging	Pennsylvania State University	Code
	Personalized Federated Learning with Moreau Envelopes	The University of Sydney	code
	Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach	MIT; UT Austin
	Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge	University of Southern California	code
	Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization	CMU; Princeton University
	Attack of the Tails: Yes, You Really Can Backdoor Federated Learning	University of Wisconsin-Madison
	Federated Accelerated Stochastic Gradient Descent	Stanford University	code
	Inverting Gradients - How easy is it to break privacy in federated learning?	University of Siegen	code
	Ensemble Distillation for Robust Model Fusion in Federated Learning	EPFL
	Throughput-Optimal Topology Design for Cross-Silo Federated Learning	INRIA	code
NeurIPS 2018	cpSGD: Communication-efficient and differentially-private distributed SGD	Princeton University; Google
NeurIPS 2017	Federated Multi-Task Learning	Stanford; USC; CMU	code

 

Top Computer Vision Conferences

In this section, we will summarize Federated Learning papers accepted by top computer vision conference, Including CVPR, ICCV, ECCV.

CVPR

Years	Title	Affiliation	Materials
CVPR 2022	FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction	National University of Defense Technology	code
	Federated Class-Incremental Learning	Chinese Academy of Sciences; Northwestern University; University of Technology Sydney	code
	Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning	Peking University; JD Explore Academy; The University of Sydney
	Differentially Private Federated Learning with Local Regularization and Sparsification	Chinese Academy of Sciences
	Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage	University of Tennessee; Oak Ridge National Laboratory; Google Research	code
CVPR 2021	Multi-Institutional Collaborations for Improving Deep Learning-Based Magnetic Resonance Image Reconstruction Using Federated Learning	Johns Hopkins University	code
	Model-Contrastive Federated Learning	National University of Singapore; UC Berkeley	code
	FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space	The Chinese University of Hong Kong	code
	Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective	Duke University	code

ECCV

Years	Title	Affiliation	Materials
ECCV 2020	Federated Visual Classification with Real-World Data Distribution	MIT; Google	Video

ICCV

Years	Title	Affiliation	Materials
ICCV 2021	Federated Learning for Non-IID Data via Unified Feature Learning and Optimization Objective Alignment	Peking University
	Ensemble Attention Distillation for Privacy-Preserving Federated Learning	University at Buffalo
	Collaborative Unsupervised Visual Representation Learning from Decentralized Data	Nanyang Technological University; SenseTime

 

Top Artificial Intelligence and Data Mining Conferences

In this section, we will summarize Federated Learning papers accepted by top AI and DM conference, Including AAAI, AISTATS, KDD.

AAAI

Years	Title	Affiliation	Materials
AAAI 2022	HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images	The Chinese University of Hong Kong; Beihang University	code
	Federated Learning for Face Recognition with Gradient Correction	Beijing University of Posts and Telecommunications
	SpreadGNN: Decentralized Multi-Task Federated Learning for Graph Neural Networks on Molecular Data	university of Southern California	code
	SmartIdx: Reducing Communication Cost in Federated Learning by Exploiting the CNNs Structures	Harbin Institute of Technology; Peng Cheng Laboratory
	Bridging between Cognitive Processing Signals and Linguistic Features via a Unified Attentional Network	Tianjin University
	Seizing Critical Learning Periods in Federated Learning	SUNY-Binghamton University; Louisiana State University
	Coordinating Momenta for Cross-silo Federated Learning	University of Pittsburgh
	FedProto: Federated Prototype Learning over Heterogeneous Devices	University of Technology Sydney; University of Washington	code
	FedSoft: Soft Clustered Federated Learning with Proximal Local Updating	Carnegie Mellon University
	Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better	The University of Texas at Austin	code
	FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition	National Taiwan University	code
	SplitFed: When Federated Learning Meets Split Learning	CSIRO; Lehigh University	code
	Efficient Device Scheduling with Multi-Job Federated Learning	Soochow University; Baidu
	Implicit Gradient Alignment in Distributed and Federated Learning	IIT Kanpur; EPFL
	Federated Nearest Neighbor Classification with a Colony of Fruit-Flies	IBM Research; Wichita State University
AAAI 2021	Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating	Xidian University; JD Tech	video
	FedRec++: Lossless Federated Recommendation with Explicit Feedback	Shenzhen University	video
	Federated Multi-Armed Bandits	University of Virginia	code video
	On the Convergence of Communication-Efficient Local SGD for Federated Learning	Temple University; University of Pittsburgh	video
	FLAME: Differentially Private Federated Learning in the Shuffle Model	Renmin University of China; Kyoto University	video code
	Toward Understanding the Influence of Individual Clients in Federated Learning	Shanghai Jiao Tong University; The University of Texas at Dallas	video
	Provably Secure Federated Learning against Malicious Clients	Duke University	video slides
	Personalized Cross-Silo Federated Learning on Non-IID Data	Simon Fraser University; McMaster University	video
	Model-Sharing Games: Analyzing Federated Learning under Voluntary Participation	Cornell University	code video
	Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning	University of Nevada; IBM Research	video
	Game of Gradients: Mitigating Irrelevant Clients in Federated Learning	IIT Bombay; IBM Research	video Supplementary
	Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models	The Chinese University of Hong Kong; Arizona State University	video code
	Adressing Class Imbalance in Federated Learning	Northwestern University	video code
	Defending against Backdoors in Federated Learning with Robust Learning Rate	The University of Texas at Dallas	video code
AAAI 2020	Practical Federated Gradient Boosting Decision Trees	National University of Singapore; The University of Western Australia	code
	Federated Learning for Vision-and-Language Grounding Problems	Peking University; Tencent
	Federated Latent Dirichlet Allocation: A Local Differential Privacy Based Framework	Beihang University
	Federated Patient Hashing	Cornell University
	Robust Federated Learning via Collaborative Machine Teaching	Symantec Research Labs; KAUST

AISTATS

Years	Title	Affiliation	Materials
AISTATS 2021	Free-rider Attacks on Model Aggregation in Federated Learning	Accenture Labs	video Supplementary
	Federated f-differential privacy	University of Pennsylvania	code video Supplementary
	Federated learning with compression: Unified analysis and sharp guarantees	The Pennsylvania State University; The University of Texas at Austin	code video Supplementary
	Shuffled Model of Differential Privacy in Federated Learning	UCLA; Google	video Supplementary
	Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning	Google	video Supplementary
	Federated Multi-armed Bandits with Personalization	University of Virginia; The Pennsylvania State University	code video Supplementary
	Towards Flexible Device Participation in Federated Learning	CMU; Sun Yat-Sen University	video Supplementary
AISTATS 2020	FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization	UC Santa Barbara; UT Austin	video Supplementary
	How To Backdoor Federated Learning	Cornell Tech	video code Supplementary
	Federated Heavy Hitters Discovery with Differential Privacy	RPI; Google	video Supplementary

KDD

Years	Sessions	Title	Affiliation	Materials
KDD 2021	Research Track	Fed2: Feature-Aligned Federated Learning	George Mason University; Microsoft; University of Maryland
		FedRS: Federated Learning with Restricted Softmax for Label Distribution Non-IID Data	Nanjing University
		Federated Adversarial Debiasing for Fair and Trasnferable Representations	Michigan State University	HomePage
		Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling	University of Southern California	code
	Application Track	AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization
		FLOP: Federated Learning on Medical Datasets using Partial Networks	Duke University	code
KDD 2020	Research Track	FedFast: Going Beyond Average for Faster Training of Federated Recommender Systems	University College Dublin	video
	Application Track	Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data	JD Tech	video

 

Books

• 联邦学习（Federated Learning）

  Chinese Version

  English Version

• 联邦学习实战（Practicing Federated Learning）

  Chinese Version

  Github

 

Papers

1. Model Aggregation

Model Aggregation (or Model Fusion) refers to how to combine local models into a shared global model.

Papers	Abbreviation	Conferences/Affiliations	Materials
Communication-Efficient Learning of Deep Networks from Decentralized Data	FedAvg	ASTATS 2017
Bayesian Nonparametric Federated Learning of Neural Networks	PFNM	ICML 2019	code
Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent	Krum	NeurIPS 2017
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates	median; trimmed mean	ICML 2018
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms	median; mean	NeurIPS 2020
The hidden vulnerability of distributed learning in byzantium	Bulyan	ICML 2018
Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance	Zeno	ICML 2019	code
Statistical Model Aggregation via Parameter Matching	SPAHM	NeurIPS 2019	code
Fed+: A Unified Approach to Robust Personalized Federated Learning	Fed+
FEDERATED OPTIMIZATION IN HETEROGENEOUS NETWORKS	FedProx	MLSys 2020	code
Separation of Powers in Federated Learning	Truda
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning	FedBE	ICLR 2021 (The Ohio State University)
Federated Learning with Matched Averaging	FedMA	ICLR 2020 (University of Wisconsin-Madison; IBM)	Code

 

2. Personalization

Personalized federated learning refers to train a model for each client, based on the client’s own dataset and the datasets of other clients. There are two major motivations for personalized federated learning：

• Due to statistical heterogeneity across clients, a single global model would not be a good choice for all clients. Sometimes, the local models trained solely on their private data perform better than the global shared model.
• Different clients need models specifically customized to their own environment. As an example of model heterogeneity, consider the sentence: “I live in .....”. The next-word prediction task applied on this sentence needs to predict a different answer customized for each user. Different clients may assign different labels to the same data.

Personalized federated learning Survey paper:

• Survey of Personalization Techniques for Federated Learning

• Three Approaches for Personalization with Applications to Federated Learning

• Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Framework

Methodology	Papers	Conferences/Affiliations	Materials
Multi-Task Learning	Federated Multi-Task Learning	NeurIPS 2017 (Stanford; USC; CMU)	code
	Decentralized Collaborative Learning of Personalized Models over Networks	AISTATS 2017 (INRIA)
	Variational Federated Multi-Task Learning	ETH Zurich
	Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs	AISTATS 2020 (INRIA)	video
	Personalized Cross-Silo Federated Learning on Non-IID Data	AAAI 2021 (Simon Fraser University; McMaster University; Huawei Technologies Canada)	video
	Ditto: Fair and Robust Federated Learning Through Personalization	ICML 2021 (CMU; Facebook AI)	code video
	Federated Multi-Task Learning under a Mixture of Distributions	NeurIPS 2021 (Inria; Accenture Labs)	code
Meta Learning	Personalized Federated Learning: A Meta-Learning Approach	MIT
	Debiasing Model Updates for Improving Personalized Federated Training	ICML 2021 (Boston University; Arm)	video
	Improving Federated Learning Personalization via Model Agnostic Meta Learning	University of Washington; Google
	Adaptive Gradient-Based Meta-Learning Methods	CMU
	Federated Meta-Learning with Fast Convergence and Efficient Communication	Huawei Noah’s Ark Lab
Mixture of Global and Local Models	Federated Learning of a Mixture of Global and Local Models	KAUST
	Federated User Representation Learning	University of Michigan Facebook
	Adaptive Personalized Federated Learning	The Pennsylvania State University
Personalization Layers	Federated Learning with Personalization Layers	Adobe Research Indian Institute of Technology
	Think Locally, Act Globally: Federated Learning with Local and Global Representations	CMU University of Tokyo Columbia University
	Exploiting Shared Representations for Personalized Federated Learning	ICML 2021 (University of Texas at Austin; University of Pennsylvania)	code video
Transfer Learning	Federated evaluation of on-device personalization	Google
	Salvaging Federated Learning by Local Adaptation	Cornell University
	Private Federated Learning with Domain Adaptation	Oracle Labs
	QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning	NeurIPS 2021 (UCLA)	HomePage
Clustering	Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints	Fraunhofer Heinrich Hertz Institute	Code
	An Efficient Framework for Clustered Federated Learning	UC Berkeley DeepMind	Code
	Robust Federated Learning in a Heterogeneous Environment	UC Berkeley
	Personalized Federated Learning with First Order Model Optimization	ICLR 2021 (Stanford University; NVIDIA)
Hypernetwork	Personalized Federated Learning using Hypernetworks	Bar-Ilan University; NVIDIA	code HomePage video

 

3. Recommender system

Recommender system (RecSys) is widely used to solve information overload. In general, the more data RecSys use, the better the recommendation performance we can obtain.

Traditionally, RecSys requires the data that are distributed across multiple devices to be uploaded to the central database for model training. However, due to privacy and security concerns, such directly sharing user data strategies are no longer appropriate.

The incorporation of federated learning and RecSys is a promising approach, which can alleviate the risk of privacy leakage.

Methodology	Papers	Conferences/Affiliations	Materials
Matrix Factorization	Secure federated matrix factorization	IEEE Intelligent Systems
	Federated Multi-view Matrix Factorization for Personalized Recommendations	ECML-PKDD 2020	video
	Decentralized Recommendation Based on Matrix Factorization: A Comparison of Gossip and Federated Learning	ECML-PKDD 2019
	Towards Privacy-preserving Mobile Applications with Federated Learning: The Case of Matrix Factorization	MobiSys 2019
	Meta Matrix Factorization for Federated Rating Predictions	ACM SIGIR 2020	code
	Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System	Arxiv
GNN	FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation	Arxiv

 

4. Security

4.1. Attack

Methodology	Papers	Conferences/Affiliations	Materials
Backdoor Attack	How To Backdoor Federated Learning	AISTATS 2020	code
	Can You Really Backdoor Federated Learning?	Arxiv
	Attack of the Tails: Yes, You Really Can Backdoor Federated Learning	NeurIPS 2020	code
	DBA: Distributed Backdoor Attacks against Federated Learning	ICLR 2020	code
Gradients Attack	Deep Leakage from Gradients	NeurIPS 2020	HomePage code
	Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix	ICML 2021	code
	Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning	ACM CCS 2017	video
	iDLG: Improved Deep Leakage from Gradients	Arxiv	code
	See through Gradients: Image Batch Recovery via GradInversion	CVPR 2021
	Inverting Gradients - How easy is it to break Privacy in Federated Learning?	NeurIPS 2020	code
	CAFE: Catastrophic Data Leakage in Vertical Federated Learning	NeurIPS 2021 (Rensselaer Polytechnic Institute; IBM)	code HomePage
	Evaluating Gradient Inversion Attacks and Defenses in Federated Learning	NeurIPS 2021 (Princeton University)
	R-GAP: RECURSIVE GRADIENT ATTACK ON PRIVACY	ICLR 2021 (KU Leuven, Belgium)	Code
	Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage	CVPR 2022 (University of Tennessee;Oak Ridge National Laboratory)	Code
Model Poison Attack	Analyzing Federated Learning through an Adversarial Lens	ICML 2019 (Princeton University; IBM)	Code

 

4.2. Defense

Methodology	Papers	Conferences/Affiliations	Materials
FL+DP	Federated Learning With Differential Privacy: Algorithms and Performance Analysis	IEEE Transactions on Information Forensics and Security
	Differentially Private Federated Learning: A Client Level Perspective	Arxiv	code
	Learning Differentially Private Recurrent Language Models	ICLR 2018
	The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation	ICML 2021 (Google)	video
	The Skellam Mechanism for Differentially Private Federated Learning	NeurIPS 2021 (Google Research; CMU)	HomePage
FL+HE	Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption	Arxiv
	BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning	USENIX 2020	code
FL+TEE	PPFL: Privacy-preserving Federated Learning with Trusted Execution Environments	ACM MobiSys 2021
	Darknetz: towards model privacy at the edge using trusted execution environments.	ACM MobiSys 2020	code video
Algorithm	A Little Is Enough: Circumventing Defenses For Distributed Learning	NeurIPS 2019
	CRFL: Certifiably Robust Federated Learning against Backdoor Attacks	ICML 2021 (UIUC; IBM)	code video
	Evaluating Gradient Inversion Attacks and Defenses in Federated Learning	NeurIPS 2021 (Princeton University)	Code HomePage

 

5. Survey

Category	Papers
General	Federated machine learning: Concept and applications
	A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
	Federated Learning in Mobile Edge Networks: A Comprehensive Survey
	Advances and Open Problems in Federated Learning
	Federated Learning: Challenges, Methods, and Future Directions
	The Internet of Federated Things (IoFT)
Security	A survey on security and privacy of federated learning
	Threats to Federated Learning: A Survey
	Vulnerabilities in Federated Learning
Personalization	Survey of Personalization Techniques for Federated Learning
	Towards Personalized Federated Learning
Aggregation	Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
Incentive	A Comprehensive Survey of Incentive Mechanism for Federated Learning
	A Survey of Incentive Mechanism Design for Federated Learning
Applications	A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
Fairness	A Survey of Fairness-Aware Federated Learning
Graph	FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks
	Federated Graph Learning - A Position Paper

 

6. System Design

Platform	Papers	Affiliations
Tensorflow-Federated	Towards Federated Learning at Scale: System Design	Google
PySyft	A generic framework for privacy preserving deep learning	OpenMined
FedML	FedML: A Research Library and Benchmark for Federated Machine Learning	fedml.ai
OpenFL	OpenFL: An open-source framework for Federated Learning	Intel
Clara		NVIDIA
IBM Federated Learning	IBM Federated Learning: an Enterprise Framework White Paper	IBM
FATE	FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection	WeBank
Fedlearner		Bytedance
Flower	Flower: A Friendly Federated Learning Research Framework	flower.dev
PaddleFL		Baidu
LEAF	LEAF: A Benchmark for Federated Settings	CMU
PyVertical	PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN	OpenMined
Sherpa.ai Federated Learning		Sherpa.ai
FedScale	FedScale: Benchmarking Model and System Performance of Federated Learning at Scale	fedscale.ai
EasyFL	EasyFL: A Low-code Federated Learning Platform For Dummies	NTU

 

7. Communication-Efficient

Papers	Conferences/Affiliations	Materials
RPN: A Residual Pooling Network for Efficient Federated Learning	ECAI 2020 (WeBank)
FedBoost: A Communication-Efficient Algorithm for Federated Learning	ICML 2021 (Google)	Video
Federated Learning: Strategies for Improving Communication Efficiency	arXiv (Google)
FetchSGD: Communication-Efficient Federated Learning with Sketching	ICML 2020 (UC Berkeley; Johns Hopkins University; Amazon)	Video Code
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization	ICML 2020 (KAUST)	Slide Video

 

8. Optimization

Papers	Application Scenarios	Conferences/Affiliations	Materials
Federated Composite Optimization	loss function contains a non-smooth regularizer	ICML 2021(Google)	code
Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	Federated Deep AUC Maximization	ICML 2021 (The University of Iowa)	video
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning	Non-Convex Objective optimization	ICML 2021 (The University of Tokyo)	video
From Local SGD to Local Fixed-Point Methods for Federated Learning	fixed-point algorithms optimization	ICML 2020 (Moscow Institute of Physics and Technology; KAUST)	Slide Video
Federated Learning Based on Dynamic Regularization	In each round, the objective function for each device dynamically updates its regularizer	ICLR 2021 (Boston University; ARM)
Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms	Formulate federated learning optimization as a posterior inference problem	ICLR 2021 (CMU; Google)	code
Adaptive Federated Optimization	Federated versions of adaptive optimizers	ICLR 2021 (Google)	code
FedBN: Federated Learning on Non-IID Features via Local Batch Normalization	How to uses local batch normalization to alleviate the feature shift before averaging models.	ICLR 2021 (Princeton University)	code
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning	SFederated versions of emi-Supervised Learning	ICLR 2021 (KAIST)	code
Sageflow: Robust Federated Learning against Both Stragglers and Adversaries	Handle both stragglers (slow devices) and adversaries simultaneously	NeurIPS 2021 (KAIST)	HomePage
STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning	Distributed stochastic non-convex optimization	University of Minnesota	HomePage
Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating	Vertical Federated Learning Optimization	AAAI 2021 (Xidian University; JD Tech)	video

 

9. Fairness

Papers	Conferences/Affiliations	Materials
Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning	NeurIPS 2021 (Tsinghua University; Alibaba)	code HomePage
Fairness-aware Agnostic Federated Learning	SDM 2021 (University of Arkansas)
Fair Resource Allocation in Federated Learning	ICLR 2020 (CMU; Facebook AI)	Code
Agnostic Federated Learning	ICML 2019 (Google)
Mitigating Bias in Federated Learning	arXiv (IBM)
Ditto: Fair and Robust Federated Learning Through Personalization	ICML 2021 (CMU; Facebook AI)	code video

 

10. Applications

Applications	Papers	Conferences/Affiliations	Materials
Computer Vision	FedVision: An Online Visual Object Detection Platform Powered by Federated Learning	WeBank (AAAI 2020)	code
Nature Language Processing	Federated learning for emoji prediction in a mobile keyboard	Google
	Federated Learning for Mobile Keyboard Prediction	Google
	Applied federated learning: Improving google keyboard query suggestions	Google
	Federated Learning Of Out-Of-Vocabulary Words	Google
Automatic Speech Recognition	A Federated Approach in Training Acoustic Models	MicroSoft (INTERSPEECH 2020)	Video
	Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?	INRIA (INTERSPEECH 2019)
	Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework	Google (ICASSP 2021)	Google Assistant Help
	Federated Evaluation and Tuning for On-Device Personalization: System Design \& Applications	Apple	Report
Healthcare	Privacy-preserving Federated Brain Tumour Segmentation	NVIDIA (MICCAI MLMI 2019)
	Advancing health research with Google Health Studies	Google	Blog
	Multi-institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation	Intel	Blog
Blockchain	FedCoin: A Peer-to-Peer Payment System for Federated Learning	Arxiv
	Blockchained On-Device Federated Learning	IEEE Communications Letters 2019

 

11. Boosting

Category	Papers	Conferences/Affiliations	Materials
Tree-Base Boosting	Practical Federated Gradient Boosting Decision Trees	AAAI 2020 (NUS)	code
	Secureboost: A lossless federated learning framework	IEEE Intelligent Systems 2021 (WeBank; HKUST)
	Large-scale Secure XGB for Vertical Federated Learning	CIKM 2021 (Ant Group)	video

 

12. Incentive mechanism

Papers	Conferences/Affiliations	Materials
One for One, or All for All: Equilibria and Optimality of Collaboration in Federated Learning	ICML 2021(Toyota; Berkeley; Cornell University)	code video
Optimality and Stability in Federated Learning: A Game-theoretic Approach	NeurIPS 2021 (Cornell University)	code HomePage

 

13. Unsupervised Learning

Category	Papers	Conferences/Affiliations	Materials
Clustering	Heterogeneity for the Win: One-Shot Federated Clustering	ICML 2021 (CMU)	video
Representations Learning	Exploiting Shared Representations for Personalized Federated Learning	ICML 2021 (University of Texas at Austin; University of Pennsylvania)	code video

 

14. Heterogeneity

Category	Papers	Conferences/Affiliations	Materials
Data Heterogeneity (NON-IID)	Data-Free Knowledge Distillation for Heterogeneous Federated Learning	ICML 2021 (Michigan State University)	code video
	Federated Deep AUC Maximization for Hetergeneous Data with a Constant Communication Complexity	ICML 2021 (The University of Iowa)	video
	Exploiting Shared Representations for Personalized Federated Learning	ICML 2021 (University of Texas at Austin; University of Pennsylvania)	code video
	SCAFFOLD: Stochastic Controlled Averaging for Federated Learning	ICML 2020 (EPFL; Google)	Video
	Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning	ICLR 2021 (The Ohio State University)
	HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients	ICLR 2021 (Duke University)	code
	FedMix: Approximation of Mixup under Mean Augmented Federated Learning	ICLR 2021 (KAIST)
	On the Convergence of FedAvg on Non-IID Data	ICLR 2020 (Peking University)	Code
	No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data	NeurIPS 2021 (NUS; Huawei)	HomePage
Model Heterogeneity	Personalized Federated Learning with Moreau Envelopes	NeurIPS 2020 (The University of Sydney)	code
	Federated Learning of a Mixture of Global and Local Models	KAUST
	Salvaging Federated Learning by Local Adaptation	Cornell University
Device Heterogeneity	Sageflow: Robust Federated Learning against Both Stragglers and Adversaries	NeurIPS 2021 (KAIST)	HomePage
	Towards Flexible Device Participation in Federated Learning	AISTATS 2021 (CMU; Sun Yat-Sen University)	video Supplementary
	Asynchronous Federated Optimization	12th Annual Workshop on Optimization for Machine Learning

 

15. Client Selection

Papers	Conferences/Affiliations	Materials
Oort: Efficient Federated Learning via Guided Participant Selection	OSDI 2021 (University of Michigan)	code video
Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning	ICML 2021 (Accenture)	code video
Federated Multi-Armed Bandits	AAAI 2021 (University of Virginia)	code video
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization	AISTATS 2020 (UC Santa Barbara; UT Austin)	video Supplementary

 

16. Graph Neural Networks

This section is partially refers to this repository: Awesome-Federated-Learning-on-Graph-and-GNN-papers

Category	Papers	Conferences/Affiliations	Materials
Knowledge Graph	FedE: Embedding Knowledge Graphs in Federated Setting	arXiv (Zhejiang University; Huawei)	code
	Improving Federated Relational Data Modeling via Basis Alignment and Weight Penalty	arXiv (Antgroup)
	Differentially Private Federated Knowledge Graphs Embedding	CIKM 2021 (Beihang University)	code
Graph Neural Network	Federated Dynamic GNN with Secure Aggregation	arXiv (University of Notre Dame)
	Vertically Federated Graph Neural Network for Privacy-Preserving Node Classification	arXiv (AntGroup)
	ASFGNN: Automated separated-federated graph neural network	arXiv (AntGroup)
	Cluster-driven Graph Federated Learning over Multiple Domains	CVPR 2021 (Politecnico di Torino)
	SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks	AAAI 2022 (university of Southern California)	code
	Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling	KDD 2021 (University of Southern California)	code
	Federated Graph Classification over Non-IID Graphs	NeurIPS 2021 (Emory University)	HomePage
	Subgraph Federated Learning with Missing Neighbor Generation	NeurIPS 2021 (Emory University; University of British Columbia; Lehigh University)	HomePage

 

Google FL Workshops

• Google Workshop on Federated Learning and Analytics 2021
• Google Workshop on Federated Learning and Analytics 2020
• Google Federated Learning workshop 2019

 

Videos and Lectures

• TensorFlow Federated Tutorial Session

• TensorFlow Federated (TFF): Machine Learning on Decentralized Data - Google, TF Dev Summit ‘19 2019

• Federated Learning: Machine Learning on Decentralized Data - Google, Google I/O 2019

• Federated Learning - Cloudera Fast Forward Labs, DataWorks Summit 2019

• GDPR, Data Shortage and AI - Qiang Yang, AAAI 2019 Invited Talk

 

Tutorials and Blogs

• What is Federated Learning - Nvidia 2019
• Online Federated Learning Comic - Google 2019
• Federated Learning: Collaborative Machine Learning without Centralized Training Data - Google AI Blog 2017
• Go Federated with OpenFL - Intel 2021

 

Open-Sources

• WeBank Federated AI Technology Enabler
• Google Tensorflow Federated
• OpenMined PySyft
• Baidu PaddleFL
• FedML
• OpenFL

⬆ Return to top