Cover image for Federated Learning

Federated Learning

Author Qiang Yang , Yang Liu , Yong Cheng , Yan
Release Date: 2019/12/01
Topic:
0%
23
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

How is it possible to allow multiple data owners to collaboratively train and use a shared prediction model while keeping all the local training data private?

Traditional machine learning approaches need to combine all data at one location, typically a data center, which may very well violate the laws on user privacy and data confidentiality. Today, many parts of the world demand that technology companies treat user data carefully according to user-privacy laws. The European Union's General Data Protection Regulation (GDPR) is a prime example. In this book, we describe how federated machine learning addresses this problem with novel solutions combining distributed machine learning, cryptography and security, and incentive mechanism design based on economic principles and game theory. We explain different types of privacy-preserving machine learning solutions and their technological backgrounds, and highlight some representative practical use cases. We show how federated learning can become the foundation of next-generation machine learning that caters to technological and societal needs for responsible AI development and application.

Table of Contents

  1. Cover
  2. Copyright
  3. Title Page
  4. Contents
  5. Preface
  6. Acknowledgments
  7. 1 Introduction
    1. 1.1 Motivation
    2. 1.2 Federated Learning as a Solution
      1. 1.2.1 The Definition of Federated Learning
      2. 1.2.2 Categories of Federated Learning
    3. 1.3 Current Development in Federated Learning
      1. 1.3.1 Research Issues in Federated Learning
      2. 1.3.2 Open-Source Projects
      3. 1.3.3 Standardization Efforts
      4. 1.3.4 The Federated AI Ecosystem
    4. 1.4 Organization of this Book
  8. 2 Background
    1. 2.1 Privacy-Preserving Machine Learning
    2. 2.2 PPML and Secure ML
    3. 2.3 Threat and Security Models
      1. 2.3.1 Privacy Threat Models
      2. 2.3.2 Adversary and Security Models
    4. 2.4 Privacy Preservation Techniques
      1. 2.4.1 Secure Multi-Party Computation
      2. 2.4.2 Homomorphic Encryption
      3. 2.4.3 Differential Privacy
  9. 3 Distributed Machine Learning
    1. 3.1 Introduction to DML
      1. 3.1.1 The Definition of DML
      2. 3.1.2 DML Platforms
    2. 3.2 Scalability-Motivated DML
      1. 3.2.1 Large-Scale Machine Learning
      2. 3.2.2 Scalability-Oriented DML Schemes
    3. 3.3 Privacy-Motivated DML
      1. 3.3.1 Privacy-Preserving Decision Trees
      2. 3.3.2 Privacy-Preserving Techniques
      3. 3.3.3 Privacy-Preserving DML Schemes
    4. 3.4 Privacy-Preserving Gradient Descent
      1. 3.4.1 Vanilla Federated Learning
      2. 3.4.2 Privacy-Preserving Methods
    5. 3.5 Summary
  10. 4 Horizontal Federated Learning
    1. 4.1 The Definition of HFL
    2. 4.2 Architecture of HFL
      1. 4.2.1 The Client-Server Architecture
      2. 4.2.2 The Peer-to-Peer Architecture
      3. 4.2.3 Global Model Evaluation
    3. 4.3 The Federated Averaging Algorithm
      1. 4.3.1 Federated Optimization
      2. 4.3.2 The FedAvg Algorithm
      3. 4.3.3 The Secured FedAvg Algorithm
    4. 4.4 Improvement of the FedAvg Algorithm
      1. 4.4.1 Communication Efficiency
      2. 4.4.2 Client Selection
    5. 4.5 Related Works
    6. 4.6 Challenges and Outlook
  11. 5 Vertical Federated Learning
    1. 5.1 The Definition of VFL
    2. 5.2 Architecture of VFL
    3. 5.3 Algorithms of VFL
      1. 5.3.1 Secure Federated Linear Regression
      2. 5.3.2 Secure Federated Tree-Boosting
    4. 5.4 Challenges and Outlook
  12. 6 Federated Transfer Learning
    1. 6.1 Heterogeneous Federated Learning
    2. 6.2 Federated Transfer Learning
    3. 6.3 The FTL Framework
      1. 6.3.1 Additively Homomorphic Encryption
      2. 6.3.2 The FTL Training Process
      3. 6.3.3 The FTL Prediction Process
      4. 6.3.4 Security Analysis
      5. 6.3.5 Secret Sharing-Based FTL
    4. 6.4 Challenges and Outlook
  13. 7 Incentive Mechanism Design for Federated Learning
    1. 7.1 Paying for Contributions
      1. 7.1.1 Profit-Sharing Games
      2. 7.1.2 Reverse Auctions
    2. 7.2 A Fairness-Aware Profit Sharing Framework
      1. 7.2.1 Modeling Contribution
      2. 7.2.2 Modeling Cost
      3. 7.2.3 Modeling Regret
      4. 7.2.4 Modeling Temporal Regret
      5. 7.2.5 The Policy Orchestrator
      6. 7.2.6 Computing Payoff Weightage
    3. 7.3 Discussions
  14. 8 Federated Learning for Vision, Language, and Recommendation
    1. 8.1 Federated Learning for Computer Vision
      1. 8.1.1 Federated CV
      2. 8.1.2 Related Works
      3. 8.1.3 Challenges and Outlook
    2. 8.2 Federated Learning for NLP
      1. 8.2.1 Federated NLP
      2. 8.2.2 Related Works
      3. 8.2.3 Challenges and Outlook
    3. 8.3 Federated Learning for Recommendation Systems
      1. 8.3.1 Recommendation Model
      2. 8.3.2 Federated Recommendation System
      3. 8.3.3 Related Works
      4. 8.3.4 Challenges and Outlook
  15. 9 Federated Reinforcement Learning
    1. 9.1 Introduction to Reinforcement Learning
      1. 9.1.1 Policy
      2. 9.1.2 Reward
      3. 9.1.3 Value Function
      4. 9.1.4 Model of the Environment
      5. 9.1.5 RL Background Example
    2. 9.2 Reinforcement Learning Algorithms
    3. 9.3 Distributed Reinforcement Learning
      1. 9.3.1 Asynchronous Distributed Reinforcement Learning
      2. 9.3.2 Synchronous Distributed Reinforcement Learning
    4. 9.4 Federated Reinforcement Learning
      1. 9.4.1 Background and Categorization
    5. 9.5 Challenges and Outlook
  16. 10 Selected Applications
    1. 10.1 Finance
    2. 10.2 Healthcare
    3. 10.3 Education
    4. 10.4 Urban Computing and Smart City
    5. 10.5 Edge Computing and Internet of Things
    6. 10.6 Blockchain
    7. 10.7 5G Mobile Networks
  17. 11 Summary and Outlook
  18. A Legal Development on Data Protection
    1. A.1 Data Protection in the European Union
      1. A.1.1 The Terminology of GDPR
      2. A.1.2 Highlights of GDPR
      3. A.1.3 Impact of GDPR
    2. A.2 Data Protection in the USA
    3. A.3 Data Protection in China
  19. Bibliography
  20. Authors’ Biographies