Federated Learning with Python: Design and implement a federated learning system and develop applications using existing frameworks
- Length: 326 pages
- Edition: 1
- Language: English
- Publisher: Packt Publishing
- Publication Date: 2022-10-28
- ISBN-10: 180324710X
- ISBN-13: 9781803247106
- Sales Rank: #338235 (See Top 100 Books)
Learn the essential skills for building an authentic federated learning system with Python and take your machine learning applications to the next level
Key Features
- Design distributed systems that can be applied to real-world federated learning applications at scale
- Discover multiple aggregation schemes applicable to various ML settings and applications
- Develop a federated learning system that can be tested in distributed machine learning settings
Book Description
Federated learning (FL) is a paradigm-shifting technology in AI that enables and accelerates machine learning (ML), allowing you to work on private data. It has become a must-have solution for most enterprise industries, making it a critical part of your learning journey. This book helps you get to grips with the building blocks of FL and how the systems work and interact with each other using solid coding examples.
FL is more than just aggregating collected ML models and bringing them back to the distributed agents. This book teaches you about all the essential basics of FL and shows you how to design distributed systems and learning mechanisms carefully so as to synchronize the dispersed learning processes and synthesize the locally trained ML models in a consistent manner. This way, you’ll be able to create a sustainable and resilient FL system that can constantly function in real-world operations. This book goes further than simply outlining FL’s conceptual framework or theory, as is the case with the majority of research-related literature.
By the end of this book, you’ll have an in-depth understanding of the FL system design and implementation basics and be able to create an FL system and applications that can be deployed to various local and cloud environments.
What you will learn
- Discover the challenges related to centralized big data ML that we currently face along with their solutions
- Understand the theoretical and conceptual basics of FL
- Acquire design and architecting skills to build an FL system
- Explore the actual implementation of FL servers and clients
- Find out how to integrate FL into your own ML application
- Understand various aggregation mechanisms for diverse ML scenarios
- Discover popular use cases and future trends in FL
Who this book is for
This book is for machine learning engineers, data scientists, and artificial intelligence (AI) enthusiasts who want to learn about creating machine learning applications empowered by federated learning. You’ll need basic knowledge of Python programming and machine learning concepts to get started with this book.
Federated Learning with Python Acknowledgments Contributors About the authors About the reviewer Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Share Your Thoughts Download a free PDF copy of this book Part 1 Federated Learning – Conceptual Foundations Chapter 1: Challenges in Big Data and Traditional AI Understanding the nature of big data Definition of big data Big data now Triple-A mindset for big data Data privacy as a bottleneck Risks in handling private data Increased data protection regulations From privacy by design to data minimalism Impacts of training data and model bias Expensive training of big data Model bias and training data Model drift and performance degradation How models can stop working Continuous monitoring – the price of letting causation go FL as the main solution for data problems Summary Further reading Chapter 2: What Is Federated Learning? Understanding the current state of ML What is a model? ML – automating the model creation process Deep learning Distributed learning nature – toward scalable AI Distributed computing Distributed ML Edge inference Edge training Understanding FL Defining FL The FL process FL system considerations Security for FL systems Decentralized FL and blockchain Summary Further reading Chapter 3: Workings of the Federated Learning System FL system architecture Cluster aggregators Distributed agents Database servers Intermediate servers for low computational agent devices Understanding the FL system flow – from initialization to continuous operation Initialization of the database, aggregator, and agent Initial model upload process by initial agent Overall FL cycle and process of the FL system Synchronous and asynchronous FL The aggregator-side FL cycle and process The agent-side local retraining cycle and process Model interpretation based on deviation from baseline outputs Basics of model aggregation What exactly does it mean to aggregate models? FedAvg – Federated averaging Furthering scalability with horizontal design Horizontal design with semi-global model Distributed database Asynchronous agent participation in a multiple-aggregator scenario Semi-global model synthesis Summary Further reading Part 2 The Design and Implementation of the Federated Learning System Chapter 4: Federated Learning Server Implementation with Python Technical requirements Main software components of the aggregator and database Aggregator-side codes lib/util codes Database-side code Toward the configuration of the aggregator Implementing FL server-side functionalities Importing libraries for the FL server Defining the FL Server class Initializing the FL server Registration function of agents The server for handling messages from local agents The global model synthesis routine Functions to send the global models to the agents Functions to push the local and global models to the database Maintaining models for aggregation with the state manager Importing the libraries of the state manager Defining the state manager class Initializing the state manager Initializing a global model Checking the aggregation criteria Buffering the local models Clearing the saved models Adding agents Incrementing the FL round Aggregating local models Importing the libraries for the aggregator Defining and initializing the aggregator class Defining the aggregate_local_models function The FedAvg function Running the FL server Implementing and running the database server Toward the configuration of the database Defining the database server efining the database with SQLite Running the database server Potential enhancements to the FL server Redesigning the database Automating the registry of an initial model Performance metrics for local and global models Fine-tuned aggregation Summary Chapter 5: Federated Learning Client-Side Implementation Technical requirements An overview of FL client-side components Distributed agent-side code Configuration of an agent Implementing FL client-side main functionalities Importing libraries for an agent Defining the Client class Initializing the client Agent participation in an FL cycle Model exchange synchronization Push and polling implementation Designing FL client libraries Starting FL client core threads Saving global models Manipulating client state Sending local models to aggregator Waiting for global models from an aggregator Local ML engine integration into an FL system Importing libraries for a local ML engine Defining the ML models, training, and test functions Integration of client libraries into your local ML engine An example of integrating image classification into an FL system Integration of client libraries into the IC example Summary Chapter 6: Running the Federated Learning System and Analyzing the Results Technical requirements Configuring and running the FL system Installing the FL environment Configuring the FL system with JSON files for each component Running the database and aggregator on the FL server Running a minimal example with the FL client Data and database folders Databases with SQLite Understanding what happens when the minimal example runs Running just one minimal agent Running two minimal agents Running image classification and analyzing the results Preparing the CIFAR-10 dataset The ML model used for FL with image classification How to run the image classification example with CNN Evaluation of running the image classification with CNN Running five agents Summary Chapter 7: Model Aggregation Technical requirements Revisiting aggregation Understanding FedAvg Dataset distributions Computational power distributions Protecting against adversarial agents Modifying aggregation for non-ideal cases Handling heterogeneous computational power Adversarial agents Non-IID datasets Summary Part 3 Moving Toward the Production of Federated Learning Applications Chapter 8: Introducing Existing Federated Learning Frameworks Technical requirements TensorFlow Federated OpenFL IBM FL Flower STADLE Introduction to FL frameworks Flower TensorFlow Federated (TFF) OpenFL IBM FL STADLE PySyft Example – the federated training of an NLP model Defining the sentiment analysis model Creating the data loader Training the model Adopting an FL training approach Integrating TensorFlow Federated for SST-2 Integrating OpenFL for SST-2 Integrating IBM FL for SST-2 Integrating Flower for SST-2 Integrating STADLE for SST-2 Example – the federated training of an image classification model on non-IID data Skewing the CIFAR-10 dataset Integrating OpenFL for CIFAR-10 Integrating IBM FL for CIFAR-10 Integrating Flower for CIFAR-10 Integrating STADLE for CIFAR-10 Summary Chapter 9: Case Studies with Key Use Cases of Federated Learning Applications Applying FL to the healthcare sector Challenges in healthcare Medical imaging Drug discovery EHRs Applying FL to the financial sector Anti-Money Laundering (AML) Proposed solutions to the existing AML approach Demo of FL in the AML space Benefits of FL for risk detection systems FL meets edge computing Edge computing with IoT over 5G Edge FL example – object detection Making autonomous driving happen with FL Applying FL to robotics Moving toward the Internet of Intelligence Introducing the IoFT Understanding the role of FL in Web 3.0 Applying FL to distributed learning for big data Summary References Further reading Chapter 10: Future Trends and Developments Looking at future AI trends The limitation of centralized ML Revisiting the benefits of FL Toward distributed learning for further privacy and training efficiency Ongoing research and developments in FL Exploring various FL types and approaches Understanding enhanced distributed learning frameworks with FL Journeying on to collective intelligence Intelligence-centric era with collective intelligence Internet of Intelligence Crowdsourced learning with FL Summary Further reading Appendix: Exploring Internal Libraries Technical requirements Overview of the internal libraries for the FL system states.py communication_handler.py data_struc.py helpers.py messengers.py Enumeration classes for implementing the FL system Importing libraries to define the enumeration classes IDPrefix defining the FL system components Client state classes List of classes defining the types of ML models and messages List of state classes defining message location Understanding communication handler functionalities Importing libraries for the communication handler Functions of the communication handler Understanding the data structure handler class Importing libraries for the data structure handler The LimitedDict class Understanding helper and supporting libraries Importing libraries for helper libraries Functions of the helper library Messengers to generate communication payloads Importing libraries for messengers Functions of messengers Summary Index Why subscribe? Other Books You May Enjoy Packt is searching for authors like you Share Your Thoughts Download a free PDF copy of this book
Donate to keep this site alive
How to download source code?
1. Go to: https://github.com/PacktPublishing
2. In the Find a repository… box, search the book title: Federated Learning with Python: Design and implement a federated learning system and develop applications using existing frameworks
, sometime you may not get the results, please search the main title.
3. Click the book title in the search results.
3. Click Code to download.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.