AI at the Edge: Solving Real World Problems with Embedded Machine Learning
Edge artificial intelligence is transforming the way computers interact with the real world, allowing internet of things (IoT) devices to make decisions using the 99% of sensor data that was previously discarded due to cost, bandwidth, or power limitations. With techniques like embedded machine learning, developers can capture human intuition and deploy it to any target–from ultra-low power microcontrollers to flexible embedded Linux devices–for applications that reduce latency, protect privacy, and work without a network connection, greatly expanding the capabilities of the IoT.
This practical guide gives engineering professionals and product managers an end-to-end framework for solving real-world industrial, commercial, and scientific problems with edge AI. You’ll explore every stage of the process, from data collection to model optimization to tuning and testing, as you learn how to design and support edge AI and embedded ML products. Edge AI is destined to become a standard tool for systems engineers. This high-level roadmap will help you get started.
- Develop your expertise in artificial intelligence and machine learning on edge devices
- Understand which projects are best solved with edge AI
- Explore typical design patterns used with edge AI apps
- Use an iterative workflow to develop an edge AI application
- Optimize models for deployment to embedded devices
- Improve model performance based on feedback from real-world use
Foreword Preface About this Book What to Expect What You Need to Know Already Responsible, Ethical, and Effective AI Further Resources Acknowledgements Conventions Used in This Book Using Code Examples O’Reilly Online Learning How to Contact Us Dedications Acknowledgments 1. A Brief Introduction to Edge AI Defining key terms Embedded The edge (and the Internet of Things) Artificial intelligence Machine learning Edge AI Embedded machine learning and tiny machine learning Digital signal processing Why do we need edge AI? To understand the benefits of edge AI, just BLERP Bandwidth Latency Economics Reliability Privacy Using BLERP Edge AI for good Key differences between edge AI and regular AI Training on the edge is rare The focus of edge AI is on sensor data ML models can get very small Learning from feedback is limited Compute is diverse and heterogeneous “Good enough” is often the goal Tools and best practices are still evolving Summary 2. Edge AI in the Real World Common Use Cases for Edge AI Green and brown field projects Real world products Preventing forest fires using power line fault detection Protecting first responders with intelligent wearables Understanding elephant behavior with smart collars Types of applications Keeping track of objects Key benefits for object tracking Understanding and controlling systems Key benefits for understanding and controlling systems Understanding people and living things Key benefits for understanding people and living things Transforming signals Key benefits for transforming signals Building applications responsibly Responsible design and AI ethics Black boxes and bias Technology that harms, not helps The costs of negligence Violations due to negligence Deliberate ethical violations Mitigating societal harms Summary 3. The Hardware of Edge AI Sensors, signals, and sources of data Types of sensors and signals Acoustic and vibration Visual and scene Motion and position Force and tactile Optical, electromagnetic, and radiation Environmental, biological, and chemical Other signals Processors for edge AI Edge AI hardware architecture Microcontrollers and digital signal processors Low-end MCUs High-end MCUs Digital signal processors (DSPs) System on Chip Deep learning accelerators FPGAs and ASICs Edge servers Multi-device architectures Devices and workloads Summary 4. Algorithms for Edge AI Feature engineering Working with data streams Digital signal processing algorithms Resampling Filtering Spectral analysis Image feature detection Combining features and sensors Artificial intelligence algorithms Algorithm types by functionality Classification Regression Object detection and segmentation Anomaly detection Clustering Dimensionality reduction Transformation Algorithm types by implementation Conditionals and heuristics Classical machine learning Deep learning Combining algorithms Post-processing algorithms Fail-safe design Optimization for edge devices Choice of algorithm Compression and optimization On-device training Summary 5. Tools and Expertise Building a team for AI at the edge Domain expertise Diversity Stakeholders Roles and responsibilities Knowledge and understanding Planning and execution Algorithm development Product engineering Technical services Hiring for edge AI Learning edge AI skills Practice Theory Tools of the trade Software engineering Operating systems Programming and scripting languages Dependency management Containerization Distributed computing Cloud providers Working with data Data capture IoT device management Data storage and management Data pipelines Algorithm development Mathematical and scientific computing libraries Data visualization Interactive computing environments Digital signal processing Deep learning frameworks Model compression and optimization Experiment tracking Automated machine learning (AutoML) Machine learning operations (MLOps) Running algorithms on-device Math and DSP libraries Machine learning inference On-device learning Embedded software engineering and electronics Embedded hardware tools Development boards Embedded software tools Emulators and simulators Embedded Linux Automated hardware testing End-to-end platforms for edge AI Summary 6. Understanding and Framing Problems The edge AI workflow Responsible AI in the edge AI workflow Do I need edge AI? Describing a problem Do I need to deploy to the edge? Things that don’t work well on the edge Disadvantages of edge compute Do I need machine learning? Reasons to use ML The drawbacks of ML Knowing when to use ML Practical exercise Determining feasibility Moral feasibility Business feasibility Proving benefit Understanding constraints Dataset feasibility Technological feasibility Framing problems Device capabilities and solution choice Making a final decision Planning an edge AI project Defining acceptable performance Understanding time and resource constraints Summary 7. How to Build a Dataset What does a dataset look like? The ideal dataset Datasets and domain expertise Data, ethics, and responsible AI Minimizing unknowns Ensuring domain expertise Data-centric machine learning Estimating data requirements A practical workflow for estimating data requirements Getting your hands on data The unique challenges of capturing data at the edge Storing and retrieving data Getting data into stores Collecting metadata Ensuring data quality Ensuring representative datasets Reviewing data by sampling Label noise Avoiding label noise Common data errors Drift and shift The uneven distribution of errors Preparing data Labelling Not all problems require labels Semi-supervised and active learning algorithms Bias in labelling Labelling tools Annotation tools Crowdsourced labelling Assisted and automated labelling Semi-supervised and active learning Formatting Data cleaning Auditing your dataset Fixing issues Amending values Substituting values Excluding records Evaluation and automation Fixing balance issues Feature engineering Splitting your data How is data split? Pitfalls when splitting data Data augmentation Data pipelines Building a dataset over time Summary 8. Designing Edge AI Applications Product and experience design Design principles Scoping a solution Setting design goals Systemic goals Technical goals Values-based design goals Architectural design Hardware, software, and services Basic application architectures Basic flow Ensemble flow Parallel flow Series flow Cascading flow Sensor fusion flow Complex application architectures and design patterns Heterogeneous cascade Multi-device cascade Cascade to the cloud Intelligent gateway Human-in-the-loop Working with design patterns Accounting for choices in design Design deliverables Summary 9. Developing Edge AI Applications An iterative workflow for edge AI development Exploration Goal-setting Calling it quits Bootstrapping Why bootstrapping is helpful Developing a baseline algorithm Our first hardware Responsible AI review Test and iterate Feedback loops Iterations in practice Updating your plans Ethical AI review Deployment Support Summary 10. Evaluating, Deploying, and Supporting Edge AI Applications Evaluating edge AI systems Ways to evaluate a system Evaluating individual components Evaluating integrated systems Simulated real-world testing Real-world testing Quality assurance testing Usability testing Monitoring a deployed system Useful metrics Algorithmic performance Loss Accuracy Confusion matrix Precision and recall Positive and negative rates F1 score and MCC ROC & AUC Error metrics Mean average precision Computational and hardware performance Memory Floating point operations Latency Duty cycle Energy Thermal Techniques for evaluation Evaluation and responsible AI Deploying edge AI applications Pre-deployment tasks Mid-deployment tasks Post-deployment tasks Supporting edge AI applications Post-deployment monitoring Types of feedback from deployed systems Data samples Distribution changes Application metrics Outcomes User reports Improving a live application Solving problems using feedback Refining an algorithm over time Supporting multiple deployed algorithms Ethics and long term support Performance degradation New information Evolving cultural norms Changing legal standards What comes next 11. Use Case: Wildlife Monitoring Problem exploration Solution exploration Goal-setting Solution design What solutions already exist? Solution design approaches Design considerations Environmental impact Bootstrapping Define your machine learning classes Dataset gathering Edge Impulse Public project Choose your hardware & sensors Hardware configuration Data collection Connecting your device directly to Edge Impulse for data collection iNaturalist Dataset limitations Dataset licensing and legal obligations Cleaning your dataset Uploading data to Edge Impulse DSP & machine learning workflow Digital signal processing block Machine learning block Visual mode EON Tuner Testing the model Live classification Model testing Test your model locally Deployment Create library Mobile phone & computer Pre-built binary flashing Impulse runnner GitHub source code Iterate & feedback loops AI for Good Related works Datasets Research 12. Use Case: Food Quality Assurance Problem exploration Solution exploration Goal-setting Solution design What solutions already exist? Solution design approaches Design considerations Environmental & social impact Bootstrapping Define your machine learning classes Dataset gathering Edge Impulse Edge Impulse public project Choose your hardware & sensors Hardware configuration Data collection Data ingestion firmware Uploading data to Edge Impulse Cleaning your dataset Dataset licensing and legal obligations DSP & machine learning workflow Digital signal processing block Machine learning block Visual mode Testing the model Live classification Model testing Deployment Pre-built binary flashing GitHub source code Iterate & feedback loops Related works Research News & other articles 13. Use Case: Consumer Products Problem exploration Goal-setting Solution design What solutions already exist? Solution design approaches Design considerations Environmental & social impact Bootstrapping Define your machine learning classes Dataset gathering Edge Impulse Edge Impulse public project Choose your hardware & sensors Hardware configuration Data collection Data ingestion firmware Mobile phone nRF Edge Impulse mobile phone application Cleaning your dataset Dataset licensing and legal obligations DSP & machine learning workflow Digital signal processing block Machine learning blocks Visual mode Anomaly detection Testing the model Live classification Model testing Deployment Pre-built binary flashing GitHub source code Iterate & feedback loops Related works Research News & other articles
How to download source code?
1. Go to:
2. Search the book title:
AI at the Edge: Solving Real World Problems with Embedded Machine Learning, sometime you may not get the results, please search the main title
3. Click the book title in the search results
Publisher resources section, click
Download Example Code.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.