Practical Simulations for Machine Learning: Using Synthetic Data for AI
Simulation and synthesis are core parts of the future of AI and machine learning. Consider: programmers, data scientists, and machine learning engineers can create the brain of a self-driving car without the car. Rather than use information from the real world, you can create artificial data using simulations to train traditional machine learning models. That’s just the beginning.
With this practical book, you’ll explore the possibilities of simulation- and synthesis-based machine learning and AI, with a focus on deep reinforcement learning and imitation learning techniques. AI and ML are increasingly data driven, and simulations are a powerful, engaging way to unlock their full potential.
With this deeply practical book, you’ll learn how to:
- Design an approach for solving ML and AI problems using simulations
- Use a game engine to synthesize images for use as training data
- Create simulation environments designed for training deep reinforcement learning and imitation learning
- Use and apply efficient general-purpose algorithms for simulation-based ML, such as proximal policy optimization (PPO) and soft actor-critic (SAO)
- Train ML models locally, concurrently, and in the cloud
- Use PyTorch, TensorFlow, the Unity ML-Agents and Perception Toolkits to enable ML tools to work with industry-standard game development tools
Preface Resources Used in This Book Audience and Approach Organization of This Book Using This Book Our Tasks Conventions Used in This Book Using Code Examples O’Reilly Online Learning How to Contact Us Acknowledgments I. The Basics of Simulation and Synthesis 1. Introducing Synthesis and Simulation A Whole New World of ML The Domains Simulation Synthesis The Tools Unity PyTorch via Unity ML-Agents Unity ML-Agents Toolkit Unity Perception The Techniques Reinforcement Learning Imitation Learning Hybrid Learning Summary of Techniques Projects Simulation Projects Synthesis Projects Summary and Next Steps 2. Creating Your First Simulation Everybody Remembers Their First Simulation Our Simulation Setting Up Creating the Unity Project Packages All the Way Down The Environment The Floor The Target The Agent Starting and Stopping the Agent Letting the Agent Observe the Environment Letting the Agent Take Actions in the Environment Giving the Agent Rewards for Its Behavior Finishing Touches for the Agent Providing a Manual Control System for the Agent Training with the Simulation Monitoring the Training with TensorBoard When the Training Is Complete What’s It All Mean? Coming Up Next 3. Creating Your First Synthesized Data Unity Perception The Process Using Unity Perception Creating the Unity Project Creating a Scene Getting the Dice Models A Very Simple Scene Preparing for Synthesis Testing the Scenario Setting Up Our Labels Checking the Labels What’s Next? II. Simulating Worlds for Fun and Profit 4. Creating a More Advanced Simulation Setting Up the Block Pusher Creating the Unity Project The Environment The Floor The Walls The Block The Goal The Agent The Environment Training and Testing 5. Creating a Self-Driving Car Creating the Environment The Track The Car Setting Up for ML Training the Simulation Training When the Training Is Complete 6. Introducing Imitation Learning Simulation Environment Creating the Ground Creating the Goal The Name’s Ball, Agent Ball The Camera Building the Simulation Agent Components Adding Heuristic Controls Observations and Goals Generating Data and Training Creating Training Data Configuring for Training Begin Training Running with Our Trained Model Understanding and Using Imitation Learning 7. Advanced Imitation Learning Meet GAIL Do What I Say and Do A GAIL Scenario Modifying the Agent’s Actions Modifying the Observations Resetting the Agent Updating the Agent Properties Demonstration Time Training with GAIL Running It and Beyond 8. Introducing Curriculum Learning Curriculum Learning in ML A Curriculum Learning Scenario Building in Unity Creating the Ground Creating the Target The Agent Building the Simulation Making the Agent an Agent Actions Observations Heuristic Controls for Humans Creating the Curriculum Resetting the Environment Curriculum Config Training Running It Curriculum Versus Other Approaches What’s Next? 9. Cooperative Learning A Simulation for Cooperation Building the Environment in Unity Coding the Agents Coding the Environment Manager Coding the Blocks Finalizing the Environment and Agents Training for Cooperation Cooperative Agents or One Big Agent 10. Using Cameras in Simulations Observations and Camera Sensors Building a Camera-Only Agent Coding the Camera-Only Agent Adding a New Camera for the Agent Seeing What the Agent’s Camera Sees Training the Camera-Based Agent Cameras and You 11. Working with Python Python All the Way Down Experimenting with an Environment What Can Be Done with Python? Using Your Own Environment Completely Custom Training What’s the Point of Python? 12. Under the Hood and Beyond Hyperparameters (and Just Parameters) Parameters Reward Parameters Hyperparameters Algorithms Unity Inference Engine and Integrations Using the ML-Agents Gym Wrapper Side Channels III. Synthetic Data, Real Results 13. Creating More Advanced Synthesized Data Adding Random Elements to the Scene Randomizing the Floor Color Randomizing the Camera Position What’s Next? 14. Synthetic Shopping Creating the Unity Environment A Perception Camera Faking It Until You Make It Using Synthesized Data Index About the Authors
How to download source code?
1. Go to:
2. Search the book title:
Practical Simulations for Machine Learning: Using Synthetic Data for AI, sometime you may not get the results, please search the main title
3. Click the book title in the search results
Publisher resources section, click
Download Example Code.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.