Introduction to Unity ML-Agents: Understand the Interplay of Neural Networks and Simulation Space Using the Unity ML-Agents Package

Table of Contents
About the Author
About the Technical Reviewers
Acknowledgments
Introduction
Chapter 1: Introduction
    What Is Machine Learning?
        How We Use Machine Learning in the  Modern Day
            Serving Content Recommendations
            Autonomous Vehicles
            Power and Electrical Grid Management
            Vaccines and Medical Drugs
            Farming
            Security and Surveillance
            Military
            City Planning
    Prerequisites
    Conclusion
Chapter 2: History of AI and Where We Are Today
    The People Who Shaped Artificial Intelligence
        Alan Mathison Turing
        John McCarthy
        Marvin Lee Minsky
        Guido van Rossum
    Modern-Day Companies Paving the Future of AI
        Python Software Foundation
        Nvidia
        IBM
        Google
        Tesla
        OpenAI
    How AI Has Evolved in Games, from Chess to Dota 2
        So, Where Are We Now with AI in Game Development?
            GitHub Copilot
            A Neural State Machine for  Character-Scene Interactions
            BLOOM, a BigScience Initiative
    Conclusion
Chapter 3: The Future of AI and Ethical Implications
    The Future of AI
        Law and Justice
        Healthcare
        Taxes and Governance
        Life Extension and Brain-Computer Interfaces
        Entertainment
    Avoiding a Bad Future
    Bias and Why We Need Diverse Datasets
        So, What Is Bias in AI?
        Why We Need Diverse Datasets
    Discussing the Moral and Ethical Implications
    Why AI?
    Flavors of AI
    AI Road Map and Classification
        Reactive Machines
        Limited Memory
        Theory of Mind
            Artificial General Intelligence (AGI)
        Self-Aware
            Artificial Superintelligence (ASI)
    Machine Learning with Unity ML-Agents
        Reinforcement Learning
        Imitation Learning
        Neuroevolution
    Practical Use Cases for Unity ML-Agents
        Learning How to Build Machine Learning Agents
        Self-Driving Cars
        Game AI
        Robotics
        Simulated Space for Agent Training
        Training Gym for Agents
    Conclusion
Chapter 4: Dopamine for  Machines
    Dopamine
        Dopamine in Humans
        Dopamine in Animals
        Dopamine in Machines
    Training Reinforcement Learning Agents
        How and When to Reward Your ML-Agents
        A Sound Reward System Makes for Great ML-Agents
        How Reward Systems Influence Training Time
        Various Aspects of Rewarding and Punishing ML-Agents
        Team-Based Rewards
        Conclusion
Chapter 5: ML-Agents Setup
    Unity Setup
        New Project Setup
    ML-Agents Unity Package Setup
        Installing the ML-Agents Extensions Package
        Opening the Example GitHub Project
            Creating a GitHub Issue
    Python Setup
        Creating a Virtual Environment
        Installing ML-Agents and Dependencies
    Validating Our ML-Agents Installation with Samples
    Conclusion
Chapter 6: Unity ML-Agents
    ML-Agent Components
        Behavior Parameters
            Behavior Name
            Vector Observations
            Actions
            Model
            Behavior Type
            Team Id
            Use Child Sensors
            Observable Attributes
        The Decision Requester
    Learning Environments
        The Agent
            Agent Override Methods
                override void Initialize()
                override void CollectObservations(VectorSensor sensor)
                override void OnActionReceived(ActionBuffers actionBuffers)
                override void OnEpisodeBegin()
                override void Heuristic(in ActionBuffers actionsOut)
                void RequestDecision()
                void AddReward(float increment)
                void SetReward(float reward)
                void EndEpisode()
    Inputs and Outputs
        Inputs, Observations, and Sensors
            So How Do We Create Observations?
                Collecting Observations
                Using the Observable Attribute
                Creating Sensors
            Building Sensors
                GetObservationSpec
                    Vector Observation Spec
                    Visual Observation Spec
                    Variable Length Observation Spec
                Write
                    ObservationWriter
                        Writer[index] = observation
                        AddList(IList<Single>, Int32)
                        Add(Vector3, Int32)
                        Add(Quaternion, Int32)
                        Add(Vector4, Int32)
                GetCompressedObservation
                Update
                Reset
                GetCompressionSpec
                GetName
            Visual Observations
    Actions
        Continuous
        Discrete
    Heuristics
    Rewards
    Training an Agent
    Conclusion
Chapter 7: Creating Your First AI in Unity
    Planning an Agent
    The Avoidance Sample
        Reward Scheme
        Observation Plans
        Actions Planning
        Expected Challenges
    Building Your First ML-Agent
        The Grid Sensor
        The Ray Perception Sensor
    Building the Environment
    Understanding Hyperparameters
    Training Your Agent
        Duplicating Your Training Zones
        TensorBoard and Why It’s Essential for Training
        Connecting Stand-Alone Builds to Python
        Exporting and Loading Your Model
    Conclusion
Chapter 8: Solve a Challenge with AI
    The Challenge
        Grazer Agents
        Predator Agents
        Bonus Objective
        Before You Start
    Other Techniques to Consider
        CL (Curriculum Learning)
        BC (Behavioral Cloning)
        Self-Play
    Tips
    Conclusion
Chapter 9: Next Steps
    Explore
        Additional ML-Agent Functionality
        Documentation
            Additional Reading
    Conclusion
Index