3D Deep Learning with Python: Design and develop your computer vision model with 3D data using PyTorch3D and more

by Lilit Yolyan, Vishakh Hegde, Xudong Ma

Length: 236 pages
Edition: 1
Language: English
Publisher: Packt Publishing
Publication Date: 2022-10-31
ISBN-10: 1803247827
ISBN-13: 9781803247823
Sales Rank: #442123 (See Top 100 Books)

Visualize and build deep learning models with 3D data using PyTorch3D and other Python frameworks to conquer real-world application challenges with ease

Key Features

Understand 3D data processing with rendering, PyTorch optimization, and heterogeneous batching
Implement differentiable rendering concepts with practical examples
Discover how you can ease your work with the latest 3D deep learning techniques using PyTorch3D

Book Description

With this hands-on guide to 3D deep learning, developers working with 3D computer vision will be able to put their knowledge to work and get up and running in no time.

Complete with step-by-step explanations of essential concepts and practical examples, this book lets you explore and gain a thorough understanding of state-of-the-art 3D deep learning. You’ll see how to use PyTorch3D for basic 3D mesh and point cloud data processing, including loading and saving ply and obj files, projecting 3D points into camera coordination using perspective camera models or orthographic camera models, rendering point clouds and meshes to images, and much more. As you implement some of the latest 3D deep learning algorithms, such as differential rendering, Nerf, synsin, and mesh RCNN, you’ll realize how coding for these deep learning models becomes easier using the PyTorch3D library.

By the end of this deep learning book, you’ll be ready to implement your own 3D deep learning models confidently.

What you will learn

Develop 3D computer vision models for interacting with the environment
Get to grips with 3D data handling with point clouds, meshes, ply, and obj file format
Work with 3D geometry, camera models, and coordination and convert between them
Understand concepts of rendering, shading, and more with ease
Implement differential rendering for many 3D deep learning models
Advanced state-of-the-art 3D deep learning models like Nerf, synsin, mesh RCNN

Who this book is for

This book is for beginner to intermediate-level machine learning practitioners, data scientists, ML engineers, and DL engineers who are looking to become well-versed with computer vision techniques using 3D data.

3D Deep Learning with Python
Contributors
About the author
About the reviewer
Preface
    Who this book is for
    What this book covers
    To get the most out of this book
    Download the example code files
    Download the color images
    Conventions used
    Get in touch
    Share Your Thoughts
    Download a free PDF copy of this book
PART 1: 3D Data Processing Basics
Chapter 1: Introducing 3D Data Processing
    Technical requirements
    Setting up a development environment
    3D data representation
        Understanding point cloud representation
        Understanding mesh representation
        Understanding voxel representation
    3D data file format – Ply files
    3D data file format – OBJ files
    Understanding 3D coordination systems
    Understanding camera models
    Coding for camera models and coordination systems
    Summary
Chapter 2: Introducing 3D Computer Vision and Geometry
    Technical requirements
    Exploring the basic concepts of rendering, rasterization, and shading
        Understanding barycentric coordinates
        Light source models
        Understanding the Lambertian shading model
        Understanding the Phong lighting model
    Coding exercises for 3D rendering
    Using PyTorch3D heterogeneous batches and PyTorch optimizers
        A coding exercise for a heterogeneous mini-batch
    Understanding transformations and rotations
        A coding exercise for transformation and rotation
    Summary
PART 2: 3D Deep Learning Using PyTorch3D
Chapter 3: Fitting Deformable Mesh Models to Raw Point Clouds
    Technical requirements
    Fitting meshes to point clouds – the problem
    Formulating a deformable mesh fitting problem into an optimization problem
    Loss functions for regularization
        Mesh Laplacian smoothing loss
        Mesh normal consistency loss
        Mesh edge loss
    Implementing the mesh fitting with PyTorch3D
        The experiment of not using any regularization loss functions
        The experiment of using only the mesh edge loss
    Summary
Chapter 4: Learning Object Pose Detection and Tracking by Differentiable Rendering
    Technical requirements
    Why we want to have differentiable rendering
    How to make rendering differentiable
        What problems can be solved by using differentiable rendering
    The object pose estimation problem
    How it is coded
        An example of object pose estimation for both silhouette fitting and texture fitting
    Summary
Chapter 5: Understanding Differentiable Volumetric Rendering
    Technical requirements
    Overview of volumetric rendering
    Understanding ray sampling
    Using volume sampling
    Exploring the ray marcher
    Differentiable volumetric rendering
        Reconstructing 3D models from multi-view images
    Summary
Chapter 6: Exploring Neural Radiance Fields (NeRF)
    Technical requirements
    Understanding NeRF
        What is a radiance field?
        Representing radiance fields with neural networks
    Training a NeRF model
    Understanding the NeRF model architecture
    Understanding volume rendering with radiance fields
        Projecting rays into the scene
        Accumulating the color of a ray
    Summary
PART 3: State-of-the-art 3D Deep Learning Using PyTorch3D
Chapter 7: Exploring Controllable Neural Feature Fields
    Technical requirements
    Understanding GAN-based image synthesis
    Introducing compositional 3D-aware image synthesis
    Generating feature fields
    Mapping feature fields to images
    Exploring controllable scene generation
        Exploring controllable car generation
        Exploring controllable face generation
    Training the GIRAFFE model
        Frechet Inception Distance
        Training the model
    Summary
Chapter 8: Modeling the Human Body in 3D
    Technical requirements
    Formulating the 3D modeling problem
        Defining a good representation
    Understanding the Linear Blend Skinning technique
    Understanding the SMPL model
        Defining the SMPL model
    Using the SMPL model
    Estimating 3D human pose and shape using SMPLify
        Defining the optimization objective function
    Exploring SMPLify
        Running the code
        Exploring the code
    Summary
Chapter 9: Performing End-to-End View Synthesis with SynSin
    Technical requirements
    Overview of view synthesis
    SynSin network architecture
        Spatial feature and depth networks
        Neural point cloud renderer
        Refinement module and discriminator
    Hands-on model training and testing
    Summary
Chapter 10: Mesh R-CNN
    Technical requirements
    Overview of meshes and voxels
    Mesh R-CNN architecture
        Graph convolutions
        Mesh predictor
    Demo of Mesh R-CNN with PyTorch
        Demo
    Summary
Index
    Why subscribe?
Other Books You May Enjoy
    Packt is searching for authors like you
    Share Your Thoughts
    Download a free PDF copy of this book

AI & Machine Learning Artificial Intelligence Intelligence & Semantics Machine Theory Neural Networks Python

Donate to keep this site alive

To access the Link, solve the captcha.

How to download source code?

1. Go to: https://github.com/PacktPublishing

2. In the Find a repository… box, search the book title: 3D Deep Learning with Python: Design and develop your computer vision model with 3D data using PyTorch3D and more, sometime you may not get the results, please search the main title.