Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, 2nd Edition

Cover
Half-title
Endorsements
Title page
Copyright information
Contents
Preface
Acknowledgments
Common Optimization Techniques, Equations, Symbols, and Acronyms
Part I Dimensionality Reduction and Transforms
	1 Singular Value Decomposition (SVD)
		1.1 Overview
		1.2 Matrix Approximation
		1.3 Mathematical Properties and Manipulations
		1.4 Pseudo-Inverse, Least-Squares, and Regression
		1.5 Principal Component Analysis (PCA)
		1.6 Eigenfaces Example
		1.7 Truncation and Alignment
		1.8 Randomized Singular Value Decomposition
			Randomized Linear Algebra
			Randomized SVD Algorithm
				Oversampling
				Power Iterations
				Guaranteed Error Bounds
				Choice of Random Matrix P
			Example of Randomized SVD
		1.9 Tensor Decompositions and N-Way Data Arrays
	2 Fourier and Wavelet Transforms
		2.1 Fourier Series and Fourier Transforms
		2.2 Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT)
		2.3 Transforming Partial Differential Equations
		2.4 Gabor Transform and the Spectrogram
		2.5 Laplace Transform
		2.6 Wavelets and Multi-Resolution Analysis
		2.7 Two-Dimensional Transforms and Image Processing
	3 Sparsity and Compressed Sensing
		3.1 Sparsity and Compression
		3.2 Compressed Sensing
		3.3 Compressed Sensing Examples
		3.4 The Geometry of Compression
		3.5 Sparse Regression
		3.6 Sparse Representation
		3.7 Robust Principal Component Analysis (RPCA)
		3.8 Sparse Sensor Placement
Part II Machine Learning and Data Analysis
	4 Regression and Model Selection
		4.1 Classic Curve Fitting
		4.2 Nonlinear Regression and Gradient Descent
		4.3 Regression and Ax = b: Over- and Under-Determined Systems
		4.4 Optimization as the Cornerstone of Regression
		4.5 The Pareto Front and Lex Parsimoniae
		4.6 Model Selection: Cross-Validation
		4.7 Model Selection: Information Criteria
	5 Clustering and Classification
		5.1 Feature Selection and Data Mining
		5.2 Supervised versus Unsupervised Learning
		5.3 Unsupervised Learning: k-Means Clustering
		5.4 Unsupervised Hierarchical Clustering: Dendrogram
		5.5 Mixture Models and the Expectation-Maximization Algorithm
		5.6 Supervised Learning and Linear Discriminants
		5.7 Support Vector Machines (SVM)
		5.8 Classification Trees and Random Forest
		5.9 Top 10 Algorithms of Data Mining circa 2008 (Before the Deep Learning Revolution)
	6 Neural Networks and Deep Learning
		6.1 Neural Networks: Single-Layer Networks
		6.2 Multi-Layer Networks and Activation Functions
		6.3 The Backpropagation Algorithm
		6.4 The Stochastic Gradient Descent Algorithm
		6.5 Deep Convolutional Neural Networks
		6.6 Neural Networks for Dynamical Systems
		6.7 Recurrent Neural Networks
		6.8 Autoencoders
		6.9 Generative Adversarial Networks (GANs)
		6.10 The Diversity of Neural Networks
Part III Dynamics and Control
	7 Data-Driven Dynamical Systems
		7.1 Overview, Motivations, and Challenges
		7.2 Dynamic Mode Decomposition (DMD)
		7.3 Sparse Identification of Nonlinear Dynamics (SINDy)
		7.4 Koopman Operator Theory
		7.5 Data-Driven Koopman Analysis
	8 Linear Control Theory
		8.1 Closed-Loop Feedback Control
		8.2 Linear Time-Invariant Systems
		8.3 Controllability and Observability
		8.4 Optimal Full-State Control: Linear–Quadratic Regulator (LQR)
		8.5 Optimal Full-State Estimation: the Kalman Filter
		8.6 Optimal Sensor-Based Control: Linear–Quadratic Gaussian (LQG)
		8.7 Case Study: Inverted Pendulum on a Cart
		8.8 Robust Control and Frequency-Domain Techniques
	9 Balanced Models for Control
		9.1 Model Reduction and System Identification
		9.2 Balanced Model Reduction
		9.3 System Identification
Part IV Advanced Data-Driven Modeling and Control
	10 Data-Driven Control
		10.1 Model Predictive Control (MPC)
		10.2 Nonlinear System Identification for Control
		10.3 Machine Learning Control
		10.4 Adaptive Extremum-Seeking Control
	11 Reinforcement Learning
		11.1 Overview and Mathematical Formulation
		11.2 Model-Based Optimization and Control
		11.3 Model-Free Reinforcement Learning and Q-Learning
		11.4 Deep Reinforcement Learning
		11.5 Applications and Environments
		11.6 Optimal Nonlinear Control
	12 Reduced-Order Models (ROMs)
		12.1 Proper Orthogonal Decomposition (POD) for Partial Differential Equations
		12.2 Optimal Basis Elements: the POD Expansion
		12.3 POD and Soliton Dynamics
		12.4 Continuous Formulation of POD
		12.5 POD with Symmetries: Rotations and Translations
		12.6 Neural Networks for Time-Stepping with POD
		12.7 Leveraging DMD and SINDy for Galerkin–POD
	13 Interpolation for Parametric Reduced-Order Models
		13.1 Gappy POD
		13.2 Error and Convergence of Gappy POD
		13.3 Gappy Measurements: Minimize Condition Number
		13.4 Gappy Measurements: Maximal Variance
		13.5 POD and the Discrete Empirical Interpolation Method (DEIM)
		13.6 DEIM Algorithm Implementation
		13.7 Decoder Networks for Interpolation
		13.8 Randomization and Compression for ROMs
		13.9 Machine Learning ROMs
	14 Physics-Informed Machine Learning
		14.1 Mathematical Foundations
		14.2 SINDy Autoencoder: Coordinates and Dynamics
		14.3 Koopman Forecasting
		14.4 Learning Nonlinear Operators
		14.5 Physics-Informed Neural Networks (PINNs)
		14.6 Learning Coarse-Graining for PDEs
		14.7 Deep Learning and Boundary Value Problems
Glossary
References
Index