Algorithms for Optimization
- Length: 520 pages
- Edition: I
- Language: English
- Publisher: The MIT Press
- Publication Date: 2019-03-12
- ISBN-10: 0262039427
- ISBN-13: 9780262039420
- Sales Rank: #83996 (See Top 100 Books)
A comprehensive introduction to optimization with a focus on practical algorithms for the design of engineering systems.
This book offers a comprehensive introduction to optimization with a focus on practical algorithms. The book approaches optimization from an engineering perspective, where the objective is to design a system that optimizes a set of metrics subject to constraints. Readers will learn about computational approaches for a range of challenges, including searching high-dimensional spaces, handling problems where there are multiple competing objectives, and accommodating uncertainty in the metrics. Figures, examples, and exercises convey the intuition behind the mathematical approaches. The text provides concrete implementations in the Julia programming language.
Topics covered include derivatives and their generalization to multiple dimensions; local descent and first- and second-order methods that inform local descent; stochastic methods, which introduce randomness into the optimization process; linear constrained optimization, when both the objective function and the constraints are linear; surrogate models, probabilistic surrogate models, and using probabilistic surrogate models to guide optimization; optimization under uncertainty; uncertainty propagation; expression optimization; and multidisciplinary design optimization. Appendixes offer an introduction to the Julia language, test functions for evaluating algorithm performance, and mathematical concepts used in the derivation and analysis of the optimization methods discussed in the text. The book can be used by advanced undergraduates and graduate students in mathematics, statistics, computer science, any engineering field, (including electrical engineering and aerospace engineering), and operations research, and as a reference for professionals.
Cover Title Page Copyright Page Dedication Table of Contents Preface Acknowledgments 1. Introduction 1.1. A History 1.2. Optimization Process 1.3. Basic Optimization Problem 1.4. Constraints 1.5. Critical Points 1.6. Conditions for Local Minima 1.7. Contour Plots 1.8. Overview 1.9. Summary 1.10. Exercises 2. Derivatives and Gradients 2.1. Derivatives 2.2. Derivatives in Multiple Dimensions 2.3. Numerical Differentiation 2.4. Automatic Differentiation 2.5. Summary 2.6. Exercises 3. Bracketing 3.1. Unimodality 3.2. Finding an Initial Bracket 3.3. Fibonacci Search 3.4. Golden Section Search 3.5. Quadratic Fit Search 3.6. Shubert-Piyavskii Method 3.7. Bisection Method 3.8. Summary 3.9. Exercises 4. Local Descent 4.1. Descent Direction Iteration 4.2. Line Search 4.3. Approximate Line Search 4.4. Trust Region Methods 4.5. Termination Conditions 4.6. Summary 4.7. Exercises 5. First-Order Methods 5.1. Gradient Descent 5.2. Conjugate Gradient 5.3. Momentum 5.4. Nesterov Momentum 5.5. Adagrad 5.6. RMSProp 5.7. Adadelta 5.8. Adam 5.9. Hypergradient Descent 5.10. Summary 5.11. Exercises 6. Second-Order Methods 6.1. Newton’s Method 6.2. Secant Method 6.3. Quasi-Newton Methods 6.4. Summary 6.5. Exercises 7. Direct Methods 7.1. Cyclic Coordinate Search 7.2. Powell’s Method 7.3. Hooke-Jeeves 7.4. Generalized Pattern Search 7.5. Nelder-Mead Simplex Method 7.6. Divided Rectangles 7.7. Summary 7.8. Exercises 8. Stochastic Methods 8.1. Noisy Descent 8.2. Mesh Adaptive Direct Search 8.3. Simulated Annealing 8.4. Cross-Entropy Method 8.5. Natural Evolution Strategies 8.6. Covariance Matrix Adaptation 8.7. Summary 8.8. Exercises 9. Population Methods 9.1. Initialization 9.2. Genetic Algorithms 9.3. Differential Evolution 9.4. Particle Swarm Optimization 9.5. Firefly Algorithm 9.6. Cuckoo Search 9.7. Hybrid Methods 9.8. Summary 9.9. Exercises 10. Constraints 10.1. Constrained Optimization 10.2. Constraint Types 10.3. Transformations to Remove Constraints 10.4. Lagrange Multipliers 10.5. Inequality Constraints 10.6. Duality 10.7. Penalty Methods 10.8. Augmented Lagrange Method 10.9. Interior Point Methods 10.10. Summary 10.11. Exercises 11. Linear Constrained Optimization 11.1. Problem Formulation 11.2. Simplex Algorithm 11.3. Dual Certificates 11.4. Summary 11.5. Exercises 12. Multiobjective Optimization 12.1. Pareto Optimality 12.2. Constraint Methods 12.3. Weight Methods 12.4. Multiobjective Population Methods 12.5. Preference Elicitation 12.6. Summary 12.7. Exercises 13. Sampling Plans 13.1. Full Factorial 13.2. Random Sampling 13.3. Uniform Projection Plans 13.4. Stratified Sampling 13.5. Space-Filling Metrics 13.6. Space-Filling Subsets 13.7. Quasi-Random Sequences 13.8. Summary 13.9. Exercises 14. Surrogate Models 14.1. Fitting Surrogate Models 14.2. Linear Models 14.3. Basis Functions 14.4. Fitting Noisy Objective Functions 14.5. Model Selection 14.6. Summary 14.7. Exercises 15. Probabilistic Surrogate Models 15.1. Gaussian Distribution 15.2. Gaussian Processes 15.3. Prediction 15.4. Gradient Measurements 15.5. Noisy Measurements 15.6. Fitting Gaussian Processes 15.7. Summary 15.8. Exercises 16. Surrogate Optimization 16.1. Prediction-Based Exploration 16.2. Error-Based Exploration 16.3. Lower Confidence Bound Exploration 16.4. Probability of Improvement Exploration 16.5. Expected Improvement Exploration 16.6. Safe Optimization 16.7. Summary 16.8. Exercises 17. Optimization under Uncertainty 17.1. Uncertainty 17.2. Set-Based Uncertainty 17.3. Probabilistic Uncertainty 17.4. Summary 17.5. Exercises 18. Uncertainty Propagation 18.1. Sampling Methods 18.2. Taylor Approximation 18.3. Polynomial Chaos 18.4. Bayesian Monte Carlo 18.5. Summary 18.6. Exercises 19. Discrete Optimization 19.1. Integer Programs 19.2. Rounding 19.3. Cutting Planes 19.4. Branch and Bound 19.5. Dynamic Programming 19.6. Ant Colony Optimization 19.7. Summary 19.8. Exercises 20. Expression Optimization 20.1. Grammars 20.2. Genetic Programming 20.3. Grammatical Evolution 20.4. Probabilistic Grammars 20.5. Probabilistic Prototype Trees 20.6. Summary 20.7. Exercises 21. Multidisciplinary Optimization 21.1. Disciplinary Analyses 21.2. Interdisciplinary Compatibility 21.3. Architectures 21.4. Multidisciplinary Design Feasible 21.5. Sequential Optimization 21.6. Individual Discipline Feasible 21.7. Collaborative Optimization 21.8. Simultaneous Analysis and Design 21.9. Summary 21.10. Exercises A. Julia A.1. Types A.2. Functions A.3. Control Flow A.4. Packages B. Test Functions B.1. Ackley’s Function B.2. Booth’s Function B.3. Branin Function B.4. Flower Function B.5. Michalewicz Function B.6. Rosenbrock’s Banana Function B.7. Wheeler’s Ridge B.8. Circle Function C. Mathematical Concepts C.1. Asymptotic Notation C.2. Taylor Expansion C.3. Convexity C.4. Norms C.5. Matrix Calculus C.6. Positive Definiteness C.7. Gaussian Distribution C.8. Gaussian Quadrature D. Solutions Bibliography Index
Donate to keep this site alive
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.