Mastering Python: Write powerful and efficient code using the full range of Python’s capabilities, 2nd Edition
- Length: 710 pages
- Edition: 2
- Language: English
- Publisher: Packt Publishing
- Publication Date: 2022-05-20
- ISBN-10: 1800207727
- ISBN-13: 9781800207721
- Sales Rank: #1087347 (See Top 100 Books)
Use advanced features of Python to write high-quality, readable code and packages
Key Features
- Extensively updated for Python 3.10 with new chapters on design patterns, scientific programming, machine learning, and interactive Python
- Shape your scripts using key concepts like concurrency, performance optimization, asyncio, and multiprocessing
- Learn how advanced Python features fit together to produce maintainable code
Book Description
Even if you find writing Python code easy, writing code that is efficient, maintainable, and reusable is not so straightforward. Many of Python’s capabilities are underutilized even by more experienced programmers. Mastering Python, Second Edition, is an authoritative guide to understanding advanced Python programming so you can write the highest quality code. This new edition has been extensively revised and updated with exercises, four new chapters and updates up to Python 3.10.
Revisit important basics, including Pythonic style and syntax and functional programming. Avoid common mistakes made by programmers of all experience levels. Make smart decisions about the best testing and debugging tools to use, optimize your code’s performance across multiple machines and Python versions, and deploy often-forgotten Python features to your advantage. Get fully up to speed with asyncio and stretch the language even further by accessing C functions with simple Python calls. Finally, turn your new-and-improved code into packages and share them with the wider Python community.
If you are a Python programmer wanting to improve your code quality and readability, this Python book will make you confident in writing high-quality scripts and taking on bigger challenges
What you will learn
- Write beautiful Pythonic code and avoid common Python coding mistakes
- Apply the power of decorators, generators, coroutines, and metaclasses
- Use different testing systems like pytest, unittest, and doctest
- Track and optimize application performance for both memory and CPU usage
- Debug your applications with PDB, Werkzeug, and faulthandler
- Improve your performance through asyncio, multiprocessing, and distributed computing
- Explore popular libraries like Dask, NumPy, SciPy, pandas, TensorFlow, and scikit-learn
- Extend Python’s capabilities with C/C++ libraries and system calls
Who this book is for
This book will benefit more experienced Python programmers who wish to upskill, serving as a reference for best practices and some of the more intricate Python techniques. Even if you have been using Python for years, chances are that you haven’t yet encountered every topic discussed in this book. A good understanding of Python programming is necessary
Preface Who this book is for What this book covers To get the most out of this book Get in touch Getting Started – One Environment per Project Virtual environments Why virtual environments are a good idea Using venv and virtualenv Creating a venv Activating a venv/virtualenv Installing packages Using pyenv Using Anaconda Getting started with Anaconda Navigator Getting started with conda Managing dependencies Using pip and a requirements.txt file Version specifiers Installing through source control repositories Additional dependencies using extras Conditional dependencies using environment markers Automatic project management using poetry Creating a new poetry project Adding dependencies Upgrading dependencies Running commands Automatic dependency tracking using pipenv Updating your packages Deploying to production Running cron commands Exercises Reading the Python Enhancement Proposals (PEPs) Combining pyenv and poetry or pipenv Converting an existing project to a poetry project Summary Interactive Python Interpreters The Python interpreter Modifying the interpreter Enabling and enhancing autocompletion Alternative interpreters bpython Rewinding your session Reloading modules ptpython IPython and Jupyter Basic interpreter usage Saving and loading sessions Regular Python prompt/doctest mode Introspection and help Autocompletion Jupyter Installing Jupyter IPython summary Exercises Summary Pythonic Syntax and Common Pitfalls A brief history of Python Code style – What is Pythonic code? Whitespace instead of braces Formatting strings – printf, str.format, or f-strings? Simple formatting Named variables Arbitrary expressions PEP 20, the Zen of Python Beautiful is better than ugly Explicit is better than implicit Simple is better than complex Flat is better than nested Sparse is better than dense Readability counts Practicality beats purity Errors should never pass silently In the face of ambiguity, refuse the temptation to guess One obvious way to do it Hard to explain, easy to explain Namespaces are one honking great idea Explaining PEP 8 Duck typing Differences between value and identity comparisons Loops Maximum line length Verifying code quality, pep8, pyflakes, and more Recent additions to the Python syntax PEP 572: Assignment expressions/the walrus operator PEP 634: Structural pattern matching, the switch statement Common pitfalls Scope matters! Global variables Mutable function default arguments Class properties Overwriting and/or creating extra built-ins Modifying while iterating Catching and storing exceptions Late binding and closures Circular imports Import collisions Summary Pythonic Design Patterns Time complexity – The big O notation Core collections list – A mutable list of items dict – A map of items set – Like a dict without values tuple – The immutable list Pythonic patterns using advanced collections Smart data storage with type hinting using dataclasses Combining multiple scopes with ChainMap Default dictionary values using defaultdict enum – A group of constants Sorting collections using heapq Searching through sorted collections using bisect Global instances using Borg or Singleton patterns No need for getters and setters with properties Dict union operators Exercises Summary Functional Programming – Readability Versus Brevity Functional programming Purely functional Functional programming and Python Advantages of functional programming list, set, and dict comprehensions Basic list comprehensions set comprehensions dict comprehensions Comprehension pitfalls lambda functions The Y combinator functools partial – Prefill function arguments reduce – Combining pairs into a single result Implementing a factorial function Processing trees Reducing in the other direction itertools accumulate – reduce with intermediate results chain – Combining multiple results compress – Selecting items using a list of Booleans dropwhile/takewhile – Selecting items using a function count – Infinite range with decimal steps groupby – Grouping your sorted iterable Exercises Summary Decorators – Enabling Code Reuse by Decorating Decorating functions Generic function decorators The importance of functools.wraps Chaining or nesting decorators Registering functions using decorators Memoization using decorators Decorators with (optional) arguments Creating decorators using classes Decorating class functions Skipping the instance – classmethod and staticmethod Properties – Smart descriptor usage Decorating classes Singletons – Classes with a single instance Total ordering – Making classes sortable Useful decorators Single dispatch – Polymorphism in Python contextmanager — with statements made easy Validation, type checks, and conversions Useless warnings – How to ignore them safely Exercises Summary Generators and Coroutines – Infinity, One Step at a Time Generators Creating generators Creating infinite generators Generators wrapping iterables Generator comprehensions Class-based generators and iterators Generator examples Breaking an iterable up into chunks/groups itertools.islice – Slicing iterables itertools.chain – Concatenating multiple iterables itertools.tee – Using an output multiple times contextlib.contextmanager – Creating context managers Coroutines A basic example Priming Closing and throwing exceptions Mixing generators and coroutines Using the state Exercises Summary Metaclasses – Making Classes (Not Instances) Smarter Dynamically creating classes A basic metaclass Arguments to metaclasses Accessing metaclass attributes through classes Abstract classes using collections.abc Internal workings of the abstract classes Custom type checks Automatically registering plugin systems Importing plugins on-demand Importing plugins through configuration Importing plugins through the filesystem Dataclasses Order of operations when instantiating classes Finding the metaclass Preparing the namespace Executing the class body Creating the class object (not instance) Executing the class decorators Creating the class instance Example Storing class attributes in definition order The classic solution without metaclasses Using metaclasses to get a sorted namespace Exercises Summary Documentation – How to Use Sphinx and reStructuredText Type hinting Basic example Custom types Generics Type checking Python type interface files Type hinting conclusion reStructuredText and Markdown Getting started with reStructuredText Getting started with Markdown Inline markup Headers Headers with reStructuredText Headers with Markdown Lists Enumerated lists Bulleted lists Option lists Definition lists (reST only) Nested lists Links, references, and labels Images Images with reStructuredText Images with Markdown Substitutions Blocks, code, math, comments, and quotes Conclusion The Sphinx documentation generator Getting started with Sphinx Using sphinx-quickstart Using sphinx-apidoc Sphinx directives Sphinx roles Documenting code Documenting a class with the Sphinx style Documenting a class with the Google style Documenting a class with the NumPy style Which style to choose Exercises Summary Testing and Logging – Preparing for Bugs Using documentation as tests with doctest A simple doctest example Writing doctests Testing with documentation The doctest flags True and False versus 1 and 0 Normalizing whitespace Ellipsis Doctest quirks Testing dictionaries Testing floating-point numbers Times and durations Testing with py.test The difference between the unittest and py.test output The difference between unittest and py.test tests Simplifying assertions Parameterizing tests Automatic arguments using fixtures Print statements and logging Plugins Mock objects Using unittest.mock Using py.test monkeypatch Testing multiple environments with tox Getting started with tox The tox.ini config file Running tox Logging Configuration Basic logging configuration Dictionary configuration JSON configuration ini file configuration The network configuration Logger Usage Formatting Modern formatting using f-strings and str.format Logging pitfalls Debugging loggers Exercises Summary Debugging – Solving the Bugs Non-interactive debugging Inspecting your script using trace Debugging using logging Showing the call stack without exceptions Handling crashes using faulthandler Interactive debugging Console on demand Debugging using Python debugger (pdb) Breakpoints Catching exceptions Aliases commands Debugging with IPython Debugging with Jupyter Other debuggers Debugging services Exercises Summary Performance – Tracking and Reducing Your Memory and CPU Usage What is performance? Measuring CPU performance and execution time Timeit – comparing code snippet performance cProfile – Finding the slowest components First profiling run Calibrating your profiler Selective profiling using decorators Using profile statistics Line profiler – Tracking performance per line Improving execution time Using the right algorithm Global interpreter lock try versus if Lists versus generators String concatenation Addition versus generators Map versus generators and list comprehensions Caching Lazy imports Using slots Using optimized libraries Just-in-time compiling Converting parts of your code to C Memory usage tracemalloc Memory Profiler Memory leaks Circular references Analyzing memory usage using the garbage collector Weak references Weakref limitations and pitfalls Reducing memory usage Generators versus lists Recreating collections versus removing items Using slots Performance monitoring Exercises Summary asyncio – Multithreading without Threads Introduction to asyncio Backward compatibility and async/await statements Python 3.4 Python 3.5 Python 3.7 A basic example of parallel execution asyncio concepts Coroutines, Futures, and Tasks Event loops Executors Asynchronous examples Processes Interactive processes Echo client and server Asynchronous file operations Creating async generators to support async for Asynchronous constructors and destructors Debugging asyncio Forgetting to await a coroutine Slow blocking functions Forgetting to check the results or exiting early Exiting before all tasks are done Exercises Summary Multiprocessing – When a Single CPU Core Is Not Enough The Global Interpreter Lock (GIL) The use of multiple threads Why do we need the GIL? Why do we still have the GIL? Multiple threads and processes Basic examples concurrent.futures threading multiprocessing Cleanly exiting long-running threads and processes Batch processing using concurrent.futures Batch processing using multiprocessing Sharing data between threads and processes Shared memory between processes Thread safety Deadlocks Thread-local variables Processes, threads, or a single thread? threading versus concurrent.futures multiprocessing versus concurrent.futures Hyper-threading versus physical CPU cores Remote processes Distributed processing using multiprocessing Distributed processing using Dask Installing Dask Basic example Running a single thread Distributed execution across multiple machines Distributed processing using ipyparallel ipython_config.py ipython_kernel_config.py ipcontroller_config.py ipengine_config.py ipcluster_config.py Summary Scientific Python and Plotting Installing the packages Arrays and matrices NumPy – Fast arrays and matrices Numba – Faster Python on CPU or GPU SciPy – Mathematical algorithms and NumPy utilities Sparse matrices Pandas – Real-world data analysis Input and output options Pivoting and grouping Merging Rolling or expanding windows Statsmodels – Statistical models on top of Pandas xarray – Labeled arrays and datasets STUMPY – Finding patterns in time series Mathematics and precise calculations gmpy2 – Fast and precise calculations Sage – An alternative to Mathematica/Maple/MATLAB mpmath – Convenient, precise calculations SymPy – Symbolic mathematics Patsy – Describing statistical models Plotting, graphing, and charting Matplotlib Seaborn Yellowbrick Plotly Bokeh Datashader Exercises Summary Artificial Intelligence Introduction to artificial intelligence Types of AI Installing the packages Image processing scikit-image Installing scikit-image Edge detection Face detection scikit-image overview OpenCV Installing OpenCV for Python Edge detection Object detection OpenCV versus scikit-image Natural language processing NLTK – Natural Language Toolkit spaCy – Natural language processing with Cython Gensim – Topic modeling for humans Machine learning Types of machine learning Supervised learning Reinforcement learning Unsupervised learning Combinations of learning methods Deep learning Artificial neural networks and deep learning Tensors PyTorch – Fast (deep) neural networks PyTorch Lightning and PyTorch Ignite – High-level PyTorch APIs Skorch – Mixing PyTorch and scikit-learn TensorFlow/Keras – Fast (deep) neural networks TensorFlow versus PyTorch Evolutionary algorithms Support-vector machines Bayesian networks Versatile AI libraries and utilities scikit-learn – Machine learning in Python Supervised learning Unsupervised learning auto-sklearn – Automatic scikit-learn mlxtend – Machine learning extensions scikit-lego – scikit-learn utilities XGBoost – eXtreme Gradient Boosting Featuretools – Feature detection and prediction Snorkel – Improving your ML data automatically TPOT – Optimizing ML models using genetic programming Exercises Summary Extensions in C/C++, System Calls, and C/C++ Libraries Setting up tooling Do you need C/C++ modules? Windows OS X Linux/Unix Calling C/C++ with ctypes Platform-specific libraries Windows Linux/Unix OS X Making it easy Calling functions and native types Complex data structures Arrays Gotchas with memory management CFFI Complex data structures Arrays ABI or API? CFFI or ctypes? Native C/C++ extensions A basic example C is not Python – Size matters The example explained static PyObject* Parsing arguments C is not Python – Errors are silent or lethal Calling Python from C – Handling complex types Exercises Summary Packaging – Creating Your Own Libraries or Applications Introduction to packages Types of packages Wheels – The new eggs Source packages Package tools Package versioning Building packages Packaging using pyproject.toml Creating a basic package Installing packages for development Adding code and data Adding executable commands Managing dependencies Building the package Building C/C++ extensions Packaging using setuptools with setup.py or setup.cfg Creating a basic package Installing the package for development Adding packages Adding package data Managing dependencies Adding executable commands Building the package Publishing packages Adding URLs PyPI trove classifiers Uploading to PyPI C/C++ extensions Regular C/C++ extensions Cython extensions Testing unittest py.test Exercises Summary Other Books You May Enjoy Index
Donate to keep this site alive
How to download source code?
1. Go to: https://github.com/PacktPublishing
2. In the Find a repository… box, search the book title: Mastering Python: Write powerful and efficient code using the full range of Python’s capabilities, 2nd Edition
, sometime you may not get the results, please search the main title.
3. Click the book title in the search results.
3. Click Code to download.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.