Advanced Analytics with PySpark: Patterns for Learning from Data at Scale Using Python and Spark

1. Analyzing Big Data
    Working with Big Data
    Introducing Apache Spark and PySpark
        Components
        PySpark
        Ecosystem
    Spark 3.0
    PySpark Addresses Challenges of Data Science
    Where to go from here
2. Introduction to Data Analysis with PySpark
    Spark Architecture
    Installing PySpark
    Setting up our data
    Analyzing Data with the DataFrame API
    Fast Summary Statistics for DataFrames
    Pivoting and Reshaping DataFrames
    Joining DataFrames and Selecting Features
    Scoring And Model Evaluation
    Where to Go from Here
3. Recommending Music and the Audioscrobbler Data Set
    Setting up the Data
    Our requirements from a recommender system
        Alternating Least Squares algorithm
    Preparing the Data
    Building a First Model
    Spot Checking Recommendations
    Evaluating Recommendation Quality
    Computing AUC
    Hyperparameter Selection
    Making Recommendations
    Where to Go from Here
4. Predicting Forest Cover with Decision Trees
    Decision Trees and Forests
    Preparing the Data
    Our First Decision Tree
    Decision Tree Hyperparameters
    Tuning Decision Trees
    Categorical Features Revisited
    Random Forests
    Making Predictions
    Where to Go from Here
5. Anomaly Detection in Network Traffic with K-means Clustering
    Anomaly Detection
    K-means Clustering
    Network Intrusion
    KDD Cup 1999 Data Set
    A First Take on Clustering
    Choosing k
    Visualization with SparkR
    Feature Normalization
    Categorical Variables
    Using Labels with Entropy
    Clustering in Action
    Where to Go from Here
6. Geospatial and Temporal Data Analysis on New York City Taxi Trip Data
    Preparing the data
        Converting datetime strings to timestamps
        Handling Invalid Records
    Geospatial Analysis
        Intro to GeoJSON
        Geopandas
    Sessionization in PySpark
        Building Sessions: Secondary Sorts in PySpark
    Where to Go from Here
7. Estimating Financial Risk
    Terminology
    Methods for Calculating VaR
        Variance-Covariance
        Historical Simulation
        Monte Carlo Simulation
    Our Model
    Getting the Data
    Preprocessing
    Determining the Factor Weights
    Sampling
        The Multivariate Normal Distribution
    Running the Trials
    Visualizing the Distribution of Returns
    Evaluating Our Results
    Where to Go from Here