Data Governance: The Definitive Guide: People, Processes, and Tools to Operationalize Data Trustworthiness

Preface
    Why Your Business Needs Data Governance in the Cloud
    Framework and Best Practices for Data Governance in the Cloud
        Data Governance Framework
        Operationalizing Data Governance in Your Organization
        The Business Benefits of Robust Data Governance
    Who Is This Book For?
    Conventions Used in This Book
    O’Reilly Online Learning
    How to Contact Us
    Acknowledgments
1. What Is Data Governance?
    What Data Governance Involves
        Holistic Approach to Data Governance
        Enhancing Trust in Data
        Classification and Access Control
        Data Governance Versus Data Enablement and Data Security
    Why Data Governance Is Becoming More Important
        The Size of Data Is Growing
        The Number of People Working and/or Viewing the Data Has Grown Exponentially
        Methods of Data Collection Have Advanced
        More Kinds of Data (Including More Sensitive Data) Are Now Being Collected
        The Use Cases for Data Have Expanded
        New Regulations and Laws Around the Treatment of Data
        Ethical Concerns Around the Use of Data
    Examples of Data Governance in Action
        Managing Discoverability, Security, and Accountability
        Improving Data Quality
    The Business Value of Data Governance
        Fostering Innovation
        The Tension Between Data Governance and Democratizing Data Analysis
        Manage Risk (Theft, Misuse, Data Corruption)
        Regulatory Compliance
            Regulation around fine-grained access control
            Data retention and data deletion
            Audit logging
            Sensitive data classes
        Considerations for Organizations as They Think About Data Governance
            Changing regulations and compliance needs
            Data accumulation and organization growth
            Moving data to the cloud
            Data infrastructure expertise
    Why Data Governance Is Easier in the Public Cloud
        Location
        Reduced Surface Area
        Ephemeral Compute
        Serverless and Powerful
        Labeled Resources
        Security in a Hybrid World
    Summary
2. Ingredients of Data Governance: Tools
    The Enterprise Dictionary
        Data Classes
        Data Classes and Policies
            Per-Use-Case Data Policies
        Data Classification and Organization
        Data Cataloging and Metadata Management
        Data Assessment and Profiling
        Data Quality
        Lineage Tracking
        Key Management and Encryption
            A Sample Key Management Scenario
        Data Retention and Data Deletion
        Workflow Management for Data Acquisition
        IAM—Identity and Access Management
        User Authorization and Access Management
    Summary
3. Ingredients of Data Governance: People and Processes
    The People: Roles, Responsibilities, and Hats
        User Hats Defined
            Legal (ancillary)
            Privacy tsar (governor)
                Privacy tsar, work example 1: Community mobility reports
                Privacy tsar, work example 2: Exposure notifications
            Data owner (approver/governor)
            Data steward (governor)
            Data analyst/data scientist (user)
            Business analyst (user)
            Customer support specialists (user/ancillary)
            C-suite (ancillary)
            External auditor (ancillary)
        Data Enrichment and Its Importance
    The Process: Diverse Companies, Diverse Needs and Approaches to Data Governance
        Legacy
        Cloud Native/Digital Only
        Retail
        Highly Regulated
        Small Companies
        Large Companies
    People and Process Together: Considerations, Issues, and Some Successful Strategies
        Considerations and Issues
            “Hats” versus “roles” and company structure
            Tribal knowledge and subject matter experts (SMEs)
            Definition of data
            Old access methods
            Regulation compliance
        Processes and Strategies with Varying Success
            Data segregation within storage systems
            Data segregation and ownership by line of business
            Creation of “views” of datasets
            A culture of privacy and security
    Summary
4. Data Governance over a Data Life Cycle
    What Is a Data Life Cycle?
    Phases of a Data Life Cycle
        Data Creation
        Data Processing
        Data Storage
        Data Usage
        Data Archiving
        Data Destruction
    Data Life Cycle Management
        Data Management Plan
            Guidance 1: Identify the data to be captured or collected
            Guidance 2: Define how the data will be organized
            Guidance 3: Document a data storage and preservation strategy
            Guidance 4: Define data policies
            Guidance 5: Define roles and responsibilities
    Applying Governance over the Data Life Cycle
        Data Governance Framework
        Data Governance in Practice
            Data creation
            Data processing
            Data storage
            Data usage
            Data archiving
            Data destruction
        Example of How Data Moves Through a Data Platform
            Scenario
    Operationalizing Data Governance
        What Is a Data Governance Policy?
        Importance of a Data Governance Policy
        Developing a Data Governance Policy
        Data Governance Policy Structure
        Roles and Responsibilities
        Step-by-Step Guidance
        Considerations for Governance Across a Data Life Cycle
            Deployment time
            Complexity and cost
            Changing regulation environment
            Location of data
            Organizational culture
    Summary
5. Improving Data Quality
    What Is Data Quality?
    Why Is Data Quality Important?
        Data Quality in Big Data Analytics
        Data Quality in AI/ML Models
    Why Is Data Quality a Part of a Data Governance Program?
    Techniques for Data Quality
        Scorecard
        Prioritization
        Annotation
        Profiling
            Data deduplication
            Data outliers
            Lineage tracking
            Data completeness
            Merging datasets
            Dataset source quality ranking for conflict resolution
    Summary
6. Governance of Data in Flight
    Data Transformations
    Lineage
        Why Lineage Is Useful
        How to Collect Lineage
        Types of Lineage
        The Fourth Dimension
        How to Govern Data in Flight
    Policy Management, Simulation, Monitoring, Change Management
    Audit, Compliance
    Summary
7. Data Protection
    Planning Protection
        Lineage and Quality
        Level of Protection
        Classification
    Data Protection in the Cloud
        Multi-Tenancy
        Security Surface
        Virtual Machine Security
    Physical Security
        Network Security
        Security in Transit
    Data Exfiltration
        Virtual Private Cloud Service Controls (VPC-SC)
        Secure Code
        Zero-Trust Model
    Identity and Access Management
        Authentication
        Authorization
        Policies
        Data Loss Prevention
        Encryption
        Differential Privacy
        Access Transparency
    Keeping Data Protection Agile
        Security Health Analytics
        Data Lineage
        Event Threat Detection
    Data Protection Best Practices
        Separated Network Designs
        Physical Security
        Portable Device Encryption and Policy
        Data Deletion Process
            Electronic medical device and OS software upgrades
            Data breach readiness
    Summary
8. Monitoring
    What Is Monitoring?
    Why Perform Monitoring?
    What Should You Monitor?
        Data Quality Monitoring
            Process and tools for monitoring data quality
        Data Lineage Monitoring
            Process and tools for monitoring data lineage
        Compliance Monitoring
            Process and tools for monitoring compliance
        Program Performance Monitoring
            Process and tools for monitoring program performance
        Security Monitoring
            Process and tools for monitoring security
    What Is a Monitoring System?
        Analysis in Real Time
        System Alerts
        Notifications
        Reporting/Analytics
        Graphic Visualization
        Customization
    Monitoring Criteria
    Important Reminders for Monitoring
    Summary
9. Building a Culture of Data Privacy and Security
    Data Culture: What It Is and Why It’s Important
    Starting at the Top—Benefits of Data Governance to the Business
        Analytics and the Bottom Line
        Company Persona and Perception
    Intention, Training, and Communications
        A Data Culture Needs to Be Intentional
            What’s important
        Training: Who Needs to Know What
            The “who,” the “how,” and the “knowledge”
            Communication
            Top-down, bottom-up, and everything in between
    Beyond Data Literacy
        Motivation and Its Cascading Effects
            Motivation and adoption
    Maintaining Agility
        Requirements, Regulations, and Compliance
        The Importance of Data Structure
        Scaling the Governance Process Up and Down
    Interplay with Legal and Security
        Staying on Top of Regulations
        Communication
        Interplay in Action
        Agility Is Still Key
    Incident Handling
        When “Everyone” Is Responsible, No One Is Responsible
    Importance of Transparency
        What It Means to Be Transparent
        Building Internal Trust
        Building External Trust
        Setting an Example
    Summary
A. Google’s Internal Data Governance
    The Business Case for Google’s Data Governance
    The Scale of Google’s Data Governance
    Google’s Governance Process
    How Does Google Handle Data?
        Privacy Safe—ADH as a Case Study
B. Additional Resources
Index