Data Management at Scale: Best Practices for Enterprise Architecture
- Length: 328 pages
- Edition: 1
- Language: English
- Publisher: O'Reilly Media
- Publication Date: 2020-08-25
- ISBN-10: 149205478X
- ISBN-13: 9781492054788
- Sales Rank: #1709291 (See Top 100 Books)
As data management and integration continue to evolve rapidly, storing all your data in one place such as a data warehouse is no longer scalable. In the very near future, data will need to be distributed and available for several technological solutions. With this practical book, you’ll learn how to migrate your enterprise from a complex and tightly coupled data landscape to a more flexible architecture ready for the modern world of data consumption.
Executives, data architects, analytics teams, and compliance and governance staff will learn how to build a modern scalable data landscape using the Scaled Architecture, which you can introduce incrementally without a large upfront investment. Author Piethein Strengholt provides blueprints, principles, observations, best practices, and patterns to get you up to speed.
- Examine data management trends and difficulties, including technological developments and regulatory and privacy requirements that puzzle enterprises
- Go deep into the Scaled Architecture and learn how the pieces fit together
- Explore data governance and data security, master data management, self-service data marketplaces, and the importance of metadata
Foreword Preface Who Is This Book For? What Will I Learn? Navigating Through This Book Conventions Used in This Book O’Reilly Online Learning How to Contact Us Acknowledgments 1. The Disruption of Data Management Data Management Analytics Is Fragmenting the Data Landscape Speed of Software Delivery Is Changing Networks Are Getting Faster Privacy and Security Concerns Are a Top Priority Operational and Transactional Systems Need to Be Integrated Data Monetization Requires an Ecosystem-to-Ecosystem Architecture Enterprises Are Saddled with Outdated Data Architectures Enterprise Data Warehouse and Business Intelligence Data Lake Centralized View Summary 2. Introducing the Scaled Architecture: Organizing Data at Scale Universally Acknowledged Starting Points Each Application Has an Application Database Applications Are Specific and Have Unique Context Golden Source There’s No Escape from the Data Integration Dilemma Applications Play the Roles of Data Providers and Data Consumers Key Theoretical Considerations Object-Oriented Programming Principles Domain-Driven Design Business Architecture Communication and Integration Patterns Point-to-Point Silos Hub-Spoke Model Scaled Architecture Golden Sources and Domain Data Stores Data Delivery Contracts and Data Sharing Agreements Eliminating the Siloed Approach Domain-Driven Design on an Enterprise Scale Read-Optimized Data Data Layer as a Holistic Picture Metadata and the Target Operating Model Summary 3. Managing Vast Amounts of Data: The Read-Only Data Stores Architecture Introducing the RDS Architecture Command and Query Responsibility Segregation What Is CQRS? CQRS at Scale Read-Only Data Store Components and Services Metadata Data Quality RDS Tiers Data Ingestion Integrating Commercial Off-the-Shelf Solutions Extracting Data from External APIs and SaaSs Historical Data Service Design Variations Data Replication Access Layer File Manipulation Service Delivery Notification Service De-Identification Service Distributed Orchestration Intelligent Consumption Services Populating RDSs on Demand RDS Direct Usage Considerations Summary 4. Services and API Management: The API Architecture Introducing the API Architecture What Is Service-Oriented Architecture? Enterprise Application Integration Service Orchestration Service Choreography Public Services and Private Services Service Models and Canonical Data Models Similarities Between SOA and Enterprise Data Warehousing Architecture Modern View on SOA API Gateway Responsibility Model The New Role of the ESB Service Contracts Service Discovery Microservices The Role of the API Gateway Within Microservices Functions Service Mesh Microservices Boundaries Microservices Within the API Reference Architecture Ecosystem Communication API-Based Communication Channels GraphQL Backend for Frontend Metadata Using RDSs for Real-Time and Intensive Reads Summary 5. Event and Response Management: The Streaming Architecture Introducing the Streaming Architecture The Asynchronous Event Model Makes the Difference What Do Event-Driven Architectures Look Like? Mediator Topology Broker Topology Event Processing Styles A Gentle Introduction to Apache Kafka Distributed Event Data Apache Kafka Features The Streaming Architecture Event Producers Event Consumers Event Platform Event Sourcing and Command Sourcing Governance Model Business Streams Streaming Consumption Patterns Event-Carried State Transfer Playing the Role of an RDS Using Streaming to Populate RDSs Controls and Policies for Guiding the Domains Streaming as the Operational Backbone Guarantees and Consistency Consistency Level “At Least Once, Exactly Once, and at Most Once” Processing Message Order Dead Letter Queue Streaming Interoperability Metadata for Governance and Self-Service Models Summary 6. Connecting the Dots Recap of the Architectures RDS Architecture API Architecture Streaming Architecture Strengthening Patterns Enterprise Interoperability Standards Stable Data Endpoints Data Delivery Contracts Accessible and Addressable Data Crossing Network Principles Enterprise Data Standards Consumption-Optimization Principles Discoverability of Metadata Semantic Consistency Supplying the Corresponding Metadata Data Origination and Movements Reference Architecture Summary 7. Sustainable Data Governance and Data Security Data Governance Organization: Data Governance Roles Processes: Data Governance Activities People: Trust and Ethical, Social, and Economic Considerations Technology: Golden Source, Ownership, and Application Administration Data: Golden Sources, Golden Datasets, and Classifications Data Security Current Siloed Approach Unified Data Security for Architectures Identity Providers Security Reference Architecture and Data Context Approach Security Process Flow Practical Guidance RDS Architecture API Architecture Streaming Architecture Intelligent Learning Engine Summary 8. Turning Data into Value Consumption Patterns Using Read-Only Data Stores Directly Domain Data Stores Target Operating Model Data Professionals as a Target User Group Business Requirements Nonfunctional Requirements Building the Data Pipeline and Data Model Distributing Integrated Data Business Intelligence Capabilities Self-Service Capabilities Analytical Capabilities Standard Infrastructure for Automated Deployments Stateless Models Prescripted and Configured Workbenches Standardize on Model Integration Patterns Automation Model Metadata Advanced Analytics Reference Architecture Summary 9. Mastering Enterprise Data Assets Demystifying Master Data Management Master Data Management Styles MDM Reference Architecture Designing a Master Data Management Solution MDM Distribution Master Identification Numbers Reference Data Versus Master Data Determining the Scope of Your Enterprise Data MDM and Data Quality as a Service Curated Data Metadata Exchange Integrated Views Reusable Components and Integration Logic Data Republishing Relation to Data Governance Summary 10. Democratizing Data with Metadata Metadata Management Enterprise Metadata Model Enterprise Knowledge Graph Architectural Approaches for Metadata Management Metadata Interoperability Metadata Repositories Marketplace to Provide Rapid Access to Authorized Data Summary 11. Conclusion Delivery Model Fully Decentralized Approach Partially Decentralized Approach Structuring Teams InnerSource Strategy Culture Technology Choices The Decline of Traditional Enterprise Architecture Blueprints and Diagrams Modern Skills Control and Governance Last Words Glossary Index
Donate to keep this site alive
How to download source code?
1. Go to: https://www.oreilly.com/
2. Search the book title: Data Management at Scale: Best Practices for Enterprise Architecture
, sometime you may not get the results, please search the main title
3. Click the book title in the search results
3. Publisher resources
section, click Download Example Code
.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.