Learn Azure Synapse Data Explorer: A guide to building real-time analytics solutions to unlock log and telemetry data
- Length: 346 pages
- Edition: 1
- Language: English
- Publisher: Packt Publishing
- Publication Date: 2023-02-17
- ISBN-10: 1803233958
- ISBN-13: 9781803233956
- Sales Rank: #1569436 (See Top 100 Books)
A hands-on guide to working on use cases helping you ingest, analyze, and serve insightful data from IoT as well as telemetry data sources using Azure Synapse Data Explorer
Free PDF included with this book
Key Features
- Augment advanced analytics projects with your IoT and application data
- Expand your existing Azure Synapse environments with unstructured data
- Build industry-level projects on integration, experimentation, and dashboarding with Azure Synapse
Book Description
Large volumes of data are generated daily from applications, websites, IoT devices, and other free-text, semi-structured data sources. Azure Synapse Data Explorer helps you collect, store, and analyze such data, and work with other analytical engines, such as Apache Spark, to develop advanced data science projects and maximize the value you extract from data.
This book offers a comprehensive view of Azure Synapse Data Explorer, exploring not only the core scenarios of Data Explorer but also how it integrates within Azure Synapse. From data ingestion to data visualization and advanced analytics, you’ll learn to take an end-to-end approach to maximize the value of unstructured data and drive powerful insights using data science capabilities. With real-world usage scenarios, you’ll discover how to identify key projects where Azure Synapse Data Explorer can help you achieve your business goals. Throughout the chapters, you’ll also find out how to manage big data as part of a software as a service (SaaS) platform, as well as tune, secure, and serve data to end users.
By the end of this book, you’ll have mastered the big data life cycle and you’ll be able to implement advanced analytical scenarios from raw telemetry and log data.
What you will learn
- Integrate Data Explorer pools with all other Azure Synapse services
- Create Data Explorer pools with Azure Synapse Studio and Azure Portal
- Ingest, analyze, and serve data to users using Azure Synapse pipelines
- Integrate Power BI and visualize data with Synapse Studio
- Configure Azure Machine Learning integration in Azure Synapse
- Manage cost and troubleshoot Data Explorer pools in Synapse Analytics
- Secure Synapse workspaces and grant access to Data Explorer pools
Who this book is for
If you are a data engineer, data analyst, or business analyst working with unstructured data and looking to learn how to maximize the value of such data, this book is for you. If you already have experience working with Azure Synapse and want to incorporate unstructured data into your data science project, you’ll also find plenty of useful information in this book. To maximize your learning experience, familiarity with data and performing simple queries using SQL or KQL is recommended. Basic knowledge of Python will help you get more from the examples.
Learn Azure Synapse Data Explorer Contributors About the author About the reviewer Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Share Your Thoughts Download a free PDF copy of this book Part 1 Introduction to Azure Synapse Data Explorer Chapter 1: Introducing Azure Synapse Data Explorer Technical requirements Understanding the lifecycle of data Introducing the Team Data Science Process Tooling and infrastructure The need for a fast and highly scalable data exploration service What is Azure Synapse? Data integration Enterprise data warehousing Exploration on the data lake Apache Spark Log and telemetry analytics Integrated business intelligence Data governance Broad support for ML Security and Managed Virtual Network Management interface What is Azure Synapse Data Explorer? Integrating Data Explorer pools with other Azure Synapse services Query experience integrated into Azure Synapse Studio’s query editor Exploring, preparing, and modeling data with Apache Spark Data ingestion made easy with pipelines Unified management experience Exploring the Data Explorer pool infrastructure and scalability Data Explorer pool architecture Scalability of compute resources Managing data on distributed clusters Mission-critical infrastructure How much scale can Data Explorer handle? What makes Azure Synapse Data Explorer unique? When to use Azure Synapse Data Explorer Summary Chapter 2: Creating Your First Data Explorer Pool Technical requirements Creating a free Azure account Creating an Azure Synapse workspace Basics tab Security tab Networking tab Tags tab Review + create tab Finding your new workspace Creating a Data Explorer pool using Azure Synapse Studio Basics tab Additional settings tab Tags tab Review + create tab Creating a Data Explorer pool using the Azure portal Creating a Data Explorer pool using the Azure CLI Summary Chapter 3: Exploring Azure Synapse Studio Technical requirements Exploring the user interface of Azure Synapse Studio Running your first query Creating a database Loading the data Verifying whether your data has loaded successfully Working with data in Azure Synapse notebooks Saving your work and configuring source control Managing and monitoring Data Explorer pools Scaling Data Explorer pools Pausing and resuming pools Monitoring Data Explorer pools Summary Chapter 4: Real-World Usage Scenarios Technical requirements Building a multi-purpose end-to-end analytics environment Sources Ingest Store Process Enrich Serve User Summary Managing IoT data Processing and analyzing geospatial data Enabling real-time analytics with big data Performing time series analytics Summary Part 2 Working with Data Chapter 5: Ingesting Data into Data Explorer Pools Technical requirements Understanding the data loading process Defining a retention policy Choosing a data load strategy Streaming ingestion Batching ingestion Performing data ingestion Using KQL control commands Building an Azure Synapse pipeline Implementing continuous ingestion Using other data ingestion mechanisms Summary Chapter 6: Data Analysis and Exploration with KQL and Python Technical requirements Analyzing data with KQL Selecting data Working with calculated columns Plotting charts Obtaining percentiles Creating a time series Detecting outliers Using linear regression Exploring Data Explorer pool data with Python Creating an Apache Spark pool Working with Azure Synapse notebooks Reading data from Data Explorer pools Plotting charts Performing data transformation tasks Creating a lake database Summary Chapter 7: Data Visualization with Power BI Technical requirements Introduction to the Power BI integration Creating a Power BI report Adding data sources to your Power BI report Connecting Power BI with your Azure Synapse workspace Authoring Power BI reports from Azure Synapse Studio Summary Chapter 8: Building Machine Learning Experiments Technical requirements Understanding the application of ML Introducing ML into your projects with AutoML Creating an Azure Machine Learning workspace Configuring the Azure Machine Learning integration Finding the best model with AutoML Exploring additional ML capabilities in Azure Synapse Using pre-trained models with Cognitive Services Finding patterns using KQL Training models with Apache Spark MLlib Building applications with SynapseML Summary Chapter 9: Exporting Data from Data Explorer Pools Technical requirements Understanding data export scenarios Exporting data with client tools Using server-side export to pull data Performing robust exports with server-side data push Exporting to cloud storage Exporting to SQL tables Exporting to external tables Configuring continuous data export Summary Part 3 Managing Azure Synapse Data Explorer Chapter 10: System Monitoring and Diagnostics Technical requirements Monitoring your environment Checking your Data Explorer pool capacity Monitoring query execution Reviewing object metadata and changes Setting up alerts Creating action groups Creating alert rules Summary Chapter 11: Tuning and Resource Management Technical requirements Implementing resource governance with workload groups Managing workload groups Classifying user requests Queuing requests for delayed execution Speeding up queries using cache policies Summary Chapter 12: Securing Your Environment Technical requirements Security overview Managing data encryption Configuring data encryption at rest Understanding data encryption in transit Authenticating users Configuring access to resources Synapse RBAC roles Reviewing role assignments Assigning RBAC roles Data Explorer database roles Implementing network security Using a managed virtual network Managed private endpoint connection Enabling data exfiltration protection Controlling public network access Protecting against external threats Summary Chapter 13: Advanced Data Management Technical requirements Managing extents Extent tagging Moving extents Dropping extents Purging personal data Enabling purge on Data Explorer pools Executing data purge operations Monitoring data purge operations Summary Index Why subscribe? Other Books You May Enjoy Packt is searching for authors like you Share Your Thoughts Download a free PDF copy of this book
Donate to keep this site alive
How to download source code?
1. Go to: https://github.com/PacktPublishing
2. In the Find a repository… box, search the book title: Learn Azure Synapse Data Explorer: A guide to building real-time analytics solutions to unlock log and telemetry data
, sometime you may not get the results, please search the main title.
3. Click the book title in the search results.
3. Click Code to download.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.