Title Page Copyright and Credits Hands-On Data Warehousing with Azure Data Factory Packt Upsell Why subscribe? PacktPub.com Contributors About the authors About the reviewer Packt is searching for authors like you Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Reviews The Modern Data Warehouse The need for a data warehouse Driven by IT Self-service BI Cloud-based BI – big data and artificial intelligence The modern data warehouse Main components of a data warehouse Staging area Data warehouse Cubes Consumption layer – BI and analytics What is Azure Data Factory Limitations of ADF V1.0 What's new in V2.0? Integration runtime Linked services Datasets Pipelines Activities Parameters Expressions Controlling the flow of activities SSIS package deployment in Azure Spark cluster data store Summary Getting Started with Our First Data Factory Resource group Azure Data Factory Datasets Linked services Integration runtimes Activities Monitoring the data factory pipeline runs Azure Blob storage Blob containers Types of blobs Block blobs Page blobs Replication of storage Creating an Azure Blob storage account SQL Azure database Creating the Azure SQL Server Attaching the BACPAC to our database Copying data using our data factory Summary SSIS Lift and Shift SSIS in ADF Sample setup Sample databases SSIS components Integration services catalog setup Sample solution in Visual Studio Deploying the project on-premises Leveraging our package in ADF V2 Integration runtimes Azure integration runtime Self-hosted runtime SSIS integration runtime Adding an SSIS integration runtime to the factory SSIS execution from a pipeline Summary Azure Data Lake Creating and configuring Data Lake Store Next Steps Ways to copy/import data from a database to the Data Lake Ways to store imported data in files in the Data Lake Easily moving data to the Data Lake Store Ways to directly copy files into the Data Lake Prerequisites for the next steps Creating a Data Lake Analytics resource Using the data factory to manipulate data in the Data Lake Task 1 – copy/import data from SQL Server to a blob storage file using data factory Task 2 – run a U-SQL task from the data factory pipeline to summarize data Service principal authentication Run U-SQL from a job in the Data Lake Analytics Summary Machine Learning on the Cloud Machine learning overview Machine learning algorithms Supervised learning Unsupervised learning Reinforcement learning Machine learning tasks Making predictions with regression algorithms Automated classification using machine learning Identifying groups using clustering methods Dimensionality reduction to improve performance Feature selection Feature extraction Azure Machine Learning Studio Azure Machine Learning Studio account Azure Machine Learning Studio experiment Dataset Module Work area Breast cancer detection Get the data Prepare the data Train the model Score and evaluate the model Summary Introduction to Azure Databricks Azure Databricks setup Prepare the data to ingest Setting up the folder in the Azure storage account Self-hosted integration runtime Linked service setup Datasets setup SQL Server dataset Blob storage dataset Linked service Dataset Copy data from SQL Server to sales-data Publish and trigger the copy activity Databricks notebook Calling Databricks notebook execution in ADF Summary Reporting on the Modern Data Warehouse Different types of BI Self-service – personal Team BI – sharing personal BI data Corporate BI Power BI Premium Power BI Report Server Power BI consumption Creating our Power BI reports Reporting with on-premise data sources Incorporating Spark data Summary