In order to be able to run a U-SQL task or job, we need to create the Data Lake Analytics resource. In the Azure dashboard, click on New to create a new resource and look for the Data Lake Analytics resource in the new window:
Press Enter, and in the new window, click on Create:
Enter the name of the new resource (note that the resource name should contain only lowercase letters and numbers) and the rest of the information:
We click on the Data Lake Store section and choose the Data Lake Store we have previously created:
And click on Create:
Find the new resource to ensure it was created:
We have created the Data Lake Analytics resource and now we can run U-SQL to manipulate or summarize data. We can run U-SQL either directly from the Data Lake Analytics Resource, via job, or from the Data Factory in a pipeline.
The next two sections will show you how to do the following:
- Run U-SQL via a job in Data Lake Analytics to copy data from SQL Server into a blob storage file.
- Run the Copy task from the Data Factory Pipeline to do the same (copy data from SQL Server into blob Storage).
- Run another task from the data factory in the same pipeline to summarize the data from the aforementioned file into another file in blob storage.