Introduction to Machine Learning and Deep Learning

At its core, ML is the process of mapping input to output without hardcoded rules. In other words, rather than using rules like “if, else, case, switch,” ML uses systems that have seen and analyzed past data (or even streaming data) to make more “educated” guesses or predictions of future events.

In classical programming, we write code (the rules or program) and provide data (input arguments and files) to receive an answer (return value). In contrast, in ML systems, labels or answers are provided along with the data to produce reusable “rules” or models, that in turn, may be used on new or “unseen” data to predict the labels or answers/return values.

The production of an ML model is through a process called training. Usually included in training a model is an evaluation of that model on the part of the original dataset that had not been used in training—sometimes called the held-out or test dataset . After the production of a satisfactory trained model (according to an evaluation), it is used to gain insights on new data in a process called inference or scoring .

There are three general paradigms in ML, as shown in Figure 16-2, within which almost all ML algorithms fall. They are ways in which we can categorize different approaches to different types of problems and datasets. These paradigms apply to traditional ML as well as deep learning.

In supervised learning, labels are provided at the beginning of training a model. Figure 16-1 depicts supervised learning, where data and the labels are provided to produce a model for tasks like classification or regression (see Figure 16-2). For instance, determining whether a patient has retinopathy, a dangerous eye condition, based on images of retinas from healthy patients and patients with this condition, is an example of a classification task. The labels for this task are binary: retinopathy or no retinopathy. Time-series analysis—where future events, like the demand for a product in a grocery store—is an example of a regression task. The demand for a product could be quantified as the number of items sold in a day. The future demand could then be predicted based on the demand from past days, or historical data. Semi-supervised learning is where a portion of the training data has labels, and the rest do not.

Unsupervised learning aims to understand the inner structure or organization in the input data and, thus, the input data does not need to be labeled. One common task is called clustering , where datapoints are formed into clusters that could answer questions such as how to group people who attend movies with input features such as age, gender, and income. Here, no label is trying to be predicted; rather, the goal is to find patterns and structure in the data. Based on how the data clusters, new characteristics may be inferred based on the age and income ranges, as well as, gender characteristics of the clusters.

Dimensionality reduction is a method to take several features and combine them in such a way that the number of features (dimensions) becomes reduced. An example of dimensionality reduction is taking a dataset such as pixel intensities from commonly sized images of the digits 0 to 9 (each pixel location is a feature or dimension, and each data point is an image). The fact that the image is a 3 or a 7 does not matter in dimensionality reduction; only the pixel intensities at each pixel location for each image. The feature count, also called feature space, is equal to the number of total pixels of the images in this case (e.g., for a set of images with a typical size of 28×28 pixels, they have 784 features each, which equates to 784 dimensions!). Dimensionality reduction reduces the number of dimensions to a smaller number. It is commonly used in conjunction with other types of ML, which, for one, often can help speed up training.

In reinforcement learning , a player (called an agent) learns how to optimally operate in an environment based on rewards and penalties assigned to it after taking actions in an iterative manner. For instance, imagine there is a maze that an agent is trying to get through to gain a big reward at the end, maybe a pile of gold or a large sum of points, but along the way, there are traps that take away points. The environment is the configuration or instance of this particular maze.

We begin with the agent at the entry-point to the maze—the initial state or location of the agent. Each turn by the agent through the maze counts as an action. At each turn, there is either a reward (award 1 point due to no traps or dead-ends encountered) or penalty (a trap or dead-end has been encountered—1 point removed). The summation of the points is the reward function in this case that the agent is trying to maximize. If the agent, through taking incremental actions (each turn or step), reaches the pile of gold, the agent has exited the maze successfully, and a new structure or set of conditions is created (new environment). The agent then begins again with some knowledge of how mazes work. After a certain number of rounds through different mazes, the agent has now learned strategies to make it through a maze to get the most points (it has maximized the reward function).

The examples in the next few sections focus on image analysis, as it is a good way to understand ML and important to many current scenarios. Text/natural-language, time-series, audio, speech, and other types of data are left for your exploration by using the resources provided on the book website at https://harris-soh-copeland-puca.github.io/.

Data Discussion

For use in an ML algorithm, data must be represented numerically, whether it’s text, audio, image, or other types. For instance, image data, which is multidimensional by nature. Commonly, it has a red, green, and blue channel (and sometimes an alpha channel) if in RGB format. Each channel is a 2D array of numbers (where each number represents the intensity of a pixel). The channel 2D array (e.g., a red array) has the same width and height in pixels as the original image. The original image data (three 2D arrays) is the starting point for further image processing, such as padding, clipping, shrinking, and normalizing. Figure 16-3 shows how an RGB image is resized from its original dimensions down to 50×50 pixels.

Figure 16-4 shows the separation of an image (a color RGB image – shown in black and white here) into the three channels, each of which are 2D arrays. It is a good example of data preprocessing for ML. The numbers equate to the pixel intensities in the range of 0–255. ML algorithms operate on all of these, and similar arrays of numbers, in different ways. Generally, they need an image as a multidimensional array of numbers, sometimes referred to as a NumPy array (because of the use of NumPy library) or a tensor (a common term in deep learning). Both terms, at their core, are simply arrays of numbers; however, they do sometimes come with a few other properties.

Note

ML algorithms that operate on other types of data such as text and speech that also need data to be represented by numerical values as arrays of numbers.

A preprocessing technique called image augmentation adds variation to the dataset by morphing a subset of the images (flipping, blurring, applying geometric transformations, lighting differences, etc.) to create a more generalized model that is used in diverse situations and usually with higher confidence. Data augmentation, more broadly, can increase the size of and add variation to a dataset. It is a very useful technique in the context of classical ML and deep learning for tasks of all kinds (image classification, sentiment analysis on text, acoustic event detection, etc.). Other types of preprocessing include extrapolation, tagging, aggregation, imputation, and probability techniques.

In supervised ML, it is extremely important to have a balanced dataset, that is, equal representation from all classes. This prevents class bias in the training process. Along those lines, the data should match the domain to get the question answered by creating the ML model. For example, to create a text-based system to answer questions from customers regarding how to file and monitor insurance claims for a particular firm, not all insurance-related documents across the Internet should be used to train the model, rather, only insurance claim questions and answers from that particular firm should be used. Also, one should always ensure to have the correct permissions to use the dataset to answer data science questions. As a final note, it is common to encounter biased datasets—ones that have inherent, built-in human prejudices. A dataset should be examined carefully, labels or feature characteristics should be checked (e.g., Do number ranges make sense? Are all classes represented equally and fairly?), and exploratory data analysis techniques should be used before using it in ML experiments.

Traditional ML

Traditional ML, also called classical ML, generally consists of simpler approaches than deep learning to solving similar data science problems (deep learning is addressed in a subsequent section). Traditional ML models often take less time to train, and the algorithms are generally easier to understand, thus, explain to others, however, at the cost of usually having to do much more manual preprocessing to address certain limitations in the algorithms.

Take, for instance, using classical ML to create an object detector. In this example, a window slides across an image incrementally, and at each increment, a classifier determines the types of objects in it. Object detection consists of two parts: localization (find the object; i.e., slide the window) and classification (name the object; i.e., run that portion of the image through a classification model).

Figure 16-5 shows the use of histogram of oriented gradients (HOG) , which transforms an image into information-rich features (the preprocessing part). A support vector machine (SVM) is used for input with a large feature space to perform classification (the machine learning part). Here, HOG is used a featurizer on the raw input images before applying the classifier (the SVM). In traditional ML, it is common to have to carefully and extensively preprocess data manually, such as using HOG or other featurizers, in addition to using normalization methods, scaling, padding, and so forth. An example of the output of the HOG featurizer is shown in Figure 16-5.

An SVM classifier, also known as a support vector classifier (SVC) , finds the best hyperplane that separates the dataset into the individual classes. As a simple example, imagine classifying flowers into three different species based on a petal length and sepal width, two features. The hyperplanes separating these classes may be visualized in Figure 16-6.

SVCs or SVMs are known to be good at dealing with larger feature spaces such as images with a height and width of 64 pixels (equating to 4096 features). SVM is one of the dominant classification algorithms falling under supervised learning techniques.

Again, in this example, a fixed-sized window incrementally slides over an image, and at each step, the window or portion of the image is classified as having the object or not. Figure 16-7 shows the resulting bounding boxes from using a trained model to detect an object in a new image that the model has never “seen” (for instance, detecting the head of a cat in a new image after training the algorithm on pictures of cat heads; the classes were “cat face” and “no cat face,” where negative samples of non-cat images were used for “no cat face”). A histogram of oriented gradients was the featurizer to train the object detection model in Figure 16-7.

For a Python Jupyter notebook code sample of Histogram of Oriented Gradients with an SVM for object detection, go to https://github.com/harris-soh-copeland-puca/SampleCode/blob/master/Ch16/Ch16.Extra_Simple_Object_Detection_HOG.ipynb. See documentation for more details on the HOG algorithm at https://scikit-image.org/docs/dev/auto_examples/features_detection/plot_hog.html and SVMs at https://scikit-learn.org/stable/modules/svm.html.

In the supervised learning case, as shown here, the ML model is trained by checking how far from the ground-truth (real) value the output is and comparing that to the model output (the prediction). This serves as feedback (through using a mathematical equation called a loss function) to the model that it is either getting better or worse at the task on which it is being trained. After some time, an ML model should perform better at the task as it converges on small loss values, an indication that the guesses are improving over iterations (called epochs) in the training process. Other types of ML models are trained similarly (iteratively and with feedback) such that a function gets minimized or maximized, which helps lead the training process to create a good model.

The Data Science Process

Just like creating and improving upon an application through cycles of development and productionizing, data science is an iterative process, as shown in Figure 16-8. The iterations of data science are a little different, however, from application development in the types of steps and resources needed. In fact, application development can very well include a data science process. This process is depicted in Figure 16-8.

The most important part of data science is asking the right question. The question usually comes from a business need or scientific study. A flawed question would be, how much money will I make next month? A better question is, by how many points will my Microsoft and Apple stocks increase over the next 28 days, and what will the mean predicted gain be at that time? Almost equally as important is to have relevant data to the question at hand.

After data acquisition, a good data science experiment begins with an exploratory data analysis step, which includes visualizing data such as the distribution of values or checking for outliers that could indicate mistakes in labels (which can happen on occasion) or calculations.

Usually, data is preprocessed in a certain way for the ML model to be trained, such as removing NaNs (for “not a number”), one type of data cleaning step. Data transformations are important, such as shaping an image from a 2D array to a long 1D array or making an RGB image grayscale. Another transformation type in the context of a time series experiment is log transforming the signal. It is said that data science is 80%–90% data processing and exploring, so becoming comfortable with data processing languages (e.g., SQL, Python, R) and ETL tools is very beneficial. Once the data is clean and transformed in a way that the algorithm can take as quality input, a model can be trained.

During the training process, different models may be tried, different algorithm parameters may be iterated over (hyperparameter tuning), and different ways of slicing and shuffling the dataset may be explored. Sometimes there is a need to go back to the cleaning and transformation data steps.

In the end, there is a better understanding of the problem space and datasets. The cycle continues as new data input changes the original dataset used to train the model; data drift may occur when the data becomes very different from the original dataset used to train a model and the model may not perform as well. A new model may need to be trained by going through the data science process periodically.

In summary, the data science process allows exploration and analysis of data and machine learning approaches in an iterative manner with improvements and better understanding along the way.

A Jupyter Notebook Overview

If your day-to-day work involves any data science, becoming familiar with Jupyter notebooks is an important skill. Jupyter notebooks are not necessarily a replacement for a full-fledged IDE or coding UI like Visual Studio, but they do have some advantages over such applications. Advantages include

• Quick and iterative prototyping

• Easy annotation that is presentation quality

• Shareable code and instructions for collaboration

• Notes and code combined in one place

• The ability to run anywhere the data science language (e.g., Python) and a browser are installed

• Integration into tools such as Visual Studio Code

Jupyter notebooks offer many different kernels for different languages and are predominantly used with data science languages like Python, R, F#, and Scala (a kernel is what interfaces with the programming language). Simply put, they provide a method of both annotating one’s work (in the Markdown language or plain text) as well as writing code in blocks or small sections. The code blocks are run one at a time, which allows quick and easy prototyping and experimentation. Once a user is satisfied with the code, it may be exported to various formats, one of which is a script file (e.g., the Python .py file). The script file may then be run as one entire program at that point if so desired.

Figure 16-9 depicts a Jupyter notebook with free-form text annotation at the beginning (white background) followed by a Python code block (gray background). The example comes from a public source created by the Azure ML team.

Note

Notebooks are used by the Azure ML team to demonstrate how to use its functionality with annotated examples. The Azure ML team maintains an official collection of Jupyter notebooks on GitHub at https://github.com/Azure/MachineLearningNotebooks.

Jupyter usage is explained in the next section, as well as how to write a simple Python program.

Hands-on with the Data Science Virtual Machine

Let’s go through provisioning an Ubuntu Linux DSVM by using JupyterHub and the command terminal for some basic Python coding in a browser. The latest information on setting up the Linux DSVM is at https://docs.microsoft.com/en-us/azure/machine-learning/data-science-virtual-machine/dsvm-ubuntu-intro.

1. 1.
   Go to the Azure portal and search for Data Science Virtual Machine. Select the Ubuntu version.

   

    

1. 2.
   When provisioning, ensure you select Password as the Authentication type in the Administrator account section on the Basics tab (the first page after clicking Create). Also, it is very important that the Username for the account is all lower-case due to a limitiation with JupyterHub.

   

    

1. 3.

   Navigate to the public IP address listed on the Overview pane of the DSVM resource in the portal once provisioning has completed. Note down this IP address. Firefox and Chrome browsers are best for use with JupyterHub. Enter the following URL and direct a browser window to it: https://THE_PUBLIC_IP_ADDRESS:8000. IMPORTANT: A certificate error may pop up; if so, simply continue to proceed to the site (this is a known issue). Next, the login screen shown in Figure 16-12 should appear. Provide your administrator username and password to proceed to the Jupyter notebooks. JupyterHub is a multitenant system that can support multiple users, but by default, it is set up for only one user. More may be added later with Unix commands.

    

1. 4.
   Open a new Python 3 Azure ML notebook by clicking the drop-down menu on the upper right and selecting Python 3.6 – Azure ML or the latest Python version with Azure ML.

   

    

1. 5.
   Git clone the https://github.com/harris-soh-copeland-puca/SampleCode repo by type the following on the command line.

   git clone https://github.com/harris-soh-copeland-puca/SampleCode.git
    
2. 6.
   Upload the Ch16/Ch16.01_Image_Manipulation.ipynb file to the DSVM through JupyterHub, confirm the upload, and then click the notebook file to open it.

   

    

An open a Jupyter notebook. It should look like what’s shown in Figure 16-15.

1. 7.

   Follow along with the notebook and read through the comments to understand how an image is read in Python. To run a code-cell, use the Run button at the top or use the keyboard shortcut Shift+Enter when the cursor is in a cell. Go through each code cell of the notebook and try to understand what is going on. In addition, the notebook contains code cells on how to separate the color channels (RGB) of the image and plot them.

    

The Azure DSVM is not your only option, however, for working in the data science domain. For those getting started, the following article describes other options for setup: https://rheartpython.github.io/navigating-ml/setup/.

Overview of Azure Machine Learning

Azure ML is a cloud-based environment for training models, managing models, monitoring services, tracking experimentation, and deploying in an iterative or automated manner. The common workflow is shown in Figure 16-16.

When training a model, think of Azure ML as your experimentation and deployment platform with integration into common frameworks like Scikit-Learn for classical ML or PyTorch and TensorFlow for deep learning. The ML practitioner has the choice to do their experimentation locally, on cloud VMs (like the DSVM) or Spark clusters like Databricks.

When the practitioner is satisfied with the model as measured by an appropriate metric (or “ruler” like accuracy), they may validate it on new, unseen data (part of our “data science process”). At this point, if the results are satisfactory, the practitioner or DevOps professional may deploy the model.

In Azure ML, there are a few different deployment paradigms. One is a cloud service and another an IoT edge device (e.g., an ARM32 IoT device like the Vision AI DevKit https://azure.github.io/Vision-AI-DevKit-Pages/). The cloud service can further be consumed by a Power BI dashboard.

Finally, a model may be monitored using Azure ML integrated with Azure Monitor for things like the health of the service. If it is an IoT Edge deployment, the device may be monitored with Azure IoT Hub from which messages and logs may then be analyzed.

The underlying Azure resources include an Azure Storage account, Azure Container Registry, Azure Key Vault, and Azure Application Insights. These resources are provisioned at the same time an Azure ML workspace is created.

The full taxonomy of the Azure ML workspace is shown in Figure 16-17.

Azure ML currently has SDKs in the R and Python programming languages. It can also be utilized with an interface called Designer, for a more of a drag-and-drop, no-code experience (however, in Designer, you can also write custom code as modules). In addition to the SDKs and Designer, Azure ML has a CLI extension to the main Azure CLI that integrates directly with the Azure ML workspace from the command line and a REST API.

Almost any Python or R training code may be converted to an Azure ML-friendly code, which allows the use of the Azure ML cloud compute to train models and services like Azure Kubernetes Service for deploying models. Not only does Azure ML have linked compute for training/deploying, but it also has mechanisms for taking a snapshot and versioning the code to train the model. Also, versioning the model itself is done by a process called registering a model . This is part of model management. Once registered, a model is associated with the Azure ML workspace for later retrieval simply by using the SDK.

Additionally, Azure ML provides automated ML (Auto ML) capabilities. Three types of tasks are automated: preprocessing (e.g., featurizing), choosing algorithms that are appropriate to use, and hyper-parameter tuning. Currently, regression, classification, and time-series forecasting are supported in Azure ML’s Auto ML capabilities.

Hands-on with Azure Machine Learning: Training a Model

1. 1.

   Provision an Azure ML workspace in the Azure portal, with the Azure ML SDK or CLI.

   In the portal, search for machine learning, select Machine Learning, and click Create when prompted. Fill out the information for provisioning, selecting Basic as the workspace edition (it can be changed later). Note, the Basic edition does not support Auto ML and a few other features. Create this resource.

   

    

Once the workspace is provisioned, go to the resource, and download the config.json file directly from the Overview pane.

1. 2.
   Start a Jupyter notebook system. Choices include

   1. a.

      JupyterHub on the DSVM

       
   2. b.

      Use Azure Notebooks (best if attached to a DSVM) at https://notebooks.azure.com

       
   3. c.

      Locally, if Python 3 and Jupyter are installed. Start the Jupyter notebook locally with the following command on the command line.

       
    

jupyter notebook

Note

The kernel may need to be set to the correct Python version and (on the DSVM, use Python 3.6 – Azure ML or the latest Python version with Azure ML).

1. 3.
   In JupyterHub, open a terminal by selecting New ➤ Terminal.

   

    

This should provide an interface that looks as follows in Figure 16-21.

1. 4.
   Change directory into the notebooks folder and git clone the following repo, if not done already, by typing the following into the terminal window.

   cd notebooks

   git clone https://github.com/harris-soh-copeland-puca/SampleCode.git

   There should now be a Ch16 folder available.

    
2. 5.
   Back in the main window of Jupyter (to get there, click the Jupyter logo in the upper-left corner), navigate to the Ch16 folder, and open Ch16.02a_Train_Digits_Classifer_Sklearn_AzureML.ipynb. The Python 3.6 – Azure ML kernel, if on the DSVM, should be chosen if prompted. If a switch of kernels is needed, the Kernel option in the menu is the correct place to go, as shown in Figure 16-22.

   

    

1. 6.

   From the Ch16 folder, upload the config.json configuration file downloaded earlier so that the notebook has access.

    
2. 7.

   Read through the introduction and then run the Import packages cell (a shortcut to run the code in a cell is to press Shift+Enter).

   The notebook trains a classical ML model (logistic regression) on the MNIST handwritten digits dataset that is now a part of Azure Open Datasets and accessible through the Python SDK. The original dataset is also accessible at http://yann.lecun.com/exdb/mnist/.

    
3. 8.

   Run the Connect to workspace cell and log in to Azure through by following the interactive authentication instructions.

    
4. 9.

   Run the Create experiment cell to create an Experiment for all the training runs for the scenario.

    
5. 10.
   Run the Create or Attach existing compute resources cell to create a managed Azure ML Compute service for training ML models in this Experiment or others. This may take some time because it is provisioning a cluster of Linux VMs on Azure with the specification shown in Figure 16-23.

   

    

1. 11.
   Run Display some sample images to view a subset of the MNIST handwritten digits dataset. Here are ten labeled sample images from this dataset.

   

    

1. 12.
   Run the Create a directory and Create a training script. Note, that the Jupyter cell magic %%writefile $script_folder/train.py appears at the beginning of the cell. This tells Jupyter to write a physical file to the path specified. Look in the folder with the Jupyter notebook to ensure the train.py was written to the sklearn-mnist subfolder. Notice the code that takes care of connecting the training process to Azure ML (Run context) as well as the code that trains a logistic regression classifier within this cell.

   

    

The sklearn-mnist script folder is a crucial element of using Azure ML as the contents of this folder are uploaded to the Azure ML Compute (target compute context) to do the training. The compute takes a snapshot of this script folder. The snapshot currently has a size limit of 300 MB or 2000 files. Because of this, if needing to train on a large amount of data, one recommendation is that one uses a Datastore object (https://docs.microsoft.com/en-us/azure/machine-learning/how-to-access-data) that utilizes either the Blob Storage attached to the workspace or a separate Blob Storage. Another option is to have the training script download the data directly with code or use a Dataset object (https://docs.microsoft.com/en-us/azure/machine-learning/how-to-train-with-datasets) which helps keep track of that process. The Dataset object is shown in this notebook.

1. 13.

   Run all the cells in the Create an estimator section to define the Python environment and create the Estimator, a wrapper object that understands the compute, training script, script folder, parameters for training script and Python environment.

    
2. 14.

   Run the Submit the job to the cluster cell to begin training the model on the MNIST digits dataset. This creates a docker image, scales the cluster, runs the training job, and saves to an output folder with any created assets and sends this to the workspace upon completion.

    
3. 15.

   Run the Jupyter widget cell to see the progress in detail of the job. The first time the job is submitted, it takes up to 10 minutes as it needs to create the docker image and scale the cluster to the right number of nodes. After running this cell, a link to the Azure portal is provided. Click this link to see how it shows in the Azure portal for monitoring. If not using a Jupyter notebook, but rather scripts, it is helpful to know about this view.

    
4. 16.

   Run Get log results upon completion to view the logs and prevent running other code until the run is complete.

    
5. 17.

   Run Display run results to see the accuracy of the model on the test dataset.

    
6. 18.

   The model is now available in the output folder as a Python pickle file. It is associated with the run in that workspace. Run the Register model cells to register the model and make it accessible programmatically (useful in future scripts that need to access/use this model). Note that the model now has a name and a version number.

    

Use Case: Image Classification with a Deep Neural Network and Azure Machine Learning

In this section, image classification with a neural network is discussed. In the sample code, the PyTorch deep learning framework is used (https://pytorch.org).

Imagine a classifier is needed in a surveillance scenario to discern when there is suspicious behavior such as fighting, lying on a floor, or leaving a package in a public space. If enough sample images are provided (10K or more), then a deep neural network architecture should be considered to train a model (e.g., ResNet or SqueezeNet). Sample images representing normal (e.g., browsing and walking) and suspicious (e.g., fighting) behavior are shown in Figure 16-27. The data used in this section originates from the CAVIAR dataset (https://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/).

IoT Devices and the Intelligent Edge

The Internet of Things, or IoT, refers to an Internet interconnected ecosystem of computing devices that are embedded into everyday objects, sending, and receiving different types of data continuously to and from other devices or the cloud. Examples of IoT devices are smartwatches, smart thermostats, network/Internet-connected cameras to monitor front doors, and temperature sensors that send data to alerting systems through the network. Devices may be controlled or monitored on cell phone, for instance, or an onsite computer system. They may be Internet-connected and monitored through a system like a Power BI dashboard in Azure. The device might not be connected to the Internet regularly, and some never connect to the Internet. Azure still helps IoT developers build entirely disconnected intelligent devices systems.

There are many Azure IoT projects available for the MXChip IoT DevKit (https://aka.ms/iot-devkit), a small, Arduino-based device, and other devices like it. The IoT DevKit, as a common example, can operate connected to wi-fi or in a completely disconnected scenario, incorporating cloud AI or running an ML model locally. Sample projects, integrated with Visual Studio Code, are provided with the Azure IoT Tools extension.

Not only is it easy to run cognitive services on constrained devices like the MXChip, but there are also advanced Azure services (e.g., Azure IoT Hub and Azure IoT Edge) that provide mechanisms to deploy ML models to devices. For instance, the NVIDIA Jetson Nano is a GPU-accelerated embedded device capable of running deep learning models efficiently. Azure IoT Edge may be set up on this type of Linux device to run AI models, coordinate their updates (as Azure IoT Edge modules), and monitor their output and health (in Azure IoT Hub). One example takes advantage of NVIDIA’s DeepStream SDK (see https://github.com/Azure-Samples/NVIDIA-Deepstream-Azure-IoT-Edge-on-a-NVIDIA-Jetson-Nano).

IoT Edge modules are docker containers with custom code, optionally supporting both Azure ML-built AI models or custom ML models trained elsewhere. They don’t necessarily have to have any ML, but it is becoming a more popular option. They are capable of running on laptops and servers, as well as constrained Windows and Linux devices, like cell phones and ARM-based computing systems.

The intelligent Edge allows the creation and deployment of modules for AI such as object detection and optical character recognition (e.g., license plate recognition), audio analysis (e.g., a dog barking), speech to text and translation (e.g., helping medical professionals with charting), the Azure IoT Edge runtime, and tight integration with the Azure cloud through IoT Hub. More information and tutorials are on the IoT Edge documentation pages at https://docs.microsoft.com/en-us/azure/iot-edge/.

As a continuation of the projects in this chapter, deploy an Azure IoT module to an Azure Edge VM (VM in Azure specifically preinstalled with Azure IoT Edge runtime) by running this tutorial: https://github.com/harris-soh-copeland-puca/Azure-AI-Camp/tree/master/day2/2.IoTEdgeModule.

Overview of Spark and Databricks

When discussing Spark, it’s helpful to know the difference between scale-up and scale-out. Scale-up and scale-out are ways to meet the increasing demand for computing resources from increasingly large workloads.

In its basic form, scale-up refers to adding more resources to a single machine or set of independent machines such as increasing memory or adding a more powerful processor.

With scale-out, there is a set of machines that share a common workload. Scale-out, in its basic form, is adding more machines to the pool that work on a single workload in unison. Parts of a workload and partitions of data are separated onto different worker nodes, which each perform calculations and, when finished with a task, return an answer to a cluster manager. This organization is shown in Figure 16-30.

This section is based on an excellent tutorial on Azure Databricks and Spark by B. Cafferky (www.youtube.com/watch?v=ofYIsToVnT0).

Spark is part of the wider Apache ecosystem or zoo, an open source collection of tools and libraries for scaling-out with distributed computing. An older approach to scale-out methodology was Hadoop Map Reduce. Spark is a newer take on this, and in some cases, hundreds of times faster than Map Reduce. Unlike Map Reduce, Spark aims to keep as much data in memory as possible to avoid I/O trips to and from the data source such as a Hadoop Distributed File System or SQL database.

In the next two sections, a quick walkthrough on how to set up a Databricks workspace and a tutorial using Databricks notebooks for image featurization and classification are shown.

Summary

This chapter laid a foundation of knowledge in machine learning, with examples in image data processing and analysis. The Data Science Virtual Machine, Jupyter notebooks, Azure ML, and Databricks for data science tasks were discussed. Images are not the only things that can be analyzed with machine learning, so you are encouraged to continue to learn and explore ways to use ML on Azure—whether on a local computer with the Azure ML SDKs, a DSVM or in an Azure Databricks workspace. Plentiful references and resources are on this book’s website at https://harris-soh-copeland-puca.github.io. Machine learning can truly be a fascinating and lifelong study.