Chapter 11: Deploying Models with Flask Applications

Over the course of this book, we explored the development of numerous robust machine learning models in areas such as breast cancer detection, scientific topic modeling, protein classification, and molecular property prediction. In each of these tutorials, we prepared and validated our models to allow them to have the best predictive power possible. We will now pivot from the development of new models to the deployment of trained models to our end users.

Within this chapter, we will explore one of the most popular frameworks for the preparation of web applications: Flask. We will use Flask to prepare a web application to serve our models to end users, and we will also prepare an Application Programming Interface (API) to serve our predictions to other web applications.

Over the course of this chapter, we will cover the following topics:

• Understanding API frameworks
• Working with Flask and Visual Studio Code
• Using Flask as an API and web application
• Tutorial – Deploying a pretrained model using Flask

With these objectives in mind, let's go ahead and get started!

Understanding API frameworks

Whether you are logging in to your email account, scrolling through social media, or even logging in to an online retailer, we use web applications on a daily basis to accomplish a variety of tasks. For example, imagine a user scrolling through an electronic laboratory notebook on their local computer. When the user logs in and sees their data, this information is retrieved using an API (that is, an application programming interface, not to be confused with an active pharmaceutical ingredient). Once the data is retrieved for the user in the backend, it populates the frontend in a beautiful User Interface (UI) that allows the user to interact with the data, make changes, and save it. We can use web applications and APIs in a variety of ways, such as transferring data, communicating with others, or even making predictions, as illustrated in Figure 11.1:

Figure 11.1 – Some examples of web application functionality

With all of these capabilities, APIs and their counterparts have provided the main tool in the web application space for creating UIs to serve data and make predictions. There are a number of useful web application frameworks available for a range of programming languages, as illustrated in Figure 11.2:

Figure 11.2 – Some examples of web application frameworks

For the purposes of this chapter, we will focus on one of the more popular machine learning deployment frameworks: Flask (https://github.com/pallets/flask). Relative to its counterparts, Flask can be thought of as a micro web framework – it is completely written in Python and highly abstracted, allowing users to get started in the model deployment process with little to no difficulties.

As we begin to deploy models using the Flask framework, it is important to ask ourselves who the end user of our application will be. In many cases, predictions using our previously trained models will be conducted by colleagues and stakeholders. Therefore, having a useable UI will be important. On the other hand, our deployed models may not be needed by a person but rather a piece of software or another web application that will need to programmatically interact with it. In that case, a UI will not be needed – however, we will need an organized way (for example, JSON) to handle the transfer of data between the two systems. We can see a depiction of these two cases in Figure 11.3:

Figure 11.3 – The two general types of web applications

In either case, we will be able to accommodate both of these cases using Flask. The Flask framework offers a variety of architectures – both simple and complex – allowing users to select the pattern that best fits their needs. Flask APIs, in a similar way to their counterparts such as Django, Node.js, and Spring, all generally operate in a similar manner using URLs. For both backend APIs and frontend UIs, we can use URLs to organize how we develop an application. For example, users can log in to a website to view and edit data within their profiles, whereas APIs can allow external entities to interact with models, as depicted in Figure 11.4:

Figure 11.4 – The two general types of web applications with examples

In order to interact with a web application, a user needs to make what is known as an HTTP request, which is usually carried out without them knowing. Each of these requests is generally associated with a URL, allowing the user to accomplish a task. The four HTTP request types are depicted in Figure 11.5:

Figure 11.5 – The four HTTP request types

For example, if a user navigating to www.website.com/profile intends to retrieve the details of their profile, they would use a GET request. On the other hand, an application using the API with the intention of classifying a segment of text would use a POST request to send the text to www.website.com/api/classify. These URL paths are known as routes within the confines of web applications, and they allow developers and data scientists to better organize their models for deployment. In the following section, we will see how routes can be used more specifically within the Flask framework.

Working with Flask and Visual Studio Code

Flask is one of the most commonly used and versatile web applications available in the Python language. Its abstract and high-level framework makes it easy for users of all levels to have an implementation up and running in no time. Over the course of this section, we will learn about the different components of a Flask application and deploy a simple model locally on our machine.

Before we can get started with Flask, we will need an Integrated Development Environment (IDE) to work with. So far, we have worked almost exclusively in Jupyter Notebook to train and develop models. When it comes to implementation, we will need another type of IDE to work with. There are numerous Python IDEs we can use, such as PyCharm, Spyder, or Visual Studio Code (VSC). I personally have found VSC to be the most user-friendly to work with, and therefore, we will use that as our primary IDE in this section. You can download VSC from their website (https://code.visualstudio.com/download) or by using Anaconda.

Go ahead and begin the installation process, which might take a few minutes. While you wait, create a new folder called flask-test on your local computer. Once the installation process is complete, open VSC. You can open the folder you just created in a few simple steps:

1. Click File on the top menu.
2. Click Open Folder.
3. Navigate to your directory and click Select Folder.

You should now see the name of your directory in the explorer pane on the left-hand side of the screen. Within the explorer pane, you will be able to see all of the files and folders relevant to your current project. Let's go ahead and populate it with a file called app.py by right-clicking in the explorer pane and selecting New File.

The app.py file is the main file that Flask uses in its framework. Everything within the application is included in this file or referenced from within it. Although its content depends on the exact implementation of the user, the file generally contains four main sections:

1. Importing libraries, data, and other resources
2. Instantiating the application and declaring other useful functions
3. Declaring the routes for the application
4. Running the __name__ == "__main__" driver piece of code

   We can see an illustration of these components in Figure 11.6:

Figure 11.6 – The main components of a Flask application

Let's now go ahead and populate app.py with some code. This is generally done in four main sections:

1. We will begin by importing the Flask class from the flask library:

   from flask import Flask

2. Next, we will need to create an instance of our Flask app:

   app = Flask(__name__)

3. We can now use the app object to create routes for our application. Routes operate by executing the function directly beneath it when that route is interacted with. Let's make a simple one that returns "Hello Biotech World!":

   @app.route('/')

   def biotech():

       return "Hello Biotech World!"

4. Finally, we will need a driver for the application that can fulfill using if __name__ == '__main__'. We will also set the debug parameter as True to help us address any potential issues, and we will set the port value to 8080:

   if __name__ == '__main__':

       app.run(debug=True, port=8080)

   From the command line in VSC, go ahead and run the Python application:

   $ python3.8 app.py

   This will run the application on your local computer. You can access it through any browser, such as Google Chrome, and by navigating to http://localhost:8080/. Upon reaching this URL, you should be greeted by our previous message. Please note that the localhost URL is a link only accessible locally on your computer and is not available to others. The concept of routes should be familiar to us from the many websites we have used in the past. We can break down a URL into its smaller components, as depicted in Figure 11.7:

Figure 11.7 – The main components of a URL

In our case, we are currently editing the path or endpoint of the application. Flask applications can handle many paths and endpoints, giving developers a great deal of flexibility.

You can stop the application from running by pressing CTRL + C in the command line, which will halt the process. With the process halted, go ahead and create a second route by copying the current route and function directly below it. Give the path a value of /lifescience (instead of just /) and give its function a unique name such as lifescience. Next, change the returned value, run the application again, and navigate to http://localhost:8080/lifescience. If all was successful, you should be able to see your new message!

Routes and Functions

Please note that routes must be unique – this means that you cannot have multiple routes in Flask pointing to /biotech. Similarly, the function beneath the route must also be unique in its name.

When deploying our models, we will work with similar architecture. However, the return statements will generally comprise either a UI for people to use or data for applications to consume. In the following section, we will explore this in a little more depth by using a Natural Language Processing (NLP) use case.

Using Flask as an API and web application

In Chapter 9, Natural Language Processing, we explored the use of the transformers library for the purposes of running text similarity search engines. By using this technology, we could have explored other models and implementations, such as sentiment analysis, text classification, and many more. One particular type of model that has gained a great deal of traction when it comes to NLP is the summarization model.

We can think of summarization models as tasks designed to reduce several paragraphs of text down to a few sentences, thereby allowing users to reduce the amount of time required to read. Luckily for us, we can implement an out-of-the-box summarization model using the transformers library and install that in our app.py file. Not only will we need to cater to human users (by using a UI), but we will also need to cater to web applications (APIs) that may be interested in using our model. In order to accommodate these two cases, we will need three files in total within our project to get us started:

• app.py: This is the main file in which the Flask framework and all NLP models are instantiated.
• styles.css: This is a CSS file that allows us to style the UI.
• index.html: This is an HTML file with a pre-built UI page that human users will interact with.

For better organization, let's add the CSS file to a directory called styles and the HTML file to a directory called templates.

When working with new Flask applications, we generally want to have a blank slate when it comes to the libraries we installed via pip. In other words, each Flask application should have its own virtual environment, where we only install libraries the application will need and use. We can accomplish this using virtualenv, which (ironically) can be installed using pip.

Once installed, we can use virtualenv on the command line to create a new environment for this project called .venv:

$ python38 virtualenv .venv

You can call your virtual environment anything you like, but most users generally default to the name in the preceding command. You will know this command was successful when you see a new directory in your current working directory with the specified name. We will now need to activate the environment, which can be a little tricky depending on the type of system you are using. Windows users can activate their environment using the following command:

> ..venvScriptsactivate

On the other hand, Linux and Mac users can activate their environments via the following command:

$ source .venvinactivate

You can confirm the environment was activated if its name appears on the left-hand side of the command line's current working directory. Go ahead and install flask and transformers, as we will need these libraries in the current environment.

With the environment set up and including the three files discussed, we should have a directory structure as depicted in Figure 11.8:

Figure 11.8 – The current folder structure of this project in VSC

With the project structure now in place, let's add some code to app.py. We can begin by importing some of the libraries we will need within this application:

from flask import Flask, jsonify, request, render_template

import json

from transformers import pipeline

import re

Now that the libraries have been imported, we can instantiate an instance of the Flask application just as before. However, we will need to specify the template folder this time:

app = Flask(__name__, template_folder='templates')

With the application instantiated, we can now create an instance of the summarizer model from the transformers pipeline class:

summarizer = pipeline("summarization")

Next, we can add our routes. We will first create a route to our home page, which displays the UI using the index.html file:

@app.route('/')

def home():

    return render_template('index.html')

We will then need to add two routes: one for the UI, and another for the API. There are many best practices that vary depending on the framework, the industry, and the use case. In most scenarios, api endpoints are generally preceded with the api word to distinguish them from others. Let's go ahead and create a route for the UI first:

@app.route('/prediction', methods = ["POST"])

def ui_prediction():

    """

    A function that takes a JSON with two fields: "text" & "maxlen"

    Returns: the summarized text of the paragraphs.

    """

    print(request.form.values())

    paragraphs = request.form.get("paragraphs")

    paragraphs = re.sub("d+", "", paragraphs)

    maxlen = int(request.form.get("maxlen"))

    summary = summarizer(paragraphs, max_length=maxlen, min_length=49, do_sample=False)

    return render_template('index.html', prediction_text = '" {} "'.format(summary[0]["summary_text"])), 200

Notice that within this function, we use the request.form.get function to retrieve the values from the form in the UI. In addition, we use some regular expressions to clean up the text, and then we summarize the contents using the summarizer model. Finally, we return the summary and the index.html file.

Let's now create the second route for the api:

@app.route('/api/prediction', methods = ["POST"])

def api_prediction():

    """

    A function that takes a JSON with two fields: "text" & "maxlen"

    Returns: the summarized text of the paragraphs.

    """

    query = json.loads(request.data)

    paragraphs = re.sub("d+", "", query["text"])

    maxlen = query["maxlen"]

    minlen = query["minlen"]

    summary = summarizer(paragraphs, max_length=maxlen, min_length=minlen, do_sample=False)

    return jsonify(summary), 200

Notice that in addition to taking the input data, cleaning the contents, and summarizing it, we can take the maxlen and minlen parameters directly from the JSON object.

Finally, we can go ahead and execute the code:

if __name__ == '__main__':

    app.run(debug=True)

With that, we have successfully developed the Flask application. Once deployed, you should be able to navigate to http://localhost:5000/ and start summarizing paragraphs of text! We can see an example of the application in Figure 11.9:

Figure 11.9 – A screenshot of the summarizer web application

In addition, we can use applications such as Postman (https://www.postman.com/) to test the API endpoint. Alternatively, we could use the requests library from Python to accomplish the same thing. In this case, we would need to make a POST request, add the URL, and then add the data in the form of a dictionary and the content type of the application/JSON:

{

    "text" : "Biotechnology is a broad area of biology, involving the use of living systems and organisms to develop or make products.

                              …

molecular biology, biochemistry, cell biology, embryology, genetics, microbiology) and conversely provides methods to support and perform basic research in biology.",

    "maxlen" : 60,

    "minlen" : 30

}

With the application now working, we managed to successfully create a solution that uses Flask to cater to both human users and other web applications. In the final chapter of this book, we will deploy this application to the cloud. However, one of the most important steps of doing this is providing a list of the libraries that need to be installed. Given that we have set up a virtual environment, we can easily transfer a list of these libraries to a requirements.txt file via pip:

$ pip freeze > requirements.txt

With that, you should now see a requirements.txt file in the same directory as app.py. It is important to ensure that the environment you use only contains the libraries you plan to use. This helps keep the application light and fast to use. In the following section, we will look at a more in-depth application – one that uses a previously trained model concerning the breast cancer dataset we saw earlier in this book.

Tutorial – Deploying a pretrained model using Flask

In the previous example of creating a Flask application, we saw how we can make use of the application in conjunction with a predictive model to deploy a solution to our end users. However, the model that we deployed was an out-of-the-box solution and not a model we developed ourselves. In this section, we will once again deploy a model within a Flask application; however, we use a model based on the cancer dataset we saw in Chapter 5, Understanding Machine Learning.

If you recall, the main idea behind this model was to take in a number of measurements for a given tumor, and based on those measurements, determine what the diagnosis will likely be, resulting in either Malignant or Benign. Within this application, we will enable users to interact with a trained model and enter measurements that the model will use to make a prediction. With this in mind, let's get started!

In the same way as before, go ahead and add a new folder and a new virtual environment to install the relevant libraries.

Using the same directory architecture and process as before, we can begin by importing the relevant libraries. Notice that we have added the pickle library here, as we will need to use the pickled models we previously created:

from flask import Flask, jsonify, request, render_template

import json

import pickle

import pandas as pd

from sklearn.preprocessing import StandardScaler

Our next step involves importing the two models we trained – the actual classification model and the standard scaler model we used for the data:

loaded_scaler= pickle.load(open("./models/ch10_scaler.pickle",'rb'))

loaded_clf= pickle.load(open("./models/ch10_rfc_clf.pickle",'rb'))

We can then define a predict_diagnosis function to clean up our code later when developing our routes. This function will take the input data in the form of a list, the scaler model, and the classification model:

def predict_diagnosis(inputData, scaler, model):

    """

    Function that takes a list of measurements, scales them, and returns a prediction

    """

    inputDataDF = pd.DataFrame([inputData])

    scaledInputData = scaler.transform(inputDataDF)

    prediction = model.predict(scaledInputData)

    return prediction[0]

Next, we will instantiate the Flask application while specifying the template folder:

app = Flask(__name__, template_folder='templates')

With these items taken care of, we can focus on our routes. First, we will create a home route that users will see first:

@app.route('/')

def home():

    return render_template('index.html')

Next, we will need a prediction route, just as before. The only difference here is that the number of input values will be greater, as we are working with a few more features now:

@app.route('/prediction', methods = ["POST"])

def prediction():

    print(request.form.values())

    radius_mean = request.form.get("radius_mean")

    texture_mean = request.form.get("texture_mean")

    smoothness_mean = request.form.get("smoothness_mean")

    texture_se = request.form.get("texture_se")

    smoothness_se = request.form.get("smoothness_se")

    symmetry_se = request.form.get("symmetry_se")

    input_features = [radius_mean, texture_mean, smoothness_mean, texture_se, smoothness_se, symmetry_se]

    prediction = predict_diagnosis(input_features, loaded_scaler, loaded_clf)

    prediction = "Malignant" if prediction == "M" else "Benign"

    

    return render_template('index.html', prediction_text = '" {} "'.format(prediction))

Finally, we can go ahead and run the application:

if __name__ == '__main__':

    app.run(debug=False, port=5000)

Upon running the model and navigating to localhost in the web browser, the application will appear. Go ahead and try making a few predictions using the UI, an example of which is displayed in Figure 11.10:

Figure 11.10 – A screenshot of the breast cancer web application

We can see that the model is able to take our input data, run a prediction, and return a result to the user. One thing we did not do here is create an API route for other web applications to interact with our model. As a challenge, go ahead and create this route, using the previous summarization application as an example.

Summary

In this chapter, we steered away from the development of models and focused more on how models can be deployed to interact with web applications. We investigated the idea of data transfer via APIs, and we also learned about some of the most common frameworks. We investigated one of the most common Python web application frameworks known as Flask. Using Flask, we developed an NLP summarization model that allows both human users and other web applications to interact with it and use its capabilities. In addition, we learned how to deploy previously trained models, such as those from scikit-learn.

In each of these instances, we launched our models locally as we developed their frameworks and capabilities. In the next chapter, we will make our model available to others by using Docker containers and AWS to deploy our model to the cloud.