Even the perfect code (if it exists) is useless if it is not being run. So in order to serve a purpose, our code needs to be installed on the target machine (computer) and executed. The process of making a specific version of your application or service available to the end users is called deployment.
In case of desktop applications, this seems to be simple—your job ends on providing a downloadable package with optional installer, if necessary. It is the user's responsibility to download and install it in his/her environment. Your responsibility is to make this process as easy and convenient as possible. Proper packaging is still not a simple task, but some tools were already explained in the previous chapter.
Surprisingly, things get more complicated when your code is not a product per se. If your application only provides a service that is being sold to the users, then it is your responsibility to run it on your own infrastructure. This scenario is typical for a web application or any "X as a Service" product. In such a situation, the code is deployed to set off remote machines that usually are hardly physically accessible to the developers. This is especially true if you are already a user of cloud computing services such as Amazon Web Services (AWS) or Heroku.
In this chapter, we will concentrate on the aspect of code deployment to remote hosts because of the very high popularity of Python in the field of building various web-related services and products. Despite the high portability of this language, it has no specific quality that would make its code easily deployable. What matters the most is how your application is built and what processes you use to deploy it to the target environments. So this chapter will focus on the following topics:
- What are the main challenges in deploying the code to remote environments
- How to build applications in Python that are easily deployable
- How to reload web services without downtime
- How to leverage Python packaging ecosystem in code deployment
- How to properly monitor and instrument code that runs remotely
The main requirement for painless deployment is building your application in a way that ensures that this process will be simple and as streamlined as possible. This is mostly about removing obstacles and encouraging well-established practices. Following such common practices is especially important in organizations where only specific people are responsible for development (developers team or Dev for short) and different people are responsible for deploying and maintaining the execution environments (operations team or Ops for short).
All tasks related to server maintenance, monitoring, deployment, configuration, and so on are often put to one single bag called operations. Even in organizations that have no separate teams for operational tasks, it is common that only some of the developers are authorized to do deployment tasks and maintain the remote servers. The common name for such a position is DevOps. Also, it isn't such an unusual situation that every member of the development team is responsible for operations, so everyone in such a team can be called DevOps. Anyway, no matter how your organization is structured and what the responsibilities of each developer are, everyone should know how operations work and how code is deployed to the remote servers because, in the end, the execution environment and its configuration is a hidden part of the product you are building.
The following common practices and conventions are important mainly for the following reasons:
- At every company people quit and new ones are hired. By using best approaches, you are making it easier for fresh team members to jump into the project. You can never be sure that new employees are already familiar with common practices for system configuration and running applications in a reliable way, but you at least make their fast adaptation more probable.
- In organizations where only some people are responsible for deployments, it simply reduces the friction between the operations and development teams.
A good source of such practices that encourage building easily deployable apps is a manifesto called Twelve-Factor App. It is a general language-agnostic methodology for building software-as-a-service apps. One of its purposes is making applications easier to deploy, but it also highlights other topics, such as maintainability and making applications easier to scale.
As its name says, the Twelve-Factor App consists of 12 rules:
- Codebase: One codebase tracked in revision control, many deploys
- Dependencies: Explicitly declare and isolate dependencies
- Config: Store config in the environment
- Backing services: Treat backing services as attached resources
- Build, release, run: Strictly separate build and run stages
- Processes: Execute the app as one or more stateless processes
- Port binding: Export services via port binding
- Concurrency: Scale out via the process model
- Disposability: Maximize robustness with fast startup and graceful shutdown
- Dev/prod parity: Keep development, staging, and production as similar as possible
- Logs: Treat logs as event streams
- Admin processes: Run admin/management tasks as one-off processes
Extending each of these rules here is a bit pointless because the official page of Twelve-Factor App methodology (http://12factor.net/) contains extensive rationale for every app factor with examples of tools for different frameworks and environments.
This chapter tries to stay consistent with the above manifesto, so we will discuss some of them in detail when necessary. The techniques and examples that are presented may sometimes slightly diverge from these 12 factors, but remember that these rules are not carved in stone. They are great as long as they serve the purpose. In the end, what matters is the working application (product) and not being compatible with some arbitrary methodology.