One of the biggest benefits of Azure is the ability for organizations that adopt the public cloud to be exponentially agile. Services and solutions can be delivered in weeks and months instead of years. This accelerated life cycle for product and service delivery is based on agile concepts like continuous integration and continuous delivery (CI/CD).
In the previous chapter, we focused on how Azure changes the landscape for developers. In this chapter, we pick up where we left off and extend the discussion beyond the actual development of specific tasks and cover the development life cycle in Azure.
When we bring up DevOps when interacting with customers, we sometimes hear that the group we are meeting with are not developers. To successfully deploy modern cloud-based applications with maximum agility, all IT roles must work in concert. Therefore, the work among project managers, developers, data engineers, security personnel, infrastructure support, and all other IT professionals must collaborate to seamlessly deliver a successful solution. This entails complex coordination of activities that must be completed on time and per specifications.
This increased collaboration between the different IT groups is known as DevOps. DevOps itself is a compound term comprised of the words developer and operations , which most accurately highlights the interdependency between developers and IT professions.
Azure DevOps is the implementation of the industry’s practice of DevOps. The official definition of Azure DevOps is “the union of people, process, and technology to continually provide value to customers.” So, although they share the same name, when we refer to “Azure DevOps,” we mean the Microsoft service offering. When we simply say “DevOps,” we are talking about DevOps as a discipline.
What Is Azure DevOps?
Azure DevOps is both a product and a practice. From a practice standpoint, Azure DevOps is designed to support the goal of breaking down the silos that we discussed earlier. It streamlines the software development/deployment life cycle (SDLC) by providing tools that facilitate planning, development, deployment, operations, and communication.
Azure Boards
Azure Repos
Azure Pipelines
Azure Test Plans
Azure Artifacts
Although we sometimes have to clarify whether we are talking about DevOps as a practice or DevOps as a set of tools, the thought behind blurring the line makes sense because it is beneficial to think of the tools and the practice as one and the same.
In this chapter, we focus on Azure Repos and Azure Pipelines as we demonstrate how to implement the basics of CI/CD. We are publishing information and exercises for Azure Boards, Azure Test Plans, and Azure Artifacts at the book’s GitHub repository (https://github.com/harris-soh-copeland-puca).
Why Azure DevOps
As with any technology, we should take a moment to discuss why Azure DevOps is important from a business standpoint.
Predictability and Repeatability
In previous chapters, we looked at virtual machines and infrastructure as code, and how a cloud-first approach changes your IT operations. In Chapter 20, we also looked at how developers need to shift and transform when developing applications for the cloud. The number of services and capabilities come with an exponential increase in the number of variables in configuration at the application and infrastructure level. Furthermore, these two areas are interdependent, so a high level of coordination needs to occur between the two groups. Manual deployment of infrastructure and code is no longer a viable or scalable approach. With automation, we can ensure a consistent and predictable deployment every time.
Agile Deployment and Continuous Improvement
The term agile is used very frequently these days, and it no longer applies to only application development. The primary premise of the Agile practice is the rolling out, or retirement, of capabilities and changes in smaller increments in scope but more frequently. For application developers, this is now a fairly common practice.
For IT pros, the fast pace of deploying and retiring infrastructure is not a traditional practice; with significantly less hardware to provision and modern infrastructure (e.g., serverless apps and web apps), the same type of agility needs to be present at the infrastructure layer to optimize Agile across an IT organization.
Planning, Collaboration, and Workflow
If the IT pros and application developers in your organization tend to operate as silos, which is common in large and complex organizations, an area of improvement is the increased efficiency through better planning, collaboration, and some form of workflow that ensures timely deployment of resources. Through proper planning and the use of modern workflow tools, application developers are empowered to securely deploy their infrastructure during the development and testing phases. These infrastructure requirements can then be better communicated to IT pros when it comes time to formally provision infrastructure resources into production.
If these areas are part of your IT transformation journey, and they should be, and then services in Azure DevOps help you achieve those goals. How would they? This question is best answered as we explore each of the services in Azure DevOps. Once you have a better understanding of the roles that each service plays, we conclude this chapter with a discussion on the issues that you need to consider as part of designing and implementing Azure DevOps.
For now, it is important to note that while your organization may gain the most benefit by implementing Azure DevOps, there is no requirement to implement all five services, and there is no particular order of implementation. Although, we are presenting the services in the order that they are generally being implemented and keep in mind that there may be some dependencies between services when it comes to certain capabilities.
Provisioning Azure DevOps
- 1.
Go to Azure DevOps at https://devops.azure.com and click Start for a free link to provision a new DevOps instance, or click the Sign in link if you already have Azure DevOps.
- 2.
For this exercise, we generate a new demo project that already has content that we can use. Open a new browser tab and go to https://azuredevopsdemogenerator.azurewebsites.net/ to generate the content.
- 3.
Give the project a name. We recommend picking a name1 that does not contain any spaces and is reasonable in length.
- 4.
Use the drop-down box to select your organization, which should show the DevOps instance you signed in or created in step 1. We chose the default SmartHotel360 template, but you can click Choose template to see the different scenarios and pick the one that is most relevant to you.
- 5.
Leave the GitHub forking checkbox off because we are going to be using Azure Pipelines. However, since GitHub integrates well in Azure DevOps, you can choose to deploy another project with this checkbox checked so you can explore GitHub’s integration with Azure DevOps.
- 6.
Click Create Project.2
- 7.
You should see the deployment progress of the demo project. Once the deployment is successful, you see a Navigate to project button.
- 8.
Take a quick look at the project stats to get an idea of this project’s artifacts, activities, and members.
- 9.
Click the Invite link to add another user and enter the user’s email address. If you are doing this exercise alone, use a different email account so that you can simulate a second user to collaborate with. Then click Add user and click Add on the next screen if you are not going to invite additional users. At some point, remember to accept the invite that was sent to the user’s email address.
Azure Repos
We start exploring Azure DevOps with the Azure Repos service because we believe that this is a service that any organization should adopt.
Azure Repos securely stores an organization’s IT assets, including source code, scripts, documentation, and any or all your IT artifacts. It includes the ability to control access and track changes to those assets over time, with the ability to keep historical information in the event you need to roll back changes. If you have heard or used GitHub before, which is also now a Microsoft company, Azure Repos is pretty much the same thing.
You can integrate GitHub with Azure DevOps; so if you already have assets stored in GitHub, you do not need to migrate them to Azure Repos. For example, when we cover CI/CD in Azure Pipelines, we highlight GitHub’s integration with Azure DevOps.
Hands-on with Azure Repos
- 1.
In the AzureHotel (Hotel 360) project, click Repos in the menu on the left.
Repos is short for repository, so this is where all the artifacts related to this project reside. Artifacts are not only source code but can contain documentation, supporting files, and a markdown file called README.md. What you see here is a well-documented README.md that provides not only the description of this project but also instructions and screenshots. The general rule of thumb for README.md is to be as detailed as possible.
- 2.
Locate and double-click .Gitignore. .Gitignore is a plain text file that is used for version control purposes. It tells Git which files it should ignore for committing and synchronization purposes. Generally, you can consider this a working file that Azure DevOps uses, and it is best you do not modify it manually.
- 3.
Click the three ellipses in the top-right corner of the dashboard. You are presented with the options fork, download, or upload content to this repo.
Git clones the repo. The Clone button provides that functionality. Azure Repos’ full source-control capabilities are also integrated into Visual Studio. Note that Azure Repos work with any Git compliant client.
- 1.
First, you need the URL for this repo, so click Repos in the menu on the left, if you are not already there.
- 2.
Copy and paste the URL into a temporary location, like WordPad. The URL should look something like https://dev.azure.com/<username>/<RepoName>.
- 3.Create a personal access token. Click the Account Profile icon on the top-right corner, and select Personal access tokens, refer to Figure 21-1, from the drop-down menu.
Figure 21-1Creating personal access tokens in GitHub desktop
- 4.
Click the New Token option and provide a name. Note that this is equivalent to a username than a description.
- 5.
Define an expiration date and review the granularity in access control you can achieve by looking at the options under Scopes ➤ Custom defined.
- 6.
For this exercise, select Full access, and then click Create.
- 7.
Copy and paste the generated token into WordPad as well. You need to use it in the GitHub desktop later. Note that you are unable to get this token again; you must regenerate a new token if you need to reauthenticate using this token.
- 8.
Once you click OK, you see the new personal access token in the list of tokens. Select the token and note that you can revoke, edit, or regenerate a token.
- 9.
Launch your preferred Git client, like GitHub desktop, which is used in this exercise.
- 10.
Click File and select Clone repository.
- 11.
Paste the URL of the Azure Repo and select the local directory to clone the repo to. Then click Clone.
This does not enable the GitHub desktop to work with this Azure Repo. All the synchronization, push, pull, and branching operations in GitHub desktop are now fully integrated with Azure Repos in this particular repo.
This exercise walked you connecting a repo in Azure Repos to a Git client. What if you have an existing project that is located somewhere else, like GitHub or GitBucket? We can also import such projects into Azure Repos. Importing a repo includes all the project’s artifacts as well as its revision history.
Importing a project from another Git repository to Azure Repos includes all artifacts, timelines, and its revision history at the time of the import. It does not continuously synchronize both repos, although there is a way to do so. This exercise does not cover synchronizing repos, but we do cover this topic in the book’s GitHub repo.
Importing a Git Repo
- 1.
Go to the Git containing the repo that you wish to clone into Azure Repos and retrieve the repo’s URL.
- 2.
Create a new Azure DevOps project, but without deploying any demo content this time.
- 3.
Click Repos in the menu on the left and select Import under Import a repository as referenced in Figure 21-2.
- 4.
Select Git as the repository type and paste the URL of the Git repo from step 1. Since our Git repo is a GitHub repo for this exercise, it is a GitHub.com URL.
- 5.
Check the Requires Authentication box if the repo requires it (e.g., a private repo in GitHub), or leave it unchecked if it is a public repo.
- 6.If you checked the box indicating the source Git repo requires authentication, the option to provide a username and password/PAT appears; otherwise, click Import.
Figure 21-2Requires authentication checked, exposing Username and Password/PAT boxes
- 7.
Provide your Git username and password, and then click Import.
Note If you are using GitHub with multifactor authentication (MFA), providing a username and password is insufficient. Go to GitHub, click your user icon on the top right, and select Settings. Then select Developer settings and generate a new token. Use it in place of the password (keep using your GitHub username). If you are using some other Git repository with MFA, the approach should be similar.
- 8.
Azure DevOps start importing all the items from the source Git into Azure Repos, including branches and the revision history.
Repository Operations
An exhaustive discussion about Git operations and concepts is beyond the scope of this book, but we want to address a few core principles that are important as you implement Azure Repos.
Azure Repo Goal
The goal of adopting Azure Repos and Azure DevOps as a whole is to streamline your development workflow through better collaboration. If you are in a development environment with many developers, this coordination is very important, so everyone does not step on each other’s toes. If you are a lone developer on a project, you may want to test things and be able to roll back changes if needed. Then, at some point, you may want to share your project with users and other developers, and the best way of doing that is by inviting them to the repo and subsequently tracking their activities against the artifacts in the repo. With these goals in mind, let’s explore the capabilities of Azure Repos that can help you achieve these goals.
Commits
- 1.
In Azure DevOps, open the Hotel 365 demo project and click Repos.
- 2.
For this exercise, you are going to select Visual Studio as your integrated development environment (IDE) to clone this repo to.
- 3.
Azure DevOps launches Visual Studio, and an Azure DevOps dialog box opens, showing the URL of the repo to clone from, and the local path to clone to. Modify the local path if desired, and then click clone.
- 4.
After the project is cloned to your local machine, locate README.md from Solution Explorer. Note that there is an icon of a lock next to the file name. This indicates that the file is under source control.
- 5.
Open README.md and make a small change. For this exercise, we are going to add a comment to line 10. Notice that the lock icon next to the filename has now changed to a red check. The red check is an indicator that a change has been made to the file.
- 6.
Click File in the Visual Studio menu and select Save All. Then right-click README.md and select Compare with Unmodified. Note that the changes you made in step 5 are highlighted.
- 7.
Right-click README.md again and select Commit.
- 8.
Provide a comment about the changes you made, and then click Commit All. This commits the changes that you just made to the local master branch on your local machine.
- 9.Note that at the top of the Team Explorer tab, which is where you should be, you are told to Sync your changes with the server if you want to implement the changes you made in step 5 with the Azure Repo for this project, as seen in Figure 21-3. Click Sync.
Figure 21-3Sync changes from the local repo to the remote repo
If you do not see this message or a message to sync all changes in all files to Azure Repos, click the Home icon in Team Explorer and select Sync, as seen in Figure 21-4.
Selecting Sync from the Home menu in Team Explorer
- 10.
The changes are pushed to Azure Repos. Go to Azure DevOps and refresh the repo page. You see the changes being reflected.
- 11.
Click Commits in the Repos menu. You should see the commit that was initiated in step 9.
Branches
A branch is a version of a project. Every project has a Master branch, which represents the current version of all artifacts in a particular location. The locations are usually remote (on the Git server, in this case, Azure Repo) and local (on your PC).
When you initially clone a project, a copy of the master on remote is cloned to a master branch on your PC.
- 1.
In Azure DevOps, select Repos.
- 2.Note that there are four branches: dev, master, searchcities, and test as referenced in Figure 21-5. The default branch that is currently selected is master.
Figure 21-5Branches shown in Azure Repos
- 3.
Look at the Behind | Ahead column. This column indicates the number of commits that have occurred in the respective branches and have not been reflected in master. For example, the searchcities branch is 14 commits behind and 2 commits ahead from master. Translated, this means that the artifacts in this branch have been modified twice since it was last synchronized with master on June 7, 2018 by Sachin Raj. Also, since June 7, 2018, master has received 14 commits, so that makes searchcities behind by 14 commits. So, there are changes in the searchcities branch that are not reflected in master, and there are even more changes in master that are not reflected in searchcities.
- 4.
Go to Visual Studio and click the Home icon in Team Explorer. Select Branches.
- 5.Expand remotes/origin, as seen in Figure 21-6. Notice that the branches are tracked in Azure Repos. Remotes/origin, as the name implies, is also the source (origin) from where you cloned this project. The master branch that is outside of the remotes/origin folder is the branch that exists on your PC, and that was where remote/origin was cloned to.
Figure 21-6Remotes/origin as seen from within Visual Studio
- 6.Double-click the searchcities branch in remotes/origin. This clones a copy of the searchcities branch to your local PC. Note the Output window to see which branch you have checked out, as seen in Figure 21-7.
Figure 21-7Output showing branch being switched
If you look at the contents of README.md in Visual Studio or via the command prompt, you notice that it changes accordingly depending on whether you have selected master or searchcities as the current branch in Visual Studio. This is because README.md differs between the two branches.
Now that we have a better understanding of branches, how would you use them?
One way may be to have different branches created for different contributors, perhaps naming each branch according to the contributor or developer. But this may still cause conflicts if two groups in different branches are working on the same concepts. Such conflicts would require intervention as to which changes to keep.
A better way uses feature branches, which are branches created to focus on specific features of a project. Look at the branches in the Hotel 360 repo. From the names of the branches, there seems to be a mix of how the branches are being utilized. The “dev” branch is presumably a work in progress, but its name does not specify what is being worked on. The branch named “Test” is probably a copy of the project that is being tested, but it is nine commits behind origin/master and one commit ahead, so the artifacts in the test branch are not the most current. Both these branches are named after activities, whereas the searchcities branch is named after a possible feature. One that perhaps incorporates the ability to search city names or zip code.
There is also a folder named features. Expanding the features folder reveals a branch named bot-widget. Now we are taking it a step further in terms of organizing our branches into folders.
- 1.
Click the New branch button at the top-right corner of Azure Repos.
- 2.
When prompted for the name of the branch, type <FolderName>/<BranchName>. For this exercise, we typed bugfix/SlowCitiesLookup. Then click the Based on drop-down menu and select master, which is the default.
- 3.
Click Create.
- 4.Your repo should look similar to Figure 21-8. Notice the new branch you created within the folder.
Figure 21-8New branch within a folder
Feature branches that are merged back to master3
Release branches are long-lived and have feature branches4
A more detailed discussion about adopting a good Git branching strategy is at https://docs.microsoft.com/en-us/azure/devops/repos/git/git-branching-guidance?view=azure-devops.
Hands-on with Azure Repos: Adding an Existing Project to Azure Repos
- 1.
From Visual Studio, open the Speech Sentiment project from Chapter 15.
- 2.
Click the File menu and select Add to Source Control.
- 3.
Click the Home icon in Team Explorer and click Sync.
- 4.Since you just added this project to Source Control, you are prompted for a destination to publish to. Click Publish Git Repo, as referenced in Figure 21-11 under the section labeled Push to Azure DevOps Services.
Figure 21-11Push to GitHub, Azure DevOps, or any other Git repo
- 5.Select or add your Azure DevOps login, and then pick the organization associated with the login, provide a name for the project, and click Publish Repository as referenced in Figure 21-12. For this exercise, name this project SentimentAnalysis.
Figure 21-12Creating and pushing to the Azure Repo
- 6.
Once the push is complete, go to Azure DevOps and browse your projects. You should see SentimentAnalysis project in the list.
There are many more features in Azure Repos that we do not have time to cover in this book, such as Repo policies and tags, but what was covered should help you get started with Azure Repos and explore the other features on your own.
Azure Pipelines
Azure pipelines are tasks that involve building, testing, and deploying your project. It is commonly known as the enablement of continuous integration/continuous delivery (CI/CD).
Azure Pipelines work with Azure Repos and any Git service, such as GitHub and BitBucket.
Key Concepts
Anatomy of a pipeline
A pipeline is comprised of one or more stages. When a pipeline is triggered manually or automatically, it executes each stage that is within the pipeline.
A stage in a pipeline is a container for one or more jobs.
A job has one or more steps, and a step can either be a task or a script. A task is the smallest building block in a pipeline.
For a job to execute its tasks or scripts, an agent needs to be assigned. The agent is responsible for executing the tasks and scripts.
Hands-on with Azure Pipelines: CI/CD
- 1.
From Azure DevOps, open the SentimentAnalysis project from the previous exercise and click Pipelines.
- 2.
Select Builds and click New pipeline.
- 3.
Azure Pipelines provide you with two options to create a new pipeline. You can have it generate a YAML file defining the pipeline, or you can choose a classic editor. For this exercise, we used the classic editor, so click the Use the classic editor link at the bottom of the screen.
- 4.
Select the SentimentAnalysis repo as the source, and then click Continue.
- 5.
Click Azure Web App for ASP.NET to select this template, and then click Apply.
Note This template builds the application and then deploys it to an Azure web app. Therefore, it is a complete CI/CD template because changes are incorporated when the master is changed (CI) and then deployed to a targeted Azure web app (CD). You could have created a CI-only pipeline if you had selected the ASP.NET template. You could also have created a CD-only pipeline. The reason why you want to do this instead of using a single CI/CD pipeline is to stage deployment separately from new updates (e.g., deploy a new version every quarter or at the end of every week).
- 6.Based on the template selected in step 5, the editor defines the tasks needed for this build job. It also highlights tasks that require attention or more information, as seen in Figure 21-14.
Figure 21-14Tasks in the build stage of the pipeline
- 7.To see the details of a task, click it, as seen in Figure 21-15. The circle at the end is checked, which indicates that the task is selected. The pane on the left displays the parameters and details related to that task. Tasks are carried out from top to bottom. To reorder the sequence of tasks, hover over the dotted bar of a task, and the cursor changes to one with up/down arrows. Click, hold down, and drag the task to rearrange the order.
Figure 21-15Select or move a task
- 8.
Select the Azure Web App Deploy task. It requires information about the subscription and web app you wish to deploy this application to. We use the web app that was created in Chapter 12. Select the subscription and the web app from the drop-down menu. Set the app type as a web app on Windows.
- 9.
Select Triggers at the top of the screen.
- 10.
Ensure that the Enable continuous integration box is checked. By default, it should be. Be aware that you can add a schedule that restricts CI to a particular day and time, but we do not do it for this exercise.
- 11.
Click Save & queue and select Save & queue from the drop-down options.
- 12.
Provide a Save comment for this pipeline. Then click Save and run.
- 13.An agent is provided, and the job runs for the first time. The status of each task is shown. You can see which tasks completed or failed and which are still pending, as seen in Figure 21-16.
Figure 21-16Verbose log for a running job
- 1.
In Visual Studio, make a small change to the SentimentAnalysis project. For this exercise, all we did was change the Default.aspx page so that there is a minor change in the text on this page.
- 2.
Click File and select Save All.
- 3.
Within Visual Studio, in Team Explorer, click the Home icon and select Changes.
- 4.
Enter a description for this commit or use the following commit message: “A minor change to the header for Default.aspx page to Chapter 21 CI Demo 2.” Then click Commit All.
- 5.
After the commit, sync the changes with the server as suggested at the top of Team Explorer.
- 6.
In Outgoing Commits, you should see the commit that you named in step 4. Click Push to send the changes up to Azure Repo.
- 7.Go to Azure DevOps and click Pipelines. You should see the latest commit being built. The description of the commit you provided in step 4 describes the build, as referenced in Figure 21-17. Note the build number as well.
Figure 21-17Pipeline automatically triggered by a change in the master
- 8.
Open a browser and view the website. It should now reflect the changes you made in step 1 without you having to publish the project like you had to in Chapter 12.
- 9.
Go back to Visual Studio and in Team Explorer, click the Home icon, and select Builds.
- 10.You should see the build history, as referenced in Figure 21-18 for this project based on the build number. The build number that you noted in step 7 should be the latest.
Figure 21-18Build history as seen in Visual Studio Team Explorer
There is a lot more that can be accomplished through Azure Pipelines. You can build complex automation, configure notifications, connect to other Git services like GitHub instead of Azure Repos, and managing libraries and releases. Covering the breadth of its capabilities is beyond the scope of this book, but you can expand your knowledge through Microsoft’s extensive online documentation or other sources. This chapter and the exercises covered here have provided you with the necessary foundation and hands-on experience to fully understand and explore Azure Pipelines and CI/CD.
The next step we recommend that you do is set up the governance for CI/CD. We saw how to add Team Members to an Azure DevOps project, so governance that identifies roles such as reviewers and administrators ensure that nothing gets inadvertently pushed to the production environment.
A good resource to explore is Azure DevOps Projects, which sets up everything you need for developing, deploying, and monitoring your application. Learn more about it at our GitHub repo at https://github.com/harris-soh-copeland-puca/azure-docs/blob/master/articles/devops-project/overview.md.
Summary
In closing, Azure Repos and Azure Pipelines are the two services that facilitate CI/CD. With proper governance in place, the product or service delivery life cycle can be significantly optimized.
Although we did not cover the other three Azure DevOps services—Azure Boards, Azure Test Plans, and Azure Artifacts, you will benefit from exploring these services because they play significant roles in facilitating modern project delivery.
We provide additional information on our GitHub repo, including examples and exercises for the other features and tools in DevOps, such as test plans, artifacts, boards, and compliance.
Managing the production of this book using Azure Boards
Azure Test Plans provide the quality assurance needed at the sprint or milestone stages of a project. Azure Test Plans allow manual or automated testing or both. Proper and timely testing ensures not only a quicker to market product, but also ensures that quality is not compromised as a result of the shortened timelines in the development life cycle.
A developer is probably aware of packages and package management and deployment services like NuGet to roll out and share code. It is a foundation for any modern development platform. In fact, we used a NuGet package for our COGS hands-on exercise in Chapter 15. Azure Artifacts provides the ability to create and distribute packages to and from feeds like NuGet and npm. It introduces the concept of connecting multiple feeds, private and public, and allowing you to organize and control access to your packages. Reusing code through packages is another key strategy in modern application development that shortens the development life cycle. Packages that have been tested ensure that quality is consistent throughout the projects that incorporate such packages.
Azure DevOps is a fully integrated suite of tools that help streamline the development life cycle, and it is not just for developers. The integrated services bring together project managers, stakeholders, and IT professionals from the planning phase through delivery and beyond.