front matter

preface

The first time I toured a data center, I was fascinated by the entrance retinal scanner, flashing lights, cooling systems, and colorful wiring. Coming from an electrical engineering background, I could appreciate the complexity of managing hardware. I came upon the confusing concept of cloud computing when a company hired me to manage a private cloud platform. I no longer plugged in wires and crafted servers. Instead, I stared at progress bars in a user interface for thousands of servers and wrote terrible scripts to provision them.

At that point, I realized I needed to learn more. I wanted to automate more infrastructure and write more sustainable code that other team members could use. My learning journey reflected the growth of cloud computing and the DevOps philosophy. We needed to learn how to change and scale our infrastructure to keep up business innovation and avoid affecting critical systems! With the public cloud making it even easier to get infrastructure resources on demand, we could almost start treating our infrastructure as an extension of our software.

I traveled a rocky learning journey by becoming a generalist. I priced out public cloud migrations, paired with senior Java developers (the challenge that made me cry), applied design patterns and software development theory to code, tried out Agile methodology, and asked quality assurance and security professionals many questions. As I soaked up different perspectives and technical experiences, I tried to help other folks on their learning journeys as a consultant and, eventually, a developer advocate for open source infrastructure tools.

I decided to write this book because enough systems administrators, security professionals, and software developers expressed that they wanted to learn infrastructure as code (IaC) and needed a resource that organized patterns and practices for writing it. This book reflects everything I wished I learned earlier about IaC and the considerations and challenges of applying specific patterns and practices over others, agnostic of the tools and technologies.

I never expected the book to have so much detail. Whenever I released a chapter, I received a note from someone about something I forgot or a recommendation to expand one subject into a chapter. Many chapters cover topics that have entire books (or documentaries, even) dedicated to them but receive a general, high-level treatment in this book. I focus on the most important things you must know to apply the topic to IaC.

You might look at the examples in this book and ask, “Why not use this other tool?” I struggled to balance high-level theories with practical examples. The code listings generated spirited discussions from my reviewers and editors, many of whom suggested expansions or substitutions in a different language, tool, and platform! I tried my best to find a combination of languages, tools, and platforms to demonstrate the patterns. At the time of writing, you’ll find the code listings written in Python, deployed by HashiCorp Terraform, and run on Google Cloud Platform (GCP). Each code listing comes with a high-level description of the pattern and practice, which you can apply irrespective of language, tool, or platform.

I hope you read this book and find one or two patterns that help you write cleaner IaC, collaborate on IaC in your team, and scale and secure your IaC across your company. Please don’t expect to use every pattern and practice or apply all of them at once. You might feel overwhelmed! As you encounter challenges in your IaC, I hope you return to this book and reference a few more patterns.

acknowledgments

It takes a community to write a book, and the one that came together to help me is exceptional.

Thank you to my partner, Adam, who helped me make the time (and plenty of coffee) to focus and work on this book. Also thank you to my family, who encouraged me to pursue my interest in infrastructure. You provided words of encouragement and a listening ear, even if you didn’t understand the technical concepts I was trying to untangle.

I am very grateful to my editors at Manning—Chris Philips, Mike Shepard, Tricia Louvar, and Frances Lefkowitz—for their patience, encouragement, guidance, and recommendations. Thank you for staying so consistent in your feedback and commitment through some very rough drafts. I want to also thank the team behind the production and promotion of this book.

A big thank you to the reviewers who read my manuscript and took the time and effort to provide me feedback: Cosimo Attanasi, David Krief, Deniz Vehbi, Domingo Salazar, Ernesto Cárdenas Cangahuala, George Haines, Gualtiero Testa, Jeffrey Chu, Jeremy Bryan, Joel Clermont, John Guthrie, Lucian Maly, Michael Bright, Ognyan Dimitrov, Peter Schott, Ravi Tamiri, Sean T. Booker, Stanford S. Guillory, Steffen Weitkamp, Steven Oxley, Sylvain Martel, and Zorodzayi Mukuya, your suggestions helped make this a better book.

To the community team at HashiCorp, thank you for lending your opinions, reviewing concepts, and encouraging me to write. You all inspired me to keep writing throughout technical challenges and the occasional bout of imposter syndrome. And a special thank you to my colleague and technical proofer, Taylor Dolezal, who bravely checked my code and read the book in various iterations.

To the system administrators, testers, product managers, infrastructure engineers, business analysts, software developers, and security engineers I have ever worked with, you might recognize some of the patterns and practices we paired on, chatted about, or debated in this book. Thank you for teaching me something about infrastructure as code. You are part of the community that made this possible.

about this book

I wrote Infrastructure as Code, Patterns and Practices to help you write IaC that changes infrastructure resources without impacting critical business systems. The book focuses on the patterns and practices you can apply as an individual, team, or company across your infrastructure system. It focuses on high-level patterns and practices that you can apply to your IaC, while offering specific examples demonstrating implementation.

Who should read this book?

This book is for anyone (software developers, security engineers, quality assurance engineers, or infrastructure engineers) beginning to use cloud infrastructure and IaC and looking to scale it across their team or company. You will have written some IaC, run it manually, and created resources on a public cloud.

However, you now face the challenges of facilitating collaboration of IaC across your team or company. You must alleviate the friction of multiple team members and other teams making infrastructure changes and requesting updates for security, compliance, or functionality. While many resources introduce IaC in the context of a specific tool, this book generalizes the patterns and practices you can apply to a variety of infrastructure use cases, tools, and systems that will evolve over time.

How this book is organized: A roadmap

I organized this book into three sections with 13 chapters.

Part 1 introduces IaC and how you, as an individual, write it.

• Chapter 1 defines IaC and its benefits and principles. The chapter explains that the book has examples in Python, run by HashiCorp Terraform, and deployed to Google Cloud Platform (GCP). I also discuss the tools and use cases you’ll encounter in your IaC journey.

• Chapter 2 dives into the principle of immutability and how you can migrate existing infrastructure resources to IaC. It also covers the practices of writing clean IaC.

• Chapter 3 offers a few patterns for dividing and grouping infrastructure resources into modules. Each pattern includes an example and a list of use cases.

• Chapter 4 covers how to manage dependencies among infrastructure resources and modules and decouple them with dependency injection and some common patterns.

Part 2 describes how to write and collaborate on IaC as a team.

• Chapter 5 organizes the practices and considerations for expressing IaC in different repository structures and sharing it across your team.

• Chapter 6 provides an infrastructure testing strategy. It describes each type of test and how to write them for IaC.

• Chapter 7 applies continuous delivery to IaC. It covers a high-level view of branching models and how your team can use them to change infrastructure.

• Chapter 8 provides techniques to build secure and compliant IaC, including testing and tagging.

Part 3 covers how to manage IaC across your company.

• Chapter 9 applies immutability to infrastructure changes, including an example for blue-green deployments.

• Chapter 10 refactors a large body of IaC to improve its maintainability and mitigate the blast radius of failed changes to one codebase.

• Chapter 11 describes reverting IaC and rolling forward changes to the system.

• Chapter 12 addresses the use of IaC to manage cloud computing costs. It includes an example for cost estimation of IaC.

• Chapter 13 completes the book with practices to manage and update IaC tools.

You will find that many concepts build on each other throughout the book, and it may help to read the chapters in order if you have not previously practiced IaC. Otherwise, you can choose the sections that best apply to the challenges you face in your IaC practice.

Before you read individual chapters for specific concepts, you may want to read chapter 1 or appendix A first to understand how to read and run the examples. Appendix A offers additional detail on the libraries, tools, and platforms related to the examples, while appendix B provides answers to the exercises.

About the code

Because of the verbosity of infrastructure configuration, some code listings in the book do not include the entire infrastructure definitions for clarity. Chapters 2 to 12 include code listings as examples of their concepts.

The existing code listings use a combination of Python 3.9, HashiCorp Terraform 1.0, and Google Cloud Platform. The appendix includes more information on how to run the examples and their tools and libraries. I update the source code on GitHub for minor versions of the tools.

This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes code is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany any of the listings, highlighting important concepts. You can get executable snippets of code from the liveBook (online) version of this book at https://livebook.manning.com/book/essential-infrastructure-as-code. The complete source code is available for download from the Manning website at https://www.manning.com/books/infrastructure-as-code-patterns-and-practices, and from GitHub at https://github.com/joatmon08/manning-book.

liveBook discussion forum

Purchase of Infrastructure as Code, Patterns and Practices includes free access to liveBook, Manning’s online reading platform. Using liveBook’s exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook.manning.com/book/infrastructure-as-code-patterns-and-practices/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/discussion.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the author some challenging questions lest her interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website for as long as the book is in print.

About the cloud provider

I had a challenge deciding which cloud provider to use for examples. While Amazon Web Services (AWS) or Microsoft Azure (Azure) may be more popular choices at publication, they require many resources to be created. For example, their networking requires a network, subnets, routing tables, gateways, and security groups before you can use it. Instead, I decided to use Google Cloud Platform (GCP) as the primary cloud provider to streamline the number of resources you need to create.

Though I use GCP for the examples, the concepts, processes, and guidelines are intended to be agnostic, and you should be able to adapt them to other cloud providers. For readers who prefer to use AWS or Azure, each example includes information on the equivalent service offerings in these two platforms. In addition, some examples include equivalents in the code repo.

In chapter 1, you can read more about my reasons for using GCP, and how to adapt the examples for Azure and AWS. And in appendix A, you’ll find instructions for setting up and running the examples on GCP, along with tips for Azure and AWS users.

Other online resources

Refer to online resources for your specific IaC tool or infrastructure provider. Many of them provide examples of practices and patterns implemented in their tool.

about the author

Rosemary Wang works to bridge the technical and cultural barriers between infrastructure, security, and application development. She is fascinated with solving intractable problems as a contributor, public speaker, writer, and advocate of open source infrastructure tools. When she is not drawing on whiteboards, Rosemary debugs stacks of various infrastructure systems on her laptop while watering her houseplants.

about the cover illustration

The figure on the cover of Infrastructure as Code, Patterns and Practices is captioned “Bayadere,” a term meaning “a female Hindu dancer in India,” taken from a collection by Jacques Grasset de Saint-Sauveur, published in 1797. Each illustration is finely drawn and colored by hand.

In those days, it was easy to identify where people lived and what their trade or station in life was just by their dress. Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional culture centuries ago, brought back to life by pictures from collections such as this one.