How many bug fixes/features should be included in a single release?

Since releases are expected to happen frequently, each release will likely be small. You probably can’t box up 50 changes to a single service component in a week. We hear most organizations have a practice of limiting the number of changes. Netflix, for example, tells teams to make only one significant change per release. For example, if your team needs to refactor some of the internal code and start using a new data store module, that would be two releases.

The biggest reason for limiting the number of changes in a release is to reduce uncertainty. If you release a component that contains multiple changes, the uncertainty is increased by the number of interactions that occur between those changes. The mathematical discipline of graph theory provides a simple formula to calculate those interactions: n(n–1)/2. Based on this, if you release a component that contains 5 changes and it causes problems in production, you know that there are 10 possible ways in which these 5 changes could interact to cause a problem. But if you release a component with 15 changes there is a potential for over 100 different ways in which those changes can interact to cause problems—and that’s just internal problems.

Limit the number of changes in each release to increase the safety of each release.

When do I know our microservice transformation is done?

Technically, creating and maintaining a vital information system is never “done.” And that is also true for one built in the microservices way. In our experience some architects and developers spend a lot of time trying to identify the ideal solution or implementation model for their system design. It rarely works. In fact, one of the advantages of microservices is that change over time is not as costly or dangerous as it might be in tightly coupled large-scope release models.

Trying to perfect “the system” is an impossible task since it will always be a moving target. Often arriving at some “final state” marks the start of accumulating “technical debt”—that status where the system is outdated and difficult to change. It helps to remember that everything you build today will likely be obsolete within a few years anyway.

Since doing things in the microservices way means lots of small releases over time, you’ll always be changing/improving something. This means you get lots of “done” moments along the way and, in keeping with the theme of microservices, are able to effect change over time “at scale.”

How do I know if my organization is ready for microservices?

You can start by assessing your organization’s structure and associated culture. In a 1967 paper titled “How Committees Invent”, computer scientist Mel Conway argued that the design of a software system will mimic the communication structure of the organization that produced it. “Conway’s law,” as it came to be known, has had a recent revival in microservices circles. There is good reason for this. As discussed earlier, the majority of microservice architecture pioneers began their quest for faster software delivery by optimizing organizational design before addressing the software architecture. Given this progression, these organizations landed on microservice architecture as a style that aligned with their small, business-aligned teams, thus revealing the wisdom of Conway’s decades-old assertion.

However, many organizations now evaluating microservice architecture are not following the same path. In those cases, it is crucial to look at the organizational structure. How are responsibilities divided between teams? Are they aligned to business domains, or technology skillsets? At what level of the organization are development and operations divided? How big are the teams? What skills do they have? How dynamic is the communication and interaction between the teams who need to be involved in the delivery lifecycle? In addition to these organizational variables, you should evaluate the culture. How is power distributed between the teams? Is it centralized at a high level, or decentralized among the delivery teams? Answering these questions will help you understand what impacts these organizational factors will have on your adoption efforts and resulting successes.

The ideal organization for microservices has small, empowered teams responsible for services that align with specific business domains. These teams feature all of the roles necessary to deliver these services, such as product owners, architects, developers, quality engineers, and operational engineers. The teams also need the right skills, such as API design and development, and knowledge of distributed applications. Organizations that mismatch any of these characteristics will pay a toll when attempting to apply microservice architecture. Teams that are not empowered will experience delays waiting for decisions to be made above their heads. Lack of business alignment will lead to cross-team dependencies, causing further delays and architectural deviation. Teams that are too large create incomprehensible code bases that impede and delay future changes. The higher up the divide between development and operations, the less motivated the operations group will be to automate and optimize and the less diligent developers will be in the operability of their software. Lastly, if the team doesn’t have the right skills to build API-fronted services using distributed concepts, costs could go up to cover training and/or contract hiring, or the solution could be dragged away from the microservices approach as existing resources retreat to their technological comfort zones.

How do I introduce change?

...the challenge is to find small changes that can unfold in a way that creates large effects...

Gareth Morgan, author of Images of Organization

If you aren’t working in a greenfield environment, chances are you’ll have inherited an existing organizational design as well. Making changes to a working organization is nontrivial and carries a much greater risk than toying with a solution architecture. After all, if we make a mistake when refactoring our software we can always undo our changes, but when we make a mistake when redesigning the reporting structure in an organization the damage is not so easily undone.

You’re unlikely to hear anyone refer to organizational design using the term refactoring. You’re far more likely to hear the term organizational transformation, and if it’s an area of interest to you, there is a wealth of material that can help you along the path to change. But within the context of this book, the term refactoring makes a lot of sense. What we are primarily interested in is a method for making changes to the organization in a way that is safe. Refactoring can help us with that.

In order to apply a refactoring strategy to the organizational design, you’ll need to:

1. Devise a way to test changes

2. Identify problem areas in your organizational design

3. Identify safe transformations (changes that don’t change existing behavior)

Refactoring the organization won’t help you do something you don’t already know how to do. Your goal should be to do the same things, but improve the design of your organization so you can do them better.

When you refactor an application you can measure and observe the performance of the application; you can audit the source code, and you can comb through logs and determine where most of the problems occur. But when dealing with the processes and people that make up an organization things are a bit less black and white. To successfully identify where the refactoring opportunities are within the organization, you’ll need to find some way to model the existing system in order to analyze and measure its performance.

SoundCloud’s Phil Calçado has written about using a lean management technique called value stream mapping as an initial step toward microservices. Value stream maps are a great tool to use for this activity, but use whichever method you are comfortable with to get a better understanding of how work is being done. Flowcharts, business process models, and activity diagrams can all do the job in the right hands.

No matter how you do it, the goal in this step is to identify how software changes are introduced to the system, who implements those changes, and the type of coordination that is required for those changes to take place. For the microservice system we are especially interested in identifying opportunities to improve the efficiency of change. Gaining a total understanding of how your organization works may be too large of an initial investment to undertake, so in practice you may need to focus only on the changes that occur the most often for the components that are the most volatile.

In particular, you should be looking for the bottlenecks that cause change to be expensive. Which processes result in a queue or backlog? Are there particular centralized functions such as audits, code reviews, and gating procedures that cause teams to have to wait? Are there any parts of your process flow that make it difficult for multiple changes to be introduced at the same time due to resource availability or a need for serialized process execution? Finding these bottlenecks will help you identify good candidates for process and organizational refactoring as they should yield a large benefit to the changeability and speed of release for the system.

Can I do microservices in a project-centric culture?

A hallmark of a microservices organization is that the teams that implement a feature, application, or service continue to support, improve, and work on the code for its lifetime. This product-centric perspective instills a sense of ownership of the component and reinforces the idea that deployed components will constantly be updated and replaced. This notion of ownership is important enough that Martin Fowler has made “products not projects” one of the primary characteristics for a microservice application.

Typical project-centric cultures operate differently. Teams are formed to address a particular problem (e.g., create a new component, add a feature, etc.) and disbanded when that problem is solved. Often a good deal of knowledge about both the problem and the solution gets lost when the team disbands. And, if there is a need to re-address the same problem, or make additional changes to the same component, it may be difficult to re-create the team or recover the lost knowledge. These challenges usually mean changes happen less often and are more likely to result in bugs or partial solutions.

In truth, it is quite difficult to adopt the microservice style if you need to operate in this type of culture. If changeability and speed of release are important properties for your system, the long-term goal should be to transition to a style of building that encourages team-based ownership of components.

Can I do microservices with outsourced workers?

A particular challenge for large companies trying to incorporate the ideal microservices system is the trend toward outsourcing technology services. The act of hiring an outside company to perform development and operations activities using workers who are external to the organization seems at odds with the culture and organizational principles we’ve described in this book. But with the right outsourcing structure, a microservice system may lend itself well to being developed by an external organization.

By embracing a decentralized way of working and standardizing on the output and processes of service teams (containers and APIs), the outsourced development team can be given enough autonomy to build a service that meets the capability requirements of the owning organization. But this is only possible if the outsourced team conforms to the principles that exemplify the microservices way: the teams should be the right size, built to last for the perpetuity of the life of the service, and composed of workers who are skilled, experienced, and capable enough to make good design and implementation decisions autonomously.

In addition to team composition, the microservice designer should acknowledge that a cross-pollination of cultures occurs whenever outsourcing is conducted. The implication is that a desired organizational culture cannot simply be adopted by the outsourced team, nor can the buying organization avoid having their culture changed by the intermingling of work. This means that culture becomes an important element in deciding which companies or people should be chosen to support the outsourcing model.

Ultimately, the selection process for a microservices outsourcing model cannot be optimized purely for low-cost work. You will need to carefully select a partner who is amenable to the cultural traits you are looking for and possesses aspects of culture you’d like to incorporate into your own system. The deal must also be structured to incentivize the team dynamic that works best for building applications the microservices way—teams should be dedicated to services, workers should be capable of working autonomously, and speed of high-quality delivery should be the primary metric for success.

What kinds of tools and technology are required for microservices?

Chapter 4 introduced the importance of the microservices platform and Chapters 5-6 identified some particularly important tools that you can use to “power up” your platform and maximize both speed and safety of change. But these aren’t the only tools you’ll need if you want to improve your chances of succeeding with microservices.

The ideal technological environment for microservices features cloud infrastructure, which facilitates rapid provisioning and automated deployment. The use of containers is particularly useful to enable portability and heterogeneity. Middleware for data storage, integration, security, and operations should be web API-friendly in order to facilitate automation and discovery, and should also be amenable to dynamic, decentralized distribution. The ideal programming languages for microservices are API friendly as well, and should be functional while also matching the skillsets of your organization. It is particularly useful to provide tools for developers that simplify their tasks yet incorporate constraints that encourage good operational behavior of their resulting code.

Straying from these technological traits can lead to adoption issues. Lack of cloud infrastructure will lead to deployment delays and inflexible scaling. Lack of containers—or reliance on older virtualization or app servers—could increase the cost of resource utilization and lead to quality issues resulting from inconsistencies across environments. Middleware that assumes strict centralized control will break the decentralized organizational model and challenge the provisioning of ephemeral environments. If used in a decentralized model, this specialized middleware could also lead to skill challenges in the organization if every team is required to cultivate expertise. Centralized or segregated data breaks the organizational model as well. It also slows down delivery and impedes evolvability. Lack of developer tooling consistency could lead to duplicate work and lack of visibility or resiliency in the overall system. Finally, a large dependency on legacy applications could limit the ability to make changes.

What kinds of practices and processes will I need to support microservices?

While we talked about the principles that underpin good microservices practices in Chapter 4, we haven’t told you which specific practices or methodologies you should use. Our advice is to focus on the principles first, but it’s worth taking a look at how the companies that are known for doing microservices well build their software.

The ideal software development lifecycle for microservices is based on a product mentality using Agile principles, which includes continuous integration and continuous delivery and features a high degree of automation in testing, deployment, and operations. Attempting to apply microservice architecture in a differing environment can subtract from its potential value. A Waterfall approach can lead to tight coupling of services, making it difficult to manage the different change rates of those services and inhibiting their evolution. Project-focused delivery assumes static requirements and heavyweight change control, both impediments to fast software delivery. Being unable to deploy frequently will lead to a “big bang” release mentality and bring with it undue ceremony. If change frequency is increased in an environment that has a legacy of change intolerance, many of those overweight processes can stick around, slowing down delivery, and introducing procedural fatigue as a new risk. Lack of automation in the deployment lifecycle will have a negative compound effect on speed to market, and lack of automation in operations will make it harder to deal with the operational complexity of a distributed environment.

How do I govern a microservice system?

Aside from regulatory issues (e.g., certification, audits, etc.) there are typically three ways in which you can address security and governance requirements in a microservice system: centralized, contextual, and decentralized.

Centralized controls

At the component level, there really isn’t anything special about securing a microservice system. If you know how to secure an operating system, secure an API, or secure an application you can apply all of the same mechanisms to a microservice system. But when security mechanisms are introduced in the manner that most experts are used to implementing them, you can inadvertently upset the system optimization goals that you’ve worked hard to design into the architecture.

This is because security, controls, and governance policies are often implemented in a centralized fashion. For example, if we have a need to authenticate, authorize, and audit messages before they are processed, the most common architecture pattern is to implement some form of central security enforcement component within the architecture. Implementing a single, scalable component that can manage a complex and expensive function like access control makes a lot of sense. Assigning a separate team to manage and implement such a service also makes a lot of sense. Unfortunately, services like access control are likely to be used by every service in the infrastructure, which results in a common component that all of other services will grow dependent on. In other words, a bottleneck can develop.

A centralized security component risks putting our system into a state of mechanical organization or centralized control. In the early days of a microservice architecture it will be easy to set up the correct access and routing rules for a handful of services, but as more services are introduced and as those services change, the demand to modify the access control component is likely to outgrow the access control team’s capacity to roll out changes in time.

For organizations that wish to prioritize control and security it may be reasonable to trade the speed of change for improved system safety and security. However, if you want to optimize for speed of implementation you’ll need to take a different decentralized approach.

Decentralized controls

The implication here is that the individual microservice teams will need to manage an infrastructure that includes security mechanisms that are bounded to the service itself. The organization may standardize on the particular components and libraries that are to be used in every microservice, but it will be the teams themselves that are responsible for implementing security components and configuring them accordingly. It naturally follows that someone on the team must also take on the role of becoming the security expert for the service.

Contextual controls

A third approach that an organization can take is to define subsystems within the microservice architecture. Each subsystem may contain multiple services and their services within the subsystem are able to share common resources such as access control. Again, the organization may mandate the nature and requirement for these security components to be in place, but it is up to a subsystem service team to own and manage the configuration for the security component.

Security will always be an important design consideration for your microservice system. Even the absence of security is an implicit trade-off. While decisions about how and what to secure will be dependent on the risk profile of your organization and nature of the application you are building, the decision about who will manage the security implementation and where it will be implemented will have a big impact on your ability to optimize for the system behavior that you want.

Designing services that may be used by all other services in the system in itself is not a problem. A commonly used service that is resilient, reliable, and available will not impede the efforts of the rest of the service teams in the system. However, a commonly used service becomes a problem if the cost of implementing, configuring, and changing it becomes so high that it reduces the ability for service teams to make changes to the system.

Should all microservices be coded in the same programming language?

The short answer is “no.” The internal language of the component is not as important as the external interface—the API—of that component. As long as two components can use the same network protocols to exchange messages in an agreed-upon format using shared terms, the programming language used to accomplish all this is not important.

At the same time, many companies we talked to constrained the number of languages supported in the organization in order to simplify support and training. While a polyglot environment has advantages, too many languages results in added nonessential complexity system developers and maintainers need to deal with.

What do I do about orphaned components?

Over the life of a microservice implementation teams will come and go, and sometimes a team might disband and this can result in an “orphaned” microservice. It’s not a good idea to just let a service run along without someone to care for it. As Martin Fowler points out in “Products not Projects”, “ownership” is an important organizational aspect of microservices.

When a team is about to disband, that team needs to designate a new “owner” of the microservice component. This might be one of the existing team members (“OK, I’ll take responsibility for it”). It might be some other team that is willing to take care of it. Or it might be someone who has taken on the special role of caring for “orphaned” services. But someone needs to be designated as the “owner.”

It’s not safe to allow orphaned services to run in your infrastructure.