Chapter 11: Driving the Platform Adoption

By now, you might be thinking about adopting Crossplane in your organization. As we’ve discussed many times, we must set up a platform team to support every product team across the organization adopting Crossplane. But many organizations fail with their technology platform projects because these platforms impact the agility of the consuming teams in many ways. Setting up proper boundaries and an interaction model between the product and platform teams is key to the success of a platform. This chapter aims to help you understand the aspects required to build and adopt a successful infrastructure platform. We will also look at a few common pitfalls that can lead to weak platform adoption and low return on investment.

The following are the topics covered in this chapter:

• Why we need an infrastructure platform as a product
• Understanding customers’ needs
• The platform prodcut life cycle and team interaction
• The OAM personas

In the following section, we will start with understanding why we need to approach infrastructure platforms as product development.

Why we need an infrastructure platform as a product

We touched on this topic a bit in Chapter 2. It is time to look back and see why we need a platform team to develop the Crossplane APIs. The following are the three key reasons:

• Cognitive load: Any organization will tend to use a vast amount of cloud resources and other external services. These resources and services consist of tens of thousands of attributes to configure them according to the organization’s requirements. Remembering the usage of each configuration attribute involves a significant cognitive load. Suppose we attempt to build this knowledge within the product team. In that case, the team will focus on technical complexity rather than product feature development, which is of direct business interest. If you look at the CNCF cloud-native landscape, it’s vast (https://landscape.cncf.io/). Not every team can tame the terrain. It requires a specialized group to build this cognitive capability into the organization. Provided every product team faces the same situation, it makes sense to abstract this effort as a shared service managed by a platform team.
• Policy management: Infrastructure and other external services’ usage policies come from security, compliance, product, and architecture teams. It requires a centralized team to track, consolidate, and encode the policies when these resources are provisioned and used. It won’t be easy to keep the policies enforced and updated if we have these resources’ automation as part of individual product teams.
• Return on investment on reuse: Generally, many tools, architecture patterns, and infrastructure setup patterns look similar across the organization. This is because of centralized IT strategies, the cross-pollination of successful designs, the organization’s cognitive capability, and so on. These commonalities will translate into the reuse of infrastructure reference architecture. Provided we have the cognitive load and policy management advantages, the economics of return on investment with reuse is an added advantage.

Now that we are convinced that an infrastructure platform team is required, the following section will explore the product teams’ expectations of the infrastructure platform team.

Understanding customers’ needs

A platform team for infrastructure means external delivery dependency. This is something that is always avoided by agile product development teams. If you have worked in a large organization, you might have noticed that the product teams are often unhappy with what the platform team offers. They have issues with the delivery timeline, backlog coupling, and weak contracts. It is critical to set up a proper interaction model between the product and platform teams to mitigate these risks. The following are qualities that a platform team should possess to match the delivery expectations of product teams in the modern software engineering era:

• Product development practices
• Self-service
• Collaborative backlog management

The following section will look at the product development engineering practices for the platform team.

Product development practices

Adopting product development practices is a critical component that can make a platform team successful. The following are some of the product development practices for adoption:

• Quick start: Platform usage must be made quick and straightforward. Crossplane being an API-based platform solves part of the problem. APIs have a clear contract, and they are easy to understand. But this is not enough. We must make additional efforts to make the platform easy to adopt. We should have a quick-start guide, code repositories with examples, a support system to help adopting teams, and extensive API documentation. More importantly, the ability to quickly start should be a critical metric measured and continuously improved. We can do this with continuous feedback cycles between the product and platform teams. The feedback should be from multiple sources, such as customer surveys, usage statistics of various quick-starter asserts, usage statistics of the platform itself, and time to adoption. There should be incentives for the platform team to improve these metrics continuously.
• Community of practice: Building a community with participants from the platform and product teams is another essential aspect. Application operators could be natural participants of the community from the product team. Being close to the product team’s requirements, they will bring a reality check to what we build as APIs in the platform. Again, there should be metrics and incentives to run the community successfully. We can use the community in many ways, such as for co-creation, as knowledge carriers, for two-way feedback flow channels, and for accelerating adoption. We will look at some of these options in detail in the following sections of the chapter.
• Composable APIs: One of the critical challenges with technology platforms is that they enforce a particular way of development that hinders innovation. The platform should have a basic set of fine-grained XRs, above which we can quickly compose the claim recipes required by the product team. It is recommended to have two-layered APIs. The first layer should have fine-grained XRs composing organization-centralized policies, and the following layer should be the recipe layer, composing product-level policies and requirements. We could have a basic set of reused recipes used across the organization and leave space for the community to innovate new recipes. Some organization setups may require Goldilocks governance to avoid the community’s proliferation of too many new recipes.

  Information

  Goldilocks architecture governance has gained traction in modern software engineering practices where a balanced approach is taken between governance and flexibility.

In the following section, we will cover the self-service aspect of platform engineering.

Self-service

Self-service is one of the critical aspects of building a platform. Think about how any cloud provider works. We have the console and the APIs to create/operate the resource required. It’s an entirely self-service model where users can manage resources according to their permissions. A refined resource granularity in a cloud provider environment will work for any organization and personas. We must have a similar experience for our platform as well. With Crossplane being an API-based resource composing platform, part of the problem is already solved. But we must put a lot of thought into defining the scope and granularity of the XR/claim APIs that the product teams will consume. What level of granularity will work for your organization is a crucial question to ask. It may depend on the organization’s size, the team boundaries, the technical maturity of the developers/platform team, and so on. Defining wrong boundaries will derail the self-service agenda.

Getting the boundary perfect at first is difficult. It keeps on improving in iterations by measuring the self-service metrics. In addition to API boundaries, we need to focus on the quick-start guide, technical support system, and building community, as discussed earlier, to enable self-service further. The whole experience should be like how a successful cloud provider platform works. The only difference is that we operate on a higher level of abstraction to tame the complexity.

Collaborative backlog management

A single centralized platform team serving multiple internal teams within the organization is not a fun place to work. We must manage the backlog to satisfy everyone’s needs. There will be continuous requests from every direction to enhance an existing XR/claim or create a new one. We may end up in many unwanted situations, such as constant delivery pressure on the platform team, impact on the product team’s delivery throughput, or frustrated product teams developing the capability locally. The following are some of the ways to mitigate these risks:

• Backlog prioritization: Score each backlog item with metrics such as time criticality, missed opportunity, and short-/long-term business value, in a prioritization session. Document the reason for each score for visibility. All product teams with backlog items should be actively involved in the prioritization session. There could be a tendency that each product team tries to push their score up in the prioritization sessions. We can tackle that with a scoring slider where a total score is set on a metric, and the sum of the scores for each backlog should not exceed the set value.
• Expectation management: Product teams are waiting for their backlog to be delivered, and they should be treated like end customers. It’s essential to set expectations for the customers on when and what they can expect. Use Scrum of Scrums or any other scale agile framework to find a way to provide regular updates. Provide an update on the progress and the delivery date. Be open and transparent about any delayed delivery.
• Community and governance: Look at how open source projects such as Kubernetes and Crossplane work. They also face the same situation where there is a lot of expectation from many users for new features, enhancements, and bug fixes. These projects were initially started with a few focused developers. When the product moves from the alpha-beta stage to general availability, many customers start adopting it. At this point, managing the backlog becomes critical. These open source projects scale up the delivery velocity by building a solid community around them. Development activities by the community are governed by technical steering groups filled with core contributors. We could adopt the same model in the internal product as well. As we have said already, application operators close to the product team can be naturally attracted to the community. Product teams with pressing needs can allocate their application operator to co-create the product.

In the following section, we will investigate various stages of platform API development and how they impact the interaction between the platform team and the product team.

The platform product life cycle and team interaction

We may start our Crossplane API development as a small proof of concept working with one of the product teams. Later, when we are comfortable with the proof of concept, we will release an alpha version asking a few teams to consume and test it. Again, when we are satisfied with the alpha version, we will ask a few more sections of the organization to adopt and go live with the API. If everything goes well, we can move the API to general availability to be consumed by anyone in the organization. It will not end there. Sometimes, we will have the requirement to deprecate the APIs and mark them for removal in the future. The following figure represents the API journey:

Figure 11.1 – Crossplane API life cycle

Every API may not go through all the phases mentioned. It may take its own path, moving from left to right and skipping some stages depending on the situation.

Information

Sometimes, we may move directly to the deprecated state from alpha or beta if we find no business value or technical feasibility.

Each step requires a different engagement model and developer capability. The following list describes each of the life cycle phases:

• Proof of concept and alpha stages: Two things are essential to consider at these stages. First, the interaction model between the product and platform teams should be collaboration. This means both teams should work closely together as much as possible. The capability of the developers involved should be rapid innovators. We don’t need the API to be scalable or reliable at this point. Our only aim should be to experiment with the feasibility and value as quickly as possible.
• Beta stage: This stage should continue the collaborative work format, but we should slowly move toward self-service. The product teams should be able to request access to the new API using some self-service portal and start using it. Creating self-service artifacts should be performed at this stage. The platform team should perfect the self-service portal and API in close collaboration with the product team. The capability of the developers involved should be a mix of rapid innovators and people who can bring stability/reliability. Our aim should be to get ready for organization usage.
• General availability: This is the stage of organization-wide usage. Two key aspects are being entirely self-service and having a stable and reliable API.
• Depreciation and removal stage: Sometimes, we need to deprecate and remove support for an API. Pushing the product team to get rid of the API usage is not easy because they are stuck with their priority. The platform team should use the facilitating interaction model to execute the activity. The platform developers perform the actual work for the product team to get rid of an API. Again, the platform developers at this stage are required to be rapid innovators.

  Acknowledgment

  Many of the concepts we discussed in this chapter are influenced by the ideas found in https://teamtopologies.com/, Evan Bottcher’s work on platform engineering (https://martinfowler.com/articles/talk-about-platforms.html), and finally, some experience in implementing these concepts at work.

The following diagram represents the preceding API stages and the team interaction model discussed:

Figure 11.2 – API stages and team interaction model

This takes us to the end of the platform life cycle discussion. In the following section, we will look at the Open Application Model (OAM) specification to take some inspiration for the organization-level team structure from the perspective of platform engineering.

The OAM personas

We’ve touched upon OAM specifications several times throughout this book. We will refresh the same topic from the perspective of organizational structure. We will take some inspiration from the OAM model to organize the platform and its ecosystem. OAM proposes the following three personas to deploy and manage cloud-native applications:

• Application developer: Concentrates on application development, keeping the entire emphasis on developing features that add value to customers directly.
• Application operator: Offloads the complexity of configuring the applications in the cloud-native ecosystem from application developers. Enables the application development team to move faster with feature development. Application operators contribute to the end consumer indirectly.
• Infrastructure operator: Offloads the complexity of configuring the cloud, other infrastructure, and services across the organization. It allows the application operator to focus on configuring and operating the application.

Overlapping the OAM personas and other concepts we learned in this chapter, we could structure the platform team and its ecosystem as illustrated in the following diagram:

Figure 11.3 – Platform ecosystem

The following aspects explain the team constructs represented in the preceding diagram:

• The organization x is split into multiple products.
• Each product is divided into multiple Scrum teams handling different subdomains.
• One or more application operators support configuring and operating all applications from a given product.
• The platform team acts as a cross-cutting service to support all product teams for infrastructure, service, and application automation.
• The platform team and application operator form the community to support activities such as co-creation and accelerating adoption.

We could explore more models to structure the platform and its ecosystem. But the core idea is to manage the platform development as a product engineering practice treating all internal consumers as customers.

Summary

We started the book with an idea to support cloud-native architects, platform engineers, infrastructure operators, and application operators interested in simplifying infrastructure, application, and service automation with the Kubernetes APIs. Throughout the book, we’ve covered different aspects of building a fully declarative self-service control plane to automate infrastructure, applications, and services using Crossplane and Kubernetes. I hope you learned about the required concepts while having fun. Let’s together take software engineering in to a new era.