Composing is a powerful construct of Crossplane that makes it unique among its peers, such as the Open Service Broker API or AWS Controllers for Kubernetes. The ability to organize infrastructure recipes in a no-code way perfectly matches the organization’s agile expectation of building a lean platform team. This chapter will take us on a journey to learn about composing from end to end. We will start with a detailed understanding of how Crossplane Composite Resources (XRs) work and then cover a hands-on journey to build an XR step by step.
The following are the topics covered in this chapter:
Traditionally, infrastructure engineers know the most profound infrastructure configuration options and different infrastructure setup patterns. But they may not have experience in building APIs. Building an infrastructure platform with Crossplane will be a shift from these usual ways. Modern infrastructure platform developers should have both pieces of knowledge, that is, infrastructure and API engineering. Building infrastructure APIs as a platform developer means implementing the following aspects:
Infrastructure recipes and shared infrastructure are vital elements in API-bounded context trade-offs. We will examine this in detail in an upcoming chapter. The following figure represents the nuances of API infrastructure engineering:
We are looking at these aspects to understand XR architecture in the best possible way. Every element of the Crossplane composite is designed to cover infrastructure engineering practices from the perspective of an API.
Tip
We can use the learnings from microservices architecture pattern to define infrastructure API boundaries. There is no perfect boundary, and every design option will have advantages and disadvantages. In Chapter 6, More Crossplane Patterns, we can look for ways to adopt microservices with the Crossplane infrastructure platform.
An XR can do two things under the hood. The first purpose is to combine related Managed Resources (MRs) into a single stack and build reusable infrastructure template APIs. When we do this, we might apply different patterns, such as shared resources between applications or cached infrastructure for faster provisioning. The second one is to expose only limited attributes of the infrastructure API to the application team after abstracting all organization policies. We will get into the details of achieving these aspects as we progress in this chapter. The following are the critical components in an XR:
Let’s start looking at the purpose of each component and how they interact with each other.
The XRD is the schema defining the infrastructure API specification. It is best to describe an XRD first. Fixing the API specification first will force us to think about end users’ needs and the different ways they will consume the API. We will also apply all the organization policies to decide what fields are to be exposed to an application team. It will clearly set the scope and the boundary of the API. CompositeResourceDefinition is the Crossplane configuration element used to define an XRD. Creating this configuration is like writing an OpenAPI Specification or Swagger API definition. The following are the critical aspects of the CompositeResourceDefinition configuration YAML:
The XRD is nothing but an opinionated Custom Resource Definition (CRD), and many parts of the configuration look like a CRD. These are just a few possible parameters. We will look at a few more parameters as we progress through the book. The complete API documentation is available at https://doc.crds.dev/github.com/crossplane/crossplane.
Tip
We are looking at v1.5.1 of the Crossplane documentation, which is the latest at the time of writing this chapter. Refer to the latest version at the time of reading for more accurate details.
Note that some of the configurations discussed previously are not part of the following YAML, such as DefaultCompositionRef and ConnectionSecretKeys. These configurations are injected by Crossplane with default behavior if not specified. Refer to the following YAML for an example:
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
#'<plural>.<group>'
name: xclouddbs.book.imarunrk.com
spec:
# API group
group: book.imarunrk.com
# Singular name and plural name.
names:
kind: xclouddb
plural: xclouddbs
# Optional parameter to create namespace proxy claim API
claimNames:
kind: Clouddb
plural: Clouddbs
# Start from alpha to beta to production to deprecated.
versions:
- name: v1
# Is the specific version actively served
served: true
# Can the version be referenced from an API implementation
referenceable: true
# OpenAPI schema
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
parameters:
type: object
properties:
storageSize:
type: integer
required:
- storageSize
required:
- parameters
Once we are done with the API specification, the next step is to build the API implementation. Composition is the Crossplane construct used for providing API implementation.
The composition will link one or more MRs with an XR API. When we create, update, and delete an XR, the same operation will happen on all the linked MRs. We can consider XRD as the CRD and composition as the custom controller implementation. The following diagram represents how XR, XRD, composition, and MRs are related:
Tip
We have referred to XR in this book in two contexts. We can use XR to refer to a new infrastructure API that we are building. Also, the composition resources list can hold both an MR and an existing XR. We will also refer to an XR from that context. Look at Figure 4.2 where XR is referred to in both dimensions.
Let’s look at some of the crucial elements from the composition configuration:
Tip
We have a few field path attributes when defining a composition. Values for these fields will follow standard JavaScript syntax to access JSON, for example, spec.parameters.storageSize or spec.versions[0].name.
We covered most of the configuration options available with the composition. Have a look at the Crossplane documentation for the complete list. The following figure represents the composition configuration options and the relationship between them:
The following is a sample composition configuration YAML:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: xclouddb-composition
spec:
# Link Composition to a specific XR and version
compositeTypeRef:
apiVersion: xclouddb.book.imarunrk.com/v1
kind: Xclouddb
# Connection secrets namespace
writeConnectionSecretsToNamespace: crossplane-system
# List of composed MRs or XRs.
resources:
- name: clouddbInstance
# Resource base template
base:
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
spec:
forProvider:
databaseVersion: POSTGRES_9_6
region: us-central
settings:
tier: db-g1-small
dataDiskSizeGb: 20
# Resource patches
patches:
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.storageSize
toFieldPath: spec.forProvider.settings.dataDiskSizeGb
# Resource secrets
connectionDetails:
- name: hostname
fromConnectionSecretKey: hostname
We will cover an example with more configuration elements in the Building an XR section. An XRD version can have more than one composition, that is, one-to-many relationships between the XRD version and composition. It provides polymorphic behavior for our infrastructure API to work based on the context. For example, we could have different compositions defined for production and staging. The CompositionRef attribute defined in the XR can refer to a specific composition. Instead of CompositionRef, we can also use CompositionSelector to match the compositions based on labels.
In the next section, we will look at XR claims, also known as claims.
A claim is a proxy API to the XR, created by providing claim name attributes in XRD configurations. As a general practice, we provide the exact name of the XR after removing the initial X. In the preceding example, xclouddb is the XR name and Clouddb is the claim name but following such naming conventions is not mandatory. Claims are very similar to the XR, and it might tempt us to think that it’s an unnecessary proxy layer. Having a claim is helpful in many ways, such as the following:
The following figure represents how claims, XR, XRD, composition, and MRs are related, giving an end-to-end view of how the whole concept works:
The following are the sample claim and XR YAML. The claim YAML is as follows:
apiVersion: book.imarunrk.com/v1
# Kind name matches the singular claim name in the XRD
kind: Clouddb
metadata:
name: cloud-db
spec:
# Parameters to be mapped and patched in the composition
parameters:
storageSize: 20
# Name of the composition to be used
compositionRef:
name: xclouddb-composition
writeConnectionSecretToRef:
namespace: crossplane-system
name: db-conn
A namespace is not part of the preceding claim YAML. Hence, it will create the resource in the default namespace, the Kubernetes standard. An equivalent XR YAML to the preceding claim YAML is as follows:
apiVersion: book.imarunrk.com/v1
kind: XClouddb
metadata:
name: cloud-db
spec:
parameters:
storageSize: 20
compositionRef:
name: xclouddb-composition
writeConnectionSecretToRef:
namespace: crossplane-system
name: db-conn
Note that the XR is always created at the cluster level and namespace configuration under metadata is not applicable. We can look at a more detailed claim and XR configurations in the Building an XR section. Let’s explore a few more XR, XRD, composition, and claim configurations from the perspective of postprovisioning requirements.
After performing CRUD operations over a claim or XR resource, the following are some critical aspects to bring the API request to a close:
Let’s start with learning about readiness checks.
The XR state will be ready by default when all the underlying resources are ready. Every resource element in the composition can define its custom readiness logic. Let’s look at a few of the custom readiness check configurations. If you want to match one of the composing resource status fields to a predefined string, use MatchString. A sample configuration for MatchString is as follows:
- type: MatchString
fieldPath: status.atProvider.state
matchString: "Online"
MatchInteger will perform a similar function when two integers are matched. The following sample configuration will check the state attribute with integer 1:
- type: MatchInteger
fieldPath: status.atProvider.state
matchInteger: 1
Use the None type to consider the readiness as soon as the resource is available:
- type: None
Use NonEmpty to make the resource ready as soon as some value exists in the field of our choice. The following example will make the readiness true as soon as some value exists under the mentioned field path:
- type: NonEmpty
fieldPath: status.atProvider.state
In the next section, we will look at an example of patching a status attribute after resource provisioning. Note that fieldPath falls under the status attribute. These are the attributes filled by MR during resource provisioning based on the values it gets back from the cloud provider.
ToCompositeFieldPath is a patch type for copying any attribute from a specific composed resource back into the XR. Generally, we use it to copy the status fields. We can look at these as a way to define the API response. While there is a set of existing default status fields, patched fields are custom defined to enhance our debugging, monitoring, and audit activities. First, we need to define the state fields as a part of openAPIV3Schema in the XRD to make the new status fields available in the XR. The next step is to define a patch under the specific composing resource. The following patch will copy the current disk size of the CloudSQLInstance to the XR:
- type: ToCompositeFieldPath
fromFieldPath: status.atProvider.currentDiskSize
toFieldPath: status.dbDiskSize
We can also use the CombineToComposite patch type if we need to copy a combination of multiple fields.
We can see that the connection secret-related configuration is part of the XRD, XR, claim, and composition. We must understand the relationship between these configurations to configure it correctly and get it working:
The following diagram can help create a mind map of these configurations:
The section covered different patterns that we can use with composition after the resources are provisioned. It is like customizing the API responses. Now we can look at the usefulness of reusing existing resources.
There are a few use cases where we may not create a new external resource and instead will reuse an existing provisioned resource. We will look at two such use cases in this section. The first use case is when we decide to cache the composed recourses because new resource provisioning may take too long to complete. The platform team can provision an XR and keep the resources in the resource pool. Then, the product team can claim these resources by adding the ResourceRef configuration under the spec of a claim YAML. With this pattern, we should ensure that the new claim attributes match the attributes in the existing pre-provisioned XR. If some of the attributes are different, Crossplane will try to update the XR specifications to match what is mentioned in the claim.
The second use case is about importing the existing resources from the external provider into the Crossplane. The crossplane.io/external-name annotation can help with this. Crossplane will look for an existing resource with the name mentioned in this configuration. The external name configuration mentioned in a claim will automatically be propagated into the XR. Still, it’s our responsibility to patch this configuration into the composing resource. The following is a sample MR YAML where we onboard an existing VPC with the name alpha-beta-vpc:
apiVersion: compute.gcp.crossplane.io/v1beta1
kind: Network
metadata:
name: alpha-beta-vpc-crossplane-ref
annotations:
# Annotation to provide existing resource named
crossplane.io/external-name: alpha-beta-vpc
spec:
providerConfigRef:
name: gcp-credentials-project-1
# Provide the required parameters same as external resource.
forProvider:
autoCreateSubnetworks: true
Once you apply the YAML, you will see that it’s ready for use in Crossplane. This can be seen in the following screenshot:
Note that the alpha-beta-vpc VPC is an existing VPC we created manually in GCP. What we achieve here is to map the manual resource to a Claim.The section covered different ways we can use preprovisioned resources with an XR/claim. The following section will be a hands-on journey to build an XR from scratch.
It’s time to go through a hands-on journey to build an XR from scratch. We will start with writing down the infrastructure API requirement at a high level, then provide an API specification with XRD and finally provide an implementation with a composition. We will cover the API requirement in such a way as to learn most of the configuration discussed in this chapter.
We will develop an API to provision a database from Google Cloud. The following are the compliance, architecture, and product team’s requirements:
The next step is to write the XRD configuration YAML.
When defining the API specification with an XRD, the following configurations should be encoded into the YAML:
As the example XRD is oversized, we will cover only the schema definition here. Refer to the entire XRD file at https://github.com/PacktPublishing/End-to-End-Automation-with-Kubernetes-and-Crossplane/blob/main/Chapter04/Hand-on-examples/Build-an-XR/xrd.yaml. Without wasting much time, let’s look at the schema:
schema:
openAPIV3Schema:
type: object
properties:
# Spec – defines the API input
spec:
type: object
properties:
parameters:
type: object
properties:
# Size will be a user input
size:
type: string
required:
- size
required:
- parameters
# status – the additional API output parameter
status:
type: object
# Recourse zone - status patch parameter.
properties:
zone:
description: DB zone.
type: string
Save the YAML from GitHub and apply it to the cluster with kubectl apply -f xrd.yaml. Refer to the following screenshot, which shows successful XRD creation:
Note that the ESTABLISHED and OFFERED flags in the screenshot are True. This means that the XRD is created correctly. If these statuses are not True, use kubectl to describe the details of the XRD and look for an error.
The next step is to provide an API implementation. As a part of the implementation, we should be providing a composition configuration. We will create two compositions, one for Postgres and the other for MySQL. It will be an example of the polymorphic behavior implementation. The following are the steps to remember when we build the composition YAML:
We will look at the Postgres composition example in four parts. The XRD and resource definition section of the composition will look like the following configuration:
spec:
# Refer to an XRD API version
compositeTypeRef:
apiVersion: alpha-beta.imarunrk.com/v1
kind: XGCPdb
writeConnectionSecretsToNamespace: crossplane-system
resources:
# Provide configuration for Postgres resource
- name: cloudsqlinstance
base:
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
spec:
# reference to GCP credentials
providerConfigRef:
name: gcp-credentials-project-1
forProvider:
databaseVersion: POSTGRES_9_6
# Compliance Policy
region: us-central1
settings:
# These are default values
# Architecture policies will be a patch
tier: db-g1-small
dataDiskSizeGb: 20
Read through the comments between the code snippets to understand concepts in detail. The following configuration uses the map transform to patch the virtual machine tier:
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.size
toFieldPath: spec.forProvider.settings.tier
# Use map transform
# If the from-field value is BIG, then
# the mapped to-field value is db-n1-standard-1
transforms:
- type: map
map:
BIG: db-n1-standard-1
SMALL: db-g1-small
policy:
# return error if there is no field.
fromFieldPath: Required
Next, we can look at the configuration to patch the disk size. The patch will have two transform operations. The first operation is to map the disk size, and the second one is to convert the mapped string to an integer:
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.size
toFieldPath: spec.forProvider.settings.dataDiskSizeGb
# If the from-field value is BIG, then
# the mapped to-field value is '40;
# Apply the second transform to convert '40' to int
transforms:
- type: map
map:
BIG: "40"
SMALL: "20"
- type: convert
convert:
toType: int
policy:
# return error if there is no field.
fromFieldPath: Required
Finally, the following patch adds the resource zone into the API response:
# Patch zone information back to the XR status
# No transformation or policy required
- type: ToCompositeFieldPath
fromFieldPath: status.atProvider.gceZone
toFieldPath: status.zone
The composition configuration for MySQL will be the same as the preceding configuration, excluding two changes. We should be changing the name of the composition in the metadata, and in the resource definition, we should change the database version to MYSQL_5_7. We can implement this with an additional parameter in the XR as well. Building two different compositions does not make sense when the difference is so small. We can capture the difference as a parameter in the XR. We are building two compositions, as an example. All composition examples and the upcoming claim examples are available for reference at https://github.com/PacktPublishing/End-to-End-Automation-with-Kubernetes-and-Crossplane/tree/main/Chapter04/Hand-on-examples/Build-an-XR.
Refer to the following screenshot, which shows the successful creation of both compositions:
The final step is to use the claim API and create the database resources.
Finally, we can start provisioning the GCP database with an XR or a claim. The CompositionRef configuration will specify which composition implementation to use. Note that the claims are namespace resources, and we provision them in the alpha namespace here. The following is a sample claim YAML for the MySQL database:
apiVersion: alpha-beta.imarunrk.com/v1
kind: GCPdb
metadata:
# Claims in alpha namespace
namespace: alpha
name: mysql-db
spec:
# Refer to the mysql composition
compositionRef:
name: mysql
# save connection details as secret - db-conn2
writeConnectionSecretToRef:
name: db-conn2
parameters:
size: SMALL
The Postgres YAML as well will look similar with minor changes. Refer to the following screenshot, which shows a successful database creation:
Note that the zone information is made available as the part of claim status:
This concludes the journey to build an XR. We will look at a few troubleshooting tips.
If we face issues with our infrastructure API, these tips could help us debug the problem in the best possible way:
Make an intentional mistake in the resource configuration of the composition to go through the debugging experience. You learn more when you debug issues. This concludes our troubleshooting section. Next, we will look at the chapter summary before moving on to the next chapter.
With this chapter, we covered one of the critical aspects of Crossplane, the XR. We started with understanding how an XR works and configuring an XR. Above all, we went through a hands-on journey to build a fresh infrastructure API from end to end. The chapter also covered some advanced XR configuration patterns and ways to approach debugging when there is an issue. This will be the base knowledge for what we will learn in the next chapter.
The next chapter will cover different advanced infrastructure platform patterns.