In this chapter, we are going to look into Helm, the popular Kubernetes package manager. Every successful and non-trivial platform must have a good packaging system. Helm was developed by Deis (acquired by Microsoft in April 2017) and later contributed to the Kubernetes project directly. It became a CNCF project in 2018. We will start by understanding the motivation for Helm, its architecture, and its components. Then, we’ll get hands-on and see how to use Helm and its charts within Kubernetes. That includes finding, installing, customizing, deleting, and managing charts. Last but not least, we’ll cover how to create your own charts and handle versioning, dependencies, and templating.
The topics we will cover are as follows:
Kubernetes provides many ways to organize and orchestrate your containers at runtime, but it lacks a higher-level organization of grouping sets of images together. This is where Helm comes in. In this section, we’ll go over the motivation for Helm, its architecture, and its components. We will discuss Helm 3. You might still find Helm 2 in the wild, but its end of life was at the end of 2020.
As you might recall, Kubernetes means helmsman or navigator in Greek. The Helm project took the nautical theme very seriously, as the project’s name implies. The main Helm concept is the chart. Just as nautical charts describe in detail an area in the sea or a coastal region, a Helm chart describes in detail all the parts of an application.
Helm is designed to perform the following:
.tgz
)Helm provides support for several important use cases:
Charts can define even the most intricate applications, offer consistent application installation, and act as a central source of authority. In-place upgrades and custom hooks allow for easy updates. It’s simple to share charts that can be versioned and hosted on public or private servers. When you need to roll back recent upgrades, Helm provides a single command to roll back a cohesive set of changes to your infrastructure.
The Helm 3 architecture relies fully on client-side tooling and keeps its state as Kubernetes secrets. Helm 3 has several components: release secrets, client, and library.
The client is the command-line interface, and often a CI/CD pipeline is used to package and install applications. The client utilizes the Helm library to perform the requested operations, and the state of each deployed application is stored in a release secret.
Let’s review the components.
Helm stores its releases as Kubernetes secrets in the target namespace. This means you can have multiple releases with the same name as long they are stored in different namespaces. Here is what a release secret looks like:
$ kubectl describe secret sh.helm.release.v1.prometheus.v1 -n monitoring
Name: sh.helm.release.v1.prometheus.v1
Namespace: monitoring
Labels: modifiedAt=1659855458
name=prometheus
owner=helm
status=deployed
version=1
Annotations: <none>
Type: helm.sh/release.v1
Data
====
release: 51716 bytes
The data is Base64-encoded twice and then GZIP-compressed.
You install the Helm client on your machine. Helm carries out the following tasks:
The Helm library is the component at the heart of Helm and is responsible for performing all the heavy lifting. The Helm library communicates with the Kubernetes API server and provides the following capabilities:
Helm 2 was great and played a very important role in the Kubernetes ecosystem. But, there was a lot of criticism about Tiller, its server-side component. Helm 2 was designed and implemented before RBAC became the official access-control method. In the interest of usability, Tiller is installed by default with a very open set of permissions. It wasn’t easy to lock it down for production usage. This is especially challenging in multi-tenant clusters.
The Helm team listened to the criticisms and came up with the Helm 3 design. Instead of the Tiller in-cluster component, Helm 3 utilizes the Kubernetes API server itself via CRDs to manage the state of releases. The bottom line is that Helm 3 is a client-only program. It can still manage releases and perform the same tasks as Helm 2, but without installing a server-side component.
This approach is more Kubernetes-native and less complicated, and the security concerns are gone. Helm users can perform via Helm only as much as their kube config allows.
Helm is a rich package management system that lets you perform all the necessary steps to manage the applications installed on your cluster. Let’s roll up our sleeves and get going. We’ll look at installing both Helm 2 and Helm 3, but we will use Helm 3 for all of our hands-on experiments and demonstrations.
Installing Helm involves installing the client and the server. Helm is implemented in Go. The Helm 2 executable can serve as either the client or server. Helm 3, as mentioned before, is a client-only program.
You must have Kubectl configured properly to talk to your Kubernetes cluster because the Helm client uses the Kubectl configuration to talk to the Kubernetes API server.
Helm provides binary releases for all platforms here: https://github.com/helm/helm/releases.
For Windows, the Chocolatey (https://chocolatey.org) package manager is the best option (usually up to date):
choco install kubernetes-helm
For macOS and Linux, you can install the client from a script:
$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh
On macOS, you can also use Homebrew (https://brew.sh):
brew install helm
$ helm version
version.BuildInfo{Version:"v3.9.2", GitCommit:"1addefbfe665c350f4daf868a9adc5600cc064fd", GitTreeState:"clean", GoVersion:"go1.18.4"}
To install useful applications and software with Helm, you need to find their charts first. Helm was designed to work with multiple repositories of charts. Helm 2 was configured to search the stable repository by default, but you could add additional repositories. Helm 3 comes with no default, but you can search the Helm Hub (https://artifacthub.io) or specific repositories. The Helm Hub was launched in December 2018 and it was designed to make it easy to discover charts and repositories hosted outside the stable or incubator repositories.
This is where the helm search
command comes in. Helm can search the Helm Hub for a specific repository.
The hub contains 9,053 charts at the moment:
$ helm search hub | wc -l
9053
We can search the hub for a specific keyword like mariadb
. Here are the first 10 charts (there are 38):
$ helm search hub mariadb --max-col-width 60 | head -n 10
URL CHART VERSION APP VERSION DESCRIPTION
https://artifacthub.io/packages/helm/cloudnativeapp/mariadb 6.1.0 10.3.15 Fast, reliable, scalable, and easy to use open-source rel...
https://artifacthub.io/packages/helm/riftbit/mariadb 9.6.0 10.5.12 Fast, reliable, scalable, and easy to use open-source rel...
https://artifacthub.io/packages/helm/bitnami/mariadb 11.1.6 10.6.8 MariaDB is an open source, community-developed SQL databa...
https://artifacthub.io/packages/helm/bitnami-aks/mariadb 11.1.5 10.6.8 MariaDB is an open source, community-developed SQL databa...
https://artifacthub.io/packages/helm/camptocamp3/mariadb 1.0.0 Fast, reliable, scalable, and easy to use open-source rel...
https://artifacthub.io/packages/helm/openinfradev/mariadb 0.1.1 OpenStack-Helm MariaDB
https://artifacthub.io/packages/helm/sitepilot/mariadb 1.0.3 10.6 MariaDB chart for the Sitepilot platform.
https://artifacthub.io/packages/helm/groundhog2k/mariadb 0.5.0 10.8.3 A Helm chart for MariaDB on Kubernetes
https://artifacthub.io/packages/helm/nicholaswilde/mariadb 1.0.6 110.4.21 The open source relational database
As you can see, there are several charts that match the keyword mariadb
. You can investigate those further and find the best one for your use case.
By default, Helm 3 comes with no repositories set up, so you can search only the hub. In the past, the stable
repo hosted by the CNCF was a good option to look for charts. But, CNCF didn’t want to pay for hosting it, so now it just contains a lot of deprecated charts.
Instead, you can either install charts from the hub or do some research and add individual repositories. For example, for Prometheus, there is the prometheus-community
Helm repository. Let’s add it:
$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
"prometheus-community" has been added to your repositories
Now, we can search the prometheus
repo:
$ helm search repo prometheus
NAME CHART VERSION APP VERSION DESCRIPTION test | default
prometheus-community/kube-prometheus-stack 39.4.1 0.58.0 kube-prometheus-stack collects Kubernetes manif...
prometheus-community/prometheus 15.12.0 2.36.2 Prometheus is a monitoring system and time seri...
prometheus-community/prometheus-adapter 3.3.1 v0.9.1 A Helm chart for k8s prometheus adapter
prometheus-community/prometheus-blackbox-exporter 6.0.0 0.20.0 Prometheus Blackbox Exporter
prometheus-community/prometheus-cloudwatch-expo... 0.19.2 0.14.3 A Helm chart for prometheus cloudwatch-exporter
prometheus-community/prometheus-conntrack-stats... 0.2.1 v0.3.0 A Helm chart for conntrack-stats-exporter
prometheus-community/prometheus-consul-exporter 0.5.0 0.4.0 A Helm chart for the Prometheus Consul Exporter
prometheus-community/prometheus-couchdb-exporter 0.2.0 1.0 A Helm chart to export the metrics from couchdb...
prometheus-community/prometheus-druid-exporter 0.11.0 v0.8.0 Druid exporter to monitor druid metrics with Pr...
prometheus-community/prometheus-elasticsearch-e... 4.14.0 1.5.0 Elasticsearch stats exporter for Prometheus
prometheus-community/prometheus-json-exporter 0.2.3 v0.3.0 Install prometheus-json-exporter
prometheus-community/prometheus-kafka-exporter 1.6.0 v1.4.2 A Helm chart to export the metrics from Kafka i...
prometheus-community/prometheus-mongodb-exporter 3.1.0 0.31.0 A Prometheus exporter for MongoDB metrics
prometheus-community/prometheus-mysql-exporter 1.9.0 v0.14.0 A Helm chart for prometheus mysql exporter with...
prometheus-community/prometheus-nats-exporter 2.9.3 0.9.3 A Helm chart for prometheus-nats-exporter
prometheus-community/prometheus-node-exporter 3.3.1 1.3.1 A Helm chart for prometheus node-exporter
prometheus-community/prometheus-operator 9.3.2 0.38.1 DEPRECATED - This chart will be renamed. See ht...
prometheus-community/prometheus-pingdom-exporter 2.4.1 20190610-1 A Helm chart for Prometheus Pingdom Exporter
prometheus-community/prometheus-postgres-exporter 3.1.0 0.10.1 A Helm chart for prometheus postgres-exporter
prometheus-community/prometheus-pushgateway 1.18.2 1.4.2 A Helm chart for prometheus pushgateway
prometheus-community/prometheus-rabbitmq-exporter 1.3.0 v0.29.0 Rabbitmq metrics exporter for prometheus
prometheus-community/prometheus-redis-exporter 5.0.0 1.43.0 Prometheus exporter for Redis metrics
prometheus-community/prometheus-snmp-exporter 1.1.0 0.19.0 Prometheus SNMP Exporter
prometheus-community/prometheus-stackdriver-exp... 4.0.0 0.12.0 Stackdriver exporter for Prometheus
prometheus-community/prometheus-statsd-exporter 0.5.0 0.22.7 A Helm chart for prometheus stats-exporter
prometheus-community/prometheus-to-sd 0.4.0 0.5.2 Scrape metrics stored in prometheus format and ...
prometheus-community/alertmanager 0.19.0 v0.23.0 The Alertmanager handles alerts sent by client ...
prometheus-community/kube-state-metrics 4.15.0 2.5.0 Install kube-state-metrics to generate and expo...
prometheus-community/prom-label-proxy 0.1.0 v0.5.0 A proxy that enforces a given label in a given ...
There are quite a few charts there. To get more information about a specific chart, we can use the show
command (you can use the inspectalias
command too). Let’s check out prometheus-community/prometheus
:
$ helm show chart prometheus-community/prometheus
apiVersion: v2
appVersion: 2.36.2
dependencies:
- condition: kubeStateMetrics.enabled
name: kube-state-metrics
repository: https://prometheus-community.github.io/helm-charts
version: 4.13.*
description: Prometheus is a monitoring system and time series database.
home: https://prometheus.io/
icon: https://raw.githubusercontent.com/prometheus/prometheus.github.io/master/assets/prometheus_logo-cb55bb5c346.png
maintainers:
- email: [email protected]
name: gianrubio
- email: [email protected]
name: zanhsieh
- email: [email protected]
name: Xtigyro
- email: [email protected]
name: naseemkullah
name: prometheus
sources:
- https://github.com/prometheus/alertmanager
- https://github.com/prometheus/prometheus
- https://github.com/prometheus/pushgateway
- https://github.com/prometheus/node_exporter
- https://github.com/kubernetes/kube-state-metrics
type: application
version: 15.12.0
You can also ask Helm to show you the README
file, the values, or all the information associated with a chart. This can be overwhelming at times.
OK. You’ve found the package of your dreams. Now, you probably want to install it on your Kubernetes cluster. When you install a package, Helm creates a release that you can use to keep track of the installation progress. We install prometheus
using the helm install
command in the monitoring namespace and instruct Helm to create the namespace for us:
$ helm install prometheus prometheus-community/prometheus -n monitoring --create-namespace
Let’s go over the output. The first part of the output lists the name of the release that we provided, prometheus
, when it was deployed, the namespace, and the revision:
NAME: prometheus
LAST DEPLOYED: Sat Aug 6 23:54:50 2022
NAMESPACE: monitoring
STATUS: deployed
REVISION: 1
TEST SUITE: None
The next part is custom notes, which can be pretty wordy. There is a lot of good information here about how to connect to the Prometheus server, the alert manager, and the Pushgateway
:
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-server.default.svc.cluster.local
Get the Prometheus server URL by running these commands in the same shell: test | default
export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 9090
The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster:
prometheus-alertmanager.default.svc.cluster.local
Get the Alertmanager URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=alertmanager" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 9093
#################################################################################
###### WARNING: Pod Security Policy has been moved to a global property. #####
###### use .Values.podSecurityPolicy.enabled with pod-based #####
###### annotations #####
###### (e.g. .Values.nodeExporter.podSecurityPolicy.annotations) #####
#################################################################################
The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster:
prometheus-pushgateway.default.svc.cluster.local
Get the PushGateway URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=pushgateway" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 9091
For more information on running Prometheus, visit:
https://prometheus.io/
Helm doesn’t wait for the installation to complete because it may take a while. The helm status
command displays the latest information on a release in the same format as the output of the initial helm install
command.
If you just care about the status without all the extra information, you can just grep
for the STATUS
line:
$ helm status -n monitoring prometheus | grep STATUS
STATUS: deployed
Let’s list all the Helm releases in the monitoring
namespace and verify that prometheus
is listed:
$ helm list -n monitoring
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
prometheus monitoring 1 2022-08-06 23:57:34.124225 -0700 PDT deployed prometheus-15.12.0 2.36.2
As you recall, Helm stores the release information in a secret:
$ kubectl describe secret sh.helm.release.v1.prometheus.v1 -n monitoring
Name: sh.helm.release.v1.prometheus.v1
Namespace: monitoring
Labels: modifiedAt=1659855458
name=prometheus
owner=helm
status=deployed
version=1
Annotations: <none>
Type: helm.sh/release.v1
Data
====
release: 51716 bytes
If you want to find all the Helm releases across all namespaces, use:
$ helm list -A
If you want to go low-level, you can list all the secrets that have the owner=helm
label:
$ kubectl get secret -A -l owner=helm
To actually extract the release data from a secret, you need to jump through some hoops as it is Base64-encoded twice (why?) and GZIP-compressed. The final result is JSON:
kubectl get secret sh.helm.release.v1.prometheus.v1 -n monitoring -o jsonpath='{.data.release}' | base64 --decode | base64 --decode | gunzip > prometheus.v1.json
You may also be interested in extracting just the manifests using the following command:
kubectl get secret sh.helm.release.v1.prometheus.v1 -n monitoring -o jsonpath='{.data.release}' | base64 --decode | base64 --decode | gunzip | jq .manifest -r
Very often as a user, you want to customize or configure the charts you install. Helm fully supports customization via config files. To learn about possible customizations, you can use the helm show
command again, but this time, focus on the values. For a complex project like Prometheus, the values
file can be pretty large:
$ helm show values prometheus-community/prometheus | wc -l
1901
Here is a partial output:
$ helm show values prometheus-community/prometheus | head -n 20
rbac:
create: true
podSecurityPolicy:
enabled: false
imagePullSecrets:
# - name: "image-pull-secret"
## Define serviceAccount names for components. Defaults to component's fully qualified name.
##
serviceAccounts:
alertmanager:
create: true
name:
annotations: {}
nodeExporter:
create: true
name:
annotations: {}
Commented-out lines often contain default values like the name of the imagePullSecrets
:
imagePullSecrets:
# - name: "image-pull-secret"
If you want to customize any part of the Prometheus installation, then save the values to a file, make any modifications you like, and then install Prometheus using the custom values file:
$ helm install prometheus prometheus-community/prometheus --create-namespace -n monitoring -f custom-values.yaml
You can also set individual values on the command line with --set
. If both -f
and --set
try to set the same values, then --set
takes precedence. You can specify multiple values using comma-separated lists: --set a=1,b=2
. Nested values can be set as --set outer.inner=value
.
The helm install
command can work with a variety of sources:
helm install foo-0.1.1.tgz
)helm install path/to/foo
)helm install https://example.com/charts/foo-1.2.3.tgz
)You may want to upgrade a package you installed to the latest and greatest version. Helm provides the upgrade
command, which operates intelligently and only updates things that have changed. For example, let’s check the current values of our prometheus
installation:
$ helm get values prometheus -n monitoring
USER-SUPPLIED VALUES:
null
So far, we haven’t provided any user values. As part of the default installation, prometheus
installed an alert manager component:
$ k get deploy prometheus-alertmanager -n monitoring
NAME READY UP-TO-DATE AVAILABLE AGE
prometheus-alertmanager 1/1 1 1 19h
Let’s disable the alert manager by upgrading and passing a new value:
$ helm upgrade --set alertmanager.enabled=false
prometheus prometheus-community/prometheus
-n monitoring
Release "prometheus" has been upgraded. Happy Helming!
NAME: prometheus
LAST DEPLOYED: Sun Aug 7 19:55:52 2022
NAMESPACE: monitoring
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
...
The upgrade completed successfully. We can see that the output doesn’t mention how to get the URL of the alert manager anymore. Let’s verify that the alert manager deployment was removed:
$ k get deployment -n monitoring
NAME READY UP-TO-DATE AVAILABLE AGE
prometheus-kube-state-metrics 1/1 1 1 20h
prometheus-pushgateway 1/1 1 1 20h
prometheus-server 1/1 1 1 20h
Now, if we check the custom values, we can see our modification:
$ helm get values prometheus -n monitoring
USER-SUPPLIED VALUES:
alertmanager:
enabled: false
Suppose we decide that alerts are kind of important and, actually, we want to have the Prometheus alert manager. No problem, we can roll back to our original installation. The helm history
command shows us all the available revisions we can roll back to:
$ helm history prometheus -n monitoring
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Sat Aug 6 23:57:34 2022 superseded prometheus-15.12.0 2.36.2 Install complete
2 Sun Aug 7 19:55:52 2022 deployed prometheus-15.12.0 2.36.2 Upgrade complete
Let’s roll back to revision 1:
$ helm rollback prometheus 1 -n monitoring
Rollback was a success! Happy Helming!
$ helm history prometheus -n monitoring
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Sat Aug 6 23:57:34 2022 superseded prometheus-15.12.0 2.36.2 Install complete
2 Sun Aug 7 19:55:52 2022 superseded prometheus-15.12.0 2.36.2 Upgrade complete
3 Sun Aug 7 20:02:30 2022 deployed prometheus-15.12.0 2.36.2 Rollback to 1
As you can see, the rollback actually created a new revision number 3. Revision 2 is still there in case we want to go back to it.
Let’s verify that our changes were rolled back:
$ k get deployment -n monitoring
NAME READY UP-TO-DATE AVAILABLE AGE
prometheus-alertmanager 1/1 1 1 152m
prometheus-kube-state-metrics 1/1 1 1 22h
prometheus-pushgateway 1/1 1 1 22h
prometheus-server 1/1 1 1 22h
Yep. The alert manager is back.
You can, of course, uninstall a release too by using the helm uninstall
command.
First, let’s examine the list of releases. We have only the prometheus
release in the monitoring namespace:
$ helm list -n monitoring
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
prometheus monitoring 3 2022-08-07 20:02:30.270229 -0700 PDT deployed prometheus-15.12.0 2.36.2
Now, let’s uninstall it. You can use any of the following equivalent commands:
uninstall
un
delete
del
Here we are using the uninstall
command:
$ helm uninstall prometheus -n monitoring
release "prometheus" uninstalled
With that, there are no more releases:
$ helm list -n monitoring
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
Helm can keep track of uninstalled releases too. If you provide --keep-history
when you uninstall, then you’ll be able to see uninstalled releases by adding the --all
or --uninstalled
flags to helm list
.
Note that the monitoring namespace remained even though it was created by Helm as part of installing Prometheus, but it is empty now:
$ k get all -n monitoring
No resources found in monitoring namespace.
Helm stores charts in repositories that are simple HTTP servers. Any standard HTTP server can host a Helm repository. In the cloud, the Helm team verified that AWS S3 and Google Cloud Storage can both serve as Helm repositories in web-enabled mode. You can even store Helm repositories on GitHub pages.
Note that Helm doesn’t provide tools for uploading charts to remote repositories because that would require the remote server to understand Helm, know where to put the chart, and know how to update the index.yaml
file.
Note that Helm recently added experimental support for storing Helm charts in OCI registries. Check out https://helm.sh/docs/topics/registries/ for more details.
On the client side, the helm repo
command lets you list, add, remove, index, and update:
$ helm repo
This command consists of multiple subcommands to interact with chart repositories.
It can be used to add, remove, list, and index chart repositories.
Usage:
helm repo [command]
Available Commands:
add add a chart repository
index generate an index file given a directory containing packaged charts
list list chart repositories
remove remove one or more chart repositories
update update information of available charts locally from chart repositories
We already used the helm repo add
and helm repo list
commands earlier. Let’s see how to create our own charts and manage them.
Helm provides several commands to manage charts.
It can create a new chart for you:
$ helm create cool-chart
Creating cool-chart
Helm will create the following files and directories under cool-chart
:
$ tree cool-chart
cool-chart
├── Chart.yaml
├── charts
├── templates
│ ├── _helpers.tpl
│ ├── deployment.yaml
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── service.yaml
│ ├── serviceaccount.yaml
│ └── tests
│ └── test-connection.yaml
Once you have edited your chart, you can package it into a tar.gz
archive:
$ helm package cool-chart
Successfully packaged chart and saved it to: cool-chart-0.1.0.tgz
Helm will create an archive called cool-chart-0.1.0.tgz
and store it in the local directory.
You can also use helm lint
to help you find issues with your chart’s formatting or information:
$ helm lint cool-chart
==> Linting cool-chart
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed
The helm create
command offers an optional --starter
flag, allowing you to specify a starter chart. Starters are regular charts located in $XDG_DATA_HOME/helm/starters
. As a chart developer, you can create charts explicitly intended to serve as starter templates for creating new charts. When developing such charts, please keep the following considerations in mind:
Presently, there is no built-in mechanism for installing starter charts. The only way to add a chart to $XDG_DATA_HOME/helm/starters
is through manual copying. If you create starter pack charts, ensure that your chart’s documentation explicitly mentions this requirement.
A chart represents a group of files that define a cohesive set of Kubernetes resources. It can range from a simple deployment of a Memcached pod to a complex configuration of a complete web application stack, including HTTP servers, databases, caches, queues, and more.
To organize a chart, its files are structured within a specific directory tree. These files can then be bundled into versioned archives, which can be easily deployed and managed. The key file is Chart.yaml
.
The Chart.yaml
file is the main file of a Helm chart. It requires a name and version fields:
apiVersion
: The API version of the chart.name
: The name of the chart, which should match the directory name.version
: The version of the chart using the SemVer
2 format.Additionally, there are several optional fields that can be included in the Chart.yaml
file:
kubeVersion
: A range of compatible Kubernetes versions specified in SemVer
format.description
: A brief description of the project in a single sentence.keywords
: A list of keywords associated with the project.home
: The URL of the project’s homepage.sources
: A list of URLs to the project’s source code.dependencies
: A list of dependencies for the chart, including the name, version, repository, condition, tags, and alias.maintainers
: A list of maintainers for the chart, including the name, email, and URL.icon
: The URL to an SVG or PNG image that can be used as an icon.appVersion
: The version of the application contained within the chart. It does not have to follow SemVer
.deprecated
: A boolean value indicating whether the chart is deprecated.annotations
: Additional key-value pairs that provide extra information.The version
field in the Chart.yaml
file plays a crucial role for various Helm tools. It is used by the helm package
command when creating a package, as it constructs the package name based on the version specified in the Chart.yaml
. It is important to ensure that the version number in the package name matches the version number in the Chart.yaml
file. Deviating from this expectation can result in an error, as the system assumes the consistency of these version numbers. Therefore, it is essential to maintain the coherence between the version field in the Chart.yaml
file and the generated package name to avoid any issues.
The optional appVersion
field is not related to the version
field. It is not used by Helm and serves as metadata or documentation for users that want to understand what they are deploying. Helm ignores it.
From time to time, you may want to deprecate a chart. You can mark a chart as deprecated by setting the optional deprecated
field in Chart.yaml
to true
. It’s enough to deprecate the latest version of a chart. You can later reuse the chart name and publish a newer version that is not deprecated. The workflow for deprecating charts typically involves the following steps:
Chart.yaml
file of the chart to indicate that it is deprecated. This can be done by adding a deprecated
field and setting it to true
. Additionally, it is common practice to bump the version of the chart to indicate that a new version with deprecation information has been released.By following these steps, you can effectively deprecate a chart and provide a clear process for users to transition to newer versions or alternative solutions.
Charts can include several metadata files, such as README.md
, LICENSE
, and NOTES.txt
, which provide important information about the chart. The README.md
file, formatted as markdown, is particularly crucial and should contain the following details:
By including these details in the README.md
file, chart users can easily understand the chart’s purpose, requirements, and how to effectively install and configure it for their specific use case.
If the chart contains a template or NOTES.txt
file, then the file will be displayed, printed out after installation and when viewing the release status, or upgraded. The notes should be concise to avoid clutter and point to the README.md
file for detailed explanations. It’s common to put usage notes and next steps in NOTES.txt
. Remember that the file is evaluated as a template. The notes are printed on the screen when you run helm install
as well as helm status
.
In Helm, a chart may depend on other charts. These dependencies are expressed explicitly by listing them in the dependencies
field of the Chart.yaml
file or copied directly to the charts/
subdirectory. This provides a great way to benefit from and reuse the knowledge and work of others. A dependency in Helm can take the form of either a chart archive (e.g., foo-1.2.3.tgz
) or an unpacked chart directory. However, it’s important to note that the name of a dependency should not begin with an underscore (_
) or a period (.
), as these files are ignored by the chart loader. Therefore, it’s recommended to avoid starting dependency names with these characters to ensure they are properly recognized and loaded by Helm.
Let’s add kube-state-metrics
from the prometheus-community
repo as a dependency to our cool-chart
's Chart.yaml
file:
dependencies:
- name: kube-state-metrics
version: "4.13.*"
repository: https://prometheus-community.github.io/helm-charts
condition: kubeStateMetrics.enabled
The name
field represents the desired name of the chart you want to install. It should match the name of the chart as it is defined in the repository.
The version
field specifies the specific version of the chart you want to install. It helps to ensure that you get the desired version of the chart.
The repository
field contains the complete URL of the chart repository that the chart will be fetched from. It points to the location where the chart and its versions are stored and can be accessed.
The condition
field is discussed in the subsequent section.
If the repository is not added yet use helm repo
to add it locally.
Once your dependencies are defined, you can run the helm dependency update
command. Helm will download all of the specified charts into the charts subdirectory for you:
$ helm dep up cool-chart
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "prometheus-community" chart repository
Update Complete. Happy Helming!
Saving 1 charts
Downloading kube-state-metrics from repo https://prometheus-community.github.io/helm-charts
Deleting outdated charts
Helm stores the dependency charts as archives in the charts/
directory:
$ ls cool-chart/charts
kube-state-metrics-4.13.0.tgz
Managing charts and their dependencies in the Chart.yaml
dependencies field (as opposed to just copying charts into the charts/
subdirectory) is a best practice. It explicitly documents dependencies, facilitates sharing across the team, and supports automated pipelines.
Each entry in the requirements.yaml
file’s requirements
entry may include optional fields such as tags
and condition
.
These fields can be used to dynamically control the loading of charts (if not specified all charts will be loaded). If tags
or condition
fields are present, Helm will evaluate them and determine if the target chart should be loaded or not.
condition
field in chart dependencies holds one or more comma-delimited YAML paths. These paths refer to values in the top parent’s values file. If a path exists and evaluates to a Boolean value, it determines whether the chart will be enabled or disabled. If multiple paths are provided, only the first valid path encountered is evaluated. If no paths exist, the condition has no effect, and the chart will be loaded regardless.tags
field allows you to associate labels with the chart. It is a YAML list where you can specify one or more tags. In the top parent’s values file, you can enable or disable all charts with specific tags by specifying the tag and a corresponding Boolean value. This provides a convenient way to manage and control charts based on their associated tags.Here is an example dependencies
field and a values.yaml
that makes good use of conditions and tags to enable and disable the installation of dependencies. The dependencies
field defines two conditions for installing its dependencies based on the value of the global enabled field and the specific subchart’s enabled field:
dependencies:
- name: subchart1
repository: http://localhost:10191
version: 0.1.0
condition: subchart1.enabled, global.subchart1.enabled
tags:
- front-end
- subchart1
- name: subchart2
repository: http://localhost:10191
version: 0.1.0
condition: subchart2.enabled,global.subchart2.enabled
tags:
- back-end
- subchart2
The values.yaml
file assigns values to some of the condition
variables. The subchart2
tag doesn’t get a value, so it is enabled automatically:
# parentchart/values.yaml
subchart1:
enabled: true
tags:
front-end: false
back-end: true
You can set tags
and condition
values from the command line too when installing a chart, and they’ll take precedence over the values.yaml
file:
$ helm install --set subchart2.enabled=false
The resolution of tags and conditions is as follows:
condition
path that exists per chart takes effect, and other conditions are ignored.true
in the top parent’s values, the chart is considered enabled. tags
and condition
values must be set at the top level of the values file. Any non-trivial application will require configuration and adaptation to the specific use case. Helm charts are templates that use the Go template language to populate placeholders. Helm supports additional functions from the Sprig library, which contains a lot of useful helpers as well as several other specialized functions. The template files are stored in the templates/
subdirectory of the chart. Helm will use the template engine to render all files in this directory and apply the provided value files.
Template files are just text files that follow the Go template language rules. They can generate Kubernetes configuration files as well as any other file. Here is the service template file of the Prometheus server’s service.yaml
template from the prometheus-community
repo:
{{- if and .Values.server.enabled .Values.server.service.enabled -}}
apiVersion: v1
kind: Service
metadata:
{{- if .Values.server.service.annotations }}
annotations:
{{ toYaml .Values.server.service.annotations | indent 4 }}
{{- end }}
labels:
{{- include "prometheus.server.labels" . | nindent 4 }}
{{- if .Values.server.service.labels }}
{{ toYaml .Values.server.service.labels | indent 4 }}
{{- end }}
name: {{ template "prometheus.server.fullname" . }}
{{ include "prometheus.namespace" . | indent 2 }}
spec:
{{- if .Values.server.service.clusterIP }}
clusterIP: {{ .Values.server.service.clusterIP }}
{{- end }}
{{- if .Values.server.service.externalIPs }}
externalIPs:
{{ toYaml .Values.server.service.externalIPs | indent 4 }}
{{- end }}
{{- if .Values.server.service.loadBalancerIP }}
loadBalancerIP: {{ .Values.server.service.loadBalancerIP }}
{{- end }}
{{- if .Values.server.service.loadBalancerSourceRanges }}
loadBalancerSourceRanges:
{{- range $cidr := .Values.server.service.loadBalancerSourceRanges }}
- {{ $cidr }}
{{- end }}
{{- end }}
ports:
- name: http
port: {{ .Values.server.service.servicePort }}
protocol: TCP
targetPort: 9090
{{- if .Values.server.service.nodePort }}
nodePort: {{ .Values.server.service.nodePort }}
{{- end }}
{{- if .Values.server.service.gRPC.enabled }}
- name: grpc
port: {{ .Values.server.service.gRPC.servicePort }}
protocol: TCP
targetPort: 10901
{{- if .Values.server.service.gRPC.nodePort }}
nodePort: {{ .Values.server.service.gRPC.nodePort }}
{{- end }}
{{- end }}
selector:
{{- if and .Values.server.statefulSet.enabled .Values.server.service.statefulsetReplica.enabled }}
statefulset.kubernetes.io/pod-name: {{ template "prometheus.server.fullname" . }}-{{ .Values.server.service.statefulsetReplica.replica }}
{{- else -}}
{{- include "prometheus.server.matchLabels" . | nindent 4 }}
{{- if .Values.server.service.sessionAffinity }}
sessionAffinity: {{ .Values.server.service.sessionAffinity }}
{{- end }}
{{- end }}
type: "{{ .Values.server.service.type }}"
{{- end -}}
It is available here: https://github.com/prometheus-community/helm-charts/blob/main/charts/prometheus/templates/service.yaml.
Don’t worry if it looks confusing. The basic idea is that you have a simple text file with placeholders for values that can be populated later in various ways as well as conditions, some functions, and pipelines that can be applied to those values.
Helm allows rich and sophisticated syntax in the template files via the built-in Go template functions, Sprig functions, and pipelines. Here is an example template that takes advantage of these capabilities. It uses the repeat
, quote
, and upper
functions for the food
and drink
keys, and it uses pipelines to chain multiple functions together:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-configmap
data:
greeting: "Hello World"
drink: {{ .Values.favorite.drink | repeat 3 | quote }}
food: {{ .Values.favorite.food | upper }}
Let’s add a values.yaml
file:
favorite:
drink: coffee
food: pizza
Now, we can use the helm template
command to see the result:
$ helm template food food-chart
---
# Source: food-chart/templates/config-map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: food-configmap
data:
greeting: "Hello World"
drink: "coffeecoffeecoffee"
food: PIZZA
As you can see, our templating worked. The drink coffee
was repeated 3 times and quoted. The food pizza
became uppercase PIZZA
(unquoted).
Another good way of debugging is to run install
with the --dry-run
flag. It provides additional information:
$ helm install food food-chart --dry-run -n monitoring
NAME: food
LAST DEPLOYED: Mon Aug 8 00:24:03 2022
NAMESPACE: monitoring
STATUS: pending-install
REVISION: 1
TEST SUITE: None
HOOKS:
MANIFEST:
---
# Source: food-chart/templates/config-map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: food-configmap
data:
greeting: "Hello World"
drink: "coffeecoffeecoffee"
food: PIZZA
You can also override values on the command line:
$ helm template food food-chart --set favorite.drink=water
---
# Source: food-chart/templates/config-map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: food-configmap
data:
greeting: "Hello World"
drink: "waterwaterwater"
food: PIZZA
The ultimate test is, of course, to install your chart into your cluster. You don’t need to upload your chart to a chart repository for testing; just run helm install
locally:
$ helm install food food-chart -n monitoring
NAME: food
LAST DEPLOYED: Mon Aug 8 00:25:53 2022
NAMESPACE: monitoring
STATUS: deployed
REVISION: 1
TEST SUITE: None
There is now a Helm release called food
:
$ helm list -n monitoring
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
food monitoring 1 2022-08-08 00:25:53.587342 -0700 PDT deployed food-chart-0.1.0 1.16.0
Most importantly, the food-configmap
config map was created with the correct data:
$ k get cm food-configmap -o yaml -n monitoring
apiVersion: v1
data:
drink: coffeecoffeecoffee
food: PIZZA
greeting: Hello World
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: food
meta.helm.sh/release-namespace: monitoring
creationTimestamp: "2022-08-08T07:25:54Z"
labels:
app.kubernetes.io/managed-by: Helm
name: food-configmap
namespace: monitoring
resourceVersion: "4247163"
uid: ada4957d-bd6d-4c2e-8b2c-1499ca74a3c3
Helm provides some built-in objects you can use in your templates. In the Prometheus chart template above, Release.Name
, Release.Service
, Chart.Name
, and Chart.Version
are examples of Helm predefined values. Other objects are:
Values
Chart
Template
Files
Capabilities
The Values
object contains all the values defined in the values
file or on the command line. The Chart
object is the content of Chart.yaml
. The Template
object contains information about the current template. Files
and Capabilities
are map-like objects that allow access via various functions to the non-specialized files and general information about the Kubernetes cluster.
Note that unknown fields in Chart.yaml
are ignored by the template engine and cannot be used to pass arbitrary structured data to templates.
Here is part of the Prometheus server’s default values
file. The values from this file are used to populate multiple templates. The values represent defaults that you can override by copying the file and modifying it to fit your needs. Note the useful comments that explain the purpose and various options for each value:
server:
## Prometheus server container name
##
enabled: true
## Use a ClusterRole (and ClusterRoleBinding)
## - If set to false - we define a RoleBinding in the defined namespaces ONLY
##
## NB: because we need a Role with nonResourceURL's ("/metrics") - you must get someone with Cluster-admin privileges to define this role for you, before running with this setting enabled.
## This makes prometheus work - for users who do not have ClusterAdmin privs, but wants prometheus to operate on their own namespaces, instead of clusterwide.
##
## You MUST also set namespaces to the ones you have access to and want monitored by Prometheus.
##
# useExistingClusterRoleName: nameofclusterrole
## namespaces to monitor (instead of monitoring all - clusterwide). Needed if you want to run without Cluster-admin privileges.
# namespaces:
# - yournamespace
name: server
# sidecarContainers - add more containers to prometheus server
# Key/Value where Key is the sidecar `- name: <Key>`
# Example:
# sidecarContainers:
# webserver:
# image: nginx
sidecarContainers: {}
That was a deep dive into creating your own charts with Helm. Well, Helm is used widely and extensively to package and deploy Kubernetes applications. However, Helm is not the only game in town. There are several good alternatives that you may prefer. In the next section, we will review some of the most promising Helm alternatives.
Helm is battle tested and very common in the Kubernetes world, but it has its downsides and critics, especially when you develop your own charts. A lot of the criticism was about Helm 2 and its server-side component, Tiller. However, Helm 3 is not a panacea either. On a large scale, when you develop your own charts and complex templates with lots of conditional logic and massive values
files, it can become very challenging to manage.
If you feel the pain, you may want to investigate some alternatives. Note that most of these projects focus on the deployment aspect. Helm’s dependency management is still a strength. Let’s look at some interesting projects that you may want to consider.
Kustomize is an alternative to YAML templating by using the concept of overlays on top of raw YAML files. It was added to kubectl in Kubernetes 1.14.
See https://github.com/kubernetes-sigs/kustomize.
Cue is a very interesting project. Its data validation language and inference were strongly inspired by logic programming. It is not a general-purpose programming language. It is focused on data validation, data templating, configuration, querying, and code generation, but has some scripting too. The main concept of Cue is the unification of types and data. That gives Cue a lot of expressive power and obviates the need for constructs like enums and generics.
See https://cuelang.org.
See the specific discussion about replacing Helm with Cue here: https://github.com/cue-lang/cue/discussions/1159.
kapp-controller provides continuous delivery and package management capabilities for Kubernetes.
Its declarative APIs and layered approach allow you to build, deploy, and manage your applications effectively. With Kapp-controller, you can package your software into distributable packages and empower users to discover, configure, and install these packages on a Kubernetes cluster seamlessly.
See https://carvel.dev/kapp-controller/.
That concludes our quick review of Helm alternatives.
In this chapter, we took a look at Helm, a popular Kubernetes package manager. Helm gives Kubernetes the ability to manage complicated software composed of many Kubernetes resources with inter-dependencies. It serves the same purpose as an OS package manager. It organizes packages and lets you search charts, install and upgrade charts, and share charts with collaborators. You can develop your own charts and store them in repositories. Helm 3 is a client-side-only solution that uses Kubernetes secrets to manage the state of releases in your cluster. We also looked at some Helm alternatives.
At this point, you should understand the important role that Helm serves in the Kubernetes ecosystem and community. You should be able to use it productively and even develop and share your own charts.
In the next chapter, we will look at how Kubernetes does networking at a pretty low level.