The principle of least privilege states that each component of an ecosystem should have minimal access to data and resources for it to function. In a multitenant environment, multiple resources can be accessed by different users or objects. The principle of least privilege ensures that damage to the cluster is minimal if users or objects misbehave in such environments.
In this chapter, we will first introduce the principle of least privilege. Given the complexity of Kubernetes, we will first look into the Kubernetes subjects, and then the privileges available for the subjects. Then, we will talk about the privileges of Kubernetes objects and possible ways to restrict them. The goal of this chapter is to help you understand a few critical concepts, such as the principle of least privilege and Role-Based Access Control (RBAC). In this chapter, we will talk about different Kubernetes objects, such as namespaces, service accounts, Roles, and RoleBindings, and Kubernetes security features, such as the security context, the PodSecurityPolicy, and the NetworkPolicy, which can be leveraged to implement the principle of least privilege for your Kubernetes cluster.
In this chapter, we will cover the following topics:
Privilege is the authority to perform an action such as accessing a resource or processing some data. The principle of least privilege is the idea that any subject, user, program, process, and so on should only have the minimum required privileges to perform its function. For example, Alice, a regular Linux user, is able to create a file under her own home directory. In other words, Alice at least has the privilege or permission to create a file under her home directory. However, Alice may not be able to create a file under another user's directory because she doesn't have the privilege or permission to do so. If none of Alice's daily tasks actually exercises the privilege to create a file in the home directory, but she does have the privilege to do so, then the administrator for the machine is not complying with the principle of least privilege. In this section, we will first introduce the concept of the authorization model from which the concept of least privilege derived, and then, we will talk about the benefits of implementing the principle of least privilege.
When we talk about least privilege, most of the time we talk in the context of authorization, and in different environments, there will be different authorization models. For example, an Access Control List (ACL) is widely used in Linux and network firewalls, while RBAC is used in database systems. It is also up to the administrator of the environment to define authorization policies to ensure least privilege based on authorization models available in the system. The following list defines some popular authorization models:
Kubernetes supports both ABAC and RBAC. Though ABAC is powerful and flexible, the implementation in Kubernetes makes it difficult to manage and understand. Thus, it is recommended to enable RBAC instead of ABAC in Kubernetes. Besides RBAC, Kubernetes also provides multiple ways to restrict resource access. Before we look into RBAC and ABAC in Kubernetes in the next sections, let's discuss the benefits of ensuring least privilege.
Though it might take quite some time to understand what the minimum privileges for subjects are in order to perform their functions, the rewards are also significant if the principle of least privilege has been implemented in your environment:
Now that you have seen the benefits for implementing the principle of least privilege, I want to introduce the challenge as well: the openness and configurability of Kubernetes makes implementing the principle of least privilege cumbersome. Let's look at how to apply the principle of least privilege to Kubernetes subjects.
Kubernetes service accounts, users, and groups communicate with kube-apiserver to manage Kubernetes objects. With RBAC enabled, different users or service accounts may have different privileges to operate Kubernetes objects. For example, users in the system:master group have the cluster-admin role granted, meaning they can manage the entire Kubernetes cluster, while users in the system:kube-proxy group can only access the resources required by the kube-proxy component. First, let's briefly talk about what RBAC is.
As discussed earlier, RBAC is a model of regulating access to resources based on roles granted to users or groups. From version 1.6 onward, RBAC is enabled by default in Kubernetes. Before version 1.6, RBAC could be enabled by running the Application Programming Interface (API) server with the --authorization-mode=RBAC flag. RBAC eases the dynamic configuration of permission policies using the API server.
The core elements of RBAC include the following:
Kubernetes RBAC defines the subjects and the type of access they have to different resources in the Kubernetes ecosystem.
Kubernetes supports three types of subject, as follows:
Cluster administrators can create new service accounts to be associated with pods by running the following command:
$ kubectl create serviceaccount new_account
A new_account service account will be created in the default namespace. To ensure least privilege, cluster administrators should associate every Kubernetes resource with a service account with least privilege to operate.
A role is a collection of permissions—for example, a role in namespace A can allow users to create pods in namespace A and list secrets in namespace A. In Kubernetes, there are no deny permissions. Thus, a role is an addition of a set of permissions.
A role is restricted to a namespace. On the other hand, a ClusterRole works at the cluster level. Users can create a ClusterRole that spans across the complete cluster. A ClusterRole can be used to mediate access to resources that span across a cluster, such as nodes, health checks, and namespaced objects, such as pods across multiple namespaces. Here is a simple example of a role definition:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: role-1
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get"]
This simple rule allows the get operation to over-resource pods in the default namespace. This role can be created using kubectl by executing the following command:
$ kubectl apply -f role.yaml
A user can only create or modify a role if either one of the following is true:
This prevents users from performing privilege escalation attacks by modifying user roles and permissions.
A RoleBinding object is used to associate a role with subjects. Similar to ClusterRole, ClusterRoleBinding can grant a set of permissions to subjects across namespaces. Let's see a couple of examples:
kubectl create rolebinding new-rolebinding-sa
--clusterrole=custom-clusterrole
--serviceaccount=default:demo-sa
kubectl create rolebinding new-rolebinding-group
--clusterrole=custom-clusterrole
--group=group-1
--namespace=namespace-1
The RoleBinding object links roles to subjects and makes roles reusable and easy to manage.
A namespace is a common concept in computer science that provides a logical grouping for related resources. Namespaces are used to avoid name collisions; resources within the same namespace should have unique names, but resources across namespaces can share names. In the Linux ecosystem, namespaces allow the isolation of system resources.
In Kubernetes, namespaces allow a single cluster to be shared between teams and projects logically. With Kubernetes namespaces, the following applies:
By default, Kubernetes has three different namespaces. Run the following command to view them:
$ kubectl get namespace
NAME STATUS AGE
default Active 1d
kube-system Active 1d
kube-public Active 1d
The three namespaces are described as follows:
Let's take a look at how to create a namespace.
A new namespace in Kubernetes can be created by using the following command:
$ kubectl create namespace test
Once a new namespace is created, objects can be assigned to a namespace by using the namespace property, as follows:
$ kubectl apply --namespace=test -f pod.yaml
Objects within the namespace can similarly be accessed by using the namespace property, as follows:
$ kubectl get pods --namespace=test
In Kubernetes, not all objects are namespaced. Lower-level objects such as Nodes and persistentVolumes span across namespaces.
By now, you should be familiar with the concepts of ClusterRole/Role, ClusterRoleBinding/RoleBinding, service accounts, and namespaces. In order to implement least privilege for Kubernetes subjects, you may ask yourself the following questions before you create a Role or RoleBinding object in Kubernetes:
This is important because once the subject has cluster-level privileges it may be able to exercise the privileges across all namespaces.
When you grant a role to a group, it means all the users in the group will automatically get the privileges from the newly granted role. Be sure you understand the impact before you grant a role to a group. Next, a user in Kubernetes is for humans, while a service account is for microservices in pods. Be sure you know what the Kubernetes user's responsibility is and assign privileges accordingly. Also, note that some microservices do not need any privilege at all as they don't interact with kube-apiserver or any Kubernetes objects directly.
When creating a role, if you don't specify the resource name or do set * in the resourceNames field, it means access is granted to all the resources of the resource type. If you know which resource name the subject is going to access, do specify the resource name when creating a role.
Kubernetes subjects interact with Kubernetes objects with the granted privileges. Understanding the actual tasks your Kubernetes subjects perform will help you grant privileges properly.
Usually, there will be a service account (default) associated with a Kubernetes workload. Thus, processes inside a pod can communicate with kube-apiserver using the service account token. DevOps should carefully grant necessary privileges to the service account for the purpose of least privilege. We've already covered this in the previous section.
Besides accessing kube-apiserver to operate Kubernetes objects, processes in a pod can also access resources on the worker nodes and other pods/microservices in the clusters (covered in Chapter 2, Kubernetes Networking). In this section, we will talk about the possible least privilege implementation of access to system resources, network resources, and application resources.
Recall that a microservice running inside a container or pod is nothing but a process on a worker node isolated in its own namespace. A pod or container may access different types of resources on the worker node based on the configuration. This is controlled by the security context, which can be configured both at the pod level and the container level. Configuring the pod/container security context should be on the developers' task list (with the help of security design and review), while pod security policies—the other way to limit pod/container access to system resources at the cluster level—should be on DevOps's to-do list. Let's look into the concepts of security context, PodSecurityPolicy, and resource limit control.
A security context offers a way to define privileges and access control settings for pods and containers with regard to accessing system resources. In Kubernetes, the security context at the pod level is different from that at the container level, though there are some overlapping attributes that can be configured at both levels. In general, the security context provides the following features that allow you to apply the principle of least privilege for containers and pods:
We will talk more about security context in Chapter 8, Securing Pods.
The PodSecurityPolicy is a Kubernetes cluster-level resource that controls the attributes of pod specification relevant to security. It defines a set of rules. When pods are to be created in the Kubernetes cluster, the pods need to comply with the rules defined in the PodSecurityPolicy or they will fail to start. The PodSecurityPolicy controls or applies the following attributes:
We will cover more about PodSecurityPolicy in Chapter 8, Securing Kubernetes Pods. A PodSecurityPolicy control is basically implemented as an admission controller. You can also create your own admission controller to apply your own authorization policy for your workload. Open Policy Agent (OPA) is another good candidate to implement your own least privilege policy for a workload. We will look at OPA more in Chapter 7, Authentication, Authorization, and Admission Control.
Now, let's look at the resource limit control mechanism in Kubernetes as you may not want your microservices to saturate all the resources, such as the Central Processing Unit (CPU) and memory, in the system.
By default, a single container can use as much memory and CPU resources as a node has. A container with a crypto-mining binary running may easily consume the CPU resources on the node shared by other pods. It's always a good practice to set resource requests and limits for workload. The resource request impacts which node the pods will be assigned to by the scheduler, while the resource limit sets the condition under which the container will be terminated. It's always safe to assign more resource requests and limits to your workload to avoid eviction or termination. However, do keep in mind that if you set the resource request or limit too high, you've caused a resource waste on your cluster, and the resources allocated to your workload may not be fully utilized. We will cover this topic more in Chapter 10, Real-Time Monitoring and Resource Management of a Kubernetes Cluster.
When pods or containers run in privileged mode, unlike the non-privileged pods or containers, they have the same privileges as admin users on the node. If your workload runs in privileged mode, why is this the case? When a pod is able to assess host-level namespaces, the pod can access resources such as the network stack, process, and Interprocess Communication (IPC) at the host level. But do you really need to grant host-level namespace access or set privileged mode to your pods or containers? Also, if you know which Linux capabilities are required for your processes in the container, you'd better drop those unnecessary ones. And how much memory and CPU is sufficient for your workload to be fully functional? Please do think through these questions for the purpose of implementing the principle of least privilege for your Kubernetes workload. Properly set resource requests and limits, use security context for your workload, and enforce a PodSecurityPolicy for your cluster. All of this will help ensure the least privilege for your workload to access system resources.
By default, any two pods inside the same Kubernetes cluster can communicate with other, and a pod may be able to communicate with the internet if there is no proxy rule or firewall rule configured outside the Kubernetes cluster. The openness of Kubernetes blurs the security boundary of microservices, and we mustn't overlook network resources such as API endpoints provided by other microservices that a container or pod can access.
Suppose one of your workloads (pod X) in namespace X only needs to access another microservice A in namespace NS1; meanwhile, there is microservice B in namespace NS2. Both microservice A and microservice B expose their Representational State Transfer (RESTful) endpoints. By default, your workload can access both microservice A and B assuming there is neither authentication nor authorization at the microservice level, and also no network policies enforced in namespaces NS1 and NS2. Take a look at the following diagram, which illustrates this:
In the preceding diagram, Pod X is able to access both microservices, though they reside in different namespaces. Note also that Pod X only requires access to Microservice A in namespace NS1. So, is there anything we can do to restrict Pod X's access to Microservice A only for the purpose of least privilege? Yes: a Kubernetes network policy can help. We will cover network policies in more detail Chapter 5, Configuring Kubernetes Security Boundaries. In general, a Kubernetes network policy defines rules of how a group of pods are allowed to communicate with each other and other network endpoints. You can define both ingress rules and egress rules for your workload.
Note
Ingress rules: Rules to define which sources are allowed to communicate with the pods under the protection of the network policy.
Egress rules: Rules to define which destinations are allowed to communicate with the pods under the protection of the network policy.
In the following example, to implement the principle of least privilege in Pod X, you will need to define a network policy in Namespace X with an egress rule specifying that only Microservice A is allowed:
In the preceding diagram, the network policy in Namespace X blocks any request from Pod X to Microservice B, and Pod X can still access Microservice A, as expected. Defining an egress rule in your network policy will help ensure least privilege for your workload to access network resources. Last but not least, we still need to bring your attention to the application resource level from a least-privilege standpoint.
Though this topic falls into the category of application security, it is worth bringing up here. If there are applications that your workload accesses that support multiple users with different levels of privileges, it's better to examine whether the privileges granted to the user on your workload's behalf are necessary or not. For example, a user who is responsible for auditing does not need any write privileges. Application developers should keep this in mind when designing the application. This helps to ensure the least privilege for your workload to access application resources.
In this chapter, we went through the concept of least privilege. Then, we discussed the security control mechanism in Kubernetes that helps in implementing the principle of least privilege in two areas: Kubernetes subjects and Kubernetes workloads. It is worth emphasizing the importance of implementing the principle of the principle of least privilege holistically. If least privilege is missed in any area, this will potentially leave an attack surface wide open.
Kubernetes offers built-in security controls to implement the principle of least privilege. Note that it is a process from development to deployment: application developers should work with security architects to design the minimum privileges for the service accounts associated with the application, as well as the minimum capabilities and proper resource allocation. During deployment, DevOps should consider using a PodSecurityPolicy and a network policy to enforce least privileges across the entire cluster.
In the next chapter, we will look at the security of Kubernetes from a different angle: understanding the security boundaries of different types of resources and how to fortify them.
You may have noticed that some of the security control mechanisms we talked about in this chapter have been around for a long time: SELinux Multi-Category Security/Multi-Level Security (MCS/MLS), AppArmor, seccomp, Linux capabilities, and so on. There are already many books or articles introducing these technologies. I would encourage you to take a look at the following materials for a better understanding of how to use them to achieve the least privilege goal in Kubernetes: