Setting resource in nodes

Computing resource management is so important in any infrastructure. We should know our application well and preserve enough CPU and memory capacity in order to prevent running out of resources. In this section, we'll introduce how to manage node capacity in the Kubernetes nodes. Furthermore, we'll also describe how to manage pod computing resources.

Getting ready

Before you start managing computing resources, you should know your applications well in order to know the maximum resources they need. Before we start, check out the current node capacity using the kubectl command described in Chapter 1, Building Your Own Kubernetes:

// check current node capacity
# kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, capacity: .status.capacity}'
{
  "name": "kube-node1",
  "capacity": {
    "cpu": "1",
    "memory": "1019428Ki",
    "pods": "40"
  }
}
{
  "name": "kube-node2",
  "capacity": {
    "cpu": "1",
    "memory": "1019428Ki",
    "pods": "40"
  }
}

You should know currently, we have two nodes with 1 CPU and 1019428 bytes memory. The node capacity of the pods are 40 for each. Then, we can start planning. How much computing resource capacity is allowed to be used on a node? How much computing resource is used in running our containers?

How to do it…

When the Kubernetes scheduler schedules a pod running on a node, it will always ensure that the total limits of the containers are less than the node capacity. If a node runs out of resources, Kubernetes will not schedule any new containers running on it. If no node is available when you launch a pod, the pod will remain pending, since the Kubernetes scheduler will be unable to find any node that could run your desired pod.

Managing node capacity

Sometimes, we want to explicitly preserve some resources for other processes or future usage on the node. Let's say we want to preserve 200 MB on all my nodes. First, we'll need to create a pod and run the pause container in Kubernetes. Pause is a container for each pod for forwarding the traffic. In this scenario, we'll create a resource reserver pod, which is basically doing nothing with a limit of 200 MB:

// configuration file for resource reserver
# cat /etc/kubernetes/reserve.yaml
apiVersion: v1
kind: Pod
metadata:
  name: resource-reserver
spec:
  containers:
  - name: resource-reserver
    image: gcr.io/google_containers/pause:0.8.0
    resources:
      limits:
        memory: 200Mi

Since it's a pod infra container, we will not use kubectl to launch it. Note that we put it in the /etc/kubernetes/ folder. You could put it under different paths; write down the path and we'll need to add it into the kubelet config file to launch it. Find the kubelet config file you specified in the Configuring nodes recipe in Chapter 1, Building Your Own Kubernetes and add the following argument when launching kubelet: --config=/etc/kubernetes/reserve.yaml. Restart kubelet. After we restart kubelet, we will see the kubelet log in the node:

I0325 20:44:22.937067   21306 kubelet.go:1960] Starting kubelet main sync loop.
I0325 20:44:22.937078   21306 kubelet.go:2012] SyncLoop (ADD): "resource-reserver-kube-node1_default"
I0325 20:44:22.937138   21306 kubelet.go:2012] SyncLoop (ADD): "mynginx-e09bu_default"
I0325 20:44:22.963425   21306 kubelet.go:2012] SyncLoop (ADD): "resource-reserver-kube-node1_default"
I0325 20:44:22.964484   21306 manager.go:1707] Need to restart pod infra container for "resource-reserver-kube-node1_default" because it is not found

kubelet will check whether the infra container exists and create it accordingly:

// check pods list
# kubectl get pods
NAME                             READY     STATUS    RESTARTS   AGE
resource-reserver-kube-node1   1/1       Running   0          3m

The Kubernetes master is aware that the resource reserver pod has been created. Let's describe the details to get deeper insight:

# kubectl describe pods resource-reserver-kube-node1
Name:        resource-reserver-kube-node1
Namespace:      default
Image(s):      gcr.io/google_containers/pause:0.8.0
Node:        kube-node1/10.0.0.224
Start Time:      Fri, 25 Mar 2016 20:44:24 +0000
Labels:        <none>
Status:        Running
IP:        192.168.99.3
Replication Controllers:  <none>
Containers:
  resource-reserver:
    ...
    QoS Tier:
      memory:  Guaranteed
    Limits:
      memory:  200Mi
    Requests:
      memory:    200Mi
    State:    Running
      Started:    Fri, 25 Mar 2016 20:44:24 +0000
    Ready:    True

We can find the limits and requests that are all set as 200Mi; it means that this container has been reserved a minimum of 200 MB and a maximum of 200 MB. Repeat the same steps in your other nodes and check the status via the kubectl command:

# kubectl get pods
NAME                             READY     STATUS    RESTARTS   AGE
resource-reserver-kube-node1   1/1       Running   0          11m
resource-reserver-kube-node2   1/1       Running   0          42m

Note

Limits or requests?

The Kubernetes scheduler schedules a pod running on a node by checking the remaining computing resources. We could specify the limits or requests for each pod we launch. Limit means the maximum resources this pod can occupy. Request means the minimum resources this pod needs. We could use the following inequality to represent their relation: 0 <= request <= the resource this pod occupies <= limit <= node capacity.

Managing computing resources in a pod

The concept for managing the capacity in a pod or node is similar. They both specify the requests or limits under the container resource spec.

Let's create an nginx pod with certain requests and limits using kubectl create -f nginx-resources.yaml to launch it:

# cat nginx-resources.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
    resources:
      requests:
        cpu: 250m
        memory: 32Mi
      limits:
        cpu: 500m
        memory: 64Mi

// create the pod
# kubectl create -f nginx-resources.yaml 
pod "nginx" created

Following are the available resources for this pod:

  • CPU: 250 milli core ~ 500 milli core
  • Memory: 32MB ~ 64 MB

Please note that the minimum CPU limit is set to 10 millicore. You cannot specify a value less than the minimum limit. Let's get more details via kubectl:

# kubectl describe pod nginx
Name:        nginx
Namespace:      default
Image(s):      nginx
Node:        kube-node1/10.0.0.224
Start Time:      Fri, 25 Mar 2016 21:12:43 +0000
Labels:        name=nginx
Status:        Running
Reason:
Message:
IP:        192.168.99.4
Replication Controllers:  <none>
Containers:
  nginx:
    ...
    QoS Tier:
      cpu:  Burstable
      memory:  Burstable
    Limits:
      memory:  64Mi
      cpu:  500m
    Requests:
      cpu:    250m
      memory:    32Mi
    State:    Running
      Started:    Fri, 25 Mar 2016 21:12:44 +0000
    Ready:    True
    Restart Count:  0

Everything is expected. QoS Tier is Burstable. Compared with Guaranteed, Burstable has a buffer to burst to the limits; however, Guaranteed will always reserve certain resources for the pod. Please note that if you specify too many pods with Guaranteed, cluster utilization would be poor, since it wastes the resources if the containers are not reaching the limits all the time.

See also

In this section, you learned how to constrain computing resources in Kubernetes. We give more control to our containers. Check out the following recipes:

  • The Preparing your environment and Configuring nodes recipes in Chapter 1, Building Your Own Kubernetes
  • The Working with namespaces recipe in Chapter 2, Walking through Kubernetes Concepts
  • The Working with configuration files recipe in Chapter 3, Playing with Containers
  • The Monitoring master and node recipe in Chapter 8, Logging and Monitoring
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset