The testing targets of the readiness probe are always containers, which means that it can't be used to disable a pod from a service by using external states. Since a service selects pods by their labels, we can control the traffic to pods by manipulating pod labels to a certain extent. However, pod labels are also read by other components inside Kubernetes, so building complex toggles with labels could lead to unexpected results.

The pod readiness gate is the feature that allows us to mark whether a pod is ready or not, based on the conditions we defined. With the pod readiness gates defined, a pod is regarded as ready only if its readiness probe passes and the status of all readiness gates associated with the pod is True. We can define the readiness gate as shown in the following snippet:

...
    spec:
      readinessGates:
      - conditionType: <value>
      containers:
      - name: main
...

The value must follow the format of a label key such as feature_1 or myorg.com/fg-2.

When a pod starts, a condition type we defined will be populated as a condition under a pod's .status.conditions[] path, and we have to explicitly set the condition to True to mark a pod ready. As for Kubernetes 1.13, the only way to edit the condition is with the patch API. Let's see an example at https://github.com/PacktPublishing/DevOps-with-Kubernetes-Second-Edition/blob/master/chapter9/9-3_on_pods/readiness_gates.yml:

$ cat chapter9/9-3_on_pods/readiness_gates.yml | grep readinessGates -C 1
    spec:
      readinessGates:
      - conditionType: "MY-GATE-1"

$ kubectl apply -f chapter9/9-3_on_pods/readiness_gates.yml
deployment.apps/my-2nd-app created
service/my-2nd-app-svc created

$ kubectl get pod -o custom-columns=NAME:.metadata.name,IP:.status.podIP
NAME                          IP
my-2nd-app-78786c6d5d-t4564   172.17.0.2

$ kubectl logs my-2nd-app-78786c6d5d-t4564
1544216932.875020742 - [app] starting server.

$ kubectl describe ep my-2nd-app-svc
Name: my-2nd-app-svc
Namespace: default
Labels: app=my-2nd-app
Annotations: <none>
Subsets:
  Addresses: <none>
  NotReadyAddresses: 172.17.0.2
...

Here, our custom condition is called MY-GATE-1 and the application is the one that we have used throughout this chapter. As we can see, even if the pod has started, its address is still listed in NotReadyAddresses. This means that the pod isn't taking any traffic. We can verify its status with describe (or wide/json/yaml):

$ kubectl describe pod my-2nd-app-78786c6d5d-t4564
...
Readiness Gates:
  Type        Status
  MY-GATE-1   <none>
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   True
  PodScheduled      True
...

The container in the pod is ready, but the pod itself isn't ready due to the readiness gates. To toggle it on, we'll need to make a request to the API server with a JSON Patch payload to /api/v1/namespaces/<namespace>/pods/<pod_name>/status:

$ kubectl proxy &
$ curl http://localhost:8001/api/v1/namespaces/default/pods/my-2nd-app-78786c6d5d-t4564/status 
-XPATCH -H "Content-Type: application/json-patch+json" -d 
'[{"op":"add","path":"/status/conditions/-","value":{"type":"MY-GATE-1", "status": "True"}}]'
...
  "status": {
    "phase": "Running",
    "conditions": [
...
      {
        "type": "MY-GATE-1",
        "status": "True",
        "lastProbeTime": null,
        "lastTransitionTime": null
      }
...

We'll see that an entry will be inserted into the .status.conditions list. Now, if we check the endpoints of the service, we can see that the pod has started to serve requests:

$ kubectl describe ep my-2nd-app-svc
Name: my-2nd-app-svc
Namespace: default
Labels: app=my-2nd-app
Annotations: <none>
Subsets:
  Addresses: 172.17.0.2
...
## also the status of the gates:
$ kubectl describe pod my-2nd-app-78786c6d5d-t4564 | grep -A2 Readiness
Readiness Gates:
  Type        Status
  MY-GATE-1   True

To put the pod in the other way around, we could use the replace or remove operation of JSON Patch to set the condition's status to False or <none>:

## we need the index of our gate, and we use jq to query it here:
$ export RG_NAME="MY-GATE-1"
$ kubectl get pod my-2nd-app-78786c6d5d-t4564 -o json | jq --arg R "$RG_NAME" 'foreach .status.conditions[] as $e (-1; .+1; select($e.type == $R))'
0
$ kubectl proxy &

## fill the queried "0" to the path parameter
$ curl http://localhost:8001/api/v1/namespaces/default/pods/my-2nd-app-78786c6d5d-t4564/status 
-XPATCH -H "Content-Type: application/json-patch+json" -d 
'[{"op":"replace","path":"/status/conditions/0","value":{"type":"MY-GATE-1", "status": "False"}}]'
...
$ kubectl describe ep my-2nd-app-svc | grep -B2 NotReadyAddresses
Subsets:
  Addresses: <none>
  NotReadyAddresses: 172.17.0.2

The pod now becomes unready again. With the readiness gate, we can nicely separate the logic of toggling business features and managing labels.