Stateless applications don't need to preserve the application or user data on the disk volume. Although stateless applications may write the data to the filesystem while a container exists, it is not important in terms of the application's life cycle.

For example, the tomcat container runs some web applications. It also writes an application log under /usr/local/tomcat/logs/, but it won't be affected if it loses a log file.

But what if you need to persist an application log for analysis or auditing? In this scenario, Tomcat can still be stateless but share the /usr/local/tomcat/logs volume with another container such as Logstash (https://www.elastic.co/products/logstash). Logstash will then send a log to the chosen analytic store, such as Elasticsearch (https://www.elastic.co/products/elasticsearch).

In this case, the tomcat container and logstash container must be in the same Kubernetes pod and share the /usr/local/tomcat/logs volume, as follows:

The preceding diagram shows how Tomcat and Logstash can share the log file using the Kubernetes emptyDir volume (https://kubernetes.io/docs/concepts/storage/volumes/#emptydir).

Tomcat and Logstash didn't use the network via localhost, but they did share the filesystem between /usr/local/tomcat/logs from the Tomcat container and /mnt from the logstash container, through Kubernetes' emptyDir volume:

$ cat tomcat-logstash.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat
spec:
  replicas: 1
  selector:
    matchLabels:
      run: tomcat
  template:
    metadata:
      labels:
        run: tomcat
    spec:
      containers:
        - image: tomcat
          name: tomcat
          ports:
            - containerPort: 8080
          env:
            - name: UMASK
              value: "0022"
          volumeMounts:
            - mountPath: /usr/local/tomcat/logs
              name: tomcat-log
        - image: logstash
          name: logstash
          args: ["-e input { file { path => "/mnt/localhost_access_log.*" } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["http://elasticsearch-svc.default.svc.cluster.local:9200"] } }"]
          volumeMounts:
            - mountPath: /mnt
              name: tomcat-log
      volumes:
        - name: tomcat-log
          emptyDir: {}

Let's create tomcat and logstash pod, and then see whether Logstash can see the Tomcat application log under /mnt:

//create Pod
$ kubectl create -f tomcat-logstash.yaml 
deployment.apps/tomcat created


//check Pod name
$ kubectl get pods
NAME                      READY     STATUS    RESTARTS   AGE
tomcat-7d99999565-6pm64   2/2       Running   0          1m


//connect to logstash container to see /mnt directory
$ kubectl exec -it tomcat-7d99999565-6pm64 -c logstash /bin/bash
root@tomcat-7d99999565-6pm64:/# ls /mnt
catalina.2018-09-20.log      localhost.2018-09-20.log   manager.2018-09-20.log
host-manager.2018-09-20.log  localhost_access_log.2018-09-20.txt

In this scenario, Elasticsearch must be stateful in the final destination, meaning that it uses a persistent volume. The Elasticsearch container must preserve the data even if the container is restarted. In addition, you do not need to configure the Elasticsearch container within the same pod as Tomcat/Logstash. Because Elasticsearch should be a centralized log datastore, it can be separated from the Tomcat/Logstash pod and scaled independently.

Once you determine that your application needs a persistent volume, there are some different types of volumes and different ways to manage persistent volumes to look at.