Error in operator logs: `volume provided in configuration is not mounted to any container, volume name:`

[andrzej-stencel]

Helm chart version: 0.5.1

When using the TailingSidecarConfig CRD to create sidecars, there is one error log for every decorated pod and every TailingSidecarConfig resource:

Example logs:

2022-07-18T13:34:58.048Z	INFO	tailing-sidecar.operator.handler.PodExtender	Handling request for Pod	{"Name": "", "Namespace": "", "GenerateName": "file-logger-646d4b5499-", "Operation": "CREATE"}
2022-07-18T13:34:58.048Z	INFO	tailing-sidecar.operator.handler.PodExtender	Found configuration for Pod	{"Pod Name": "", "Namespace": "", "GenerateName": "file-logger-646d4b5499-"}
2022-07-18T13:34:58.048Z	ERROR	tailing-sidecar.operator.handler.PodExtender	Failed to prepare volume	{"Pod Name": "", "Namespace": "", "GenerateName": "file-logger-646d4b5499-", "error": "volume provided in configuration is not mounted to any container, volume name: "}
github.com/go-logr/zapr.(*zapLogger).Error
	/go/pkg/mod/github.com/go-logr/zapr@v0.2.0/zapr.go:132
sigs.k8s.io/controller-runtime/pkg/log.(*DelegatingLogger).Error
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/log/deleg.go:144
github.com/SumoLogic/tailing-sidecar/operator/handler.PodExtender.extendPod
	/workspace/handler/handler.go:149
github.com/SumoLogic/tailing-sidecar/operator/handler.(*PodExtender).Handle
	/workspace/handler/handler.go:96
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/webhook/admission/webhook.go:140
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/webhook/admission/http.go:100
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1
	/go/pkg/mod/github.com/prometheus/client_golang@v1.7.1/prometheus/promhttp/instrument_server.go:40
net/http.HandlerFunc.ServeHTTP
	/usr/local/go/src/net/http/server.go:2047
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1
	/go/pkg/mod/github.com/prometheus/client_golang@v1.7.1/prometheus/promhttp/instrument_server.go:100
net/http.HandlerFunc.ServeHTTP
	/usr/local/go/src/net/http/server.go:2047
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2
	/go/pkg/mod/github.com/prometheus/client_golang@v1.7.1/prometheus/promhttp/instrument_server.go:76
net/http.HandlerFunc.ServeHTTP
	/usr/local/go/src/net/http/server.go:2047
net/http.(*ServeMux).ServeHTTP
	/usr/local/go/src/net/http/server.go:2425
net/http.serverHandler.ServeHTTP
	/usr/local/go/src/net/http/server.go:2879
net/http.(*conn).serve
	/usr/local/go/src/net/http/server.go:1930

The sidecar creation still works fine, but the error message is worrying.

Steps to reproduce:

1. Create a Kubernetes cluster, e.g. EKS 1.22
2. Install the Tailing Sidecar Helm chart in the cluster helm install ts tailing-sidecar/tailing-sidecar-operator -n tso --create-namespace
3. Create a tailing sidecar configuration in CRD kubectl apply -f tailing-config1.yaml

tailing-config1.yaml

apiVersion: tailing-sidecar.sumologic.com/v1
kind: TailingSidecarConfig
metadata:
  name: tsconfig1
spec:
  configs:
    logs1:
      path: /var/log/file1.log
      volumeMount:
        mountPath: /var/log
        name: logs-dir
  podSelector:
    matchLabels:
      app: file-logger

5. Create a deployment that writes logs to files kubectl apply -f file-logger.yaml

file-logger.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: file-logger
spec:
  selector:
    matchLabels:
      app: file-logger
  replicas: 1
  template:
    metadata:
      labels:
        app: file-logger
    spec:
      containers:
      - name: loggercontainer
        image: busybox
        args:
        - /bin/sh
        - -c
        - >
          i=0;
          while true;
          do
            echo "log1 $i $(date)" >> /var/log/file1.log;
            echo "log2 $i $(date)" >> /var/log/file2.log;
            echo "log3 $i $(date)" >> /var/log/file3.log;
            i=$((i+1));
            sleep 5;
          done
        resources: {}
        volumeMounts:
        - name: logs-dir
          mountPath: /var/log
      volumes:
      - name: logs-dir
        emptyDir: {}

5. Look at the tailing sidecar operator's logs (substitute pod name with yours) kubectl -n tso logs ts-tailing-sidecar-operator-cb9576d59-ssrq6

Actual result:

• ✅ A tailing container logs is correctly created in the file-logger pod.
• ❌ There is one error log Failed to prepare volume similar to that at the top of this issue.

Expected result

• No error log

6. Create another tailing config in CRD kubectl apply -f tailing-config2.yaml

tailing-config2.yaml

apiVersion: tailing-sidecar.sumologic.com/v1
kind: TailingSidecarConfig
metadata:
  name: tsconfig2
spec:
  configs:
    logs2:
      path: /var/log/file2.log
      volumeMount:
        mountPath: /var/log
        name: logs-dir
  podSelector:
    matchLabels:
      app: file-logger

7. Delete the file-logger pod for it to be recreated
8. Notice how two sidecar containers are correctly created for it.
9. Notice two error logs instead of one in tailing sidecar operator's logs.
10. Create a third tailing configuration, delete the file-logger pod again and notice three error logs.