Kubediff find differences on missing defaults
nmiculinic opened this issue · 13 comments
nmiculinic commented
kubediff should ignore the differences on missing defaults in the .yaml files. Poking through the source code you're getting the object (kubectl get), whilst instead should do something similar to kubectl diff
nmiculinic commented
Here is what kubectl diff does for -v=9. Basically gets the object json and pathes with dry-run the file:
I0218 14:02:58.423434 46043 round_trippers.go:438] GET https://10.251.0.133:6443/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy 200 OK in 33 milliseconds
I0218 14:02:58.423526 46043 round_trippers.go:444] Response Headers:
I0218 14:02:58.423549 46043 round_trippers.go:447] Content-Type: application/json
I0218 14:02:58.423588 46043 round_trippers.go:447] Content-Length: 766
I0218 14:02:58.423663 46043 round_trippers.go:447] Date: Mon, 18 Feb 2019 13:02:58 GMT
I0218 14:02:58.423729 46043 request.go:942] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"gitlab-deploy","namespace":"ascalia-dev","selfLink":"/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy","uid":"70371a68-2878-11e9-80f5-0242b121b722","resourceVersion":"466307","creationTimestamp":"2019-02-04T12:28:55Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"rbac.authorization.k8s.io/v1\",\"kind\":\"Role\",\"metadata\":{\"annotations\":{},\"name\":\"gitlab-deploy\",\"namespace\":\"ascalia-dev\"},\"rules\":[{\"apiGroups\":[\"extensions\"],\"resources\":[\"deployments\"],\"verbs\":[\"update\",\"get\"]}]}\n"}},"rules":[{"verbs":["update","get","patch"],"apiGroups":["extensions"],"resources":["deployments"]}]}
I0218 14:02:58.425157 46043 request.go:942] Request Body: {"metadata":{"resourceVersion":"466307"},"rules":[{"apiGroups":["extensions"],"resources":["deployments"],"verbs":["get","update","patch"]}]}
I0218 14:02:58.425218 46043 round_trippers.go:419] curl -k -v -XPATCH -H "Accept: application/json" -H "Content-Type: application/strategic-merge-patch+json" -H "User-Agent: kubectl/v1.13.3 (darwin/amd64) kubernetes/721bfa7" 'https://10.251.0.133:6443/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy?dryRun=All'
I0218 14:02:58.463888 46043 round_trippers.go:438] PATCH https://10.251.0.133:6443/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy?dryRun=All 200 OK in 38 milliseconds
I0218 14:02:58.463933 46043 round_trippers.go:444] Response Headers:
I0218 14:02:58.463951 46043 round_trippers.go:447] Content-Length: 766
I0218 14:02:58.463966 46043 round_trippers.go:447] Date: Mon, 18 Feb 2019 13:02:58 GMT
I0218 14:02:58.463983 46043 round_trippers.go:447] Content-Type: application/json
I0218 14:02:58.464045 46043 request.go:942] Response Body: {"kind":"Role","apiVersion":"rbac.authorization.k8s.io/v1","metadata":{"name":"gitlab-deploy","namespace":"ascalia-dev","selfLink":"/apis/rbac.authorization.k8s.io/v1/namespaces/ascalia-dev/roles/gitlab-deploy","uid":"70371a68-2878-11e9-80f5-0242b121b722","resourceVersion":"466307","creationTimestamp":"2019-02-04T12:28:55Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"rbac.authorization.k8s.io/v1\",\"kind\":\"Role\",\"metadata\":{\"annotations\":{},\"name\":\"gitlab-deploy\",\"namespace\":\"ascalia-dev\"},\"rules\":[{\"apiGroups\":[\"extensions\"],\"resources\":[\"deployments\"],\"verbs\":[\"update\",\"get\"]}]}\n"}},"rules":[{"verbs":["get","update","patch"],"apiGroups":["extensions"],"resources":["deployments"]}]}
nmiculinic commented
Related to #37
nmiculinic commented
Or maybe this is a repeat of #37 , since all errors are those "missing" elements in the array. Anyhow using kubectl diff is much more stable and maintained solution anyhow, or even accessing the API server endpoints directly.
samiyaakhtar commented
+1, looking for a response to this as well.
bboreham commented
Can you give an example of some input data that causes this?
nmiculinic commented
Actually here you go:
fn-k8s-ops (master) ✔ kubediff k8s-manifests/devtools/gitlab/charts/gitlab-runner/templates/deployment.yaml k8s [devtools@fashionnetwork]
## devtools/gitlab-gitlab-runner (Deployment.v1beta1.extensions)
.spec.template.spec.containers: 'element [0]' missing
## devtools/gitlab-gitlab-runner (Deployment.v1beta1.extensions)
.spec.template.spec.initContainers: 'element [0]' missing
fn-k8s-ops (master) ✔ k8s [devtools@fashionnetwork]
And the yaml file:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: gitlab-gitlab-runner
labels:
app: gitlab-gitlab-runner
chart: gitlab-runner-0.3.0
release: gitlab
heritage: Tiller
namespace: devtools
spec:
replicas: 1
template:
metadata:
labels:
app: gitlab-gitlab-runner
annotations:
checksum/configmap: dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7
checksum/secrets: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
prometheus.io/scrape: "true"
prometheus.io/port: "9252"
spec:
securityContext:
runAsUser: 100
fsGroup: 65533
initContainers:
- name: configure
command:
- sh
- /config/configure
image: gitlab/gitlab-runner:alpine-v11.9.0
env:
- name: CI_SERVER_URL
value: https://gitlab.fashionnetwork.com
- name: CLONE_URL
value: ""
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
value: docker
- name: KUBERNETES_IMAGE
value: ubuntu:16.04
- name: KUBERNETES_PRIVILEGED
value: "true"
- name: KUBERNETES_NAMESPACE
value: gitlab-jobs
- name: KUBERNETES_CPU_LIMIT
value: 200m
- name: KUBERNETES_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_CPU_REQUEST
value: 100m
- name: KUBERNETES_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_SERVICE_ACCOUNT
value: ""
- name: KUBERNETES_SERVICE_CPU_LIMIT
value: 200m
- name: KUBERNETES_SERVICE_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_SERVICE_CPU_REQUEST
value: 100m
- name: KUBERNETES_SERVICE_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_CPU_LIMIT
value: 200m
- name: KUBERNETES_HELPER_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_HELPER_CPU_REQUEST
value: 100m
- name: KUBERNETES_HELPER_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_IMAGE
value: ""
- name: KUBERNETES_PULL_POLICY
value: always
- name: CACHE_TYPE
value: s3
- name: CACHE_PATH
value: gitlab_runner
- name: CACHE_SHARED
value: "true"
- name: CACHE_S3_SERVER_ADDRESS
value: s3.amazonaws.com
- name: CACHE_S3_BUCKET_NAME
value: gitlab-fn-runner-cache
- name: CACHE_S3_BUCKET_LOCATION
value: eu-central-1
volumeMounts:
- name: runner-secrets
mountPath: /secrets
readOnly: false
- name: scripts
mountPath: /config
readOnly: true
- name: init-runner-secrets
mountPath: /init-secrets
readOnly: true
resources: {}
serviceAccountName: gitlab-gitlab-runner
containers:
- name: gitlab-gitlab-runner
image: gitlab/gitlab-runner:alpine-v11.9.0
imagePullPolicy: IfNotPresent
command:
- /bin/bash
- /scripts/entrypoint
env:
- name: CI_SERVER_URL
value: https://gitlab.fashionnetwork.com
- name: CLONE_URL
value: ""
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
value: docker
- name: KUBERNETES_IMAGE
value: ubuntu:16.04
- name: KUBERNETES_PRIVILEGED
value: "true"
- name: KUBERNETES_NAMESPACE
value: gitlab-jobs
- name: KUBERNETES_CPU_LIMIT
value: 200m
- name: KUBERNETES_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_CPU_REQUEST
value: 100m
- name: KUBERNETES_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_SERVICE_ACCOUNT
value: ""
- name: KUBERNETES_SERVICE_CPU_LIMIT
value: 200m
- name: KUBERNETES_SERVICE_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_SERVICE_CPU_REQUEST
value: 100m
- name: KUBERNETES_SERVICE_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_CPU_LIMIT
value: 200m
- name: KUBERNETES_HELPER_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_HELPER_CPU_REQUEST
value: 100m
- name: KUBERNETES_HELPER_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_IMAGE
value: ""
- name: KUBERNETES_PULL_POLICY
value: always
- name: CACHE_TYPE
value: s3
- name: CACHE_PATH
value: gitlab_runner
- name: CACHE_SHARED
value: "true"
- name: CACHE_S3_SERVER_ADDRESS
value: s3.amazonaws.com
- name: CACHE_S3_BUCKET_NAME
value: gitlab-fn-runner-cache
- name: CACHE_S3_BUCKET_LOCATION
value: eu-central-1
livenessProbe:
exec:
command:
- /bin/bash
- /scripts/check-live
initialDelaySeconds: 60
timeoutSeconds: 1
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
readinessProbe:
exec:
command:
- /usr/bin/pgrep
- gitlab.*runner
initialDelaySeconds: 10
timeoutSeconds: 1
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
ports:
- name: metrics
containerPort: 9252
volumeMounts:
- name: runner-secrets
mountPath: /secrets
- name: etc-gitlab-runner
mountPath: /home/gitlab-runner/.gitlab-runner
- name: scripts
mountPath: /scripts
resources: {}
volumes:
- name: runner-secrets
emptyDir:
medium: Memory
- name: etc-gitlab-runner
emptyDir:
medium: Memory
- name: init-runner-secrets
projected:
sources:
- secret:
name: gitlab-aws-s3
- secret:
name: gitlab-gitlab-runner-secret
items:
- key: runner-registration-token
path: runner-registration-token
- key: runner-token
path: runner-token
- name: scripts
configMap:
name: gitlab-gitlab-runner
bboreham commented
Can you post what kubectl get -o yaml returns for that object so I can see the comparison, please.
bboreham commented
Within a dictionary, kubediff only compares items that are present in the source, i.e. in the file it is given. In this way, most items defaulted by Kubernetes are ignored.
nmiculinic commented
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "11"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"gitlab-gitlab-runner","chart":"gitlab-runner-0.3.0","heritage":"Tiller","release":"gitlab"},"name":"gitlab-gitlab-runner","namespace":"devtools"},"spec":{"replicas":1,"template":{"metadata":{"annotations":{"checksum/configmap":"dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7","checksum/secrets":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","prometheus.io/port":"9252","prometheus.io/scrape":"true"},"labels":{"app":"gitlab-gitlab-runner"}},"spec":{"containers":[{"command":["/bin/bash","/scripts/entrypoint"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["/bin/bash","/scripts/check-live"]},"failureThreshold":3,"initialDelaySeconds":60,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"name":"gitlab-gitlab-runner","ports":[{"containerPort":9252,"name":"metrics"}],"readinessProbe":{"exec":{"command":["/usr/bin/pgrep","gitlab.*runner"]},"failureThreshold":3,"initialDelaySeconds":10,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets"},{"mountPath":"/home/gitlab-runner/.gitlab-runner","name":"etc-gitlab-runner"},{"mountPath":"/scripts","name":"scripts"}]}],"initContainers":[{"command":["sh","/config/configure"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","name":"configure","resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets","readOnly":false},{"mountPath":"/config","name":"scripts","readOnly":true},{"mountPath":"/init-secrets","name":"init-runner-secrets","readOnly":true}]}],"securityContext":{"fsGroup":65533,"runAsUser":100},"serviceAccountName":"gitlab-gitlab-runner","volumes":[{"emptyDir":{"medium":"Memory"},"name":"runner-secrets"},{"emptyDir":{"medium":"Memory"},"name":"etc-gitlab-runner"},{"name":"init-runner-secrets","projected":{"sources":[{"secret":{"name":"gitlab-aws-s3"}},{"secret":{"items":[{"key":"runner-registration-token","path":"runner-registration-token"},{"key":"runner-token","path":"runner-token"}],"name":"gitlab-gitlab-runner-secret"}}]}},{"configMap":{"name":"gitlab-gitlab-runner"},"name":"scripts"}]}}}}
creationTimestamp: "2019-02-20T10:03:29Z"
generation: 11
labels:
app: gitlab-gitlab-runner
chart: gitlab-runner-0.3.0
heritage: Tiller
release: gitlab
name: gitlab-gitlab-runner
namespace: devtools
resourceVersion: "8914046"
selfLink: /apis/extensions/v1beta1/namespaces/devtools/deployments/gitlab-gitlab-runner
uid: c5f662ea-34f6-11e9-95c5-061409733802
spec:
progressDeadlineSeconds: 2147483647
replicas: 1
revisionHistoryLimit: 2147483647
selector:
matchLabels:
app: gitlab-gitlab-runner
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
checksum/configmap: dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7
checksum/secrets: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
prometheus.io/port: "9252"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app: gitlab-gitlab-runner
spec:
containers:
- command:
- /bin/bash
- /scripts/entrypoint
env:
- name: CI_SERVER_URL
value: https://gitlab.fashionnetwork.com
- name: CLONE_URL
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
value: docker
- name: KUBERNETES_IMAGE
value: ubuntu:16.04
- name: KUBERNETES_PRIVILEGED
value: "true"
- name: KUBERNETES_NAMESPACE
value: gitlab-jobs
- name: KUBERNETES_CPU_LIMIT
value: 200m
- name: KUBERNETES_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_CPU_REQUEST
value: 100m
- name: KUBERNETES_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_SERVICE_ACCOUNT
- name: KUBERNETES_SERVICE_CPU_LIMIT
value: 200m
- name: KUBERNETES_SERVICE_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_SERVICE_CPU_REQUEST
value: 100m
- name: KUBERNETES_SERVICE_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_CPU_LIMIT
value: 200m
- name: KUBERNETES_HELPER_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_HELPER_CPU_REQUEST
value: 100m
- name: KUBERNETES_HELPER_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_IMAGE
- name: KUBERNETES_PULL_POLICY
value: always
- name: CACHE_TYPE
value: s3
- name: CACHE_PATH
value: gitlab_runner
- name: CACHE_SHARED
value: "true"
- name: CACHE_S3_SERVER_ADDRESS
value: s3.amazonaws.com
- name: CACHE_S3_BUCKET_NAME
value: gitlab-fn-runner-cache
- name: CACHE_S3_BUCKET_LOCATION
value: eu-central-1
image: gitlab/gitlab-runner:alpine-v11.9.0
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- /bin/bash
- /scripts/check-live
failureThreshold: 3
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: gitlab-gitlab-runner
ports:
- containerPort: 9252
name: metrics
protocol: TCP
readinessProbe:
exec:
command:
- /usr/bin/pgrep
- gitlab.*runner
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /secrets
name: runner-secrets
- mountPath: /home/gitlab-runner/.gitlab-runner
name: etc-gitlab-runner
- mountPath: /scripts
name: scripts
dnsPolicy: ClusterFirst
initContainers:
- command:
- sh
- /config/configure
env:
- name: CI_SERVER_URL
value: https://gitlab.fashionnetwork.com
- name: CLONE_URL
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
value: docker
- name: KUBERNETES_IMAGE
value: ubuntu:16.04
- name: KUBERNETES_PRIVILEGED
value: "true"
- name: KUBERNETES_NAMESPACE
value: gitlab-jobs
- name: KUBERNETES_CPU_LIMIT
value: 200m
- name: KUBERNETES_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_CPU_REQUEST
value: 100m
- name: KUBERNETES_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_SERVICE_ACCOUNT
- name: KUBERNETES_SERVICE_CPU_LIMIT
value: 200m
- name: KUBERNETES_SERVICE_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_SERVICE_CPU_REQUEST
value: 100m
- name: KUBERNETES_SERVICE_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_CPU_LIMIT
value: 200m
- name: KUBERNETES_HELPER_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_HELPER_CPU_REQUEST
value: 100m
- name: KUBERNETES_HELPER_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_IMAGE
- name: KUBERNETES_PULL_POLICY
value: always
- name: CACHE_TYPE
value: s3
- name: CACHE_PATH
value: gitlab_runner
- name: CACHE_SHARED
value: "true"
- name: CACHE_S3_SERVER_ADDRESS
value: s3.amazonaws.com
- name: CACHE_S3_BUCKET_NAME
value: gitlab-fn-runner-cache
- name: CACHE_S3_BUCKET_LOCATION
value: eu-central-1
image: gitlab/gitlab-runner:alpine-v11.9.0
imagePullPolicy: IfNotPresent
name: configure
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /secrets
name: runner-secrets
- mountPath: /config
name: scripts
readOnly: true
- mountPath: /init-secrets
name: init-runner-secrets
readOnly: true
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65533
runAsUser: 100
serviceAccount: gitlab-gitlab-runner
serviceAccountName: gitlab-gitlab-runner
terminationGracePeriodSeconds: 30
volumes:
- emptyDir:
medium: Memory
name: runner-secrets
- emptyDir:
medium: Memory
name: etc-gitlab-runner
- name: init-runner-secrets
projected:
defaultMode: 420
sources:
- secret:
name: gitlab-aws-s3
- secret:
items:
- key: runner-registration-token
path: runner-registration-token
- key: runner-token
path: runner-token
name: gitlab-gitlab-runner-secret
- configMap:
defaultMode: 420
name: gitlab-gitlab-runner
name: scripts
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2019-02-20T10:03:29Z"
lastUpdateTime: "2019-02-20T10:03:29Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 11
readyReplicas: 1
replicas: 1
updatedReplicas: 1
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "11"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"gitlab-gitlab-runner","chart":"gitlab-runner-0.3.0","heritage":"Tiller","release":"gitlab"},"name":"gitlab-gitlab-runner","namespace":"devtools"},"spec":{"replicas":1,"template":{"metadata":{"annotations":{"checksum/configmap":"dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7","checksum/secrets":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","prometheus.io/port":"9252","prometheus.io/scrape":"true"},"labels":{"app":"gitlab-gitlab-runner"}},"spec":{"containers":[{"command":["/bin/bash","/scripts/entrypoint"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["/bin/bash","/scripts/check-live"]},"failureThreshold":3,"initialDelaySeconds":60,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"name":"gitlab-gitlab-runner","ports":[{"containerPort":9252,"name":"metrics"}],"readinessProbe":{"exec":{"command":["/usr/bin/pgrep","gitlab.*runner"]},"failureThreshold":3,"initialDelaySeconds":10,"periodSeconds":10,"successThreshold":1,"timeoutSeconds":1},"resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets"},{"mountPath":"/home/gitlab-runner/.gitlab-runner","name":"etc-gitlab-runner"},{"mountPath":"/scripts","name":"scripts"}]}],"initContainers":[{"command":["sh","/config/configure"],"env":[{"name":"CI_SERVER_URL","value":"https://gitlab.fashionnetwork.com"},{"name":"CLONE_URL","value":""},{"name":"RUNNER_EXECUTOR","value":"kubernetes"},{"name":"REGISTER_LOCKED","value":"false"},{"name":"RUNNER_TAG_LIST","value":"docker"},{"name":"KUBERNETES_IMAGE","value":"ubuntu:16.04"},{"name":"KUBERNETES_PRIVILEGED","value":"true"},{"name":"KUBERNETES_NAMESPACE","value":"gitlab-jobs"},{"name":"KUBERNETES_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_SERVICE_ACCOUNT","value":""},{"name":"KUBERNETES_SERVICE_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_SERVICE_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_SERVICE_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_SERVICE_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_CPU_LIMIT","value":"200m"},{"name":"KUBERNETES_HELPER_MEMORY_LIMIT","value":"256Mi"},{"name":"KUBERNETES_HELPER_CPU_REQUEST","value":"100m"},{"name":"KUBERNETES_HELPER_MEMORY_REQUEST","value":"128Mi"},{"name":"KUBERNETES_HELPER_IMAGE","value":""},{"name":"KUBERNETES_PULL_POLICY","value":"always"},{"name":"CACHE_TYPE","value":"s3"},{"name":"CACHE_PATH","value":"gitlab_runner"},{"name":"CACHE_SHARED","value":"true"},{"name":"CACHE_S3_SERVER_ADDRESS","value":"s3.amazonaws.com"},{"name":"CACHE_S3_BUCKET_NAME","value":"gitlab-fn-runner-cache"},{"name":"CACHE_S3_BUCKET_LOCATION","value":"eu-central-1"}],"image":"gitlab/gitlab-runner:alpine-v11.9.0","name":"configure","resources":{},"volumeMounts":[{"mountPath":"/secrets","name":"runner-secrets","readOnly":false},{"mountPath":"/config","name":"scripts","readOnly":true},{"mountPath":"/init-secrets","name":"init-runner-secrets","readOnly":true}]}],"securityContext":{"fsGroup":65533,"runAsUser":100},"serviceAccountName":"gitlab-gitlab-runner","volumes":[{"emptyDir":{"medium":"Memory"},"name":"runner-secrets"},{"emptyDir":{"medium":"Memory"},"name":"etc-gitlab-runner"},{"name":"init-runner-secrets","projected":{"sources":[{"secret":{"name":"gitlab-aws-s3"}},{"secret":{"items":[{"key":"runner-registration-token","path":"runner-registration-token"},{"key":"runner-token","path":"runner-token"}],"name":"gitlab-gitlab-runner-secret"}}]}},{"configMap":{"name":"gitlab-gitlab-runner"},"name":"scripts"}]}}}}
creationTimestamp: "2019-02-20T10:03:29Z"
generation: 11
labels:
app: gitlab-gitlab-runner
chart: gitlab-runner-0.3.0
heritage: Tiller
release: gitlab
name: gitlab-gitlab-runner
namespace: devtools
resourceVersion: "8914046"
selfLink: /apis/extensions/v1beta1/namespaces/devtools/deployments/gitlab-gitlab-runner
uid: c5f662ea-34f6-11e9-95c5-061409733802
spec:
progressDeadlineSeconds: 2147483647
replicas: 1
revisionHistoryLimit: 2147483647
selector:
matchLabels:
app: gitlab-gitlab-runner
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
annotations:
checksum/configmap: dcf8b57898c96448dec3b044e2e528dc25fb8b75583f186b86fb8ad54f8f34d7
checksum/secrets: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
prometheus.io/port: "9252"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app: gitlab-gitlab-runner
spec:
containers:
- command:
- /bin/bash
- /scripts/entrypoint
env:
- name: CI_SERVER_URL
value: https://gitlab.fashionnetwork.com
- name: CLONE_URL
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
value: docker
- name: KUBERNETES_IMAGE
value: ubuntu:16.04
- name: KUBERNETES_PRIVILEGED
value: "true"
- name: KUBERNETES_NAMESPACE
value: gitlab-jobs
- name: KUBERNETES_CPU_LIMIT
value: 200m
- name: KUBERNETES_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_CPU_REQUEST
value: 100m
- name: KUBERNETES_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_SERVICE_ACCOUNT
- name: KUBERNETES_SERVICE_CPU_LIMIT
value: 200m
- name: KUBERNETES_SERVICE_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_SERVICE_CPU_REQUEST
value: 100m
- name: KUBERNETES_SERVICE_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_CPU_LIMIT
value: 200m
- name: KUBERNETES_HELPER_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_HELPER_CPU_REQUEST
value: 100m
- name: KUBERNETES_HELPER_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_IMAGE
- name: KUBERNETES_PULL_POLICY
value: always
- name: CACHE_TYPE
value: s3
- name: CACHE_PATH
value: gitlab_runner
- name: CACHE_SHARED
value: "true"
- name: CACHE_S3_SERVER_ADDRESS
value: s3.amazonaws.com
- name: CACHE_S3_BUCKET_NAME
value: gitlab-fn-runner-cache
- name: CACHE_S3_BUCKET_LOCATION
value: eu-central-1
image: gitlab/gitlab-runner:alpine-v11.9.0
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- /bin/bash
- /scripts/check-live
failureThreshold: 3
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: gitlab-gitlab-runner
ports:
- containerPort: 9252
name: metrics
protocol: TCP
readinessProbe:
exec:
command:
- /usr/bin/pgrep
- gitlab.*runner
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /secrets
name: runner-secrets
- mountPath: /home/gitlab-runner/.gitlab-runner
name: etc-gitlab-runner
- mountPath: /scripts
name: scripts
dnsPolicy: ClusterFirst
initContainers:
- command:
- sh
- /config/configure
env:
- name: CI_SERVER_URL
value: https://gitlab.fashionnetwork.com
- name: CLONE_URL
- name: RUNNER_EXECUTOR
value: kubernetes
- name: REGISTER_LOCKED
value: "false"
- name: RUNNER_TAG_LIST
value: docker
- name: KUBERNETES_IMAGE
value: ubuntu:16.04
- name: KUBERNETES_PRIVILEGED
value: "true"
- name: KUBERNETES_NAMESPACE
value: gitlab-jobs
- name: KUBERNETES_CPU_LIMIT
value: 200m
- name: KUBERNETES_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_CPU_REQUEST
value: 100m
- name: KUBERNETES_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_SERVICE_ACCOUNT
- name: KUBERNETES_SERVICE_CPU_LIMIT
value: 200m
- name: KUBERNETES_SERVICE_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_SERVICE_CPU_REQUEST
value: 100m
- name: KUBERNETES_SERVICE_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_CPU_LIMIT
value: 200m
- name: KUBERNETES_HELPER_MEMORY_LIMIT
value: 256Mi
- name: KUBERNETES_HELPER_CPU_REQUEST
value: 100m
- name: KUBERNETES_HELPER_MEMORY_REQUEST
value: 128Mi
- name: KUBERNETES_HELPER_IMAGE
- name: KUBERNETES_PULL_POLICY
value: always
- name: CACHE_TYPE
value: s3
- name: CACHE_PATH
value: gitlab_runner
- name: CACHE_SHARED
value: "true"
- name: CACHE_S3_SERVER_ADDRESS
value: s3.amazonaws.com
- name: CACHE_S3_BUCKET_NAME
value: gitlab-fn-runner-cache
- name: CACHE_S3_BUCKET_LOCATION
value: eu-central-1
image: gitlab/gitlab-runner:alpine-v11.9.0
imagePullPolicy: IfNotPresent
name: configure
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /secrets
name: runner-secrets
- mountPath: /config
name: scripts
readOnly: true
- mountPath: /init-secrets
name: init-runner-secrets
readOnly: true
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65533
runAsUser: 100
serviceAccount: gitlab-gitlab-runner
serviceAccountName: gitlab-gitlab-runner
terminationGracePeriodSeconds: 30
volumes:
- emptyDir:
medium: Memory
name: runner-secrets
- emptyDir:
medium: Memory
name: etc-gitlab-runner
- name: init-runner-secrets
projected:
defaultMode: 420
sources:
- secret:
name: gitlab-aws-s3
- secret:
items:
- key: runner-registration-token
path: runner-registration-token
- key: runner-token
path: runner-token
name: gitlab-gitlab-runner-secret
- configMap:
defaultMode: 420
name: gitlab-gitlab-runner
name: scripts
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2019-02-20T10:03:29Z"
lastUpdateTime: "2019-02-20T10:03:29Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 11
readyReplicas: 1
replicas: 1
updatedReplicas: 1bboreham commented
The one thing I can see is that empty environment variables, e.g.
- name: CLONE_URL
value: ""
come back with a missing value which causes a diff.
What made you think it was defaults?
nmiculinic commented
IIRC I think I observed similar behavior when timeoutSeconds wasn't specified in the liveness/rediness probe. (( default being 1s IIRC ))
bboreham commented
If you have an example, please post. Within a dictionary such as .spec.template.spec.containers[0].readinessProbe, kubediff only compares items that are present in the source.
bboreham commented
Closing - if you get some concrete evidence please post and re-open.