Use docker_resource as a resource to "pin" a dependency
keymon opened this issue · 4 comments
I want to suggest an improvement for your pipelines to pin the version of the containers.
I suggest this here because there is no other official channel to communicate.
In concourse, you can use a resource as image for a task, by using the get instead of specify task.config.docker_resource.
This way you are pining the version of the container to use:
diff --git a/concourse/pipelines/create-cloudfoundry.yml b/concourse/pipelines/create-cloudfoundry.yml
index 810c3d7..21af57d 100644
--- a/concourse/pipelines/create-cloudfoundry.yml
+++ b/concourse/pipelines/create-cloudfoundry.yml
@@ -55,6 +55,7 @@ groups:
- name: credentials
jobs:
- clear-cloudfoundry-credentials
+
resource_types:
- name: s3-iam
type: docker-image
@@ -67,6 +68,11 @@ resource_types:
repository: governmentpaas/semver-resource
resources:
+ - name: cf-acceptance-tests-container
+ type: docker-image
+ source:
+ repository: governmentpaas/semver-resource
+
- name: pipeline-trigger
type: semver-iam
source:
@@ -1129,6 +1135,8 @@ jobs:
- get: graphite-nozzle
- get: datadog-tfstate
- get: paas-rubbernecker
+ - get: cf-acceptance-tests-container
+ passed: ['cf-deploy']
- aggregate:
- task: extract-cf-terraform-outputs
@@ -1931,7 +1939,8 @@ jobs:
- get: bosh-CA
- get: cf-secrets
passed: ['cf-deploy']
-
+ - get: cf-acceptance-tests-container
+ passed: ['cf-deploy']
- do:
- task: create-temp-user
file: paas-cf/concourse/tasks/create_admin.yml
@@ -1972,7 +1981,7 @@ jobs:
./cf-release/jobs/acceptance-tests/spec \
acceptance_test_properties.yml \
> ./test-config/run
-
+ t
chmod +x ./test-config/run
./paas-cf/platform-tests/bosh-template-renderer/render.rb \
@@ -1983,11 +1992,9 @@ jobs:
- task: run-tests
config:
+ image: cf-acceptance-tests-container
platform: linux
- image_resource:
- type: docker-image
- source:
- repository: governmentpaas/cf-acceptance-tests
+g
params:
DISABLE_CF_ACCEPTANCE_TESTS: {{disable_cf_acceptance_tests}}
inputs:May I say what an amazing issue this is? It's quite remarkable. Thanks!
Since 467c751 we've been using tags to pin our docker image versions and it seems to be working.
good!
Be aware that in that case the semantics of pinning one resource to a specific version are not the same than consuming the outputs between the job.
For instance, in your pipeline, you want your continuous-smoke-test job to run using the "latest valid version for the latest sucessful job of cf-deploy". As all the deployment of CF might take up to 1 hour, by using version pinning your continuous-smoke-test will start using the new version immediately, even when the changes were not deployed. That means that you might be losing your monitoring capabilities for 1 hours. Also, if the deployment fails, you need to revert the pinned version.
If you do use dependencies as I suggest above, your continuous-smoke-test will always run the expected version for the latest valid deployment of CF.
The best approach is that you pin the version in the cf-deploy job, but the continous-smoke-tests use the dependency as I suggested initially.