GoCD server plugin to keep pipelines and environments configuration in source-control in YAML. See this document to find out what are GoCD configuration repositories.
This is second GoCD configuration plugin, which enhances some of shortcomings of JSON configuration plugin
- Format is concise. Unlike JSON, there are no unnecessary quotations
"
, brackets{}
and commas,
- Tries to enforce correctness where possible. By using maps instead of lists and shorter object graphs.
- Allows to have multiple pipelines and environments in single file. But doesn't force it.
- Comments in configuration files - YAML supports comments, so you can explain why pipeline/environment it is configured like this.
- Shorter syntax for declaring single-job stages
- Very short syntax for declaring tasks with script executor plugin
Download plugin from releases page and place in your Go server (>= 16.7.0
)
in plugins/external
directory.
Add config-repos
element right above first <pipelines />
. Then you can
add any number of YAML configuration repositories as such:
<config-repos>
<config-repo plugin="yaml.config.plugin">
<git url="https://github.com/tomzo/gocd-yaml-config-example.git" />
</config-repo>
</config-repos>
More examples are in test resources.
#ci.gocd.yaml
environments:
testing:
environment_variables:
DEPLOYMENT: testing
secure_variables:
ENV_PASSWORD: "s&Du#@$xsSa"
pipelines:
- example-deploy-testing
- build-testing
pipelines:
mypipe1: # definition of mypipe1 pipeline
group: mygroup # note that the group name can contain only of alphanumeric & underscore characters
label_template: "${mygit[:8]}"
locking: off
materials:
mygit: # this is the name of material, the name can contain only of alphanumeric & underscore characters
# keyword git says about type of material and url at once
git: http://my.example.org/mygit.git
branch: ci
myupstream:
# type is optional here, material type is implied based on presence of pipeline and stage fields
# type: dependency
pipeline: pipe2
stage: test
stages: # list of stages in order
- build: # name of stage
clean_workspace: true
jobs:
csharp: # name of the job
resources:
- net45
artifacts:
- build:
source: bin/
destination: build
- test:
source: tests/
destination: test-reports/
tabs:
report: test-reports/index.html
tasks: # ordered list of tasks to execute in job csharp
- fetch:
pipeline: pipe2
stage: build
job: test
source: test-bin/
destination: bin/
- exec: # indicates type of task
command: make
arguments:
- "VERBOSE=true"
# shorthand for script-executor plugin
- script: ./build.sh ci
By default GoCD configuration files should end with .gocd.yaml
.
See official GoCD XML configuration reference for details about each element. Below is a reference of format supported by this plugin. Feel free to improve it!
A minimal pipeline configuration must contain:
mypipe:
materials:
mygit:
git: http://example.com/mygit.git
stages:
- build:
jobs:
build:
tasks:
- exec:
command: make
All elements available on a pipeline object are:
group
label_template
locking
- tracking_tool or
mingle
- timer
- environment_variables
secure_variables
- materials
- stages
pipe2:
group: group1
label_template: "foo-1.0-${COUNT}"
locking: on
tracking_tool:
link: "http://your-trackingtool/yourproject/${ID}"
regex: "evo-(\\d+)"
timer:
spec: "0 15 10 * * ? *"
environment_variables:
DEPLOYMENT: testing
secure_variables:
ENV_PASSWORD: "s&Du#@$xsSa"
materials:
...
stages:
...
Please note:
- templates are not supported
- parameters are not supported
- pipeline declares a group to which it belongs
tracking_tool:
link: "http://your-trackingtool/yourproject/${ID}"
regex: "evo-(\\d+)"
timer:
spec: "0 15 10 * * ? *"
only_on_changes: yes
See XML reference for more information.
A minimal stage must contain jobs:
element or tasks:
in single-job stage case.
build:
jobs:
firstJob:
...
secondJob:
...
A custom stage:
test:
fetch_materials: yes
keep_artifacts: yes
clean_workspace: yes
approval:
type: manual
roles:
- manager
users:
- john
environment_variables:
TEST_NUM: 1
secure_variables:
PASSWORD: "!@ESsdD323#sdu"
jobs:
one:
...
two:
...
Stage can have approval,
which is success
by default. There are 2 ways to declare approval:
approval: manual
If you need to set associated users or roles:
approval:
type: manual
roles:
- manager
users:
- john
Job is a hash starting with jobs name:
test:
timeout: 5
run_instances: 7
environment_variables:
LD_LIBRARY_PATH: .
tabs:
test: results.xml
resources:
- linux
artifacts:
- test:
source: src
destination: dest
- build:
source: bin
properties:
perf:
source: test.xml
xpath: "substring-before(//report/data/all/coverage[starts-with(@type,\u0027class\u0027)]/@value, \u0027%\u0027)"
tasks:
...
Note: timeout is added since 0.2.0 version of yaml plugin
Job configuration may define elastic agents profile id, as such:
elastic_profile_id: "docker.unit-test"
It MUST NOT be specified along with resources
.
Available in GoCD server since v16.12.0, yaml plugin 0.4.0.
Part of job object can be number of job to runs:
run_instances: 7
Or to run on all agents:
run_instances: all
Tabs are a hash with <tab-name>: <path>
pairs.
Path should exist in Go servers artifacts.
tabs:
tests: test-reports/index.html
gauge: functional-reports/index.html
Job can have properties, declared as a hash:
properties:
cov1: # this is the name of property
source: test.xml
xpath: "substring-before(//report/data/all/coverage[starts-with(@type,\u0027class\u0027)]/@value, \u0027%\u0027)"
performance.ind1.mbps:
source: PerfTestReport.xml
xpath: "//PerformanceSuiteReport/WriteOnly/MBps"
A common use case is that stage has only one job. This plugin provides a shorthand
to declared such stages - just omit the jobs:
and job name from configuration tree.
You can then declare job and stage options on the same (stage) level:
stages:
- build:
approval: manual
resources:
- cpp
tasks:
- exec:
command: make
Above configuration declares build
stage with build
job which executes make
task.
Minimal configuration of a git pipeline material:
- material name is
mygit
- git repository url is
http://example.com/mygit.git
mygit:
git: http://example.com/mygit.git
Above can be also written more explicitly:
mygit:
type: git
url: http://example.com/mygit.git
More customized git material is possible:
gitMaterial1:
git: "http://my.git.repository.com"
branch: feature12
blacklist:
- externals/**/*.*
- tools/**/*.*
destination: dir1
auto_update: false
shallow_clone: true
Since Go >= 16.7.0
whitelist is also supported,
you can specify whitelist
instead of blacklist
, as such
gitMaterial1:
git: "git@my.git.repository.com"
branch: "feature12"
whitelist:
- src/**/*.*
For details about each option, see GoCD XML reference
svnMaterial1:
svn: "http://svn"
username: "user1"
password: "pass1"
check_externals: true
blacklist:
- tools
- lib
destination: destDir1
auto_update: false
hgMaterial1:
hg: repos/myhg
blacklist:
- externals
- tools
destination: dir1
auto_update: false
p4Material1:
p4: "host.domain.com:12345"
username: johndoe
password: pass
use_tickets: false
view: |
//depot/external... //ws/external...
//depot/tools... //ws/external...
blacklist:
- externals
- tools
auto_update: false
TODO
myPluggableGit:
scm: someScmGitRepositoryId
destination: destinationDir
blacklist:
- dir1
- dir2
To add a dependency on another pipeline stage:
mydependency:
pipeline: upstream-pipeline-1
stage: test
Note: mydependency
is the name of material - it must be unique
myapt:
package: apt-repo-id
Every task is a hash starting with its type.
Type can be exec
, ant
, nant
, rake
, fetch
, plugin
or script
.
<type>:
<task-prop1>: <prop1-value>
Optionally any task can have run_if
and on_cancel
.
run_if
is a string. Valid values arepassed
,failed
,any
on_cancel
is a task object. Same rules apply as to tasks described on this page. See rake example.
exec:
run_if: any
working_directory: dir
command: make
arguments:
- -j3
- docs
- install
ant:
build_file: mybuild.xml
target: compile
run_if: passed
nant:
run_if: passed
working_directory: "script/build/123"
build_file: FilePath
target: Build
nant_path: NantExe
A minimal rake task with default values is very short
rake:
A complete rake example:
rake:
run_if: passed
working_directory: sample-project
build_file: SomeRakefile
target: build
on_cancel:
rake:
working_directory: sample-project
build_file: CancelRakefile
target: cancel
fetch:
run_if: any
pipeline: myupstream
stage: upstream_stage
job: upstream_job
is_file: yes
source: result
destination: test
plugin:
options:
ConverterType: jsunit
secure_options:
password: "ssd#%fFS*!Esx"
run_if: failed
configuration:
id: xunit.converter.task.plugin
version: 1
Because script-executor plugin requires quite a lot of boiler-plate configuration there is a shorthand for defining tasks with it:
script: ./build.sh compile
Above is equivalent of
plugin:
options:
script: ./build.sh compile
configuration:
id: script-executor
version: 1
You can declare multi-line scripts too:
script: |
./build.sh compile
make test
Above executes a 2-line script:
./build.sh compile
make test
If you want to execute a single long line script, but break it into multiple lines in YAML,
you can do this with >
as such:
script: >
./build.sh compile &&
make test
Above executes a single line script:
./build.sh compile && make test
NOTE: The agents should be a guid, which is currently impossible to get for user
testing:
environment_variables:
DEPLOYMENT: testing
secure_variables:
ENV_PASSWORD: "s&Du#@$xsSa"
pipelines:
- example-deploy-testing
- build-testing
agents:
- 123
Environment variables can be declared in Environments, Pipelines, Stages and Jobs.
In YAML you have 2 keywords for secure (encrypted) variables and standard variables.
environment_variables:
DEPLOYMENT: testing
FOO: bar
secure_variables:
# this value is encrypted by Go's private key (Note in 16.7.0 there is no easy way to obtain such value yet)
MY_PASSWORD: "s&Du#@$xsSa"
There is no easy way to generate obtain encrypted value from GoServer, alternatively you can login into go-server and execute the following command to generate encrypted value
sudo -u go bash -c 'echo -n 'YOUR-INPUT' | openssl enc -des-cbc -a -iv 0 -K $(cat /etc/go/cipher)'
Among all configuration elements there are boolean values, which can be defined using any of the keywords below (as in yaml specs):
- true -
y|Y|yes|Yes|YES|true|True|TRUE|on|On|ON
- false -
n|N|no|No|NO|false|False|FALSE|off|Off|OFF
Run all tests and create a ready to use jar
./gradlew test fatJar
There are examples of yaml partials and their resulting json to be sent to Go server. If something is not working right we can always add a new case covering exact yaml that user has and json that we expect on server side.