Seed project for Go language.
This project aims to provide an archetype of Go project, with a common scheleton that includes a development environment and a continuous delivery pipeline (integrated with dcip). It also builds the service with govice to generate JSON logs, manage the configuration, handle errors and alarms, and validate JSON documents.
Usage: make <command>
Commands:
help: Show this help information
clean-build: Clean the project (remove build directory and clean golang packages)
clean-vendor: Remove vendor directory
clean: Clean the project (both clean-build and clean-vendor)
build-deps: Install the golang dependencies with deps
build-config: Copy the configuration into build/bin directory
build-install: Build the application into build/bin directory
build-test: Pass unit tests
build-cover: Create the coverage report (in build/cover)
build: Build the application. Launches: build-install, build-test and build-cover
test-e2e-local: Pass e2e tests locally (launching the service, passing e2e tests, and stopping the service)
test-e2e: Pass e2e tests
package: Create the docker image
publish: Publish the docker image
deploy: Deploy the service (see delivery/deploy/docker-compose.yml)
undeploy: Undeploy the service (see delivery/deploy/docker-compose.yml)
promote: Promote a docker image using the environment DOCKER_PROMOTION_TAG
release: Create a new release (tag and release notes)
run: Launch the service with docker-compose (for testing purposes)
pipeline-pull: Launch pipeline to handle a pull request
pipeline-dev: Launch pipeline to handle the merge of a pull request
pipeline: Launch the pipeline for the selected environment
develenv-up: Launch the development environment with a docker-compose of the service
develenv-sh: Access to a shell of a launched development environment
develenv-down: Stop the development environment
The source code of golang projects needs to be downloaded under GOPATH
. Otherwise, the project packages cannot be found.
Prerequirements:
- Golang 1.9
- GNU Make
git clone git@github.com:Telefonica/seed-golang.git "$GOPATH/src/github.com/Telefonica/seed-golang"
cd "$GOPATH/src/github.com/Telefonica/seed-golang"
make build
This project provides a development environment based on docker. It provides some benefits:
- Common environment with required dependencies already installed
- Pipelines to build the source code, pass the acceptance tests, package a docker image and publish the docker image.
Prerequirements:
- Docker v17
- GNU Make
# Launch the development environment
make develenv-up
# Access to the development environment
make develenv-sh
# Now you can launch any make task (e.g. pipeline-pull)
make pipeline-pull
Pipeline | Description |
---|---|
pipeline-pull | Pipeline triggered by a pull request. |
pipeline-dev | Pipeline triggered by a merge to master branch. It will publish a docker image in dockerhub.hi.inet |
A release will generate:
- A new git tag. The tag is increased automatically taking the previous existing tag as reference and incrementing the second number. For example, if the latest tag is 0.3, a release would create a new tag 0.4.
- A docker image versioned with this tag.
- Release notes in github.
make pipeline-dev RELEASE=true
Json Parameter | Env Parameter | Default value | Description |
---|---|---|---|
address | ADDRESS | :9000 | Address ip:port (ip is optional) where the service is listening |
basePath | BASE_PATH | /seed | API base path. |
logLevel | LOG_LEVEL | INFO | Log level. Possible values are: DEBUG , INFO , WARN , ERROR , FATAL |
mongoUrl | MONGO_URL | 127.0.0.1 | Mongo URL |
mongoDatabase | MONGO_DATABASE | seed | Mongo database |
The default configuration is:
{
"address": ":9000",
"basePath": "/seed",
"logLevel": "INFO",
"mongoUrl": "127.0.0.1",
"mongoDatabase": "seed"
}
Logs are written to console using a JSON format to make easier that log aggregators process them.
It is possible to override the default log level INFO
by using the environment variable LOG_LEVEL
.
The following log entries are written when the service is started up with default configuration:
{"time":"2017-11-26T10:44:38.2435417Z","lvl":"INFO","op":"init","svc":"seed","msg":"Configuration: {\"address\":\":9000\",\"basePath\":\"/seed\",\"logLevel\":\"INFO\",\"mongoUrl\":\"127.0.0.1\",\"mongoDatabase\":\"seed\"}"}
{"time":"2017-11-26T10:44:38.2548223Z","lvl":"INFO","op":"init","svc":"seed","msg":"Starting server at :9000"}
The following log entries correspond to a user creation flow. Note that this flow includes a patch to niji backend (updateBundle
operation to update the line_id in the bundle) and a post to nijiHome backend to create the user (createUser
operation):
{"time":"2017-11-26T10:45:56.0756457Z","lvl":"INFO","trans":"fb471406-d296-11e7-a2a4-0242ac1a0002","corr":"fb471406-d296-11e7-a2a4-0242ac1a0002","op":"createUser","svc":"seed","method":"POST","path":"/seed/users","remoteaddr":"172.26.0.1:39802","msg":"Request"}
{"time":"2017-11-26T10:45:56.0773839Z","lvl":"INFO","trans":"fb471406-d296-11e7-a2a4-0242ac1a0002","corr":"fb471406-d296-11e7-a2a4-0242ac1a0002","op":"createUser","svc":"seed","status":201,"latency":1,"location":"/seed/users/5a1a9b64edde3709d1feef8d","msg":"Response"}
Relevant log entry fields:
- svc. It is the service name. All the log entries generated by the service are identified with
seed
service. - op. There are several operations:
createUser
,getUser
,deleteUser
,updateUser
,notAllowed
, andnotFound
.
Alarm ID | Start condition | Stop condition | Description |
---|---|---|---|
SEED_INIT_01 | Log entry with alarm field to SEED_INIT_01 |
Log entry with op to init , and msg starts with Starting service at |
The service cannot be initialised by a configuration issue or because the port is already used. Review the configuration. |
The following log traces correspond to a conflict with a port that is already in use when the service is launched. It generates an alarm SEED_INIT_01
:
{"time":"2017-11-26T10:49:06.6620696Z","lvl":"INFO","op":"init","svc":"seed","msg":"Configuration: {\"address\":\":9000\",\"basePath\":\"/seed\",\"logLevel\":\"DEBUG\",\"mongoUrl\":\"127.0.0.1\",\"mongoDatabase\":\"seed\"}"}
{"time":"2017-11-26T10:49:06.6775121Z","lvl":"INFO","op":"init","svc":"seed","msg":"Starting server at :9000"}
{"time":"2017-11-26T10:49:06.6862503Z","lvl":"FATAL","op":"init","svc":"seed","alarm":"SEED_INIT_01","msg":"Error starting server. listen tcp :9000: bind: address already in use"}
- Create a component to host the project jobs by executing
_component-setup
- Build
_docker-slave-setup
(with branch: "master" and RAM: 2048) - Configure
_docker-slave-builder
:- Disable automatic hook. Avoid creating the docker image whenever there is a merge in master branch.
- Configure the location to the Dockerfile for development. Set as
Build Context
inDocker Build and Publish
: delivery/docker/dev.
- Execute
_docker-slave-builder
job to create the dockerhub.hi.inet/dcip/seed-golang image that is going to be used in all the dcip jobs. - Create
pipeline-01-pull
. This pipeline is triggered by pull requests:- Github project: https://github.com/Telefonica/seed-golang
- Restrict project to: seed-golang
- Use a custom workspace directory: /home/contint/go/src/github.com/Telefonica/seed-golang
- Git:
- Repository URL: https://github.com/Telefonica/seed-golang/
- Refspec: +refs/pull/:refs/remotes/origin/pr/
- Branches to build: ${sha1}
- Repository URL: https://github.com/Telefonica/seed-golang/
- Trigger:
- Check "GitHub Pull Request Builder"
- GitHub API credentials: https://api.github.com
- Check "Use github hooks for build triggering"
- Trigger phrase: "\Qtest\E"
- Check: "Build every pull request automatically without asking (Dangerous!)."
- Check "GitHub Pull Request Builder"
- Execution environment:
- Check "Color ANSI Console Output"
- Add execute step: "Execute command line (shell)" with:
#!/bin/bash source /home/contint/.bashrc delivery/docker/dev/entrypoint.sh make pipeline-pull
- Actions:
- Add "Set build status on Github commit"
- Publish Cobertura report:
- Pattern: build/cover/cover.xml
- Create
pipeline-02-dev
by cloningpipeline-01-pull
. This pipeline is trigger after a merge in master branch.- Execution is parameterized:
- Add boolean value. Name: RELEASE.
- Add text parameter. Name: GIT_REVISION. Default: master.
- Add password parameter. Name: GITHUB_TOKEN. Default: Token generated with all repo permissions.
- Git:
- Empty Refspec
- Branches to build: ${GIT_REVISION}
- Trigger:
- Uncheck: "GitHub Pull Request Builder"
- Check: "GitHub hook trigger for GITScm polling"
- Update execute step: "Execute command line (shell)" with:
#!/bin/bash source /home/contint/.bashrc delivery/docker/dev/entrypoint.sh make pipeline-dev
- Actions:
- Remove "Set build status on Github commit"
- Execution is parameterized:
See Git WebHooks in DCIP.
- Add team
CI (w+r)
with "write" permissions - Generate a personal access token with all the repo permissions. Note that this token is required to generate the tag and publish the release notes. This is configured with environment variable
GITHUB_TOKEN
. - Add webhook for Pull request:
- Payload URL: "http://dcip.tid.es/ghprbhook/"
- Content type: "application/x-www-form-urlencoded"
- Let me select individual events:
- Check "Issue comment" and "Pull request"
- Uncheck "Push"
- Add webhook for merge:
- Payload URL: "http://dcip.tid.es/github-webhook/"
- Content type: "application/json"
- Just the push event
Copyright 2017 Telefónica Investigación y Desarrollo, S.A.U
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.