snahelou/gitlab-ci-pipelines-exporter

mvisonneau/gitlab-ci-pipelines-exporter

gitlab-ci-pipelines-exporter is a very small binary that allows you to monitor your GitLab CI pipelines with Prometheus or any monitoring solution supporting the OpenMetrics format.

TL:DR

# Write your config file somewhere on disk
~$ cat <<EOF > $(pwd)/config.yml
# URL and Token with sufficient permissions to access your GitLab's projects
# pipelines informations
gitlab:
  url: https://gitlab.example.com
  token: xrN14n9-ywvAFxxxxxx

# Waiting time between polls for each projects
polling_interval_seconds: 30

# The list of the projects you want to monitor
projects:
  - name: foo/project
    ref: master
  - name: bar/project
    ref: master
  # Wildcard is supported if you want to monitor all owned projects
  #- name: '*'
  #  ref: master
EOF

# If you have docker installed, it is as easy as :
~$ docker run -d \
   --name gitlab-ci-pipelines-exporter \
   -v $(pwd)/config.yml:/etc/config.yml \
   -p 8080:80 \
   mvisonneau/gitlab-ci-pipelines-exporter:latest \
   -config /etc/config.yml

You should then be able to see the following logs

~$ docker logs -f gitlab-ci-pipelines-exporter
2018/09/21 12:44:05 -> Starting exporter
2018/09/21 12:44:05 -> Polling https://gitlab.example.com every 30s
2018/09/21 12:44:05 -> 2 project(s) configured
2018/09/21 12:44:05 --> Polling ID: 1 | foo/project:master
2018/09/21 12:44:05 --> Polling ID: 2 | bar/project:master

And this is an example of the metrics you should expect to retrieve

~$ curl -s localhost:8080/metrics | grep gitlab_ci_pipeline
# HELP gitlab_ci_pipeline_last_run_duration_seconds Duration of last pipeline run
# TYPE gitlab_ci_pipeline_last_run_duration_seconds gauge
gitlab_ci_pipeline_last_run_duration_seconds{project="bar/project",ref="master"} 676
gitlab_ci_pipeline_last_run_duration_seconds{project="foo/project",ref="master"} 33
# HELP gitlab_ci_pipeline_run_count GitLab CI pipeline run count
# TYPE gitlab_ci_pipeline_run_count counter
gitlab_ci_pipeline_run_count{project="bar/project",ref="master"} 0
gitlab_ci_pipeline_run_count{project="foo/project",ref="master"} 0
# HELP gitlab_ci_pipeline_status GitLab CI pipeline current status
# TYPE gitlab_ci_pipeline_status gauge
gitlab_ci_pipeline_status{project="bar/project",ref="master",status="failed"} 0
gitlab_ci_pipeline_status{project="bar/project",ref="master",status="running"} 0
gitlab_ci_pipeline_status{project="bar/project",ref="master",status="success"} 1
gitlab_ci_pipeline_status{project="foo/project",ref="master",status="failed"} 0
gitlab_ci_pipeline_status{project="foo/project",ref="master",status="running"} 0
gitlab_ci_pipeline_status{project="foo/project",ref="master",status="success"} 1
# HELP gitlab_ci_pipeline_time_since_last_run_seconds Elapsed time since most recent GitLab CI pipeline run.
# TYPE gitlab_ci_pipeline_time_since_last_run_seconds gauge
gitlab_ci_pipeline_time_since_last_run_seconds{project="bar/project",ref="master"} 87627
gitlab_ci_pipeline_time_since_last_run_seconds{project="foo/project",ref="master"} 29531

Usage

~$ gitlab-ci-pipelines-exporter -h
Usage of .gitlab-ci-pipelines-exporter:
  -config string
    	Config file path (default "~/.gitlab-ci-pipelines-exporter.yml")
  -listen-address string
    	Listening address (default ":80")

HELM

If you want to make it run on kubernetes, there is a helm chart for that!

~$ git clone git@github.com:mvisonneau/gitlab-ci-pipelines-exporter.git
~$ cd gitlab-ci-pipelines-exporter/charts
~$ cat <<EOF > values.yml
gitlab:
  url: https://gitlab.example.com
  token: xrN14n9-ywvAFxxxxxx
polling_interval_seconds: 30
projects:
  - name: foo/project
    ref: master
  - name: bar/project
    ref: master
EOF
~$ helm package gitlab-ci-pipelines-exporter
~$ helm upgrade -i gitlab-ci-pipelines-exporter ./gitlab-ci-pipelines-exporter-0.0.0.tgz -f values.yml

Examples

Here are a couple of Grafana dashboards I've been able to craft with those metrics :

Develop / Test

If you use docker, you can easily get started using :

~$ make dev-env
# You should then be able to use go commands to work onto the project, eg:
~docker$ make fmt
~docker$ gitlab-ci-pipelines-exporter

Contribute

Contributions are more than welcome! Feel free to submit a PR.