/dockerfile-for-dev-ci-image

Primary LanguageShellMIT LicenseMIT

Dockerfile for day8au/dev-ci Image

The docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci Docker image is the reference development and testing environment used throughout Day8's development pipeline.

There are two main use cases for the docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci Docker image:

  1. During development Okteto is used to run this Docker image in Kubernetes.
  2. On pushing code to GitHub, this Docker image is the environment used to execute tests and build releases for deployment with GitHub Actions.

This repository contains the Dockerfile to build the docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci Docker image. It also contains the GitHub Actions that test and deploy the Docker image.

Ubuntu 20.04 LTS was chosen as the base image as it is a long term stable release of a widely understood and supported distribution.

Quick Start

To run an interactive terminal:

$ docker login docker.pkg.github.com
$ docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0.1

Build Requirements

On image startup the exact versions of important software are printed to the console.

Name Version Description Origin
Leiningen 2.9.x Clojure(Script) build tool. Day8's main build tool. GitHub Releases Assets
Clojure 1.10.x 'Official' Clojure CLI tools. Clojure Website
Boot Alternative to Leiningen. Day8 does not use it but several important 3rd party projects do use it. GitHub Release Assets
OpenJDK 11.x (LTS) Java runtime. Dependency of Leiningen, clojure CLI, Boot etc. Ubuntu Package: openjdk-11-headless
Node.js 12.x (LTS) JavaScript runtime. Dependency of shadow-cljs, lumo. NodeSource Package Repository
NPM 6.x (LTS) JavaScript package manager. Dependency of shadow-cljs. Bundled with Node.js
Yarn 1.x ('Classic') JavaScript package manager. Alternative to npm. Yarn Package Repository
Python 2 2.7.x Python 2 runtime. Ubuntu Package: python2
Python 3 3.8.x Python 3 runtime. Ubuntu Package: python3
pip Latest at build time. Python package manager. Ubuntu Package: python3-pip
pipenv Latest at build time. Python package manager. Python Package: pipenv
pytest Latest at build time. Python test runner. Python Package: pytest
flake8 Latest at build time. Python source code checker/linter. Python Package: flake8
Git Latest at build time. Dependency of actions/checkout and day8/lein-git-inject 'Git stable releases' Ubuntu PPA
Git LFS Latest at build time. Required to clone Git repositories using Large File Storage (LFS). PackageCloud
aws Latest at build time. Interface to Amazon Web Services. Dependency of S3 deployments. Python Package: awscli
GNU Compiler Collection 9.3.x C (gcc) and C++ (g++) compiler. Dependency of npm install... and therefore shadow-cljs. Ubuntu Package: build-essential
make Latest at build time. Build automation tool, esp common for older C/C++ projects. Dependency of space-vim. Ubuntu Package: build-essential
cmake Latest at build time. Build automation tool, esp common for newer C/C++ projects. Dependency of gitstatusd. Ubuntu Package: cmake
zstd Latest at build time. Fast, lossless compression. Dependency of actions/cache@v2. Ubuntu Package: zstd
Xvfb Latest at build time. X virtual framebuffer. Dependency of running Chrome without a display in GitHub Actions. Ubuntu Package: xvfb
karma CLI 2.0.0 Dependency of builds that use the Karma test runner. npm: karma-cli
Chromium 56.0.2924.0 Used as a browser to execute Karma tests. This specific old version tracks the version of Electron that Day8 has deployed. Long storey
ChromeDriver 2.29 Dependency of executing Karma tests in Chromium. Project Website
PhantomJS 2.1.1 Obsolete headless browser to execute Karma tests. As of 2020 still used by cljs-oss/canary builds. Bitbucket

REPLs

The following REPLs are available for exploratory programming and debugging of code snippets.

Name Language Example
'Official' Clojure Clojure docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 clojure
Leiningen Clojure docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 lein repl
Planck ClojureScript docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 planck
Lumo ClojureScript docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 lumo
Node.js JavaScript docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 node
Python 2 Python 2 docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 python2
Python 3 Python 3 docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 python3
Bash Bash docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 bash
PowerShell PowerShell Core docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0 pwsh
ZSH (default) ZSH docker run -it --rm docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0

Command-Line Tools

The following tools are not usually required for builds (e.g. GitHub Actions). These are included for convenience and usability when using the image as an interactive shell.

Tool Description Origin
clj-kondo A linter for Clojure code that sparks joy. GitHub Releases Assets
babashka Native Clojure interpreter for scripting. GitHub Releases Assets
gh GitHub's official command line tool. GitHub Releases Assets: gh_N.N.N_linux_amd64.deb
exa Modern replacement for ls. GitHub Releases Assets: exa-linux-x86_64-N.N.N.zip
bat cat clone with syntax highlighting and Git integration. GitHub Releases Assets: bat_N.N.N_amd64.deb
fd Simple, fast and user-friendly alternative to find. GitHub Releases Assets: fd_N.N.N_amd64.deb
fzf Fuzzy finder. Ubuntu Package: fzf
rgrep ('ripgrep') grep that respects .gitignore and automatically skips hidden files/directories and binary files. GitHub Releases Assets: ripgrep_N.N.N_amd64.deb
ag The silver searcher, a code-searching tool similar to ack. Ubuntu Package: silversearcher-ag
websocat Client for WebSockets, like curl for ws://. GitHub Releases Assets: websocat_N.N.N_ssl1.1_amd64.deb
pueue Task management for sequential and parallel execution of long-running tasks. GitHub Releases Assets: pueue-linux-x86_64 and pueued-linux-x86_64
tmux Terminal multiplexer. Ubuntu Package: tmux
rlwrap A 'readline wrapper' to allow the editing of keyboard input for any command. Ubuntu Package: rlwrap
diff-so-fancy Human readable diffs. npm: diff-so-fancy
diffstat Make a histogram of diffs. Ubuntu Package: diffstat
jq Like sed for JSON. Ubuntu Package: jq
yq jq wrapper for YAML and XML. Python Package: yq
hexyl Hex viewer. GitHub Releases Assets: hexyl_N.N.N_amd64.deb
neofetch System information tool. Ubuntu Package: neofetch
htop Interactive process viewer. Ubuntu Package: htop
ncdu Interactive disk usage analyzer. Ubuntu Package: ncdu
ssh OpenSSH client. Ubuntu Package: openssh-client
mosh More robust and responsive SSH client. Ubuntu Package: mosh
pngnq Lossy PNG compressor. Ubuntu Package: pngnq
pngquant Lossy PNG compressor. Ubuntu Package: pngquant
pngcrush Lossless PNG compressor. Ubuntu Package: pngcrush
pngtools Series of tools for PNGs. Ubuntu Package: pngtools
pngmeta Extracts metadata from PNGs. Ubuntu Package: pngmeta
pngcheck Verifies integrity of PNGs. Ubuntu Package: pngcheck
jpegoptim Lossy JPEG compressor. Ubuntu Package: jpegoptim
jhead JPEG metadata manipulation tool. Ubuntu Package: jhead
jpegpixi Removes defects from JPEGs. Ubuntu Package: jpegpixi
jpeginfo Verifies integrity of JPEGs. Ubuntu Package: jpeginfo

Editors

Usually editing of source files is done outside of the container and either synced with Okteto or checked out with Git in the case of GitHub Actions.

However, on rare occasions it may be useful to shell into the container to edit a file. To make such situations less painful we include the following common editors:

Name Configuration Description Origin
nano Ubuntu Defaults The simplest editor. Good for beginners. Ubuntu Package: nano
nvim ('NeoVim') space-vim A better vim. 'Neovim Unstable' Ubuntu PPA
emacs Spacemacs For the grey-beards. Ubuntu Package: emacs-nox

Why Not Docker Hub ?

In 2020 Docker Hub introduced rate limiting and around the same time GitHub Actions finally fixed use of GitHub Container Registry.

Therefore we no longer publish images to Docker Hub and use GitHub Container Registry instead.

GitHub Actions Example

jobs:
  test:
    name: Test
    runs-on: ubuntu-18.04
    container:
      image: docker.pkg.github.com/day8/dockerfile-for-dev-ci-image/dev-ci:0.1
      credentials:
        username: ${{ github.actor }}
        password: ${{ secrets.GLOBAL_TOKEN_FOR_GITHUB }} # <-- you need to create a GitHub Secret with a manual token that has global access as github.token only has access to the current repo!

Troubleshooting

Upgrading Chrome / Chromium

Problem

The version of Chromium used needs to be upgraded to match an Electron upgrade.

Solution

Hold on to your hat and walk this path.

Karma - 'Gtk: cannot open display: :99'

Problem

When executing Karma tests an error similar to the following is displayed:

21 08 2020 12:10:13.052:ERROR [launcher]: ChromeHeadless stdout:
21 08 2020 12:10:13.052:ERROR [launcher]: ChromeHeadless stderr: [0821/121012:ERROR:nacl_helper_linux.cc(311)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
[2442:2442:0821/121013:ERROR:browser_main_loop.cc(272)] Gtk: cannot open display: :99

Solution

xvfb, the X virtual framebuffer, is not running. This should be started by the /docker-entrypoint.sh script in the container which is the ENTRYPOINT of the Docker image.

If this error is occurring on GitHub Actions it may be because:

  1. GitHub Actions overrides the Docker image ENTRYPOINT with the --entrypoint CLI option to be /usr/bin/tail.
  2. We replace /usr/bin/tail with an intercept script that executes /docker-entrypoint.sh before executing the original tail at /usr/bin/tail.original.
  3. GitHub Actions has changed the --entrypoint option to something other than /usr/bin/tail.

Deployment

Simply push a semver tag of the form v1.2.3. GitHub Actions will publish Docker images for day8au/dev-ci:1.2.3, day8au/dev-ci:1.2 and day8au/dev-ci:1. E.g.:

$ git tag v1.2.3 HEAD
$ git push --tags

License

This repository is MIT licenced