An opinionated introduction to using Docker as a software development tool
Disclaimer: This is a collection of examples of how you can use Docker as a supporting tool for software development. I make no claim as to if this follows best practices of software engineering. Exploration of uses on your own should be considered mandatory follow-up to reading.
- A computer with Docker installed on it
To install Docker Community Edition on your Linux, Mac, or Windows machine follow the instructions in the Docker docs.
The official Docker documentation and tutorial can be found on the Docker website. It is quite thorough and useful. It is an excellent guide that should be routinely visited, but the emphasis of this introduction is on using Docker, not how Docker itself works.
A note up front, Docker has very similar syntax to Git and Linux, so if you are familiar with the command line tools for them then most of Docker should seem somewhat natural (though you should still read the docs!).
It is still important to know what Docker is and what the components of it are. Docker images are executables that bundle together all necessary components for an application or an environment. Docker containers are the runtime instances of images — they are images with a state.
Importantly, containers share the host machine's OS system kernel and so don't require an OS per application. As discrete processes containers take up only as much memory as necessary, making them very lightweight and fast to spin up to run.
Much like GitHub allows for web hosting and searching for code, Docker Hub allows the same for Docker images. Hosting and building of images is free for public repositories and allows for downloading images as they are needed. Additionally, through integrations with GitHub and Bitbucket, Docker Hub repositories can be linked against Git repositories so that automated builds of Dockerfiles on Docker Hub will be triggered by pushes to repositories.
To begin with we're going to pull down the Docker image we're going to be working in for the tutorial
docker pull matthewfeickert/intro-to-docker
and then list the images that we have available to us locally
docker images
If you have many images and want to get information on a particular one you can apply a filter, such as the repository name
docker images matthewfeickert/intro-to-docker
To use a Docker image as a particular instance on a host machine you run it as a container. You can run in either a detached or foreground (interactive) mode.
Run the image we pulled as an interactive container
docker run -it matthewfeickert/intro-to-docker:latest /bin/bash
You are now inside the container in an interactive bash session. Check the file directory
pwd
revealing that you are in /root and check the host to see that you are not in your local host system
hostname
Further, check the os-release to see that you are actually inside a release of Debian (given the Docker Library's Python image Dockerfile choices)
cat /etc/os-release
Open up a new terminal tab on the host machine and list the containers that are currently running
docker ps
Notice that the name of your container is some randomly generated name. To make the name more helpful, rename the running container
docker rename <CONTAINER ID> my-example
and then verify it has been renamed
docker ps
As a test, create a file in your container
touch test.txt
In the container exit at the command line
exit
You are returned to your shell. If you list the containers you will notice that none are running
docker ps
but you can see all containers that have been run and not removed with
docker ps -a
To restart your exited Docker container start it again and then attach it to your shell
docker start <CONTAINER ID>
docker attach <CONTAINER ID>
Starting and attaching by name
You can also start and attach containers by their name
docker start <NAME>
docker attach <NAME>
Notice that your entry point this time was at / not /root, so navigate to /root (which is $HOME) and then check that your test.txt still exists
cd
ls
So this shows us that we can exit Docker containers for arbitrary lengths of time and then return to our working environment inside of them as desired.
Clean up a container
If you want a container to be cleaned up — that is deleted — after you exit it then run with the --rm option flag
docker run --rm -it <IMAGE> /bin/bash
Copying files between the local host and Docker containers is possible. On your local host find a file that you want to transfer to the container and then
docker cp <file path> <CONTAINER ID>:/
and then from the container check and modify it in some way
echo "This was written inside Docker" >> example_file.txt
and then on the local host copy the file out of the container
docker cp <CONTAINER ID>:/example_file.txt .
and verify if you want that the file has been modified as you wanted
tail example_file.txt
This I/O allows for Docker images to be used for specific tasks that may be difficult to do with the tools or software installed on only the local host machine. For example, debugging problems with software that arise on cross-platform software, or even just having a specific version of software perform a task (e.g., using Python 2 when you don't want it on your machine, or using a specific release of TeX Live when you aren't ready to update your system release).
You can run a Jupyter server from inside of your Docker container. First run a container while exposing the container's internal port 8888 with the -p flag
docker run --rm -it -p 8888:8888 matthewfeickert/intro-to-docker /bin/bash
Then start a Jupyter server with the server listening on all IPs
jupyter notebook --allow-root --no-browser --ip 0.0.0.0
Finally, copy and paste the following with the generated token from the server as <token> into your web browser on your local host machine
http://localhost:8888/?token=<token>
You now have access to Jupyter running on your Docker container.
If Docker is run in a detached state then the container will exit as soon as it has executed the commands given to it. If the clean up option is given as well, then the container will be removed once it exits.
docker run --rm matthewfeickert/intro-to-docker:latest /bin/bash -c 'cat /etc/os-release && echo "hello" && python --version'
This allows for containers to serve as either full pieces of infrastructure (e.g., reana) or as individual instances that are spun up, used, and spun down (think something somewhat along the lines of AWS Lambda).
To be added for a future extension
If you would like to contribute please read the CONTRIBUTING.md and then open up an Issue or a PR based on its recommendations. Contributions are welcome and encouraged!