/vagrant-docker-python3

Project that automates the creation of a Python 3.10 + Jupyter local environment, meant to be used for learning and testing purposes.

Primary LanguageShellMIT LicenseMIT

Vagrant Docker Python3

⭐️ Local, automated and isolated environment for learning Python 3.

This project automates the creation of a Python 3.10 + Jupyter local environment, meant to be used for learning and testing purposes.

It creates a VM using Vagrant, that has Docker and Docker Compose installed.

Python3 + Jupyter are installed and executed inside a Docker container, inside the VM.

With this, you don't need to install Docker or Python on your local computer.
You just need to install Vagrant and VirtualBox.

All the images and containers are created in an isolated environment that can be deleted at any time.

Architecture

Vagrant creates an Ubuntu VM that installs Docker, pulls Docker images from DockerHub, and runs containers with their corresponding port mappings.

Jupyter website will be accessible to the host's web browser through port 8888.

The automation process is specified using the following files:

  1. Vagrantfile: Tells Vagrant how to create and configure the VM
  2. docker-compose.yml: Tells Docker Compose which and how containers should be executed

The following diagram shows the architecture:

Architecture diagram

Prerequisites

Verify installation

Note

Execute these steps only if it's the first time that you use Vagrant with VirtualBox.
If not, you can skip them. They only serve to test the Vagrant + VirtualBox installation.
If Vagrant and VirtualBox are installed and configured correctly, then the environment will work fine (it has already been tested, and is repeatable).

Check that the vagrant executable was added correctly to the PATH variable:

vagrant version

Check that vagrant is able to create a VM:

mkdir test-vagrant
cd test-vagrant
vagrant init ubuntu/jammy64
vagrant up
vagrant ssh
pwd
exit
vagrant destroy --force
cd ..
rm -rf test-vagrant

Warning

If the following error appears after executing vagrant up:
No usable default provider could be found for your system.

  1. Verify that VirtualBox was installed correctly
  2. Obtain more info about the error: 
    vagrant up --provider=virtualbox

Warning

If the following error appears after executing vagrant up:
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005)

  • Reinstall VirtualBox

Warning

If Vagrant gets stuck on the following line after executing vagrant up:
SSH auth method: private key

  • Windows users: Open cmd as admin and execute: 
    bcdedit /set hypervisorlaunchtype off
    This disables Hyper-V.

Warning

For other issues:

Steps to run the environment

All the vagrant commands must be executed in the host machine from the folder that contains the Vagrantfile (in this case, the project root folder).

Note

For Windows users:
If Vagrant doesn't show any output in the stdout for a Vagrant command after some time, press the Enter key or right click in the console window. See this post for more info about this problem.

1. Start the VM [host]

This will:

  1. Install Docker inside the VM
  2. Pull the Docker images from DockerHub
  3. Run the containers All with the corresponding port mappings.

Note

Docker images/containers will only be downloaded/executed if the Docker Compose up line in the Vagrantfile is uncommented.

vagrant up

2. Check the status of the VM [host]

vagrant status

3. Connect to the VM [host]

This connection is done via SSH.

vagrant ssh

Tip

Some interesting commands to execute inside the VM:

Commmand Description
free -h Display amount of free and used memory in the VM
docker stats Display a live stream of container(s) resource usage statistics.
Useful to monitor Docker containers memory usage.
docker container ls --all List all Docker containers (running or not).
If both containers specify "Up" in the status column, everything is running fine.
docker logs <containerid> Fetch the logs of a container.
Really useful to see what's going on.
docker top <containerid> Display the running processes of a container
docker exec -it <containerid> <command> Run a command in a running container (in interactive mode)
docker images List images
docker version Show the Docker version information
docker info Display system-wide information
netstat -tulpn | grep LISTEN Display network connections (listening TCP or UDP).
Useful to check that Jupyter port (8888) is listening.

4. Create tmux session [vm]

The VM welcome message shows the command for connecting to the tmux session.

The tmux window is divided in panes with the following layout:

┌─────────────────┬──────────────────────────┐
│    LINUX CLI    │                          │
├─────────────────┤ IPYTHON 3.10 INTERPRETER │
│ JUPYTER-LAB URL │                          │
└─────────────────┴──────────────────────────┘

5. Access JupyterLab in your web browser [host]

The URL for accesing JupyterLab will be shown in the JUPYTER-LAB URL pane of the tmux session.

Simply copy that URL and paste it in your web browser.

Note

It can also be obtained executing the following command:

docker logs vagrant-python-1 2>&1 | grep -o '[^ ]*127.0.0.1[^ ]*'

(Optional) Detach from tmux session [vm]

Ctrl-B + d

(Optional) Attach again to tmux session [vm]

tmux list-sessions
tmux attach-session -t <session-name>

If the tmux session is deleted (for example, using Ctrl-D several times), you may need to restart the tmux server in order to be able to connect again to the tmux session:

tmux kill-server
tmux attach-session -t <session-name>

(Optional) Remove and start containers to clean data [vm]

Note

Only if containers where executed using Docker Compose.

This is useful if you want to clean the data inside the containers.

cd /vagrant
docker compose rm --stop --force
docker compose up -d

(Optional) Connect to one of the Docker containers [vm]

Obtain the name of the container you want to connect to:

docker container ls --all

The name is the last column.

Execute the bash command in that container to connect to it:

docker exec -it <container-name> bash

Stop the VM (keeps data) [host]

Stopping the VM will stop the Docker containers and turn off the VM.
All the data is persisted inside the containers, and a subsequent turn on of the VM (and the containers) will have access to that data.

Stop the VM:

vagrant halt

Check the status of the VM:

vagrant status

Start the VM and the containers again:

vagrant up

Destroy the VM (removes data) [host]

Destroying the VM will remove all the VM data, and therefore, the containers inside it.

This should be the option used if you do not want to keep the data, and you want to have a "clean" environment in the next turn on of the VM (because the VM and the containers will be created from scratch).

vagrant destroy

Additional notes

Whenever you change the docker-compose.yml file, you need to run vagrant reload to redefine the Vagrant box.

Jupyter project

IPython

Enhanced interactive Python shell.

JupyterLab

The latest web-based interactive development environment.

Jupyter Notebook

The original web application for creating and sharing computational documents.

Voilà

Runs the code in the Jupyter notebooks and transforms them to standalone web applications and dashboards.

References