/ber-lab-jupyter-ansible

Primary LanguageHCLMIT LicenseMIT

Jupyter notebook deployment for Jugend Hackt Labs

Foreword

This repository holds terraform and Ansible configuration files to set up Jupyter notebooks on a Hetzner Cloud instance. It uses traefik as a reverse proxy and docker with docker-compose to orchestrate the containers which will run the jupyter notebooks.

Warning

A word of warning: You should be familiar with using the command line. The steps described here try to help you through the process but we can not cover everything.

Prerequisites

  • Computer running macOS, Linux or Windows with WSL
  • A ssh key (create on using ssh-keygen -t ed25519) in ~/.ssh/id_ed25519
  • A Domain name like alpacabook.de with access to its DNS entries.
  • A Hetzner Cloud account.
  • Python
  • terraform
  • Ansible

Setup

Step 1: Setup Hetzner Cloud instance

  1. Create a new project on the Hetzner Cloud Console
  2. Create a new token for the project with read/write access (open left sidebar > Security > API-Tokens > add API-Token)
  3. Copy this token into your password manager of choice as it will only be displayed once.

Step 2: Create a new configuration file.

  1. Copy the template config to a new file cp config.tmpl.yaml config.yaml
  2. Replace the placeholder for hcloud_token with your token from Hetzner Cloud.
  3. Replace the placeholder for domain with your domain name like alpacabook.de.
  4. Choose a hostname. This will be the subdomain your server will be reachable by. This could just be jupyter for example. So the server will be reachable using jupyter.alpacabook.de and the jupyter notebooks will be reachable under 00[1,2,3...].jupyter.alpacabook.de
  5. Add the location of your public key into ssh_key_location like ~/.ssh/id_ed25519. This will be the key used to access the server initially.
  6. Add an email for the email field under which you will get your certificates from Letsencrypt
  7. Fill the admins array. These will be the accounts that will be created on the server. Use your the public part of your ssh-key from above or any other ssh-key from your friends who want to access this server. Regarding the shell: if you know, you know otherwise just leave it as is.

Step 3: Setup

  1. install requirements

    make install

  2. run generate.py with number of instances you need

    python generate.py -n 10

Step 4: Create your Hetzner Cloud instance

  1. run terraform

    make tf_init && make tf_plan

    Verify that the settings shown in this plan is what you are expecting.

  2. apply terraform

    make tf_apply

    this will print the public IPv4 and IPv6 addresses

Step 4: Add IP Adresses to DNS records of your domain

  1. navigate to your registrar find your chosen domain's DNS records
  2. add A and AAAA records for *.hostname and hostname
  3. the Arecords take the IPv4 address
  4. the AAAArecords take the IPv6 address
  5. verify that the records are published using nslookup or dig

Step 5: Deploy the docker containers using Ansible

  1. Run Ansible

    make a_run

    This might take a while.

  2. in case any step fails, use tags in ansible/playbook.yaml to run single steps

  3. to debug you can access the server via ssh -i ~/.ssh/private_ssh_key <admin_user>@<hostname>.<domain>

Step 6: extract passwords and domain names

  1. go to ansible/config.yaml
  2. get urls and passwords

Step 7: Use jupyter notebooks

The jupyter notebooks can be reached under 001.<hostname>.<domain> where 001 indicates the instance number.

Step 8: (later) destroy server

  1. open your Hetzner Cloud Console
  2. deactivate protection
  3. delete server