LibreTexts JupyterHub Default Environment

This repository contains the repo2docker config files needed to build the default Jupyter environment available at jupyter.libretexts.org. The goal of this environment is to contain as many of the most popular software packages needed for scientific computing from a variety of languages: Python, R, Octave, Sage, Julia, etc.

Repo2docker Configuration Files

environment.yml. This file contains all the packages to be installed by conda. It is generally used for R, Python and C/C++ packages. We only install packages available from the conda-forge and default channels here.
apt.txt. Here we install Debian packages through apt-get which otherwise are not available through conda, such as texlive and some base functionality tools. We will only install from Ubuntu 18.04 APT repositories.
Project.toml. This is the file where repo2docker installs Julia packages.
postBuild. This is a shell script which runs after everything else has been built and installed. We install SageMath and enable JupyterLab extensions here.

You can read more about each configuration file's purpose from the repo2docker documentation.

How to Build a New Image

Step 1: Pre-Setup

Install repo2docker from source using pip install git+https://github.com/jupyterhub/repo2docker.git. The stable releases on PyPI are often out of date and the developers themselves have suggested to install from source.
Install and enable Docker if you have not already. Repo2docker requires Docker to be running in order to function.
Update the Docker image which repo2docker builds on top of with docker pull buildpack-deps:bionic.
Clone the default-env repository with git clone https://github.com/LibreTexts/default-env.git. You may want do this on one of the test servers we have available so that you can take advantage of FAST internet speeds for image builds and pushes.

Step 2: Making Changes

Before beginning, it is usually good practice to create a new git branch and make your changes in there. If you are only doing something minor, and have someone else available to immediately review, you may find it easier to make the changes directly in the master branch. Use your best judgement and ask if uncertain.
Using nano, vim, or another text-editor of your choice, make changes to add/remove/alter packages within the relevant repo2docker configuration file(s). For the most part you should be able to follow the intallation pattern already present within the file. For instance, all packages installed via conda (such as those for Python and R) will go into environment.yml under the dependencies: section with a format of -<package-name>=<version-number. When adding packages, be sure to add a comment in the file about what the package is and why it was included.
If the package requires a jupyter labextension, then be sure to place it in postBuild. Don't forget to add --no-build at the end of the command or else JupyterHub will try to rebuild itself after each jupyter labextension command.

Step 3: Building, Tagging and Pushing a Test Image

After your changes have been made, run repo2docker --user-name jovyan --user-id 1000 . in the default-env/ directory to start the build. We ensure that we do not break the filesystem by specifying jovyan as the username, --user-id 1000 specifies a normal user account (not root), and . starts the build in the current working directory.
Once finished, you will have an image that looks like r2d-<string>:latest, where <string> could look something like 2e1598919088. If you do not see this, look for it with docker images. You will want to tag this image with docker tag r2d-<string>:latest libretexts/default-test:<tagname> so that you can later push this to our testing dockerhub repository . Be sure to fill in the <string> and <tagname> fields with the actual string value given by repo2docker and the tagname that you desire. An example testing tagname would be something like pandas-test.
After the image has been tagged, push it to the testing dockerhub repository wth docker push libretexts/default-test:<tagname>. Use the Libretexts dockerhub login credentials when pushing.

Step 4: Testing the Image

Before deploying the image to the production environment at jupyter.libretexts.org, you will want to test the image on staging.jupyter.libretexts.org.

ssh into rooster and edit the helm values for staging.jupyter.libretexts.org located at ~/jupyterhub-staging/staging-config.yaml. Look for the following lines and update the tag: <tagname> value with your chosen tagname;

image:
  name: libretexts/default-test
  tag: "<tagname>"

In order to apply your helm value changes, you will need to upgrade the helm chart for staging-jhub with these new values. Navigate to ~/jupyterhub-staging/ and use the command ./upgrade.sh to run the upgrade shell script.
After the upgrade is complete, you can go to staging.jupyter.libretexts.org, spawn the environment selected by default, and you should now be able to test your image in JupyterLab.

Step 5: Deploying to Production

If you find that the test on staging was successful, you can now deploy the image on production.

Retag the image with your desired tag (2.x in this case) using docker tag libretexts/default-test:<tagname> libretexts/default-env:2.x. Starting with our images built using repo2docker, we are using 2.x (where x is some number) notation to denote the version number.
Push this image to the default-env dockerhub repository using docker push libretexts/default-env:2.x.
ssh into rooster and edit the helm values for jupyter.libretexts.org located at ~/jupyterhub/config.yaml. Look for a similar tag: 2.x line as you did in Step 4, and change it to your new 2.x value.

image:
  name: libretexts/default-env
  tag: "2.x"

Before upgrading, check if there are any users of the Hub with kubectl get pods -n jhub before running the upgrade script, as it could cause problems for active users. Get confirmation from another team member before upgrading if there are active users.
Within the ~/jupyterhub/ directory, run ./upgrade.sh to apply the helm values for jhub, similar to how you did before on staging-jhub.
Once the upgrade is complete, your new image will be deployed and available at jupyter.libretexts.org. Spawn the environment selected by default to ensure it deployed properly. Great work!

Step 6: Committing to Github

If you made your changes within a branch other than master, be sure to commit your changes made within ~/jupyterhub/config.yaml and ~/jupyterhub-staging/staging-config.yaml and then push and make a pull request on the default-env repository. Otherwise, simply push straight to the master branch.

Legacy environment

Before our current setup, there has been a legacy default enviornment used before Aug. 11th, 2020. The Dockerfile for this image is available in the legacy-rich-default branch. This is kept for reference purposes.

Requesting additional installed packages

In general, we will install any user requested software available in the Ubuntu 18.04 LTS apt repositories or from the conda-forge Conda channel. The conda installed packages, in general, will be the default programs accessible at the JupyterLab command line. The latest compatible versions of the requested packages will be installed. To request packages you can:

Make a pull request to this repository, adding your desired packages to the environment.yml or apt.txt,
Open an issue on this repository with the desired packages, or
Send an email to jupyterteam@ucdavis.edu with a list of the desired packages.

If your desired package is not available in the default apt repositories or the Conda Forge Conda channel, you have these options for supporting your software needs:

Submit the package for inclusion in Conda Forge channel,
Install the package manually after launching the default environment (see our FAQ page for instructions on how to install custom packages), or
create your own repo2docker compatible environment and send us the URL of the Git repository with your configuration files.

Note that we will not install "un-released" software from arbitrary locations in the default environment, but you may do so using options 2 and 3 above. We will consider installing packages from PyPi, CRAN, and other language specific package managers on a case-by-case basis for temporary use until it is added to Conda Forge.

celine168/default-env