The paper is here.
For first time usage, create the two overlay files that contains all the packages needed:
-
Clone this repo to greene scratch.
-
In the main directory of nlu_project, run
bash ./scripts/create_base_overlay.sh
andbash ./scripts/create_package_overlay.sh
.
If you already have the overlay files:
-
Connect to NYU vpn.
-
Go to https://ood-3.hpc.nyu.edu/ -> Interactive Apps -> Jupyter Notebook
-
Follow the instructions under 'How to use your singularity+conda environment in jupyterhub'. The python wrapper for step 2 should look like this:
singularity exec $nv \
--overlay /scratch/zh2095/nlu_project/overlay-base.ext3:ro \
--overlay /scratch/zh2095/nlu_project/overlay-packages.ext3:ro \
/scratch/wz2247/singularity/images/pytorch_21.06-py3.sif \
/bin/bash -c "source ~/.bashrc; conda activate /ext3/conda/bootcamp; $cmd $args"
- Launch the notebook, and select the kernel 'my_env'.