This repository is for tutorials on simulating and working with DESI data. There are additional more detailed tutorials within many of the DESI code repositories; tutorials here are generally for:
- Topics that span individual repositories, e.g. working with data challenge outputs
- Tutorials to be presented at a collaboration meeting where we want to decouple updates to the tutorial itself from the desispec, desisim, etc. code versions that the tutorial is describing.
- Some tutorials are grouped under a dated directory, e.g. Dec2020, for tutorials presented at a specific DESI meeting.
These tutorials use a Jupyter server at NERSC, which provides pre-installed DESI code and access to all DESI data at NERSC without having to download or install anything locally.
First, get a NERSC account following the instructions on the DESI wiki. Then login to cori:
ssh cori.nersc.gov
Jupyter "kernels" define a specific set of code versions to use. To install DESI jupyter kernels from cori.nersc.gov (you only need to do this once):
source /project/projectdirs/desi/software/desi_environment.sh 20.8
$DESIMODULES/install_jupyter_kernel.sh 20.8
$DESIMODULES/install_jupyter_kernel.sh master
Get a copy of the tutorials on cori.nersc.gov:
mkdir -p $HOME/desi/git
cd $HOME/desi/git
git clone https://github.com/desihub/tutorials
Login at https://jupyter.nersc.gov and enter your credentials and OTP (One Time Password).
Then, click on the Cori Shared CPU Node "Start" button.
In the file browser on the left, navigate to wherever you cloned the
tutorials repository. ($HOME/desi/git/tutorials in the above commands).
Click on a tutorial to try it out.
If you click on a tutorial and it prompts you to "Select Kernel" from a drop down list, it means that the default tutorial kernel in GitHub isn't one of the ones that you have already installed and it is asking you to select from one of the kernels that you do have installed. In most cases the tutorials will work with the latest release (e.g. 20.8) and the "master" release (updated nightly, name to switch to "main" in 2021). Otherwise the text at the top of the tutorial will tell you which release is needed. Go back to cori.nersc.gov, run "$DESIMODULES/install_jupyter_kernel.sh VERSION" for the required version, and reload the Jupyter page.
Tutorials in this repository include
- spectro_nz.ipynb: making an n(z) histogram from the spectro production zbest files.
- Intro_to_DESI_SV_spectra.ipynb: working with real DESI data from minisv2
- Intro_to_DESI_spectra.ipynb: how to find and read DESI spectra (based on sims, similar to what main survey will be)
- simulating-desi-spectra.ipynb: how to simulate your own spectra
- survey-simulations.ipynb: how to use "survey simulation" outputs
- RedrockOutputs.ipynb: Understanding redrock outputs and connecting the coefficients to template spectra.
- Fiber assignment
- FiberAssign.ipynb: fiber assignment basics
- FiberAssignDECaLS.ipynb: use observational data from DR8 to prepare it for fiberassign
- FiberAssignMocks.ipynb: Generate mock files from simulations to feed into fiberassign
- FiberAssignAlgorithms_Part1.ipynb: a detailed look at fiberassign algorithms
- FiberAssignAlgorithms_Part2.ipynb: fiberassign with multiple survey passes
- redshift-database.ipynb : interacting with DESI pipeline and redshift data in a database (work in progress).
These links allow you to browse the results of these tutorials from GitHub; to run them yourself follow the instructions in the "Getting started" section above.
Tutorials hosted in other packages include (but haven't been recently vetted):
- How to run survey simulations.
- How to convert an SED into a simulated DESI spectrum
- How to run fiber assignment
- How to run quicksurvey catalog-level simulations
- How to make all-sky plots
- Working with DESI target bits (Main, CMX, SV)
Include at the beginning of the tutorial what is needed as a prerequisite for running the tutorial, e.g. specific codes, environment variables, datasets. It's OK to link elsewhere for detailed instructions (e.g. for how to install DESI code in general).