The Quick Look Framework is part of the DES Instrument Control System (ICS) and provide interfaces for executing the quick look pipeline and display data quality information in realtime.
In the current version it is possible to control the execution of the quicklook pipeline through the interface, process multiple cameras in paralel and follow the progress of the data reduction. The interfaces for QA display are still in initial stage of development. Your suggestions and comments to improve this system are very welcome!
See a presentation from June 2017 here.
-
Install the Quick Look Framework
# We assume you are using bash export QLF_ROOT=$HOME/quicklook mkdir -p $QLF_ROOT cd $QLF_ROOT git clone https://github.com/desihub/qlf.git
-
Install software dependencies
source ~/miniconda3/bin/activate root
NOTE: if you don't have conda installed we recommend installing miniconda3 follow the instructions here and make sure ~/miniconda3/bin is in your PATH
conda config --add channels conda-forge conda create --name quicklook python=3.5 --yes --file qlf/requirements.txt source activate quicklook pip install -r qlf/extras.txt
-
Install the DESI Quick Look pipeline
cd $QLF_ROOT git clone https://github.com/desihub/desispec.git git clone https://github.com/desihub/desiutil.git
Test if DESI Quick Look pipeline is available (optional)
for package in desispec desiutil; do echo "Setting $package..." export PATH=$QLF_ROOT/$package/bin:$PATH export PYTHONPATH=$QLF_ROOT/$package/py:$PYTHONPATH done desi_quicklook --help
-
Get test data
mkdir -p $QLF_ROOT/spectro # or any other place, make sure this path is consistent with qlf.cfg in step 5. cd $QLF_ROOT/spectro # Test data for local run of QLF: night 20190101, exposures 3 and 4. wget -c http://portal.nersc.gov/project/desi/data/quicklook/20190101_small.tar.gz tar xvzf 20190101_small.tar.gz
-
Configure QLF
Create your
qlf.cfg
filecd $QLF_ROOT/qlf/config cp qlf.cfg.template qlf.cfg
Edit
qlf.cfg
(follow the instructions there)NOTE: in development mode, QLF will process the data in the path specified in the
qlf.cfg
. Each time you run QLF a fresh database is created and the Quick Look outputs are ingested after each exposure is processed.
-
Activate the
quicklook
environmentsource ~/miniconda3/bin/activate quicklook export QLF_ROOT=$HOME/quicklook
-
Start QLF
cd $QLF_ROOT/qlf/qlf ./run.sh
NOTE: you can follow the progress of the data processing watching
qlf.log
or from the web application.
Contact: helpdesk@linea.gov.br