Dask_k8s

This repository will contain the files related to the deployment of Dask Helm charts on the OpenStack Kubernetes cluster.

1. Infrastructure setup

Dask cluster setup.

Check here.

Prometheus monitoring setup

Check here for the Promethus stand-alone or here for the Promethus stack (Graphana included).

2. Custom Dask Docker images

The Dask base image and the dask-notebook image were modified to be able to run the "SCENIC" protocol. The corresponding Dockerfiles are accessible here.

3. Example notebooks

To test the execution of pySCENIC, the mouse brain dataset and the PBMC dataset were used. The custom notebooks adapted from the ones that were published in the "SCENIC" protocol repository can be found here and here.