Speaker: David Grossman (CG/UM)
Title: Parallel Computing in Python with Dask on Pegasus
Powerpoint Presentation: You can find the PowerPoint presentation on Google Slides here and a PDF of the presentation here.
Abstract: You’ve built THE program that will change how we study volcanic prediction in the northern tip of Isabela Island. There’s only one problem: it takes 10 hours to run one dataset... and you have 750 datasets to analyze. So you just parallelize the algorithm. Easy, right?
In this talk, we will explore the Dask parallelization framework with the goal of writing parallel algorithms on Pegasus. We will cover three main topics:
- Basic parallelization with Dask
- How to configure dask.distributed on Pegasus
- Tricks and tips for speeding up your parallel algorithm.
Please have a version of Anaconda installed and take a look at Tim Norris’s Introduction to Dask.