pangeo-data/pangeo-stacks

Noisy Dask JupyterLab Warnings

scottyhq opened this issue · 0 comments

New images with Dask>2.5 have lots of warning messages being displayed in JupyterLab. We attempted to fix this with #87, but it still seems to be an issue with the latest image (2019-11-14):

from dask_kubernetes import KubeCluster
cluster = KubeCluster()
cluster.scale(2);
cluster

Results in:

distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at: tcp://192.168.140.221:42003
distributed.scheduler - INFO -   dashboard at:                     :8787
cluster.adapt(minimum=2, maximum=10);

Results in an infinite output loop with these messages:

distributed.scheduler - INFO - Suggest closing workers: ['tcp://192.168.125.103:46797', 'tcp://192.168.103.24:34165']

@TomAugspurger and @jacobtomlinson - seems like these messages should not be appearing by default. Are we enabling them somehow in our dask configuration? https://github.com/pangeo-data/pangeo-stacks/blob/master/base-notebook/binder/dask_config.yaml