dask/distributed

Deserialization warning with Bokeh 2.1

Closed this issue · 18 comments

mrocklin commented

The dashboard broke under 2.1. It has been resolved in #3904

However, we're still getting a deserialization error like the following:

 message: Message 'PATCH-DOC' content: {'references': [], 'events': [{'kind': 'ModelChanged', 'model': {'id': '1683'}, 'attr': 'text_align', 'new': {'value': 'right'}}, {'kind': 'ModelChanged', 'model': {'id': '1684'}, 'attr': 'text_align', 'new': {'value': 'right'}}, {'kind': 'ModelChanged', 'model': {'id': '1628'}, 'attr': 'reset_start', 'new': -8}, {'kind': 'ModelChanged', 'model': {'id': '1628'}, 'attr': 'reset_end', 'new': 0}]} 
 error: DeserializationError("Enum('left', 'right', 'center') expected str, got {'value': 'right'} of type dict")
Traceback (most recent call last):
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/server/protocol_handler.py", line 90, in handle
    work = await handler(message, connection)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/server/session.py", line 67, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/server/session.py", line 261, in _handle_patch
    message.apply_to_document(self.document, self)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 100, in apply_to_document
    doc._with_self_as_curdoc(lambda: doc.apply_json_patch(self.content, setter))
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/document/document.py", line 1150, in _with_self_as_curdoc
    return f()
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 100, in <lambda>
    doc._with_self_as_curdoc(lambda: doc.apply_json_patch(self.content, setter))
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/document/document.py", line 411, in apply_json_patch
    patched_obj.set_from_json(attr, value, models=references, setter=setter)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/core/has_props.py", line 341, in set_from_json
    descriptor.set_from_json(self, json, models, setter)
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/core/property/descriptors.py", line 611, in set_from_json
    self.property.from_json(json, models),
  File "/home/mrocklin/miniconda/envs/test-3714/lib/python3.8/site-packages/bokeh/core/property/bases.py", line 463, in from_json
    raise DeserializationError("%s expected %s, got %s of type %s" % (self, expected, json, type(json).__name__))
bokeh.core.property.bases.DeserializationError: Enum('left', 'right', 'center') expected str, got {'value': 'right'} of type dict

This comes about from the TaskProgress plot available here:

distributed/distributed/dashboard/components/scheduler.py

Lines 1488 to 1662 in 1d7640b

class TaskProgress(DashboardComponent):
""" Progress bars per task type """
def __init__(self, scheduler, **kwargs):
self.scheduler = scheduler
data = progress_quads(
dict(all={}, memory={}, erred={}, released={}, processing={})
)
self.source = ColumnDataSource(data=data)
x_range = DataRange1d(range_padding=0)
y_range = Range1d(-8, 0)
self.root = figure(
id="bk-task-progress-plot",
title="Progress",
name="task_progress",
x_range=x_range,
y_range=y_range,
toolbar_location=None,
tools="",
**kwargs,
)
self.root.line( # just to define early ranges
x=[0, 0.9], y=[-1, 0], line_color="#FFFFFF", alpha=0.0
)
self.root.quad(
source=self.source,
top="top",
bottom="bottom",
left="left",
right="right",
fill_color="#aaaaaa",
line_color="#aaaaaa",
fill_alpha=0.1,
line_alpha=0.3,
)
self.root.quad(
source=self.source,
top="top",
bottom="bottom",
left="left",
right="released-loc",
fill_color="color",
line_color="color",
fill_alpha=0.6,
)
self.root.quad(
source=self.source,
top="top",
bottom="bottom",
left="released-loc",
right="memory-loc",
fill_color="color",
line_color="color",
fill_alpha=1.0,
)
self.root.quad(
source=self.source,
top="top",
bottom="bottom",
left="memory-loc",
right="erred-loc",
fill_color="black",
fill_alpha=0.5,
line_alpha=0,
)
self.root.quad(
source=self.source,
top="top",
bottom="bottom",
left="erred-loc",
right="processing-loc",
fill_color="gray",
fill_alpha=0.35,
line_alpha=0,
)
self.root.text(
source=self.source,
text="show-name",
y="bottom",
x="left",
x_offset=5,
text_font_size=value("10pt"),
)
self.root.text(
source=self.source,
text="done",
y="bottom",
x="right",
x_offset=-5,
text_align="right",
text_font_size=value("10pt"),
)
self.root.ygrid.visible = False
self.root.yaxis.minor_tick_line_alpha = 0
self.root.yaxis.visible = False
self.root.xgrid.visible = False
self.root.xaxis.minor_tick_line_alpha = 0
self.root.xaxis.visible = False
hover = HoverTool(
point_policy="follow_mouse",
tooltips="""
<div>
<span style="font-size: 14px; font-weight: bold;">Name:</span>&nbsp;
<span style="font-size: 10px; font-family: Monaco, monospace;">@name</span>
</div>
<div>
<span style="font-size: 14px; font-weight: bold;">All:</span>&nbsp;
<span style="font-size: 10px; font-family: Monaco, monospace;">@all</span>
</div>
<div>
<span style="font-size: 14px; font-weight: bold;">Memory:</span>&nbsp;
<span style="font-size: 10px; font-family: Monaco, monospace;">@memory</span>
</div>
<div>
<span style="font-size: 14px; font-weight: bold;">Erred:</span>&nbsp;
<span style="font-size: 10px; font-family: Monaco, monospace;">@erred</span>
</div>
<div>
<span style="font-size: 14px; font-weight: bold;">Ready:</span>&nbsp;
<span style="font-size: 10px; font-family: Monaco, monospace;">@processing</span>
</div>
""",
)
self.root.add_tools(hover)
@without_property_validation
def update(self):
with log_errors():
state = {
"memory": {},
"erred": {},
"released": {},
"processing": {},
"waiting": {},
}
for tp in self.scheduler.task_prefixes.values():
active_states = tp.active_states
if any(active_states.get(s) for s in state.keys()):
state["memory"][tp.name] = active_states["memory"]
state["erred"][tp.name] = active_states["erred"]
state["released"][tp.name] = active_states["released"]
state["processing"][tp.name] = active_states["processing"]
state["waiting"][tp.name] = active_states["waiting"]
state["all"] = {
k: sum(v[k] for v in state.values()) for k in state["memory"]
}
if not state["all"] and not len(self.source.data["all"]):
return
d = progress_quads(state)
update(self.source, d)
totals = {
k: sum(state[k].values())
for k in ["all", "memory", "erred", "released", "waiting"]
}
totals["processing"] = totals["all"] - sum(
v for k, v in totals.items() if k != "all"
)
self.root.title.text = (
"Progress -- total: %(all)s, "
"in-memory: %(memory)s, processing: %(processing)s, "
"waiting: %(waiting)s, "
"erred: %(erred)s" % totals
)

cc @bryevdv @jsignell

jsignell commented

It looks to me that if you remove the text_align = "right" line you will be good, but I don't know if that is desirable.

mrocklin commented

Is there a new way to align text?

bryevdv commented
mrocklin commented

OK great. It sounds like the thing for us to do is to sit tight on the Dask side. Correct?

bryevdv commented

Yes seems we broke a few eggs with this release unfortunately.

mrocklin commented

It's ok, you all make a mean omelet.

  • 😄3
bryevdv commented

@mrocklin @jsignell Bokeh 2.1.1 is available on PyPI and the bokeh channel on anaconda.org (presumably soon on C-F also) It should resolve this issue (but please test in your actual usage first!)

Tangentially: how can we help augment the downstream tests? I plan to raise the visibility of the adivsory downstream tests soon, but that said, they were all still passing even with both issues 2.1 introduced

  • 👍2
jsignell commented

Would it make sense for dask to mark some tests as visualization tests and then bokeh could run them on some schedule? Probably released dask + latest bokeh is good enough for testing right?

bryevdv commented

@jsignell That's what we do already:

https://github.com/bokeh/bokeh/runs/798587187?check_suite_focus=true

But evidently not enough already, since it did not catch either of the issues with the 2.1 release.

jsignell commented

Yes sorry should have circled back after the dask dev meeting yesterday. I am going to look over the tests in dask and see if there are any holes in the testing that would be easy to fix.

  • 👍1
jakirkham commented

Did we wind up resolving this or is this still an issue?

bryevdv commented

@jakirkham AFAIK Bokeh 2.1.1 resolved the immediate problem on the Bokeh side, I am not sure if version 2.1 can/was specifically excluded from requirements on the Dask side. Another option would be to set Bokeh 2.1.1 a a new min version which I would certainly endorse 😄 (it is ~18 months old at this point which seem like a lot in dask-time)

  • 👍1
jakirkham commented

Guessing we would want to update this then

@jrbourbeau feel free to point any other spots I've missed :)

bryevdv commented

RIght, there's at least that and some docs spots IIRC, and there may be some old version-detection code that could be chucked out. If there's agreement to set 2.1.1 as a new minimum, I am happy to take a comprehensive look and submit a PR.

jsignell commented

Yeah I'd be happy to have the min be 2.1.1

jrbourbeau commented

Yeah, if bumping the minimum supported version to 2.1.1. will cut down on user pain or maintainer burden, then let's do it

jakirkham commented

Thanks Bryan! 😄

  • 👍1