"Too many positional arguments" in pipeline when line_width>0
jobh opened this issue · 5 comments
ALL software version info
datashader version 0.14.4
Description of expected behavior and the observed behavior
When setting non-zero line_width on LineAxis0
glyphs (passed to Pipeline
), an error is raised. With set_line_width(0)
, or not calling set_line_width
at all, it works fine.
In earlier versions (0.14.2 I think) line_width>0
did not produce an error. However, it did not work correctly either (line width was respected but constant color/not shaded), which was why I tried to upgrade.
Complete, minimal, self-contained example code that reproduces the issue
import datashader as ds
import pandas as pd
df = pd.DataFrame(dict(x=[0,1],y=[0,1]))
glyph = ds.glyphs.LineAxis0('x','y')
glyph.set_line_width(1)
pipeline = ds.Pipeline(df, glyph)
pipeline(x_range=(0,1), y_range=(0,1), width=400, height=400)
Stack traceback and/or browser JavaScript console output
The stack is shown below, but the relevant bit seems to be:
too many positional arguments
During: resolving callee type: type(CPUDispatcher(<function append at 0x7f82b5fc9ab0>))
During: typing of call at <_full_antialias> (151)
Full trace:
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
Input In [5], in <cell line: 12>()
8 dfs = pd.DataFrame(dict(x=[0,1],y=[0,1]))
10 traj_pipeline = ds.Pipeline(dfs, glyph)#, color_fn=lambda agg: tf.shade(agg, cmap=cc.fire, how=cnorm, span=span))
---> 12 traj_pipeline(x_range=(0,1), y_range=(0,1), width=400, height=400)
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/pipeline.py:69, in Pipeline.__call__(self, x_range, y_range, width, height)
56 """Compute an image from the specified pipeline.
57
58 Parameters
(...)
64 The shape of the image
65 """
66 canvas = core.Canvas(plot_width=int(width*self.width_scale),
67 plot_height=int(height*self.height_scale),
68 x_range=x_range, y_range=y_range)
---> 69 bins = core.bypixel(self.df, canvas, self.glyph, self.agg)
70 img = self.color_fn(self.transform_fn(bins))
71 return self.spread_fn(img)
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/core.py:1260, in bypixel(source, canvas, glyph, agg, antialias)
1258 with np.warnings.catch_warnings():
1259 np.warnings.filterwarnings('ignore', r'All-NaN (slice|axis) encountered')
-> 1260 return bypixel.pipeline(source, schema, canvas, glyph, agg, antialias=antialias)
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/utils.py:109, in Dispatcher.__call__(self, head, *rest, **kwargs)
107 typ = type(head)
108 if typ in lk:
--> 109 return lk[typ](head, *rest, **kwargs)
110 for cls in getmro(typ)[1:]:
111 if cls in lk:
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/data_libraries/pandas.py:17, in pandas_pipeline(df, schema, canvas, glyph, summary, antialias)
15 @bypixel.pipeline.register(pd.DataFrame)
16 def pandas_pipeline(df, schema, canvas, glyph, summary, *, antialias=False):
---> 17 return glyph_dispatch(glyph, df, schema, canvas, summary, antialias=antialias)
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/utils.py:112, in Dispatcher.__call__(self, head, *rest, **kwargs)
110 for cls in getmro(typ)[1:]:
111 if cls in lk:
--> 112 return lk[cls](head, *rest, **kwargs)
113 raise TypeError("No dispatch for {0} type".format(typ))
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/data_libraries/pandas.py:48, in default(glyph, source, schema, canvas, summary, antialias, cuda)
44 y_axis = canvas.y_axis.compute_index(y_st, height)
46 bases = create((height, width))
---> 48 extend(bases, source, x_st + y_st, x_range + y_range)
50 return finalize(bases,
51 cuda=cuda,
52 coords=OrderedDict([(glyph.x_label, x_axis),
53 (glyph.y_label, y_axis)]),
54 dims=[glyph.y_label, glyph.x_label])
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/datashader/glyphs/line.py:110, in LineAxis0._internal_build_extend.<locals>.extend(aggs, df, vt, bounds, plot_start)
107 do_extend = extend_cpu
109 # line may be clipped, then mapped to pixels
--> 110 do_extend(
111 sx, tx, sy, ty, xmin, xmax, ymin, ymax,
112 xs, ys, plot_start, antialias_stage_2, *aggs_and_cols
113 )
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/numba/core/dispatcher.py:468, in _DispatcherBase._compile_for_args(self, *args, **kws)
464 msg = (f"{str(e).rstrip()} \n\nThis error may have been caused "
465 f"by the following argument(s):\n{args_str}\n")
466 e.patch_message(msg)
--> 468 error_rewrite(e, 'typing')
469 except errors.UnsupportedError as e:
470 # Something unsupported is present in the user code, add help info
471 error_rewrite(e, 'unsupported_error')
File ~/mambaforge/envs/transact/lib/python3.10/site-packages/numba/core/dispatcher.py:409, in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
407 raise e
408 else:
--> 409 raise e.with_traceback(None)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Internal error at <numba.core.typeinfer.CallConstraint object at 0x7f82b56daa10>.
too many positional arguments
During: resolving callee type: type(CPUDispatcher(<function append at 0x7f82b5fc9ab0>))
During: typing of call at <_full_antialias> (151)
Enable logging at debug level for details.
File "<_full_antialias>", line 151:
<source missing, REPL/exec in use?>
During: resolving callee type: type(CPUDispatcher(<function _full_antialias at 0x7f82b5ff9ab0>))
During: typing of call at <draw_segment> (47)
During: resolving callee type: type(CPUDispatcher(<function _full_antialias at 0x7f82b5ff9ab0>))
During: typing of call at <draw_segment> (47)
File "<draw_segment>", line 47:
<source missing, REPL/exec in use?>
During: resolving callee type: type(CPUDispatcher(<function draw_segment at 0x7f82b5ff9cf0>))
During: typing of call at <perform_extend_line> (21)
During: resolving callee type: type(CPUDispatcher(<function draw_segment at 0x7f82b5ff9cf0>))
During: typing of call at <perform_extend_line> (21)
File "<perform_extend_line>", line 21:
<source missing, REPL/exec in use?>
During: resolving callee type: type(CPUDispatcher(<function perform_extend_line at 0x7f82b5ff9e10>))
During: typing of call at <extend_cpu> (10)
During: resolving callee type: type(CPUDispatcher(<function perform_extend_line at 0x7f82b5ff9e10>))
During: typing of call at <extend_cpu> (10)
File "<extend_cpu>", line 10:
<source missing, REPL/exec in use?>
Thanks for the report @jobh.
Antialiasing doesn't currently work using the Pipeline
approach as there is extra antialiasing information that doesn't get through to the low-level calculation functions this way. It can be fixed though.
Is there a particular reason that you are using the Pipeline
approach? This equivalent code using Canvas.line
works fine:
import datashader as ds
import pandas as pd
df = pd.DataFrame(dict(x=[0,1],y=[0,1]))
cvs = ds.Canvas(plot_width=400, plot_height=400)
agg = cvs.line(source=df, x='x', y='y', line_width=1)
im = ds.transfer_functions.shade(agg)
Thanks @ianthomas23.
What I have is basically a dynamic map-tile server using datashader. The pipeline API is "just right" for this: Data is parsed once, up-front, while images are generated on-the fly based on viewport (location+size).
The Canvas
approach — I'm not too familiar with it, but specifying the size (and maybe location?) before adding data is probably not performant enough.
Edit: ...but actually timing it on [1k lines of 10k points each] indicates I'm wrong about that... don't know if that is expected.
Btw; I'm not too worried about the anti-aliasing, only the line width. If that makes a difference.
The Pipeline
approach is really just a thin wrapper around the Canvas
approach:
datashader/datashader/pipeline.py
Lines 66 to 71 in f7de271
You might be saving some time by using a specific
Glyph
object rather than letting Canvas.line
work it out, but this should negligible compared to the computation time.
Nowadays a line_width
other than zero implies antialiasing. If you are happy without antialiasing then you can leave line_width
at its default of 0
and it will definitely run faster.
Regardless, this bug is quite simple to fix so it should be in the next release, possibly in the next couple of weeks.
I will probably switch to Canvas
as you suggest, but thanks for fixing anyway :-)