ipywidgets-runner
Supercharge your ipywidgets dashboards and apps in Jupyter
Ipywidgets is an excellent and popular (~4M downloads/per month from PyPI) dashboarding tool for data visualization in Jupyter.
However, in typical setups the whole dashboard gets recalculated with every change in the configuration: even if you're changing a small parameter in the dashboard (such as the date range), the entire data analysis script runs again, greatly slowing down data exploration.
Ipywidgets-runner alleviates this issue by providing a framework for attaching ipywidgets widgets to different stages in your data analysis.
Then, when a user changes a parameter, ipywidgets-runner intelligently re-executes those parts of your data analysis that actually need re-executing.
And what's better, if the dashboard has seen that parameter before, it pulls the output from a cache rather than recalculating anything. So flip-flopping between two months in your timeseries analysis? Instant.
Using ipywidgets-runner
ipywidgets-runner was designed to be easily integrated into existing Jupyter notebooks.
You use it by specifying a directed acyclic graph, shown as the middle layer in this diagram:
An Example
It's very easy to integrate ipywidgets-runner into your existing data analysis.
-
Data analysis typically takes the form of one big function f which maps parameters to an collection of outputs, such as tables, charts, maps, or images. The first step is to split this function f into a collection of smaller, interdependent functions.
-
Express the interdependence of these functions by linking them up in ipywidgets-runner nodes.
import ipywidgets as w
import ipywidgets_runner as wr
first_name_widget = w.Text()
last_name_widget = w.Text()
age_widget = w.Date()
output_widget = w.Output()
full_name_node = wr.Node(
input=[first_name_widget, last_name_widget],
f=lambda x, y: x+y
)
name_and_time_node = wr.Node(
input=[full_name_node, age_widget],
f=lambda x, y: x + str(datetime.now()-y)
)
output_node = wr.OutNode(
input=[name_and_time_node],
f=lambda x: plot(x),
out=output_widget
)
container_widget = w.VBox([first_name_widget, last_name_widget, age_widget, output_widget])
wr.start(container_widget)Complete:
- DAG-based runtime core
- Debug mode
- Worker threading
In progress:
- Styling of widgets to show stale vs fresh outputs
- Worker multiprocessing (optionally)
- Config # of worker threads
- Allow for better diffing than just "=="
Future Work and Design Considerations:
- Automatically construct DAG via static analysis
- Disk cache previous computations
- Allow for subsetting: (e.g., if the new time range is a subinterval of the old one, maybe you don't need to hit the SQL data store again)