/jupyterlab_spark

Spark Application UI extension for JupyterLab

Primary LanguageTypeScriptApache License 2.0Apache-2.0

jupyterlab_spark

npm version Build status

A JupyterLab extension to show Spark Application UI in a JupyterLab panel

Prerequisites

  • JupyterLab

Installation

jupyter labextension install jupyterlab_spark

Usage

Note: the extension only works with Spark application running in client mode.

  1. Add the following configuration to spark-defaults.conf to allow framing from localhost
spark.ui.allowFramingFrom    http://localhost
  1. Click on the "Application UI" item from the new "Spark" menu

spark_menu

  1. Input the Spark application id in the popped up dialog and click on "CREATE"

input_app_id

The id can be found in driver log or output of executing sparkContext.applicationId

  1. Here is your Spark application UI. Enjoy!

spark_app_ui

Development

For a development install (requires npm version 4 or later), do the following in the repository directory:

npm install
npm run build
jupyter labextension link .

To rebuild the package and the JupyterLab app:

npm run build
jupyter lab build

Acknowledgement

Thanks to the xkcd exntension tutorial and cookiecutter template. I'm especially inspired by the jupyterlab_tensorboard project where I also shamelessly borrowed some codes.