/enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

Primary LanguagePythonOtherNOASSERTION

Technical Overview | Features | Installation | System Architecture | Contributing |

Jupyter Enterprise Gateway

PyPI version Build Status Documentation Status Coverage Status Google Group

Jupyter Enterprise Gateway enables Jupyter Notebook to launch remote kernels in a distributed cluster, including Apache Spark managed by YARN, IBM Spectrum Conductor, Kubernetes or Docker Swarm.

It provides out of the box support for the following kernels:

  • Python using IPython kernel
  • R using IRkernel
  • Scala using Apache Toree kernel

Full Documentation for Jupyter Enterprise Gateway can be found here

Jupyter Enterprise Gateway does not manage multiple Jupyter Notebook deployments, for that you should look for JupyterHub.

Technical Overview

Jupyter Enterprise Gateway is a web server that provides headless access to Jupyter kernels within an enterprise. Built directly upon Jupyter Kernel Gateway, Jupyter Enterprise Gateway leverages all of the Kernel Gateway functionality in addition to the following:

  • Adds support for remote kernels hosted throughout the enterprise where kernels can be launched in the following ways:
    • Local to the Enterprise Gateway server (today's Kernel Gateway behavior)
    • On specific nodes of the cluster utilizing a round-robin algorithm
    • On nodes identified by an associated resource manager
  • Provides support Apache Spark managed by YARN, IBM Spectrum Conductor, Kubernetes or Docker Swarm out of the box.
    Others can be configured via Enterprise Gateway's extensible framework.
  • Secure communication from the client, through the Enterprise Gateway server, to the kernels
  • Multi-tenant capabilities
  • Ability to associate profiles consisting of configuration settings to a kernel for a given user
  • Persistent kernel sessions

Deployment Diagram

Features

See Enterprise Gateway Features in the documentation for a list of Jupyter Enterprise Gateway features.

Installation

Detailed installation instructions are located in the Getting Started page of the project docs. Here's a quick start using pip:

# install from pypi
pip install --upgrade jupyter_enterprise_gateway

# show all config options
jupyter enterprisegateway --help-all

# run it with default options
jupyter enterprisegateway

Please check the Configuration Options page for information about the supported options.

System Architecture

The System Architecture page includes information about Enterprise Gateway's remote kernel, process proxy, and launcher frameworks.

Contributing

The Contribution page includes information about how to contribute to Enterprise Gateway along with our roadmap. While there, you'll want to set up a development environment and check out typical developer tasks.