/gtfs_onboarding.nix

A Jupyter Notebook workspace for exploring GTFS data via SQLite3 and optionally Python. Configured and provided via Nix.

Primary LanguageJupyter Notebook

gtfs_onboarding

A Jupyter Notebook workspace for exploring GTFS data via SQLite3 and optionally Python. Configured and provided via Nix.

Table of Contents

  1. Getting Started
    1. Requirements
    2. (Very) Quick Start
    3. Local Quick Start with persistance (save and resume)
    4. Opening the notebook
  2. Notes
    1. Running specific kernels
      1. Running the SQLite Kernel without Python and the Python Kernel
      2. Running the Python Kernel without the SQLite3 Kernel
    2. Loading different GTFS feeds
    3. Firefox Slowness

Getting Started

Requirements

  • a Nix installation 1
    • must have the following experimental-features2
      • flakes
      • nix-command

(Very) Quick Start

Warning

Using this method, you cannot save your progress.

Use the Local Quick Start method if you wish to save your progress.

This method is great for quickly getting the notebook up and running with all data populated, but not great for learners who wish to save their progress.

This is because the notebook will be downloaded to the nix store and will be read-only.

Using Nix, we can start JupyterLab and load the notebook with a single command.

: nix run github:firestack/gtfs_onboarding.nix#interactive-read-only

This command does the following:

  1. Runs the nix flake app named interactive-read-only which
    1. downloads the flake repository into the nix store then:
      • builds the jupyter lab and kernels needed.
      • downloads the associated GTFS data and:
        • creates a SQLite3 database using the downloaded GTFS data.
    2. starts jupyter-lab in a directory containing:
      • the notebooks and associated data.
      • the GTFS SQLite database.
  2. next, follow the instructions in the section: opening the notebook.

Local Quick Start with persistance (save and resume)

  1. Clone this repository 
    : git clone https://github.com/firestack/gtfs_onboarding.nix.git
  2. Enter the directory 
    : cd ./gtfs_onboarding.nix

Nix Flakes

    1. Run via nix flakes 
      : nix run

Direnv

    1. load the necessary dependencies into your shell using direnv

      : direnv allow

    2. Then run the jupyter-lab command to start the notebook server.

      : jupyter-lab

Opening the notebook

Important

(Prerequisite) Start Jupyter Notebook

  1. In the terminal output, find and click on link with authentication token

Tip

afterwards, you can just navigate to localhost:8888 in that same browser.

Here's an example of the links to look out for in the terminal window.

To access the server, open this file in a browser:
	file:///.jupyter/runtime/jpserver-00000-open.html
Or copy and paste one of these URLs:
	http://localhost:8888/lab?token=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
	http://127.0.0.1:8888/lab?token=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
  1. Open file GTFS Onboarding.sqlite3.ipynb
    1. For a Python notebook, you can also open GTFS Onboarding.python.ipynb
  2. Enjoy!

Notes

Running specific kernels

Running the SQLite Kernel without Python and the Python Kernel

Runs the notebook with only the SQLite kernel loaded. This will skip downloading the python dependencies and building the Python environment.

: nix run .#lab-sqlite

Running the Python Kernel without the SQLite3 Kernel

Runs the notebook with only the Python kernel loaded. This will download the Python dependencies and configure the Python environment and will not download or build the SQLite kernel.

: nix run .#lab-python

Loading different GTFS feeds

The gtfs.nix file contains content addressed2 definitions for other GTFS feeds which can be loaded. These can be loaded into the environment using the Local start method and building the relevant GTFS database.

For example, the command

: nix build -o mbta-20201002.db .#mbta-gtfs-20201002
# OR
: nix build -o mbta-20201002.db github:firestack/gtfs_onboarding.nix#mbta-gtfs-20201002

will download the archived GTFS feed and build a SQLite3 database, and then link it to a file in your working directory named mbta-20201002.db.

Then, in the notebook, you'll need to change the name of the database passed to the open command.

Alternatively, you can build the database and overwrite the feed.db link:

: nix build -o feed.db .#mbta-gtfs-20201002

Firefox Slowness

JupyterLab seems to have difficulty running on Firefox, with symptoms of being extremely slow and unresponsive. Using Safari or Chrome have been a stable alternative for running the notebooks2.

Footnotes

  1. Nix installer which enables flakes and nix-command by default ↩

  2. At time of writing. ↩ ↩2 ↩3