/mbtaccess

Web app for public transit wheelchair access

Primary LanguagePythonMIT LicenseMIT

README

Grow with Google - New England collaborative project

license

Travis (.org)

Table of Contents

Description

This app integrates the Google Maps and MBTA (Massachusetts Bay Transportation Authority) APIs to provide wheelchair accessibility data for MBTA stops.

MBTA recently released their MBTA V3 API that provides public transportation data in JSON API format. One of the under-utilized datasets in their API is the wheelchair accessibility of the stops. Google Maps recently started providing wheelchair accessibility info, but the implementation is not particularly extensive.

We aim to create a web app that quickly and conveniently identifies wheelchair accessible stops near the user.

There are two builds available, a stable build at mbtaccess.org and a canary build at latest.mbtaccess.org.

Contributing

Application stack

Flask

Flask summary

  • The application fetches data from the MBTA V3 API.
  • The application originally fetched JSON data directly from the MBTA API, and then used client-side JavaScript to iterate over the stops and drop map pins for stops near the user.
  • After experiencing slow speeds, we rebuilt our app with Python 3 and the Python framework Flask. Instead of fetching data directly from MBTA, we store data in our own back-end database API. We then perform the calculations with Python and structure the application with Flask. Python code has been formatted according to the PEP 8 specification, with line length extended to 100 characters.
  • utilities.py contains distance calculation functions. The get_distance function uses the Haversine formula to calculate the distance between two points, such as between the user's location and an MBTA stop.
  • models.py sets up the data model for the MBTA stop data.
  • migrate.py creates a database, fetches data from the MBTA API, and stores it in the database.
  • app.py contains the main Flask application code. This file controls the app, with Flask routing functions to render the pages of the web application and access app content. The database is accessed by SQLAlchemy from within app.py.
  • App pages are templated and stored in the templates directory. Code common to all application pages is stored in {% include %}, and code unique to each page is stored in {% block %} content.

Running Flask

The Flask application should run within a virtual environment. Python 3 is bundled with the venv module for creation of virtual environments.

  1. Install and activate Python virtual environment: The shell commands in the code block below will create a Python virtual environment, activate the virtual environment and display a modified virtual environment prompt

  2. Install required Python modules into the virtual environment: Use pip to install required modules listed in requirements.txt. The modules will be installed locally within the virtual environment. The requirements.txt file was generated by running pip freeze > requirements.txt.

  3. Set up database: Run python migrate.py from within the virtual environment. This needs to be run only once to populate the mbta_stops table in the SQLite database.

  4. Start application: Run python app.py or python -m flask run from within the virtual environment. Flask will start a local web server.

    cd mbtaccess
# 1. install and activate virtual env
python3 -m venv venv
. venv/bin/activate
# 2. install modules
(venv) <path> $ pip install -r requirements.txt
# 3. set up database once
(venv) <path> $ python migrate.py
# 4. run app
(venv) <path> $ python app.py

    • Using Flask in VS Code:

      • Command Palette -> Python: Select Interpreter. Select virtual environment.
      • Command Palette -> Python: Create Terminal. Creates a terminal and automatically activates the virtual environment. Using the Python: Create Terminal command is preferred. Simply creating a new terminal window will not activate the virtual environment.

    • Flask, by default, will bind the application to port 5000. If you wish to change this, you can export an APP_PORT environment variable and call the script:

      export APP_PORT=7000 && python app.py

  5. Test the back-end API locally: In a web browser, navigate to http://127.0.0.1:5000/stops?lat=42.35947&lon=-71.09296 to see the data returned from the API. If you're adventurous, change the values for lat and lon in the query string to see new results.

    • The API endpoints have not yet been hosted anywhere. The only way to test them is to run the application locally.
    • The goal is to deploy the API endpoints to Google Cloud.

  6. Test the app pages locally: Navigate to the map page and the universities page

Testing

  • We use Python's unittest module for unit testing.

  • Python unit testing in VS Code:

    • Use unittest settings in .vscode/settings.json
    • Command Palette -> Run All Unit Tests. The result will show up in the status bar.
    • Command Palette -> View Test Results.
    • View results in the output panel (Cmd+shift+u).
    • Individual tests can also be run. A popup will show above each test unit in the test module file.

  • Unit testing Python from the command line:

    # activate virtual env
. venv/bin/activate
# Discover and run tests
(venv) <path> $ python -W ignore -m unittest discover -vv -s tests -p "*test.py"
    • This calls python with the following twists:
      1. -W ignore: Tells the python interpreter to ignore warnings since those are written to the standard error and can act like errors, when they're usually just deprecation warnings.
      2. -m unittest: Tells the python interpreter to import the unittest module
      3. discover -vv -s tests -p "*test.py": Tells the unittest module to discover and run test cases (with a verbosity level of 2) located within the tests directory inside python files with the pattern "*test.py"

  • Tests are run automatically on the GitHub repo by Continuous Integration (CI).

  • We use Travis CI. Build information is available on the MBTAccess Travis CI page.

Debugging

Two VSCode debug configurations are provided: one for debugging the database migration, and one for running the Flask app.

Front-end

Styling

Search

Geolocation
  • Users can provide their location, and the app will locate stops around them. Geolocation comes from the W3C geolocation standard, not from Google Maps.
  • Successful geolocation calls the showStops(location) to return results.
  • Users who deny the geolocation prompt will see a dismissable alert.
Google Places
  • Users can search the Google Places database using the input field above the map. Google Places Autocomplete will return results as users type.
  • Selection of an autocomplete item clears previous map markers, and calls the showStops(location) function to return results.

Docker

App container

The application is assembled into a Docker container.

  • An image is the executable set of files used by Docker.
  • A container is a running image.
  • The Dockerfile tells Docker how to build the container.
  • Visual Studio Code has built-in Docker features. See Working with Docker in VS Code.

To build and run the Docker application container locally:

  1. Clone, or fork and clone, the GitHub repository to your machine.

  2. Install Docker Desktop on your machine.

  3. Build the image and run the container.

    cd mbtaccess
docker build -t web .
docker run -d -p 80:80 mbtaccess:latest

    -t web tells Docker to name the image web. A tag can be specified with mbtaccess:tag, otherwise, the tag latest will be used. Adding . builds from the current directory.

    -d runs the container in detached mode. Docker will display the container SHA and return the terminal prompt. -p 80:80 maps the http port 80 from your local machine to port 80 on the container.

    Other useful commands:

    docker container ls
docker container stop <SHA or container name>
docker container rm <SHA or container name>
docker image ls
docker image rm <SHA or container name>

  4. Browse to http://localhost:80 to see the app running in the Docker container.

Docker Compose

Docker Compose can create multi-container applications. We don't currently use Docker Compose in production, but the docker-compose.yml file is included for local testing.

To build and run with Docker Compose:

cd mbtaccess
docker-compose build
docker-compose up -d

Deployment

  • The app is deployed with Google Cloud Platform (GCP).
  • Cloud Build reads the cloudbuild.yml files and builds a Docker container from the GitHub repository each time a commit is pushed.
  • Containers built with Cloud Build are stored in the GCP Container Registry (similar to Docker Cloud/Docker Store).
  • Google Cloud Platform has three options for Python apps.
  • We first tried using Kubernetes Engine. Kubernetes (Greek for 'helmsman') is an open-source container management system.
    • The Docker container for our app is considered a pod.
    • The pod is deployed as a workload on Kubernetes Engine. Horizontal pod autoscaling adjusts the number of pod replicas based on traffic.
    • The deployment is configured for rolling updates and imagePullPolicy: Always.
    • Cloud Build triggers automatic updates to the Kubernetes deployment with an annotation patch in the cloudbuild.yml files. We use the $REVISION_ID default variable substitution, but a timestamp annotation can also be used.
  • The Kubernetes setup was about $100 month, so we switched to the "free tier" base-level single VM (which is actually not free).

Networking

  • The domain name mbtaccess.org was purchased through Hover.
  • The domain name directs web traffic to Cloudflare nameservers for security.
  • Cloudflare sends traffic to GCP with DNS A records that point to a static IP address. When we were using Kubernetes, the static IP address passed network requests to the Kubernetes cluster with a Kubernetes ingress and load balancer service.
  • The Google Maps JavaScript API requires the API key to be embedded in the source code. The API key only accepts network requests from the Geocoding, Maps and Places APIs and the following HTTP referrers:
    • http://localhost/*
    • https://growwithgooglema.github.io/mbtaccess/*
    • https://mbtaccess.org/*
    • https://*.mbtaccess.org/*

Repository contents

  • data/: JSON data for the list of universities.
  • kubernetes/: Kubernetes deployment configuration files.
  • scripts/
  • static/: CSS, image, and JavaScript files for the application.
    • css/
      • custom.css: Custom CSS for the app, in addition to Bootstrap.
    • img/
    • js/
      • about
      • date: Footer date calculation
      • google-maps: JavaScript for map page. Sends a location query to the database, and displays results on the page.
      • sw-install: Installation script for the Service Worker script sw.js.
      • universities: JavaScript for universities page. Shows universities on map, with results in table below.
  • templates/: Webpage HTML. Templates are assembled by Flask.
  • tests/: Application tests.
  • .dockerignore: Instructions to Docker to exclude certain files from commits.
  • .gitignore: Instructions to Git to exclude certain files from commits.
  • .mailmap: Email configuration.
  • .travis.yml: Travis CI build configuration.
  • app.py: Python Flask application module. This is the main file that runs the app, templates the pages, and coordinates the app's functions.
  • CODE_OF_CONDUCT.md: Guidelines for conduct.
  • CONTRIBUTING.md: Instructions for contributing to the repository.
  • docker-compose-debug.yml and docker-compose.yml: Instructions for coordinated builds of multi-container applications.
  • Dockerfile: Instructions for Docker container builds.
  • LICENSE: This file describes how the repository can be used by others. We have provided the repository under the MIT license, a permissive and widely-used license. See the choose a license page for more info on licenses.
  • migrate.py: Python script that fetches data from the MBTA API and stores it in the stops database.
  • models.py: Python script that sets up the MBTA stops database model.
  • README.md: This file, a concise description of the repository.
  • stops.sqlite: Database of MBTA stops.
  • sw.js: This is the Service Worker file. Service Worker is a JavaScript file that sits between the browser and network requests. It helps to coordinate retrieval of information from the network and cache.
  • utilities.py: Python module with helper functions for the application. The functions calculate distances in order to display stop information based on the location searched.

Maintainers

  • @AbdouSeck
    • Chief Data Wrangler
    • Location Services
  • @br3ndonland
    • Nominated Benevolent Dictator
    • Dr. Documentation
  • @jethridge13
    • Cartesian Contributor
    • Interstate Transportation Logistics