developers.democracyclub.org.uk API

About

This project provides an API gateway in front of other Democracy Club APIs.

The /postcode and /address APIs provide a nice abstraction over calling both the WDIV and WCIVF APIs and combine the responses.
The /elections endpoint is a straight proxy for EE's /elections endpoint.
The project hosts user-facing documentation for the API and sandbox outputs to help data consumers develop applications when real data is not available.
As time goes on, developers.democracyclub.org.uk should become a one-stop-shop for DC data, additionally proxying additional EE/YNR endpoints.

Local Development

Application

cp aggregator-api/aggregator/settings/local.example.py aggregator-api/aggregator/settings/local.py
Install Python dependencies: pipenv install --dev
Run the test suite: pytest
Run lint checks: pytest --flakes
Auto-format: black .

Documentation

We build the docs locally instead of rendering on-the-fly and then commit the compiled documentation so it can be served as a static template. Compile the docs with ./manage.py build_docs. The API docs use drafter 3 for parsing API Blueprint. On linux this will be installed automatically with the python dependencies. On OSX/Windows, this needs to be installed seperately.

Configuration

Local Settings

For local development, the only required setting is SECRET_KEY. In local.py, it may also be useful to set:

EE_BASE_URL - e.g: to run against a local WDIV install instead of elections.democracyclub.org.uk
WCIVF_BASE_URL - e.g: to run against a local WDIV install instead of whocanivotefor.co.uk
WDIV_BASE_URL - e.g: to run against a local WDIV install instead of wheredoivote.co.uk
WDIV_API_KEY - WDIV API key. In local dev, anonymous access is generally sufficient. You are unlikely to exceed the rate limit.

Production Settings

In production, settings are obtained from environment variables. In production, we need:

SECRET_KEY - Django secret key.
WDIV_API_KEY - WDIV API key. For production usage, we must call WDIV with an API key to ensure we are not rate limited.
SENTRY_DSN - If set, exceptions will be logged here.
ENV - "prod" or "dev" - used by and sentry. FIXME: not currently propagated via the SAM CLI

Deployment

The API is hosted on AWS Lambda and built with the AWS SAM CLI
Deployment of 3 parallel environments is handled by Circle CI
If you merge to master and all the tests pass, deployments will be triggered:
- to development and staging in parallel
- to production, if staging smoke tests pass

Development deployments to AWS Lambda

Development deployments are described in detail in a separate document.

Here are the happy-path steps to create a deployment in DC's development AWS account.

Before following these steps:

Choose a deployment name and use it wherever these steps mention <NAME>
Install and configure the local development prerequisites
Activate the Python virtualenv that contains the local development prerequisites you installed in the previous step

NEW_ENV_NAME=<NAME> python samconfig.toml.d/new-dev-env.py >>samconfig.toml
AWS_DEFAULT_REGION=eu-west-2 pipenv run sam validate
make all
sam build  --config-env <NAME> --use-container --cached
sam deploy --config-env <NAME>

These steps should have deployed the app to Lambda, accessible via AWS API Gateway on the 'AggregatorApiFqdn' domain mentioned near the end of the deployment output, but only when accessed the path '/Prod'.

You can continue and add TLS, caching, and a custom domain to this deployment by following the rest of the deployment document.

GeoWill/aggregator-api