/postmangovsg

Templated message sending for public officers

Primary LanguageTypeScript

Postman.gov.sg

Postman.gov.sg

Postman.gov.sg enables public officers to send templated messages to many recipients.

Table of Contents

Features

  • Passwordless login: login with your .gov.sg email
  • Easily customize messages to reach a wide audience: create a message template, upload a file containing customization parameters and we will handle the customization for you
  • Send emails: Just click send and Postman will send those messages out to your intended audience via email
  • Send SMSes: Enter your twilio credentials, and Postman will send those messages via SMS. No integration with twilio is necessary
  • View stats: Keep track of your campaign's progress as it's sending, and check back when it's done.

API Usage

Just want to use the API? See api-usage.md for details.

Development

Install and run required services

Set up a postgresql@11 database, redis cache and localstack server. Postgresql and redis can be started either natively or using Docker.

Starting postgresql and redis natively:

# Install postgres
brew install postgresql@11
brew services start postgresql@11

# Create the database
createdb postmangovsg_dev

# Check if you can connect to the database
psql -h localhost -p 5432 postmangovsg_dev

# Install redis
brew install redis
brew services start redis

# Check that redis is running
redis-cli ping

# Start localstack container
export AWS_ENDPOINT=http://localhost:4566
export FILE_STORAGE_BUCKET_NAME=localstack-upload
export AWS_LOG_GROUP_NAME=postmangovsg-beanstalk-localstack

npm run dev:localstack

Starting all services using Docker:

export AWS_ENDPOINT=http://localhost:4566
export FILE_STORAGE_BUCKET_NAME=localstack-upload
export AWS_LOG_GROUP_NAME=postmangovsg-beanstalk-localstack

npm run dev:services

Optionally, run the following to install and use cw to tail Cloudwatch logs:

brew tap lucagrulla/tap
brew install cw

# Tail logs on localstack
cw tail -r ap-northeast-1 -u $AWS_ENDPOINT -f $AWS_LOG_GROUP_NAME:`node --eval='console.log(require("os").hostname())'`

Secrets detection

This project makes of detect-secrets to prevent secrets and credentials from being committed to the repository. It runs as a pre-commit hook and it needs to be installed if you intend to make commits to the repo. Run the following to install:

pip install detect-secrets

Set environment variables

Example environment variables can be found in

Set the environment variables in a file named .env in each folder

Install dependencies

cd postmangovsg
npm install

Compile frontend translations

lingui is used for internationalization. Read this for more info.

cd postmangovsg/frontend
npm run extract
npm run compile

Run the app

cd postmangovsg
npm run dev

You should find the

Deployment

We use TravisCI to simplify our deployment process:

  • backend is deployed on Elastic Beanstalk
  • frontend is deployed on AWS Amplify
  • worker is deployed on Elastic Container Service
  • serverless is deployed on AWS Lambda

The environment variables on Travis are:

  • AWS_ACCESS_KEY_ID : access key for the travis IAM user
  • AWS_SECRET_ACCESS_KEY : access key for the travis IAM user
  • AWS_DEFAULT_REGION : region where your infrastructure is deployed (ap-northeast-1 for us)
  • REPO: Path to the elastic container registry (<account_id>.dkr.ecr.ap-northeast-1.amazonaws.com/<repo_name>)
  • PRODUCTION_BRANCH : branch that is deployed to production
  • STAGING_BRANCH: branch that is deployed to staging. Change this variable to test different branches.

To deploy workers, trigger a custom build on Travis with the Custom Config set to

env:
  - DEPLOY_WORKER=true

Releasing

  1. Checkout a new release branch from develop.
  2. Bump the version of each of the subpackages (frontend, backend, worker).
  3. Stage changes for subpackage package.json and package-lock.json.
  4. Bump the version of the main package.
    • You will need to use --force as the working directory is not clean.
  5. Push both the version commit and tag to GitHub.
  6. Open a new pull request and consolidate changelist from the commits made since the previous release.
  7. Merge pull request into master to trigger Travis build and deploy. Note that we do not squash commits when merging into master.
    • Manually trigger a custom build on Travis if there were changes made to the worker.
  8. Create a new pull request to merge release branch back into develop branch.

Example:

# Create a new release branch for a new minor version (e.g. v1.6.0 -> v1.7.0)
$ git checkout develop
$ git checkout -b release-v1.7.0

# Bump shared module version
$ cd ../ && cd shared/
$ npm version minor

# Bump frontend version
$ cd frontend/
$ npm version minor

# Bump backend version
$ cd ../ && cd backend/
$ npm version minor

# Bump worker version
$ cd ../ && cd worker/
$ npm version minor

# Stage changes for subpackage package.json and package-lock.json
$ cd ../
$ git add */package.json */package-lock.json

# Create a version commit for main package
$ npm version minor --force

# Push version commit and tag to GitHub
$ git push -u origin release-v1.7.0
$ git push origin v1.7.0

Serverless

We make use of AWS lambda to handle the callbacks from Twilio, as well as updating email delivery status.

See serverless-docs for details

Downtime procedure

See downtime-procedure for steps on how to bring the app down in the event that we need to make database changes

Infrastructure customizations

Amplify rewrite rule

[
    {
        "source": "</^[^.]+$|\\\\\\\\.(?!(css|gif|ico|jpg|js|png|txt|svg|woff|ttf)$)([^.]+$)/>",
        "target": "/index.html",
        "status": "200",
        "condition": null
    }
]

Elastic Container Service

Create a cluster with four services. These names are currently hardcoded for deployment in .travis.yml

Cluster Name: postmangovsg-workers
Service Name LaunchType Platform version
staging-sending FARGATE 1.4.0
staging-logger FARGATE 1.4.0
prod-sending FARGATE 1.4.0
prod-logger FARGATE 1.4.0

Architecture

See architecture for details

Local node module

See local-module.md for details

How messages are sent

See sending.md for details

Forking and configuring this product

Disclaimer of Liability. This product is pending Vulnerability Assessment and Penetration Testing (VAPT). You should conduct your own security assessment prior to using code provided in this repository. Open Government Products (OGP) makes no representations or warranties of any kind, expressed or implied about the completeness, accuracy, reliability, suitability or availability of this codebase. Any usage is at your own risk.

Backend

See configure/backend for details

Frontend

See configure/frontend for details

Worker

See configure/worker for details

Contributions

The production branch is master, and the development branch is develop.

If you have write access to this repository

  • Check out your feature branch from develop
  • Make changes, and commit those changes
  • Push these changes to Github
  • Submit a pull request against develop, filling in the standard template

If you do not have write access to this repository

  • Fork this repository
  • Clone the forked repository to your machine
  • Create a branch, make changes and commit those changes.
  • Push these changes to Github
  • Submit a pull request against basefork/develop (that's us!)
  • Describe the issue as thoroughly as possible, and with screenshots if applicable. A picture speaks a thousand words!

For more information, see CONTRIBUTING.md