Early years child development training

This is a Rails 7 application using the DfE template.

Optionally create .env to override or set default variables like DATABASE_URL.

Getting Started

Clone the repository
Install git-secrets
Obtain the master keys
Start the server

Useful Links

Rails Credentials

We use rails credentials to manage secrets; obtain the encryption keys from the dev team.

To edit, use either:

EDITOR=vi rails credentials:edit --environment <env>.
./bin/docker-rails credentials:edit --environment <env>

Full instructions can be found by running rails credentials:help

Git Secrets

This will help to prevent unintentional commits of access keys.

brew install git-secrets
cd /path/to/my/repo
git secrets --install
git secrets --register-aws

Find advanced settings and other installation options at the git-secrets project.

Working locally

$ asdf plugin add ruby
$ asdf install ruby
$ asdf plugin add postgres
$ asdf install postgres
$ asdf plugin add nodejs
$ asdf install nodejs

Development

Gemfile group :development

Use bin/dev to start the process workers (watching for changes to asset files).

Testing

Gemfile group :test

Use bin/rspec to run the test suite under /spec. Rails system specs use RackTest only for efficiency.

Production

Running locally in the production rails environment requires generating a self-signed certificate. Use bin/docker-certs

UI Framework

Gemfile group :ui

Use bin/qa to run the test framework under /ui against a given URL. These tests have additional dependencies:

brew install chromedriver geckodriver
xattr -d com.apple.quarantine /usr/local/bin/chromedriver on Intel
xattr -d com.apple.quarantine /opt/homebrew/bin/chromedriver on ARM

Using Docker

Containerised services are available for use by developers and testers. There are a number of convenience scripts to make working with Docker easier. All containers for the project are named with the prefix recovery_. The project uses chained Docker Compose files to prepare different environments.

These commands help maintain your containerised workspace:

bin/docker-build creates tagged images for all the services
bin/docker-certs generates a self-signed certificate for running the app in production
bin/docker-files changes the ownership of files to your current user, files generated inside containers are created by root
bin/docker-down stop any active services
bin/docker-prune purge project containers, volumes and images

The commands run common tasks inside containers:

bin/docker-adr rebuilds the architecture decision records table of contents
bin/docker-dev starts Procfile.dev, containerised equivalent of bin/dev, using the docker-compose.dev.yml override Additionally, it will install bundle and yarn dependencies.
bin/docker-rails erd generate an Entity Relationship Diagram
bin/docker-rails db:seed populates the containerised postgres database
bin/docker-rails console drops into a running development environment or starts one, containerised equivalent of bin/rails console
bin/docker-rspec -f doc runs the test suite with optional arguments, containerised equivalent of bin/rspec
bin/docker-doc runs a YARD documentation server
bin/docker-uml exports UML diagrams as default PNGs
bin/docker-prod starts rails in production under https
bin/docker-qa runs the browser tests against a running production application, a containerised equivalent of bin/qa
bin/docker-pa11y runs WCAG checks against a generated sitemap.xml

These commands can be used to debug problems:

docker ps lists all active Docker processes
docker system prune tidies up your system
docker-compose -f docker-compose.yml -f docker-compose.<FILE>.yml --project-name recovery run --rm app can help identify why the application is not running in either the dev, test, or qa contexts
BASE_URL=https://app:3000 docker-compose -f docker-compose.yml -f docker-compose.qa.yml --project-name recovery up app debug the UAT tests

Using Rake

Custom tasks are namespaced under eyfs, list them using rake --tasks eyfs.

rake eyfs:bot # Generate secure bot user
rake eyfs:jobs:plug_content # Que job to insert page view events for injected module items
rake eyfs:user_progress # Recalculate module completion time
rake eyfs:whats_new # Enable the post login 'What's new' page

Trigger a task on a deployed application in either the ey-recovery-content or ey-recovery-staging spaces, not production, using bin/cf-task.

./bin/cf-task pr-123 eyfs:bot
./bin/cf-task dev eyfs:whats_new[email1@example.com,email2@example.com]

Refer to this script for the steps necessary to ssh in and run a task manually.

Deployment Pipeline

Visit the Github Container Registry.

Development Space

Development is deployed automatically with the latest commit from main.

Content Space

Manually adding the "deployed" label to a pull request in Github will cause it to be deployed. This supports manual testing and content review in a production environment.

When a feature branch review application is deployed, the URL to access it is added as a comment in the PR conversation in the format: https://ey-recovery-pr-##.london.cloudapps.digital/

Review applications are deployed with 3 seeded user accounts that share a restricted password. This facilitates team members demoing content and functionality, so registration is not required.

Staging Space

Staging is deployed automatically when a candidate tag is pushed.

git checkout <ref/branch>
git tag --force rc0.0.x
git push --force origin rc0.0.x

A tag can also be created and a deployment run from this workflow. We intend to use semantic versioning.

Production Space

Production is deployed automatically when a version tag is pushed.

git checkout rc0.0.x
git tag v0.0.x
git push origin v0.0.x

A tag can also be created and a deployment run from this workflow.

Autoscaling

Auto scaling policy is

Scale up when CPU > 85%
Scale down when CPU < 30%

Created with the following commands:

Install autoscaling plugin:

cf install-plugin -r CF-Community app-autoscaler-plugin

Create an autoscaler service:

cf create-service autoscaler autoscaler-free-plan scale-ey-recovery

Bind the autoscaler service to the app:

cf bind-service ey-recovery scale-ey-recovery

Policy is found in policy.json

Attach the autoscaling policy to the app:

cf attach-autoscaling-policy ey-recovery policy.json

Observe the app scaling automatically:

cf autoscaling-history ey-recovery

For further information see [Managing apps - autoscaling]: https://docs.cloud.service.gov.uk/managing_apps.html#autoscaling

The link includes additional examples for policies e.g. adding a schedule to scale at certain times or days of the month

Monitoring

Sentry is used to monitor production environments

$ brew install getsentry/tools/sentry-cli

$ sentry-cli projects list --org early-years-foundation-reform

+---------+--------------+-------------------------------+--------------+
| ID      | Slug         | Team                          | Name         |
+---------+--------------+-------------------------------+--------------+
| 6274627 | eyf-reform   | early-years-foundation-reform | Rails        |
| 6274651 | eyf-recovery | early-years-foundation-reform | eyf-recovery |
+---------+--------------+-------------------------------+--------------+

Quality Assurance

The UI/UA test suite can be run against any site. A production-like application is available as a composed Docker service for local development. To run a self-signed certificate must first be generated.

./bin/docker-certs (Mac users can trust the certificate in Keychain Access)
./bin/docker-qa (this will build and bring up the application with a clean database)
./bin/docker-prod-reset (clean and reseed the prerequisite user accounts)
./bin/docker-qa (retest)
BASE_URL=https://deployment ./bin/docker-qa (alternative test against another server)

Accessibility Standards

An automated accessibility audit can be run against a development server running in Docker using ./bin/docker-pa11y. The test uses pa11y-ci and a dynamic sitemap.xml file to ensure the project meets WCAG2AA standards. A secure HTTP header BOT is used to provide access to pages that require authentication. The secret $BOT_TOKEN environment variable defines the account to seed.

curl -i -L -H "BOT: ${BOT_TOKEN}" http://localhost:3000/my-account

docker-pa11y accepts an optional argument to test external sites.

Emails

Emails are sent using the GOV.UK Notify.

Getting a GovUK Notify account

You need an account before you can use GOV.UK Notify to send emails. To obtain one ask a current member of the team to add you to the "Early Years Foundation Recovery" service, by navigating to the Team members page and clicking the Invite a team member button and entering a government email address. This will send an email inviting you to use the service.

The credentials file for each environment holds an API key for Notify:

railsdevelopment-... Team and guest list (limits who you can send to)
railstest-... Test (pretends to send messages)
railsproduction-... Live (sends to anyone)

It is possible to temporarily override the key by defining GOVUK_NOTIFY_API_KEY in .env.

Accessing information in the Notify service

Once you have an account you can view the Dashboard with details of how many emails have been sent out and any that have failed to send.

You can update the content of the emails in the Templates section.

For more information

Documentation for GovUK Notify can be found here: https://docs.notifications.service.gov.uk/ruby.html

The status of GovUK notify can be checked here: https://status.notifications.service.gov.uk/

For more information the Notify team can be contacted here: https://www.notifications.service.gov.uk/support, or in the UK Government digital slack workspace in the #govuk-notify channel.

Service Dashboard

Key performance metrics are surfaced in a Looker Studio dashboard and refreshed daily. User service accounts can authenticate using the Google Cloud SDK.

Storage and Reporting

Downloading exported data

gcloud auth login
gcloud config set project eyfsdashboard
gsutil ls (list buckets)
gsutil -m cp -r "gs://eyfs-data-dashboard-live/eventsdata" "gs://eyfs-data-dashboard-live/useranswers" . (export folders recursively)

Cloning production data

Export production database as plaintext dump: $ cf conduit --space ey-recovery ey-recovery-db -- pg_dump --file "./tmp/ey-recovery-db-<DATE>.sql" --no-acl --no-owner -Z 9
Clone exported live data to an existing empty local development database: $ docker exec -it recovery_dev psql prod_clone < ./tmp/ey-recovery-db-<DATE>.sql
Optional: Restart dev server pointing to the prod_clone database

User experience

Session timeout functionality:

default timeout period is 25 minutes
default timeout warning appears after 5 minutes
screen readers announce every time the timeout refreshes every 15 secs

Hotjar

This project uses Hotjar for user insight. Hotjar records user journeys and automatically redacts certain user information on recordings. All personally identifiable information should be redacted. In order to override the default settings the following classes can be added:

data-hj-suppress to redact additional user information
data-hj-allow to allow data that is automatically redacted

Azure

Production console access

DFE-Digital/early-years-foundation-recovery