laa-crime-equinity-historical-data

This is a Java based Spring Boot application hosted on MOJ Cloud Platform.

Application Set up

Clone Repository

git clone git@github.com:ministryofjustice/laa-crime-equinity-historical-data.git

cd laa-crime-equinity-historical-data

Make sure all tests are passed by running following ‘gradle’ Command

./gradlew clean test

You will need to build the artifacts for the source code, using gradle.

./gradlew clean build

docker-compose build
docker-compose up

laa-crime-equinity-historical-data application will be running on http://localhost:8089

Crime Apps repository template guide

Filling docker-compose.override.yml

In order to run docker image on local, a docker-compose.override.yml file is required. This file will contain sensible data and therefore is not available on git. However, a template is provided. The first thing to do is to define the environment variables the application will expectect (look for $ on applicaiton.yaml file)

Database

This project runs in a MS-SQL Server database. Currently, there is no option to run it locally. Unit and Integration tests are running using mocked data over an H2 (memory) database with JPA driver to emulate MS-SQL one.

Application can run locally (java or docker) connecting to the RDS database via a tunnel.

How to connect to a database tunnel

Connection to a database needs to be done from a tunnel. This tunnel is done from a pod that will serve a bridge between our local connection and the RDS on cloud platform.

Crime Apps guide to local RDS connections

See example below:

Create port-forward-pod

kubectl \
 -n laa-crime-equinity-historical-data-uat \
 run rds-port-forward-pod \
 --image=ministryofjustice/port-forward \
 --port=1433 \
 --env="REMOTE_HOST=cloud-platform-somethingherecheckcloudplatform.rds.amazonaws.com" \
 --env="LOCAL_PORT=1433" \
 --env="REMOTE_PORT=1433"

Open tunnel from local (on port 9433) to port-forward (on port 1433)

kubectl \
 -n laa-crime-equinity-historical-data-uat \
 port-forward \
 rds-port-forward-pod 9433:1433

This tunnel will redirect all connections to localhost:9433 to the port-forward:1433, and then the port forwards would redirect them to the RDS database server at port 1433

Scripts

Create port-forward pod

kubectl \
 -n {namespace} \
 run {rds-pod-name} \
 --image=ministryofjustice/port-forward \
 --port={target-port} \
 --env="REMOTE_HOST=cloud-platform-34f4d161bec89a00.cdwm328dlye6.eu-west-2.rds.amazonaws.com" \
 --env="LOCAL_PORT={target-port}" \
 --env="REMOTE_PORT={target-port}"

The por-forward port only needs to be created once. And there is no need to create if it has been already created.

Open a tunnel to port-forward

kubectl \
 -n {namespace} \
 port-forward \
 rds-port-forward-pod {local-port}:{target-port}

Where:

{namespace} : use the valid namespace (e.g. laa-crime-equinity-historical-data-uat)
{rds-pod-name} : give a name easy to remember and identify, and make sure it matches on both creation and opening (e.g. rds-port-forward-pod)
{target-port} : this is teh port where usually the database is running (e.g. 1433 as it is the default for MSSQL Server)
{local-port} : this is the port we want to have our local tunnel listening on (e.g. use same 1433, if changed to 9433 then use 9433 for localhost connections but it will redirect to {target-port} at the end of teh tunnel)

CI/CD

We have configured a GitHub Actions code pipelines. You can log in from here to access the pipeline.

To make any changes, create a branch and submit the PR. Once the PR is submitted the branch checks will kick off, and on success, it will wait for approval before merged. Once PR is merged, image will be build and pushed automatically to AWS ECR. On success, the image will deploy automatically into UAT. After successful deployment to UAT, deployment to higher environments (staging and prod) start automatically, but require manual approval.

Open API

We have implemented the Open API standard (with Swagger 3). The web link provides a details Rest API with a schema definition. The link can only from local or from dev environment. The swagger link can be found from here

Application Monitoring and Logs

Logs on Kibana
Alerts on Sentry
Metrics on Prometheus

Crime Apps guide on logging and metrics

Kubernetes

Useful commands to access pods and kubernetes cluster information

How to access the pods:

In order to access the pods, it is required to have Kubernetes installed and configured in your local machine. If you need help, check these documents:

Assuming Kubernetes is all setup, follow these steps to access the pods.

Use the following command from terminal:

kubectl get pods -n {nameSpace}

Possible values for nameSpace are:

laa-crime-equinity-historical-data-uat
laa-crime-equinity-historical-data-staging
laa-crime-equinity-historical-data-prod

Check response from command below, you will need that for the following step

kubectl get pods -n {nameSpace}

Output:

NAME                                 READY   STATUS    RESTARTS   AGE
{poddName}                           1/1     Running   0          18m

Access the pod console using the following command:

kubectl exec -it {podName} -n {nameSpace} -- sh

Example:

kubectl get pods -n laa-crime-equinity-historical-data-readme

Output should be similar to this:

NAME                                             READY   STATUS    RESTARTS   AGE
laa-crime-equinity-historical-data-00000-xxxxx   1/1     Running   0          18m

kubectl exec -it laa-crime-equinity-historical-data-00000-xxxxx -n laa-crime-equinity-historical-data-readme -- sh

That should give you access to the pods terminal.

Additional guides

Ministry of Justice Compliant Repository

The current repository used this template to create a repository with the default initial files for a Ministry of Justice Github repository, including:

The correct LICENSE
Github Action example
A .gitignore file
A CODEOWNERS file
A dependabot.yml file
The MoJ Compliant Badge (Public repositories only)