HMDA Platform

This project is a work in progress

Information contained in this repository should be considered provisional and a work in progress, and not the final implementation for the HMDA Platform, unless otherwise indicated.

Introduction

For more information on HMDA, checkout the About HMDA page on the CFPB website.

The HMDA Platform

This repository contains the code for the entirety of the HMDA platform backend. This platform has been designed to accommodate the needs of the HMDA filing process by financial institutions, as well as the data management and publication needs of the HMDA data asset.

The HMDA Platform is composed of the following modules:

Parser

Module responsible for reading incoming data and making sure that it conforms to the HMDA File Specification

Data Validation

Module responsible for validating incoming data by executing validation rules as per the Edit Checks documentation

Persistence

Module responsible for persisting information into the system. It becomes the system of record for HMDA data

API

This module contains both public APIs for HMDA data for general use by third party clients and web applications, as well as endpoints for receiving data and providing information about the filing process for Financial Institutions

Dependencies

Java 8 SDK

The HMDA Platform runs on the Java Virtual Machine (JVM), and requires the Java 8 JDK to build and run the project. This project is currently being built and tested on Oracle JDK 8. See Oracle's JDK Install Overview for install instructions.

The HMDA Platform should also run on OpenJDK 8.

Scala

The HMDA Platform is written in Scala. To build it, you will need to download and install Scala 2.11.x

In addition, you'll need Scala's interactive build tool sbt. Please refer to sbt's installation instructions to get started.

Building and Running

The HMDA Platform uses sbt's multi-project builds, each project representing a specific task.

Interactive

Start sbt

$ sbt

Select project to build and run.This will retrieve all necessary dependencies, compile Scala source, and start a local server. It also listens for changes to underlying source code, and auto-deploys to local server.

> projects
[info] In file:/Users/marinj/Development/hmda-platform/
[info]     api
[info]   * hmda
[info]     model
[info]     parser
[info]     platformTest
[info]     validation

> ~re-start

Confirm that the platform is up and running by browsing to http://localhost:8080

To build JVM artifacts (the default, includes all projects), from the sbt prompt:

> clean assembly

This task will create a fat jar, which can be executed directly on any JDK8 compliant JVM:

java -jar target/scala-2.11/hmda.jar

Docker

First, make sure that you have the Docker Toolbox installed.

If you don't have a Docker machine created, you can create one by issuing the following:

docker-machine create --driver virtualbox dev

After the machine is created, make sure that you connect your shell with the newly created machine

$ eval "(docker-machine env dev)"

Ensure there's a compiled jar to create the Docker image with:

sbt clean assembly

To run only the API

Build the docker image

docker build -t hmda-api .

Then, run the docker image

docker run -d -p "8080:8080" hmda-api

The API will run on $(docker-machine ip):8080

To run the entire platform

Clone the HMDA Platform UI directory into a sibling directory of this one. Your directory structure should look like this:

~/dev/hmda-project$ ls -la
total 16
drwxr-xr-x   6 lortone  staff   204B Jul 25 17:44 ./
drwxr-xr-x   9 lortone  staff   306B Jul 25 17:50 ../
drwxr-xr-x  22 lortone  staff   748B Jul 27 16:28 hmda-platform/
drwxr-xr-x  25 lortone  staff   850B Jul 25 17:13 hmda-platform-ui/

From hmda-platform's root directory, run the following:

docker-compose up -d --build

This will bring up all the HMDA Platform services. The first run may take several minutes.

For convenience when doing development on both the UI and the API, the docker-compose file uses a volumes which mounts the local directory into the hmda-platform-ui container and the hmda.jar into hmda-platform container. This means you can make changes to either the UI or API and view them without needing to rebuild their respective containers.

To build the front-end and allow "watching" for changes you can run:

# while still in the hmda-platform directory
cd ../hmda-platform-ui
npm install # optional, to make sure you get the dependencies
npm run watch

If you don't need to "watch" for changes you can run:

# while still in the hmda-platform directory
cd ../hmda-platform-ui
npm install # optional, to make sure you get the dependencies
npm run build

This will simply build the front-end, still taking advantage of the mounted volume.

In order to view changes in the API you need to rebuild the jar and then restart the container:

# while still in the hmda-platform directory
sbt clean assembly
docker-compose stop
docker-compose up

View the app by visiting your docker machine's endpoint in the browser. To find your docker machine endpoint:

docker-machine ip dev

Contributing

CFPB is developing the HMDA Platform in the open to maximize transparency and encourage third party contributions. If you want to contribute, please read and abide by the terms of the License for this project.

We use GitHub issues in this repository to track features, bugs, and enhancements to the software. Pull Requests are welcome

wpears/hmda-platform