/banking-microservices-tutorial

Tutorial project for learning how to test microservices

Primary LanguageJavaApache License 2.0Apache-2.0

Banking Microservices Tutorial

Build Status Coverage Status FOSSA Status

The Banking Microservices Tutorial project is a small system used to show how microservices can be implemented and tested with Micronaut, Consul, Express Gateway, and Axon's Event Sourcing framework. The system can be run in multiple configurations using Docker.

Architecture

Build Status

The overall Banking Example architecture is broken into two sub-domains, Accounts and People.

Build Status

Example of flow of data after issueing a withdraw command.

Build Status

Routing configuration of gateways.

Configuration

The services can be configured in three ways, a local default configuration under each project resources/application.yml, a development configuration under resources/application-dev.yml, and the centralized configuration service.

Requirements

See each services readme for detailed requirement information

Compose

Java 8

IntelliJ IDEA

Lombok

Running the Project

Start the Microservices

Build JARs for each project (You will need to build a JAR anytime changes are made to a project, then rebuild either the container or all containers)

# Assemble the binaries
./gradlew assemble
# Start the backing services: service discovery, configuration, authentication, edge service
docker-compose up --build
# After verifying everything spun up correctly tear it down.
# Press Control C to shut down the docker containers

Running with Mocks

To download the Mock images and test running on your machine use the following commands.

docker-compose -f docker-compose-subdomain-testing.yml up
# After verifying everything spun up correctly tear it down.
# Press Control C to shut down the docker containers

Running the PACT Broker

Exercise the PACT Broker service.

docker-compose -f ./docker/pact-broker/docker-compose.yml up
# After verifying everything spun up correctly tear it down.
# Press Control C to shut down the docker containers

Rebuild Containers

docker-compose build

Executing Tests

The following guides are meant to get your environment up and running tests, not necessarily a guide to the most effective way to execute the tests while you are developing them.

To see detailed logs for any of these tests, you may execute them from IntelliJ or view the test reports from the terminal execution within <PROJECT>/build/reports/tests/test/index.html

Windows Users

The following examples use shell scripts, just replace the .sh extensions in the examples with .bat to execute them in Command Prompt or PowerShell.

Running Unit and Integration Tests

The Gradle task 'test' executes the JUnit tests for each project.

sh ./scripts/run-unit-tests.sh

Running Code Coverage: Unit and Integration Tests

JaCoCo is used for code coverage and can be run after the unit and integration tests for each service have been executed. You can find a JaCoCo coverage report under the "coverage" in transaction service after running the unit tests.

Running Service Isolation Tests

The documentation here provides a guide on creating new isolation tests with HTTP stubs.

Running Service Isolation Tests with All External Dependencies Mocked

Mocking all external dependencies to the services allows for very rapid execution of tests and alleviates the need for configuring or utilizing resources for the external dependencies. In memory databases are used in the place of Mongo, though the same Mongo code dependencies are used to connect to these in-memory databases. HTTP mock server stubs are used to provide stubbed responses for external services. Externally Mocked Services Docker is not required to run these tests as all external dependencies are mocked.

sh ./scripts/run-isolation-tests-mocked.sh

Running Service Isolation Tests with External Databases, Caches, and Etc...

Here only the calls to other services are mocked, but external dependencies like databases, caches, and discovery services are deployed. For this guide, we will run the Transaction service isolation tests. We use Docker Compose to stand up Mongo. Transactions is the only service demoed here because in an actual product you will most likely have a cloud deployment infrastructure where you can dynamically configure the HTTP stubs, here we use a Docker Compose configuration. Externally Mocked Services Start the services database using the backing services.

docker-compose -f docker-compose-mongo-axon.yml up -d

Execute the tests in a new terminal once external dependencies have started.

sh ./scripts/run-isolation-tests.sh

Tear down the external dependencies.

docker-compose -f docker-compose-mongo-axon.yml down

If you modify or add an HTTP stub under ./test/resources/wiremock then you will need to restart the instances so they refresh their mappings. You can read more about the WireMock API here.

If you update the WireMock request journal validations under ./domain-services/account-transactions/src/tests/resources/wiremock you will not need to restart the instances, only the tests use these. More documentation on WireMock verification can be found here.

Running Contract Tests

Ideally, these tests would run in a continuous integration system and not require the Docker Compose steps provided. Start the domain services with internal mocks so that only the endpoints will be tested. Internally Mocked Services To read more on implementing PACT contract tests we have provided a guide here.

docker-compose -f docker-compose-internal-mocked.yml up -d

Start the PactBroker service and check http://localhost:8089 that it is live.

docker-compose -f ./docker/pact-broker/docker-compose.yml up -d

Generate the PACTs and execute them. Note, if you have not completed the PACT tests in all the projects then you will see build failures during the first step here, these can be ignored.

sh ./scripts/generate-publish-pact-tests.sh
sh ./scripts/run-pact-tests.sh

Stop the PactBroker.

docker-compose -f ./docker/pact-broker/docker-compose.yml down

Stop the services with internal mocks.

docker-compose -f docker-compose-internal-mocked.yml down

Note, if you want to examine the individual PACTs, these are generated in tests/pact-tests as JSON files.

Running Service Integration Tests

Sub-Domain Service Integration Testing

To run the sub-domain service integration tests, all of the dependencies must be available for the given service under test. For this case, we will be running integration tests for the Account sub-domain, which means that the gateway for the People domain will be mocked, but all Account related services should be up. Sub-Domain Integration Testing

Use docker to stand up the supporting services, databases, and etc...

docker-compose -f docker-compose-subdomain-testing.yml up

Once the services stabilize, you should see a message like o.a.a.c.AxonServerConnectionManager - Re-subscribing commands and queries, at this point you can open a new terminal and run the tests.

sh ./scripts/run-sub-domain-integration-tests.sh

Take down the services in the other terminal window.

docker-compose -f docker-compose-subdomain-testing.yml down

Pairwise Service Integration Testing

To run the pairwise service integration tests you will need to have the appropriately configured environment for the particular tests. Here, we demo pairwise testing of the Account Transactions and Account Cmd pair, with all other domain services mocked, notice that unlike subdomain testing, the domain gateway is not present. This type of testing requires many configurations and thus should be used for complicated interactions between two services, not for every service pair. Sub-Domain Integration Testing

Use docker to stand up the supporting services, databases, etc...

docker-compose -f docker-compose-pairwise-account-cmd-transaction.yml up

Once the services stabilize, you should see a message like o.a.a.c.AxonServerConnectionManager - Re-subscribing commands and queries, at this point you can open a new terminal and run the tests.

sh ./scripts/run-transaction-pairwise-tests-with-cmd.sh

Take down the services in the other terminal window.

docker-compose -f docker-compose-pairwise-account-cmd-transaction.yml down

Note on Event Sourcing and Hydration

If Axon Server is running it will automatically attempt to rehydrate event listeners, if you have run tests before this means you will see accounts created, transactions go through, and then accounts get deleted. Your tests should not be affected by this but it can create noise or potentially cause side effects when creating new tests. To remove these events you will need to delete the axon-server-controldb and axonserver-eventstore folders in the root of this project.

API Documentation:

Each service publishes a Swagger YAML configuration, if you are familiar with Swagger UI you can consume the following configurations:

It is in the roadmap to expose a Swagger UI endpoint on each service in the future.

Troubleshooting

Docker Issues

Orphaned Docker Containers Are Still Running

If you are seeing issues with port allocations and Docker then try running docker ps, if you see something running that should not be you can kill it with docker rmi --force <ID>. A useful Docker command for killing all live containers is docker kill $(docker ps -q)

Containers Keep Restarting or Failing

Try increasing your Docker memory, more than 2 CPUs and 4GB assigned to Docker is preferable for this project.

Trouble Building

General Build Issues

Try clearing your global Gradle cache by deleting ~/.gradle and the local ./.gradle in the project.

Duplicate class found

Check that you have the proper version of Java installed java -version. If it is not 1.8 then set your JAVA_HOME to 1.8.

IntelliJ can't find getter methods

You are probably missing the Lombok annotation plugin listed in the project requirement section or haven't turned on the annotation processor setting in IntelliJ.

Tests Issues

Test Not Running

Check your imports for JUnit, if you don't see juniper for your Test annotation then you are using JUNit 4 and the tests won't run until you fix the imports.

Mocks are Null

Check that you are using Mockito the JUnit 5 way, with the MockitoExtension and ExtendWith annotations.

Services are Rehydrated After Restart and Clearing Mongo

If you have a lot of events then services are going to be rehydrated when you bring everything up. To stop this you can delete the event folders axonserver-eventstore and axonserver-controldb in the root of the project and then bring the environment up.

Test Do Not Rerun

Gradle caches outputs from tasks, if it sees an input (in this case the source code) hasn't changed then it won't rerun the tests. You can add cleanTest to the scripts in order to force reruns without changes.

Incorrect Array Size on GET Requests

There may be orphaned results in Mongo, try tearing it down and removing the volumes.

Parallization

In the root build.gradle there is a parallization line for JUnit, but beware, some tests spin up mock servers off their process, this will result in a port conflict if two tests use the same mock server port. This is only a consideration for service isolation tests and contract test generation.

License

FOSSA Status