🏈 A microservice to issue, register and manage boletos
Here's a brief overview of our technology stack:
- Docker and Docker Compose to create our development and test environments.
- Serverless Framework to manage and deploy our AWS Lambda functions.
- AWS SQS as a queue manager to process things like boletos to register.
- Postgres as to store our data Sequelize as a Node.js ORM.
- Babel to transpile our code written in modern Javascript and we use multiple Webpack configurations to bundle our code for production, test and development.
- Ava as a test runner and Chai to do some more advanced test assertions.
- Yarn to install npm dependencies.
In order to develop for this project you must have Docker and Docker Compose installed.
If you never developed in this repo before:
- Clone the repository:
$ git clone git@github.com:pagarme/superbowleto
- Build the base image:
$ docker-compose build test
Tests are separate in functional
, integration
and unit
. You can either run them separately or run them all.
-
Run all tests:
$ docker-compose run test
-
Run only
functional
tests:$ docker-compose run test yarn run test-functional
-
Run only
integration
tests:$ docker-compose run test yarn run test-integration
-
Run only
unit
tests:$ docker-compose run test yarn run test-unit
We install our dependencies (aka npm dependencies) inside the Docker image (see our Dockerfile to understand it better).
This gives us the advantage of caching the dependencies installation process, so when we build the image again, it's already cached by Docker and the image can be easily distributed with all its dependencies installed.
However, if you need to install any new dependency, you must rebuild the image, otherwise, your dependency will not be available inside the container.
You can install dependencies and rebuild the image by running:
$ docker-compose run test yarn add jquery
$ docker-compose build test
Tests are found inside the test/
directory and are separate by type: functional
, integration
and unit
. It's also common to have some helpers
folders alongside the tests.
-
Unit
tests are used to test the smallest units of functionality, typically a method or a function ref.The folder structure of the unit tests tend to mirror the folder structure of the
src
folder. For instance, we generally see the following folder structure:├── src │ ├── index.js │ └── lib │ └── http.js └── test └── unit ├── index.js └── lib └── http.js
-
Integration
tests build on unit tests by combining the units of code and testing that the resulting combination functions correctlyref.The folder structure of the integration tests tend to mirror the folder structure of the
src
folder. For instance, we generally see the following folder structure:├── src │ ├── index.js │ └── lib │ └── http.js └── test └── integration ├── index.js └── lib └── http.js
-
Functional
tests check a particular feature for correctness by comparing the results for a given input against the specification. Functional tests don't concern themselves with intermediate results or side-effects, just the resultref.The folder structure of functional tests does not need to mirror the source folder, and the files can be organized as they seem fit. One way to organize this files is by
feature
oruser-story
. For instance, take a look at the example below, whereboleto/create.js
andboleto/register.js
are complete user stories:├── test └── functional └── boleto └── create.js └── register.js
-
Helpers
do not test anything, but instead provide tools for the tests. Inside thehelpers
folders one can havefixtures
(also know as "mocks"), or some util functions.For instance, if you need credit card information to perform various tests in many different places, or if you need an util function that is called before your tests are ran, you could place them inside a
helpers
folder in order to not repeat yourself:export const creditCardMock = { number: 4242424242424242, holder_name: 'David Bowie', expiration_date: 1220, cvv: 123, } export const cleanUpBeforeTests = () => { db.reset(); }
Helpers
folders can be created at any level within thetest
folder structure. If some helper is used only for unit tests, it should reside withintest/unit/helpers
. If the helpers is used across all tests, it should reside withintest/helpers
. If there's a helper that is used only for testing the http module on integration tests, then it should reside withintest/integration/http/helpers
.
This project is designed in AWS Lambda functions and has some logical branches and business domain rules that can be hard to understand. This section documents what every Lambda function does how it fits in the entire project.
Create a new boleto.
After creating the boleto (on our database), we will try to register the boleto withing the provider. Here, there's two possible outcomes: a) the provider could be reached, could process the boleto and gave us a status (either registered
or refused
); or b) the provider could not be reached or could not process the boleto (giving us an unknown
/undefined
/try_later
status).
The following steps illustrate the case where the provider could be reached and it could process the boleto.
- The
Client
makes an HTTP request to create a boleto. - We create the boleto in the
Database
with statusissued
. - We try to register the boleto within the Provider.
- The provider returns an answer (either
registered
orrefused
). - We update the boleto status in the
Database
. - We return the response to the
Client
(HTTP response).
Diagram built with mermaid.js. Check out the source code at docs/diagrams
The following steps illustrate the case where the provider could be reached and it could process the boleto.
- The
Client
makes an HTTP request to create a boleto. - We create the boleto in the
Database
with statusissued
. - We try to register the boleto within the Provider.
- The provider could not be reached or could not process the boleto.
- We update the boleto status in the
Database
topending_registration
. - We send the boleto (
boleto_id
andissuer
) to an SQS queue calledboletos-to-register
. This queue will be processed by another Lambda later. - We return the response to the
Client
(HTTP response) with thestatus = pending_registration
.
Diagram built with mermaid.js. Check out the source code at docs/diagrams
Example:
POST /boletos
Content-Type: application/json
{
"queue_url": "http://yopa/queue/test",
"expiration_date": "Tue Apr 18 2017 18:46:59 GMT-0300 (-03)",
"amount": 2000,
"instructions": "Please do not accept after expiration_date",
"issuer": "bradesco",
"payer_name": "David Bowie",
"payer_document_type": "cpf",
"payer_document_number": "98154524872"
}
201 Created
Content-Type: application/json
{
"queue_url": "http://yopa/queue/test",
"status": "issued | registered | refused",
"expiration_date": "Tue Apr 18 2017 18:46:59 GMT-0300 (-03)",
"amount": 2000,
"instructions": "Please do not accept after expiration_date",
"issuer": "bradesco",
"issuer_id": null,
"title_id": "null",
"payer_name": "David Bowie",
"payer_document_type": "cpf",
"payer_document_number": "98154524872"
}
Retrieve all boletos.
Diagram built with mermaid.js. Check out the source code at docs/diagrams
Example:
GET /boletos
Content-Type: application/json
{
"count": "10",
"page": "1"
}
200 Ok
Content-Type: application/json
[{
"id": "bol_cj1o33xuu000001qkfmlc6m5c",
"status": "issued",
"queue_url": "http://yopa/queue/test",
"expiration_date": "Tue Apr 18 2017 18:46:59 GMT-0300 (-03)",
"amount": 2000,
"instructions": "Please do not accept after expiration_date",
"issuer": "bradesco",
"issuer_id": null,
"title_id": "null",
"payer_name": "David Bowie",
"payer_document_type": "cpf",
"payer_document_number": "98154524872"
}]
Find one boleto by id.
Diagram built with mermaid.js. Check out the source code at docs/diagrams
Example:
GET /boletos/:id
Content-Type: application/json
{
"id": "bol_cj1o33xuu000001qkfmlc6m5c"
}
200 Ok
Content-Type: application/json
{
"id": "bol_cj1o33xuu000001qkfmlc6m5c",
"status": "issued",
"queue_url": "http://yopa/queue/test",
"expiration_date": "Tue Apr 18 2017 18:46:59 GMT-0300 (-03)",
"amount": 2000,
"instructions": "Please do not accept after expiration_date",
"issuer": "bradesco",
"issuer_id": null,
"title_id": "null",
"payer_name": "David Bowie",
"payer_document_type": "cpf",
"payer_document_number": "98154524872"
}
Process the boletos-to-regiter-queue
. This Lambda function is triggered by the Schedule Event (runs every n
minutes).
When a boleto can't be registered within the provider at the moment of its creation, it will be posted to a SQS Queue called boletos-to-register
. This function is responsible for processing this queue. Here are the steps:
- This function is triggered by a Schedule Event that runs every
n
minutes. - Using
sqs-quooler
we then start to poll items from SQS (sqs.receiveMessage
) - For each message received we invoke a Lambda function called
register
with the boleto information and the SQS Message as parameters. sqs-quooler
package will repeat thispollItems->receive->invokeRegisterLambda
cycle until this Lambda dies (max 5 minutes).
Diagram built with mermaid.js. Check out the source code at docs/diagrams
Register a boleto. This Lambda function is invoked by another Lambda function: process boletos-to-register
queue.
This function registers boletos that went to the boletos-to-register
queue. Here are the steps of execution:
- The function is invoked by the queueProcessor and receives
boleto
({ id, issuer}) andmessage
(the raw SQS message fromboletos-to-register
queue). - We use the boleto
id
to find the boleto onDatabase
. - We check if the boleto can be registered, i.e. if the status of the boleto is either
issued
orpending_registration
. - If the boleto can be registered, we try to register it within the provider.
- If the provider could not process the boleto, we stop executing here. The SQS Message will then go back to
boletos-to-register
queue and will be later processed. - We update the boleto status with either
registered
orrefused
. - IMPORTANT: After the boleto is updated, we notify the boleto owner by sendin an SQS message to the queue the owner specified on the boleto creation (aka:
boleto.queue_url
). The owner will then handle the processing of these SQS Messages. That's the only way we can notify the boleto owner that a boleto that went toboletos-to-register
queue was updated. That's why it's mandatory to pass a queue at the moment of the boleto creation.
Diagram built with mermaid.js. Check out the source code at docs/diagrams