The aim of this project is to demonstrate the followings;
- An Apache Camel poller integrated with Spring Boot
- A Spring Boot backend API that serves the persisted polled documents
The tech stack is as follows;
- Spring Boot
- Apache Camel
- MySQL Database
The application is dockerized.
Functionally the application is controlled via Postman. The controller has two operations;
- A GET endpoint to retrieve the persisted governmental public documentations
- A POST endpoint to trigger the polling of the governmental public documentations
To give a demonstration, I'm using officially public documentation of the Dutch goverment, called as "OfficielebekendMakingen", based on OASIS standard
Camel Routes are consuming the XML data, parsing with the use of XPath, and persisting the parsed data to the mysql database.
Here is an overview of the structure of the project;
In this part I'm going to show you how to run the project via docker file.
We have two components, the mysql databse and the backend.
Simply the steps are as follows;
- Build the docker image of the backend
- Create the network bridge
- Run the whole stack via docker compose
- Build the docker image of the backend;
docker build --tag=oep-poller:latest .
- Create the network;
docker network create -d bridge poller-app-net
- Run the yaml file for whole stack;
docker compose up
In this scenario, we only need the database, thus we are only going to start the database. Still it's important
- Create the network;
docker network create -d bridge poller-app-net
- Run the yaml file for only database;
docker compose up -d db
- Run the application via IDE;
In this section, I'm going to show you how you can test the running application.
When the application is running, you can see the swagger API documentation via `` as follows;
If you have Postman which is a mainstream tool to test web-service endpoints, you can import the [Postman Collection]index.html from the misc directory.
Once you do it, you can trigger the polling process via the POST operation, and/or retrieve the the polled persisted data via the GET operation;
- Triggering the route;
This is done via a POST operation, and the payload should be as follows;
{
"dtDate": "2024-03-27",
"dtModified": "2024-03-27"
}
dtDate: Stands for the since date of the documents. If empty or null, the default value will be used. dtModified: Stands for the since modified date of the documents. If empty or null, the default value will be used.
When you are done with Postman, the POST operation should succeed as below;
- Retrieving the data;
Retrieve is simple, you should be able to call the GET operation as follows;
Testing via curl is easy;
- Triggering the route;
In windows the following curl command should work;
curl -d "{\"dtDate\":\"2024-03-27\",\"dtModified\":\"2024-03-27\"}" -H "Content-Type: application/json" -X POST http://localhost:8080/api/v1/poll
In *nix systems, the following should also work;
curl -d '{"dtDate":"2024-03-27","dtModified":"2024-03-27"}' -H "Content-Type: application/json" -X POST http://localhost:8080/api/v1/poll
- Retrieving the data;
In both Windows and *nix systems, the following curl command should work;
curl -H "Content-Type: application/json" -X GET http://localhost:8080/api/v1/documents