The main goal of this project is to create an environment for learning and tests which includes the group of microservices. To make this project more “real” I developed a group of microservices which main task is to observe stackoverflow topics and download interesting questions. The project is intentionally overcomplicated.
For each observed stackoverflow topic/tag one instance of go.stack-app is being run. Grabbed questions are moved to RabbitMq. Because the services can grab the same message (questions can be overlapped between different tags), questions are deduplicated in the next step.
Service py.deduplicator is responsible for deduplication. This service reads messages from the queue, checks their id in RedisDB and if the id exists the message is dropped. Otherwise the message is sent to exchange for deduplicated messages and the ID of message is saved in DB
In the third step, the questions are being read, analyzed and tagged. They are put to the exchange with the routing key which depends on the question tag received in this step. Service go.message.tagger is responsible for this step.
As a final step, messages with additional information are sent by go.discord-publisher to the user.
Diagram:
First download the main repository:
git clone git@github.com:JacekKorta/microservices-training-ground.git
cd microservices-training-groundThen update/init all submodules
git submodule update --initSetup all sub services. See each services's readme file for more information.
Finally you can start:
docker-compose upThe configuration is not production ready.