rizkisunaryo/fabelio-backend

Architecture

By API

URL can be inserted by API. The flow is shown with green line in the picture above

1. User insert URL. The API will then insert the URL to MongoDB
2. API will add the URL to the Queue as well
3. Queue will send the URL to Crawler
4. Crawler will crawl the details of the URL
5. URL details will be saved to MongoDb

By Scheduler

1. Scheduler get the URLs from MongoDB
2. And then, the Scheculer will add the URLs to the Queue
3. Queue will send the URL to Crawler
4. Crawler will crawl the details of the URL
5. URL details will be saved to MongoDb

Installation

• Install yarn https://yarnpkg.com/en/docs/install
• Install Docker https://www.docker.com/products/docker-desktop
• Run yarn install in terminal, in this project root folder

How to Run

./run.sh

How to Stop

./kill.sh