nodejs-crawler

Crawler using terraform and Logstash for logs

Project

A simple crawler that will retrieve data from Mercado Livre(e-commerce).

Infra as a code!

terraform

We are going to use terraform to set our infrastructure! There will be a folder called infra in our project and it will be responsible for deploying our virtual machine and our Kafka.

Crawler

For this one, we will be using cheerio + request(node), to have a stack with a lot of content on the community and also that will not increase so much the size of our packages.

Logging

The famous ELK stack.

CI/CD

circle-ci