TvSeries Scraper
This is a component of the PopCorn Planner application.
This service aims to populate a "tvseries" collection with the information about the Tv Series it's required.
It listens on a queue on RabbitMQ when a message with the name of a Tv Serie is received, it searches for information about that series using the IMDb API. Once the information has been found, it updates a collection on MongoDB.
Example of a message received on RabbitMQ.
{ "name": "Supernatural" }
Example of response from the IMDb API: look at "find-title.example.json" file.
Example of document saved on MongoDB:
{
"serieId":"tt0460681",
"createdAt":"1631644913465",
"numberOfEpisodes": "327",
"title":"Supernatural",
"updatedAt": "1631644913465"
}
Local Testing
Install the dependencies
npm ci
Run the docker images of MongoDB and RabbitMQ
docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.9
docker run -it --rm --name mongo4.4 -p 27017:27017 mongo:4.4
Run postgres
Run the image
docker pull postgres
docker run -d \
--name postgres -e POSTGRES_USER=user \
-e POSTGRES_PASSWORD=password123 \
-p 5432:5432 postgres
Enter into the container
docker exec -it postgres bash
Execute the psql cli inside the container with the superuser defined previously
psql -U user
Create the database
CREATE DATABASE dbtest;
Check that the database has been created
\l
Example:
user=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+------------+------------+-------------------
dbtest | user | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | user | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | user | UTF8 | en_US.utf8 | en_US.utf8 | =c/user +
| | | | | user=CTc/user
template1 | user | UTF8 | en_US.utf8 | en_US.utf8 | =c/user +
| | | | | user=CTc/user
user | user | UTF8 | en_US.utf8 | en_US.utf8 |
Close the cli
\q
Exit the container
exit
Run the tests
npm t