miliar/tech-stack-demand

An example microservice pipeline to visualize the tech stack demand on the current job market

Python

Tech stack demand

An example microservice pipeline to visualize the tech stack demand on the current job market.

Show who is using certain technologies:

Search for company:

Installation

Install docker/docker-compose
Assure docker has enough resources (Preferences > Advanced; tested with 4 CPUs and 6 GiB RAM)
Run docker-compose up
Open in your browser http://localhost:5000/
Collect new data (reload page to see progress; this can take up to 20 min)

How to use the UI

Check out this basic action wiki

Technical sidenotes

Architecture

Browse kafka topics:

Kafka logs are not mounted
While collecting new data, check http://localhost:9000/

Manage Neo4j data:

Data mounted to /neo4j_data_loader/neo4j_data
Open Neo4j browser: http://localhost:7474/
Login with user: neo4j password: password

Run tests:

docker-compose -f docker-compose-tests.yml up

Run keywords api update:

Delete old data saved in keywords_api/redis_data
Run: docker-compose -f docker-compose-keywords-update.yml up