greyhub/job_center

Crawl, matching and explore data about jobs in Viet Nam.

Jupyter Notebook

Job Center

@team:  Luong (greyhub)
        Nam (Namdv99)
        Minh (MinhPN101)
        Manh (htlmm99)

Main workflow

Structure

.
├── app
│   ├── README.md
│   └── website
├── database
│   ├── data_center
│   ├── raw_data
│   └── temp_storage
├── README.md
└── workflow
    ├── config.py
    ├── crawler
    ├── exploratory
    ├── __init__.py
    ├── matching
    ├── pipeline.py
    ├── __pycache__
    ├── README.md
    ├── requirements.txt
    ├── scrapy.cfg
    ├── test_scrapy
    ├── timer.py
    └── utils.py

Requirements

Conda
Python
Scrapy

Guideline

Run pipeline & more

Run exploratory & more

Job domains

We focus on bellow domains:

References

Crawl with Scrapy

Schedule in python

Matplotlib: Visualization with Python