/etl-example-in-python

A Data pipeline example (MySQL to MongoDB), used with MovieLens Dataset.

Primary LanguagePython

The application runs on python 3.5

About

This is a command line application, It takes a MySQL dataset that is provided by https://grouplens.org/datasets/movielens/.

Prerequisties

Make sure the following conditions are met:

  • MovieLens Database is loaded into MySQL https://grouplens.org/datasets/movielens/
  • A MySQL server is running and has MovieLens database ready
  • A MongoDB server is running
  • Make sure you configure config.py with appropirate variables
  • Install the dependencies
pip install -r requirements.txt

How to run

To run, simply type

python3 pipeline.py

Author

Balraj Singh Bains