/data-science-things

Primary LanguageJupyter Notebook

Personal Things

This project is for personal investigations if you find any use in this, excellent!.

Set up the environment

  1. Install Poetry

  2. Set up the environment:

make setup
make activate

Install new packages

To install new PyPI packages, run:

poetry add <package-name>

Run Python scripts

To run the Python scripts to process data, train model, and run a notebook, type the following:

make pipeline

View all flow runs

A flow is the basis of all Prefect workflows.

To view your flow runs from a UI, sign in to your Prefect Cloud account or spin up a Prefect Orion server on your local machine:

prefect orion start

Open the URL http://127.0.0.1:4200/, and you should see the Prefect UI:

Run flows from the UI

After creating a deployment, you can run a flow from the UI with default parameters:

API documentation

To auto-generate API document for your project, run:

make docs_save

Run tests when creating a PR

When creating a PR, the tests in your tests folder will automatically run.