Inspect ML Pipelines in Python in the form of a DAG
Prerequisite: Python 3.9
-
Clone this repository
-
Set up the environment
cd mlinspect
python -m venv venv
source venv/bin/activate
-
If you want to use the visualisation functions we provide, install graphviz which can not be installed via pip
Linux:
apt-get install graphviz
MAC OS:
brew install graphviz
-
Install pip dependencies
pip install -e .[dev]
-
To ensure everything works, you can run the tests (without graphviz, the visualisation test will fail)
python setup.py test
Make it easy to analyze your pipeline and automatically check for common issues.
from mlinspect import PipelineInspector
from mlinspect.inspections import MaterializeFirstOutputRows
from mlinspect.checks import NoBiasIntroducedFor
IPYNB_PATH = ...
inspector_result = PipelineInspector\
.on_pipeline_from_ipynb_file(IPYNB_PATH)\
.add_required_inspection(MaterializeFirstOutputRows(5))\
.add_check(NoBiasIntroducedFor(['race']))\
.execute()
extracted_dag = inspector_result.dag
inspection_to_annotations = inspector_result.inspection_to_annotations
check_to_check_results = inspector_result.check_to_check_results
We prepared a demo notebook to showcase mlinspect and its features.
- For debugging in PyCharm, set the pytest flag
--no-cov
(Link) - This is a research project, so comprehensive coverage of all possible ML APIs will not be possible in the current initial step. We will try to tell you if we encounter APIs we can not handle yet.
This library is licensed under the Apache 2.0 License.