The idea: Extensible and parametrized inference framework to use with mlserver (KServe compliant webservers).
The user configures a source and a destination using env-vars, akin to the way that for example Kafka Connect sources data from one place and inserts into another place.
Currently geared towards synchronous batch inference, but an asynchronous Kafka-based service can easily be implemented on top of this.
from pathlib import Path
from mlserver_inference_pipeline.predict import MlserverPredictor
from mlserver_inference_pipeline.extractors.random import RandomFeatureExtractor
from mlserver_inference_pipeline.destinations.csv import CsvPredictionDestination
MlserverPredictor(
"http://my.model.example.com",
"mlflow-model",
RandomFeatureExtractor(n_features=10),
CsvPredictionDestination(outpath=Path("file.csv")),
).predict()
Very much a WIP or PoC project.
Connectors that get features from a datastore are referred to as Extractors. An extractor must
implement AbstractFeatureExtractor
.
Connectors that insert predictions into a datastore are referred to as Destinations. A destination
must implement AbstractPredictionDestination
.
poetry install
poetry run pytest
- CLI
- Docker Image
- Airflow DAG