Integration of ROCK images into Seldon Core Operator
Opened this issue · 4 comments
Description
Integration of ROCK images.
https://github.com/canonical/seldonio-rocks
The following ROCK images need to be integrated into Seldon Core Operator:
Charm managed:
- seldonio-rocks/seldon-core-operator: Rock is done, integration tests are done, integrated into CKF
Workload managed:
- seldonio-rocks/sklearnserver: Rock is done, integration tests are done, integrated into CKF
- seldonio/mlserver-sklearn
- seldonio/mlserver-xgboost
- seldonio/mlserver-mlflow
- seldonio/mlserver-huggingface
seldonio/mlserver-slim- seldonio/mlflowserver
seldonio-rocks/xgboostserver v1- seldonio-rocks/xgboostserver v2
- seldonio-rocks/seldon-core-s2i-python3
seldonio-rocks/seldon-core-s2i-python36 (could be drop candidate)seldonio-rocks/seldon-core-s2i-python37 (could be drop candidate)- seldonio-rocks/seldon-core-s2i-python38
seldonio-rocks/seldon-core-s2i-python36-gpu (could be drop candidate)seldonio-rocks/seldon-core-s2i-python37-gpu (could be drop candidate)- seldonio-rocks/seldon-core-s2i-python38-gpu
- seldonio-rocks/tensorflow/serving
- seldonio-rocks/seldonio/tfserving-proxy
Design
Design of how ROCKs are built, tested and integrated is captured in related specification (KF-044).
Main design points for Seldon ROCKs:
- metadata.yaml is updated with references to new ROCK image.
- Seldon Core Operator ConfigMap updated with new ROCK images.
Testing
Existing integration tests are to be re-used to test functionality of new ROCK images.
Follow Seldon documenation for testing:
https://docs.seldon.io/projects/seldon-core/en/latest/reference/apis/index.html
https://docs.seldon.io/projects/seldon-core/en/latest/nav/config/servers.html
Integration tests
Implementation
- Create ROCK images:
- seldonio-rocks/seldon-core-operator
- SKLEARN_SERVER: seldonio-rocks/sklearnserver (seldon)
- SKLEARN_SERVER-V2: seldonio-rocks/mlserver-sklearn (v2) PR
-
XGBOOST_SERVER: seldonio-rocks/xgboostserver (seldon) - XGBOOST_SERVER-V2: seldonio-rocks/mlserver-xgboost (v2)
- MLFLOW_SERVER: seldonio-rocks/mlflowserver (seldon) PR
- MLFLOW_SERVER-V2: seldonio-rocks/mlserver-mlflow (v2) PR
-
TEMPO_SERVER-V2: seldonio/mlserver-slim - HUGGINGFACE_SERVER-V2: seldonio/mlserver-huggingface PR
- seldonio-rocks/seldon-core-s2i-python3
- seldonio-rocks/seldon-core-s2i-python38
- seldonio-rocks/seldon-core-s2i-python38-gpu
- seldonio-rocks/tensorflow/serving
- seldonio-rocks/seldonio/tfserving-proxy
- Create integration tests to test each image (charm and workload managed).
- seldonio-rocks/seldon-core-operator
- SKLEARN_SERVER: seldonio-rocks/sklearnserver (seldon)
- SKLEARN_SERVER-V2: seldonio-rocks/mlserver-sklearn (v2) PR
-
XGBOOST_SERVER: seldonio-rocks/xgboostserver (seldon) - XGBOOST_SERVER-V2: seldonio-rocks/mlserver-xgboost (v2)
- MLFLOW_SERVER: seldonio-rocks/mlflowserver (seldon)
- MLFLOW_SERVER-V2: seldonio-rocks/mlserver-mlflow (v2) PR
-
TEMPO_SERVER-V2: seldonio/mlserver-slim - HUGGINGFACE_SERVER-V2: seldonio/mlserver-huggingface (no test container)
- seldonio-rocks/seldon-core-s2i-python3
- seldonio-rocks/seldon-core-s2i-python38
- seldonio-rocks/seldon-core-s2i-python38-gpu
- seldonio-rocks/tensorflow/serving
- seldonio-rocks/seldonio/tfserving-proxy
- Update required charm's resources to include/deploy ROCK images.
- seldonio-rocks/seldon-core-operator
- SKLEARN_SERVER: seldonio-rocks/sklearnserver (seldon)
- SKLEARN_SERVER-V2: seldonio-rocks/mlserver-sklearn (v2)
-
XGBOOST_SERVER: seldonio-rocks/xgboostserver (seldon) - XGBOOST_SERVER-V2: seldonio-rocks/mlserver-xgboost (v2)
- MLFLOW_SERVER: seldonio-rocks/mlflowserver (seldon)
- MLFLOW_SERVER-V2: seldonio-rocks/mlserver-mlflow (v2) (broken model in test container)
-
TEMPO_SERVER-V2: seldonio/mlserver-slim - HUGGINGFACE_SERVER-V2: seldonio/mlserver-huggingface (no test container)
- seldonio-rocks/seldon-core-s2i-python3
- seldonio-rocks/seldon-core-s2i-python38
- seldonio-rocks/seldon-core-s2i-python38-gpu
- seldonio-rocks/tensorflow/serving
- seldonio-rocks/seldonio/tfserving-proxy
- Publish ROCK images.
- seldonio-rocks/seldon-core-operator
- SKLEARN_SERVER: seldonio-rocks/sklearnserver (seldon)
- SKLEARN_SERVER-V2: seldonio-rocks/mlserver-sklearn (v2)
-
XGBOOST_SERVER: seldonio-rocks/xgboostserver (seldon) - XGBOOST_SERVER-V2: seldonio-rocks/mlserver-xgboost (v2)
- MLFLOW_SERVER: seldonio-rocks/mlflowserver (seldon)
- MLFLOW_SERVER-V2: seldonio-rocks/mlserver-mlflow (v2) (broken model in test container)
-
TEMPO_SERVER-V2: seldonio/mlserver-slim - HUGGINGFACE_SERVER-V2: seldonio/mlserver-huggingface (no test container)
- seldonio-rocks/seldon-core-s2i-python3
- seldonio-rocks/seldon-core-s2i-python38
- seldonio-rocks/seldon-core-s2i-python38-gpu
- seldonio-rocks/tensorflow/serving
- seldonio-rocks/seldonio/tfserving-proxy
Recording status of s2i.
Updated ROCK:
https://github.com/canonical/seldonio-rocks/blob/feat-s2i-rock/seldon-core-s2i-python3.8/rockcraft.yaml
Use existing example to buid. When using the ROCK container it fails. Evenlocation of those assemble/run scripts is specified with --image-scripts-url, it still refuses to run:
$ git clone https://github.com/seldonio/seldon-core.git
$ cd seldon-core
$ s2i build --image-scripts-url "image:///s2i/bin" wrappers/s2i/python/test/model-template-app charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1 test-image
WARNING: could not inspect the builder image for labels: reading manifest v1.16.0_22.04_1 in docker.io/charmedkubeflow/seldon-core-s2i: requested access to the resource is denied
warning: Image sha256:9d65f600b06a0a916b70a3d48f16d1b4197549a63e7cc5f5b9014ce26228ad8d does not contain a value for the io.openshift.s2i.scripts-url label
Build failed
ERROR: An error occurred: failed to install [assemble run]
ERROR: Suggested solution: set the scripts URL parameter with the location of the S2I scripts, or check if the image has the "io.openshift.s2i.scripts-url" label set
ERROR: If the problem persists consult the docs at https://github.com/openshift/source-to-image/tree/master/docs. Eventually reach us on freenode #openshift or file an issue at https://github.com/openshift/source-to-image/issues providing us with a log from your build using log output level 3.
And those script do exist in the ROCK container:
$ docker run charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1 exec ls -la /s2i/bin
drwxr-xr-x 2 root root 4096 Jul 14 15:11 .
drwxr-xr-x 3 root root 4096 Jul 14 15:11 ..
-rwxr-xr-x 1 root root 2399 Jul 14 15:10 assemble
-rwxr-xr-x 1 root root 1495 Jul 14 15:10 run
-rwxr-xr-x 1 root root 406 Jul 14 15:10 save-artifacts
-rwxr-xr-x 1 root root 1867 Jul 14 15:10 usage
Add label to existin container to enable s2i to find scripts:
echo "FROM charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1" | docker build --label io.openshift.s2i.scripts-url="image:///s2i/bin" -t "charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1" -
Problem: openshift/source-to-image#475
Update ROCK:
echo '''FROM charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1
ENTRYPOINT []''' | docker build --label io.openshift.s2i.scripts-url="image:///s2i/bin" -t "charmedkubeflow/seldon-core-s2i:v1.16.0_22.04_1" -
It is possible to build S2I base container image using rockcraft.yaml as a source, however it will require some adjustmetns and basically a rebuild using Docker.
The process is as follows:
- Build ROCK using
rockcraftbased onrockcaft.yaml - Copy ROCK to Docker daemon using
skopeo - Update label and entrypoint in the image and rebuild
This will create a container image suitable to be used in S2I build process describe in Seldon documentation: https://docs.seldon.io/projects/seldon-core/en/latest/python/python_wrapping_s2i.html
This method does not really produce an S2I base ROCK image based on rockcraft.yaml, but rather Dockerfile based container image.