[P0] Create CAIKit-nlp community images/CI for Open Data Hub
heyselbi opened this issue · 7 comments
@vaibhavjainwiz @xianli123 I think we don't need image/ci for caikit-nlp because we will use caikit-nlp from pypi library. Am I right?
Hi, @Jooho I am sorry, but I don't have the answer. I am a UX designer and not so familiar with the question. We may need to wait for @vaibhavjainwiz 's reply.
@Jooho I have same understanding we don't need image/ci for caikit-nlp. @danielezonca @Xaenalt Could you please confirm.
We need to decide how we prefer to proceed here: caikit-nlp is a python project so we can publish it to pypi but at the same time we need to build the actual runtime image with the entry point.
At the moment @Xaenalt created a custom Dockerfile that has a simple start-serving.sh entry point
We can keep the image creation part in another repo but I think it is easier to add the dockerfile + scripts (if necessary) directly in the caikit-nlp (Note: there is this entry-point script already in the repo that can be used)
Long story short: I expect we will publish caikit-nlp to pypi but we will also create the image to be used for the serving. I think we can focus directly on the image part
@Xaenalt Have you already discussed with caikit team if/when they plan to publish caikit-nlp to pypi?
We do need a CI image for the caikit runtime, which includes caikit-nlp
Separately, caikit-nlp will eventually be pushed to pypi (there are some ongoing efforts around this work, check the IBM slack)
As for the entrypoint script, I only took the smaller part for two reasons: one is that it was a test image, and two is that they are actively refactoring the handling of connections, caikit/caikit#268 is the issue for that. Based on that work, the start script will need some reworking, for now it's just an example of how to launch it in a minimal configuration
So, current working images that we use for KServe testing are:
TGIS: quay.io/opendatahub/text-generation-inference@sha256:04e3a0d84b17614853990955a6372b7647ad66ee663fd5d162fae406efadd42b
Caikit-nlp: quay.io/opendatahub/caikit-tgis-serving@sha256:20cbc7b312e0dc63903d4a2c7d70b741135ffea31113af63dd8691da4baae0f9