'Service Unavailable' with REST API Model Serving

[thinkall]

Hi team,

I just tried the example in https://github.com/bentoml/gallery/blob/master/paddlehub/image-segmentation/image-segmentation.ipynb , only I ran the code in Pycharm Python Console (not jupyterbook).

Everything went fine till

# Start a dev model server to test out everything
bento_svc.start_dev_server()

I could get correct results in postman, the code was like

curl --location --request POST 'http://0.0.0.0:5000/predict' \
--header 'Content-Type: image/jpeg' \
--data-binary '@test_image.jpg'

But the code bento_svc.stop_dev_server() made Pycharm quit. Which was not expected. It should only stop bento_svc.

Another issue is that after saving the service with saved_path = bento_svc.save(), I could lauch API model serving with bentoml serve PaddleHubService:latest, but when I sent request with postman, it returned Service Unavailabel. However, in the meantime, bentoml run PaddleHubService:latest predict --input-file test_image.jpg returned the correct results.

One more thing, it looks like you are using original Flask as the rest api server, is it a good choice for production services?

Thanks a lot！

[yubozhao]

Hi @thinkall Thank you for reporting it. I am going to look into this issue and will get back to you.

[parano]

Update: start_dev_server was an experimental API and has been replaced by the new APIs provided in BentoML 1.0, namely the new bentoml serve command can be used to load a bentoml service definition python file directly and spin up a development server, which also supports auto-reload when code is changed.