tensorflow/serving

Is there a way to graceful shutdown of Tensorflow Serving Server?

sskgit opened this issue · 7 comments

sskgit commented

Tensorflow Serving Server right now has to be stopped by killing the process or logging of the VM. Is there a better way to graceful shutdown of the Serving server?

chrisolston commented

To implement this we can call server->Shutdown():
https://github.com/grpc/grpc/blob/master/include/grpc%2B%2B/impl/codegen/server_interface.h#L83

The question is what would trigger our main.cc to call that? An RPC?

I'm not sure what is the standard way to structure this for open-source servers. Maybe I'll open an issue on the grpc github for suggestions.

chrisolston commented

I posted an inquiry to grpc: grpc/grpc#10223

yupbank commented

how about a SIGTERM or SIGINT ? we are deploying tf/serving from docker, and it would benefit us from being able to handle such signal

  • 👍6
chrisolston commented

If you can send a PR that adds a SIGTERM handler we can review it. Thanks!

ymodak commented

Closing this issue since inquiry to grpc provides the solution.

Deleted user commented

So has a SIGTERM handler been implemented? or is there a separate issue tracking that?

Thanks!

junneyang commented

why close, how to gracefully shutdown the server

  • 👍1