triton-inference-server/server

Thread safety in channel map

Closed this issue · 0 comments

Is your feature request related to a problem? Please describe.
The channel map for the gRPC client is not thread-safe: https://github.com/NVIDIA/triton-inference-server/blob/466dbdf96518e3a16fb4e39a97e29254affbe25b/src/clients/c%2B%2B/api_v1/library/request_grpc.cc#L49. It is effectively a global variable, though not directly accessible because it's hidden in an anonymous namespace. Therefore, if multiple threads create clients at the same time, data races and corruption are possible.

Describe the solution you'd like
Operations on the channel map should be atomic or thread_local to avoid data races.

Describe alternatives you've considered
Putting locks in user code requires more maintenance and is not foolproof (if unknown operations also interact with the channel map, the user may not know those operations should be locked).

Additional context
Some processes inherently run multithreaded, so it's best not to assume a single-threaded environment.