fastmachinelearning/SonicCMS

Add shared memory options

Closed this issue · 1 comments

kpedro88 commented

Using shared memory for a local GPU is expected to be faster than using gRPC to communicate. There is an example here: https://github.com/triton-inference-server/server/blob/v2.3.0/src/clients/c++/examples/simple_grpc_shm_client.cc. This could be an option in TritonClient/TritonData.

Assigned to: @yongbinfeng