Update SonicTriton to Triton 2.X and v2 API
Closed this issue · 1 comments
kpedro88 commented
The triton-inference-server 2.X release series includes resolutions of several relevant issues:
- triton-inference-server/server#1425: Support client-side deadlines/timeouts for AsyncRun callback
- triton-inference-server/server#1426: Support https connections
- triton-inference-server/server#1427: Thread safety in channel map
- triton-inference-server/server#1428: Don't require curl CMake config
- triton-inference-server/server#1429 (partial): Improve RapidJSON support in build
It also changes the client API from v1 to v2. Currently, Triton 2.X is described as "beta quality", with indications that APIs might change. Therefore, we are currently waiting to observe the stability of the API over multiple releases before upgrading.
This update will required simultaneous changes in cms-sw/cmsdist and cms-sw/cmssw, as well as any currently operating servers.
Assigned to: @kpedro88