astramind-ai/BitMat

Create a Triton backend

Closed this issue · 3 comments

EwoutH commented

Would it be possible to create a Triton backend from this implementation?

A Triton backend is the implementation that executes a model. A backend can be a wrapper around a deep-learning framework, like PyTorch, TensorFlow, TensorRT or ONNX Runtime. Or a backend can be custom C/C++ logic performing any operation (for example, image pre-processing).

mlinmg commented

I don't have direct experience with Triton-Inference-Server, I'll look into it in the nex days

  • 👍1
michaelfeil commented

@EwoutH I think you confused OpenAI Triton (the Language) with Nvidia Triton (a API server in C++)

  • 👍1
EwoutH commented

Right, from the Readme I didn’t figure that. Thanks for clearing that up!

  • 👍1