/bisheng-rt

bisheng model services backend

Primary LanguageC++Apache License 2.0Apache-2.0

What is bisheng-rt?

bisheng-rt is an open source inference serving framework that power the model inference and resource allocation. bisheng-rt makes the different model efficient deployment and provides a consistent user experience regardless of any model types.

The project is a sub-project of bisheng.

Key features

  • High performance
  • Comaptible with most computing cards (nv, atlas, cambricon, enflame)
  • Friendly model management
  • Easily integrate for new model

Quick start

Start With Bisheng Platform

Use in Bisheng Platform Model Manager

Start with DataElem Services.

We provide a open cloud service for easily use. See free trial.

Install bisheng-rt

Using from pre-builded image

Images Release Notes

Documentation

For guidance on installation, development, deployment, and administration, check out bisheng-rt Dev Docs.

Issues

Reporting problems, asking questions We appreciate any feedback, questions or bug reporting regarding this project.

User can post Issues, follow the process outlined in the Stack Overflow document.

For questions, we recommend posting in our community GitHub Discussions.

Acknowledgments

bisheng-rt adopts dependencies from the following: