A Golang-based service that serves machine learning model for inference. Access a RESTful endpoint to run an inference on an instance of a YOLOv8 model.
Because of onnxruntime bindings for Go, if you are on Windows, you will need to be using a version of Go with cgo enabled---meaning that you'll need to have gcc available on your PATH. For this I recommend using TDM-GCC-64, since it was the only one that worked out for me.
You also need to provide the shared library file for onnxruntime ("onnxruntime.dll" for Windows, "onnxruntime.so" for Linux, etc). I gave you two standard options to provide them:
- Create an environment variable with the key "ONNXRUNTIME_SHARED_LIBRARY_PATH" pointing to the path where the shared library file is stored. Or...
- Put the shared library file for onnxruntime in a folder that respects the following path: "../third_party/onnxruntime/" (the lib file will be inside this folder), because that is how I implemented the loading of the dynamic library initially (as a second option).
When you build the executable, place the model file (for now, yolov8s.onnx) in the same folder.
To execute the service, run the command:
~$ ./inference-service --ip IP_NUMBER
PS > inference-service.exe --ip IP_ADDRESS
If no ip address is provided, it runs on localhost:8080.
- /inference : get a json with a list of previously ran inferences
-
/detect_objects : send an image and get a json with the descriptions of the bounding boxes with the detected objects inside
Request body "key":value to send: "image" : jpeg or png image file "name" : string with the name of the image (human friendly name or alias)
To contact me, write me an email at almir.filho77@gmail.com.