/onnxruntime-cuda-cpp-example

Examples for inference models with ONNXRuntime and CUDA

Primary LanguageDockerfileMIT LicenseMIT

onnxruntime-cuda-cpp-example

I am still working on this...

Will it be finished?

Examples

  • Image Classification
    • single_image_model_inference
    • fixed_batch_image_model_inference
    • dynamic_batch_image_model_inference
  • Object Detection
    • dynamic_batch_yolov8
    • dynamic_batch_yolov8_tensorrt
    • dynamic_batch_yolov8_with_cuda_based_opencv_image_preprocessing
    • dynamic_batch_yolov8_single_video_single_frame_inference
    • dynamic_batch_yolov8_single_video_single_frame_inference_tensorrt
    • dynamic_batch_yolov8_single_video_multiple_frames_inference
    • dynamic_batch_yolov8_single_video_multiple_frames_inference_tensorrt
    • dynamic_batch_yolov8_multiple_videos_inference_per_video(not yet implemented)
    • dynamic_batch_yolov8_multiple_videos_parallel_inference_with_queue(not yet implemented)
  • Utilities
    • parse_metadata
    • cuda_based_opencv_image_preprocessing

Setup

git clone https://github.com/developer0hye/onnxruntime-cuda-cpp-example
cd onnxruntime-cuda-cpp-example
docker build . -t onnxruntime-cuda:1.0
sudo docker run -it --runtime=nvidia --gpus all onnxruntime-cuda:1.0

Run

(on a container)

./{example_you_want_to_run}