PaddleSharp 🌟

English | 简体中文

💗 .NET Wrapper for PaddleInference C API, support Windows(x64) 💻, NVIDIA Cuda 10.2+ based GPU 🎮 and Linux(Ubuntu-22.04 x64) 🐧, currently contained following main components:

PaddleOCR 📖 support 14 OCR languages model download on-demand, allow rotated text angle detection, 180 degree text detection, also support table recognition 📊.
PaddleDetection 🎯 support PPYolo detection model and PicoDet model 🏹.
RotationDetection 🔄 use Baidu's official text_image_orientation_infer model to detect text picture's rotation angle(0, 90, 180, 270).
Paddle2Onnx 🔄 Allow user export ONNX model using C#.

NuGet Packages/Docker Images 📦

Release notes 📝

Please checkout this page 📄.

Infrastructure packages 🏗️

NuGet Package 💼	Version 📌	Description 📚
Sdcb.PaddleInference		Paddle Inference C API .NET binding ⚙️

Native packages 🏗️

Package	Version 📌	Description
Sdcb.PaddleInference.runtime.win64.mkl		win64+mkldnn
Sdcb.PaddleInference.runtime.win64.openblas		win64+openblas
Sdcb.PaddleInference.runtime.win64.openblas-noavx		win64+openblas(no AVX, for old CPUs)
Sdcb.PaddleInference.runtime.win64.cuda102_cudnn76_tr72_sm61_75		win64/CUDA 10.2/cuDNN 7.6/TensorRT 7.2/sm61+sm75
Sdcb.PaddleInference.runtime.win64.cuda118_cudnn86_tr85_sm86_89		win64/CUDA 11.8/cuDNN 8.6/TensorRT 8.5/sm86+sm89

Some of packages already deprecated(Version <= 2.5.0):

Package	Version 📌	Description
Sdcb.PaddleInference.runtime.win64.cuda117_cudnn84_tr84_sm86		win64/CUDA 11.7/cuDNN 8.4/TensorRT 8.4/sm86
Sdcb.PaddleInference.runtime.win64.cuda102_cudnn76_sm61_75		win64/CUDA 10.2/cuDNN 7.6/sm61+sm75
Sdcb.PaddleInference.runtime.win64.cuda116_cudnn84_sm86_onnx		win64/CUDA 11.6/cuDNN 8.4/sm86/onnx

Any other packages that starts with Sdcb.PaddleInference.runtime might deprecated.

Baidu packages were downloaded from here: https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#windows

My Sdcb packages were self compiled.

Baidu official GPU packages are too large(>1.5GB) to publish to nuget.org, there is a limitation of 250MB when upload to Github, there is some related issues to this:

PaddlePaddle/Paddle#43874 ❌
NuGet/Home#11706 (comment) ❌

But You're good to build your own GPU nuget package using 01-build-native.linq 🛠️.

Note: Linux does not need a native binding NuGet package like windows(Sdcb.PaddleInference.runtime.win64.mkl), instead, you can/should based from a Dockerfile🐳 to development:

Docker Images 🐳	Version 📌	Description 📚
sdflysha/dotnet6-paddle		PaddleInference 2.5.0, OpenCV 4.7.0, based on official Ubuntu 22.04 .NET 6 Runtime 🌐
sdflysha/dotnet6sdk-paddle		PaddleInference 2.5.0, OpenCV 4.7.0, based on official Ubuntu 22.04 .NET 6 SDK 🌐

You can also build your docker images using these dockerfiles.

Paddle Devices

Mkldnn - PaddleDevice.Mkldnn()

Based on Mkldnn, generally fast
Openblas - PaddleDevice.Openblas()

Based on openblas, slower, but dependencies file smaller and consume lesser memory
Onnx - PaddleDevice.Onnx()

Based on onnxruntime, is also pretty fast and consume less memory
Gpu - PaddleDevice.Gpu()

Much faster but relies on NVIDIA GPU and CUDA

If you wants to use GPU, you should refer to FAQ How to enable GPU? section, CUDA/cuDNN/TensorRT need to be installed manually.
TensorRT - PaddleDevice.Gpu().And(PaddleDevice.TensorRt("shape-info.txt"))

Even faster than raw Gpu but need install TensorRT environment.

Please refer to tensorrt section for more details

FAQ ❓

Why my code runs good in my windows machine, but DllNotFoundException in other machine: 💻

Please ensure the latest Visual C++ Redistributable was installed in Windows (typically it should automatically installed if you have Visual Studio installed) 🛠️ Otherwise, it will fail with the following error (Windows only):
```
DllNotFoundException: Unable to load DLL 'paddle_inference_c' or one of its dependencies (0x8007007E)
```
If it's Unable to load DLL OpenCvSharpExtern.dll or one of its dependencies, then most likely the Media Foundation is not installed in the Windows Server 2012 R2 machine:
Many old CPUs do not support AVX instructions, please ensure your CPU supports AVX, or download the x64-noavx-openblas DLLs and disable Mkldnn: PaddleDevice.Openblas() 🚀
If you're using Win7-x64, and your CPU does support AVX2, then you might also need to extract the following 3 DLLs into C:\Windows\System32 folder to make it run: 💾
- api-ms-win-core-libraryloader-l1-2-0.dll
- api-ms-win-core-processtopology-obsolete-l1-1-0.dll
- API-MS-Win-Eventing-Provider-L1-1-0.dll
You can download these 3 DLLs here: win7-x64-onnxruntime-missing-dlls.zip ⬇️

How to enable GPU? 🎮

Enable GPU support can significantly improve the throughput and lower the CPU usage. 🚀

Steps to use GPU in Windows:

(for Windows) Install the package: Sdcb.PaddleInference.runtime.win64.cuda* instead of Sdcb.PaddleInference.runtime.win64.mkl, do not install both. 📦
Install CUDA from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) 🔧
Install cuDNN from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) 🛠️
Install TensorRT from NVIDIA, and configure environment variables to PATH or LD_LIBRARY_PATH (Linux) ⚙️

You can refer to this blog page for GPU in Windows: 关于PaddleSharp GPU使用常见问题记录 📝

If you're using Linux, you need to compile your own OpenCvSharp4 environment following the docker build scripts and the CUDA/cuDNN/TensorRT configuration tasks. 🐧

After these steps are completed, you can try specifying PaddleDevice.Gpu() in the paddle device configuration parameter, then enjoy the performance boost! 🎉

TensorRT 🚄

To use TensorRT, just specify PaddleDevice.Gpu().And(PaddleDevice.TensorRt("shape-info.txt")) instead of PaddleDevice.Gpu() to make it work. 💡

Please be aware, this shape info text file **.txt is bound to your model. Different models have different shape info, so if you're using a complex model like Sdcb.PaddleOCR, you should use different shapes for different models like this:

using PaddleOcrAll all = new(model,
   PaddleDevice.Gpu().And(PaddleDevice.TensorRt("det.txt")),
   PaddleDevice.Gpu().And(PaddleDevice.TensorRt("cls.txt")),
   PaddleDevice.Gpu().And(PaddleDevice.TensorRt("rec.txt")))
{
   Enable180Classification = true,
   AllowRotateDetection = true,
};

In this case:

DetectionModel will use det.txt 🔍
180DegreeClassificationModel will use cls.txt 🔃
RecognitionModel will use rec.txt 🔡

NOTE 📝:

The first round of TensorRT running will generate a shape info **.txt file in this folder: %AppData%\Sdcb.PaddleInference\TensorRtCache. It will take around 100 seconds to finish TensorRT cache generation. After that, it should be faster than the general GPU. 🚀

In this case, if something strange happens (for example, you mistakenly create the same shape-info.txt file for different models), you can delete this folder to generate TensorRT cache again: %AppData%\Sdcb.PaddleInference\TensorRtCache. 🗑️

Thanks & Sponsors 🙏

崔亮 https://github.com/cuiliang
梁现伟
深圳-钱文松
iNeuOS工业互联网操作系统：http://www.ineuos.net

Contact 📞

QQ group of C#/.NET computer vision technical communication (C#/.NET计算机视觉技术交流群): 579060605

quejuwen/PaddleSharp