/SAM-ONNX-AX650-CPP

SAM and lama inpaint，包含QT的GUI交互界面，实现了交互式可实时显示结果的画点、画框进行SAM，然后通过进行Inpaint，具体操作看readme里的视频。

Primary LanguageC++

Inpaint Anything

sam_inpaint.mp4

Build

mkdir build
cd build

if x86 onnxruntime

cmake -DONNXRUNTIME_DIR=${onnxruntime_dir} -DOpenCV_DIR=${opencv_cmake_file_dir} ..

else if ax650

cmake -DONNXRUNTIME_DIR=${onnxruntime_dir} -DOpenCV_DIR=${opencv_cmake_file_dir} -DBSP_MSP_DIR=${msp_out_dir} -DBUILD_WITH_AX650=ON ..
make -j4

Build with QT

cd qtproj
mkdir build

build for x86 (for example qt5.14.2,ubuntu 20.04,you need to download opencv and onnxruntime)

cmake -DONNXRUNTIME_DIR=${onnxruntime_dir} -DOpenCV_DIR=${opencv_cmake_file_dir} -DQt5_DIR=${qt5_dir}/5.14.2/gcc_64/lib/cmake/Qt5  ../SAMQT/

build for AX650 on board(爱芯派pro，you need to download opencv onnxruntime in this repos releases)

apt install cmake qt6-base-dev qtcreator
cmake -DONNXRUNTIME_DIR=${onnxruntime_dir} -DOpenCV_DIR=${opencv_cmake_file_dir} -DBSP_MSP_DIR=${msp_out_dir} -DBUILD_WITH_AX650=ON ../SAMQT/
make -j4

Usage

Segment Anything

/opt/test/sam # ./main -e ax_models/sam-encoder.axmodel -d ax_models/sam_vit_b_01ec64_decoder.onnx -i test.jpg
Engine creating handle is done.
Engine creating context is done.
Engine get io info is done.
Engine alloc io is done.
[I][                            init][ 233]: BGR MODEL
Encoder Inference Cost time : 0.737574s
Decoder Inference Cost time : 0.448962s
0.96
0.95
0.96
0.86
Decoder Inference Cost time : 0.438249s
0.98
1.00
0.98
0.91

Inpaint

/opt/test/sam # ./main_inpaint -m big-lama-regular.axmodel  -i test.jpg --mask result_1
.jpg
image 1920x1282
mask 1920x1282
Engine creating handle is done.
Engine creating context is done.
Engine get io info is done.
Engine alloc io is done.
[I][                            init][ 275]: RGB MODEL
Inpaint Inference Cost time : 0.582908s

Reference

segment-anything
MobileSAM
lama
SegmentAnything-OnnxRunner