Check out the latest releases for downloads and install instructions.
This plugin is meant to make it easy to replace the background in portrait images and video. It is using a neural network to predict the mask of the portrait and remove the background pixels. It's easily composable with other OBS plugins to replace the background with e.g. an image or a transparent color.
The models used for background detection are SINet: https://arxiv.org/abs/1911.09099 and MODNet: https://arxiv.org/pdf/2011.11961.pdf The pre-trained model weights were taken from:
- https://github.com/anilsathyan7/Portrait-Segmentation/tree/master/SINet
- https://github.com/ZHKKKe/MODNet
Some more information about how I built it: https://www.morethantechnical.com/2021/04/15/obs-plugin-for-portrait-background-removal-with-onnx-sinet-model/
The plugin was built and tested on Mac OSX, Windows and Ubuntu Linux. Help is appreciated in building on other OSs and formalizing the one-click installers.
- OpenCV v4.2+: https://github.com/opencv/opencv/
- ONNXRuntime: https://github.com/microsoft/onnxruntime
You may use homebrew:
$ brew install opencv onnxruntime
Or - you may also build a (very minimal) version of OpenCV and ONNX Runtime for static-linking, instead of the homebrew ones:
<root>/build/ $ ../scripts/makeOpenCV_osx.sh
<root>/build/ $ ../scripts/makeOnnxruntime_osx.sh
Static linking should be more robust across versions of OSX, as well as building for 10.13.
If you install the desktop OBS app (https://obsproject.com/download) you already have the binaries
for libobs (e.g. /Applications/OBS.app/Contents/Frameworks/libobs.0.dylib)
But you don't have the headers - so clone the main obs repo e.g. git clone --single-branch -b 27.0.1 git@github.com:obsproject/obs-studio.git (match the version number to your OBS install. Right now on OSX it's 27.0.1)
$ mkdir build && cd build
$ cmake .. -DobsLibPath=/Applications/OBS.app/Contents/Frameworks -DobsIncludePath=~/Downloads/obs-studio/libobs
$ cmake --build . --target dist
$ cpack
Unpack the package to the plugins directory of the system's Library folder (which is Apple's preffered way)
$ unzip -o obs-backgroundremoval-macosx.zip -d "/Library/Application Support/obs-studio/plugins"
or directly to your OBS install directory, e.g.
$ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/
The first is recommended as it preserves the plugins over the parallel installation of OBS versions (i.e. running the latest productive version and a release candidate) whereas the latter will also remove the plugin if you decide to delete the OBS application.
$ apt install -y libobs-dev libopencv-dev language-pack-en wget git build-essential cmake
$ wget https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-1.7.0.tgz
$ tar xzvf onnxruntime-linux-x64-1.7.0.tgz --strip-components=1 -C /usr/local/ --wildcards "*/include/*" "*/lib*/"
Then build and install:
$ mkdir build && cd build
$ cmake .. && cmake --build . && cmake --install .
A PKGBUILD file is provided for making the plugin package
$ cd scripts
$ makepkg -s
Building for Arch in Docker (host OS e.g. MacOSX):
$ docker pull archlinux:latest
$ docker run -it -v $(pwd):/src archlinux:latest /bin/bash
# pacman -Sy --needed --noconfirm sudo fakeroot binutils gcc make
# useradd builduser -m
# passwd -d builduser
# printf 'builduser ALL=(ALL) ALL\n' | tee -a /etc/sudoers
# sudo -u builduser bash -c 'cd /src/scripts && makepkg -s'
We will use static linking (as much as possible) to aviod having to lug around .DLLs with the plugin.
Install OpenCV via vcpkg:
$ mkdir build
$ cd build
$ git clone https://github.com/microsoft/vcpkg
$ cd vcpkg
$ .\bootstrap-vcpkg.bat
$ .\vcpkg.exe install opencv[core]:x64-windows-static
Install Onnxruntime with NuGet:
$ cd build
$ mkdir nuget
$ Invoke-WebRequest https://dist.nuget.org/win-x86-commandline/latest/nuget.exe -UseBasicParsing -OutFile nuget.exe
$ nuget.exe install Microsoft.ML.OnnxRuntime.DirectML -Version 1.7.0
$ nuget.exe install Microsoft.ML.OnnxRuntime.Gpu -Version 1.7.1
Clone the OBS repo, Downloads\ $ git clone --single-branch -b 27.0.1 git@github.com:obsproject/obs-studio.git, to e.g. Downloads.
$ cmake .. -DobsPath="$HOME\Downloads\obs-studio\"
$ cmake --build . --config Release
$ cpack
$ Expand-Archive .\obs-backgroundremoval-win64.zip -DestinationPath 'C:\Program Files\obs-studio\' -Force
To build with CUDA support, tell cmake to use the CUDA version of OnnxRuntime
$ cmake .. -DobsPath="$HOME\Downloads\obs-studio\" -DWITH_CUDA=ON
The rest of the build process is similar, but the result archive will be
obs-backgroundremoval-win64-cuda.zip.