How to set up various vtuber productions
- OBS: https://obsproject.com/
- VSeeFace: https://www.vseeface.icu/
- Best for web cam face tracking
- VMagicMirror: https://malaybaku.github.io/VMagicMirror/en/index
- Best desktop mode support
Double check the terms of use and licenses that come with your avatar.
- Purchase
- Create
- https://vroid.com/en/studio
- https://readyplayer.me/ (need to convert to VRM)
- Collect
- Load the VRM into VMagicMirror, check "Load current VRM on next startup"
- Open the Setting Window and go to Layout, check
Free Camera Mode
and move your character around (hold right click + drag / mouse wheel) - Position your avatar directly in the middle of the screen surrounded by green, then uncheck
Free Camera Mode
when done
Tips
If you want to record your avatar standing somewhere, position it to capture the entire body. If you want to be sitting behind a desk or podium, capture half the body.
- Check
Free Layout
on Device and drag the arrows to position the mouse / keyboard for your character
- If its on, go to Effects and disable shadows
- Make sure the right microphone is set by going to the Face tab and looking at LipSync
VSeeFace works great with a webcam for face tracking while vtubing. Many great tutorials are listed here: https://www.vseeface.icu/#tutorials
When you're finished configuring, click this tiny x in the corner to hide the UI:
Note: You can load into OBS as a transparent video, no need for green screen filter.
- Create a new scene, add a source for window capture with VMagicMirror selected.
- Make sure the green covers the entire window without clipping the character
- Make sure your microphone is picking up in the Audio Mixer
- Go to Settings, Output, and Recording to see where videos get saved
When your OBS looks like this, you are ready to record. Hop into a Discord call and start talking.
If using screenshare to present while vtubing, you can setup OBS like the picture below.
There's an overlay template included in this repo that you can add as an OBS source to make life easier. Feel free to modify it using the Gimp save file.
ffmpeg command to split 1080p recording with background overlay into 2 videos:
ffmpeg -i test.mp4 -filter_complex "[0:v]crop=540:520:0:145[face]" -map "[face]" -map 0:a face.mp4 -filter_complex "[0:v]crop=1377:780:542:135[pres]" -map "[pres]" -map 0:a pres.mp4
Example ffmpeg command to do the conversion to alpha webm:
ffmpeg -i face.mp4 -vf "chromakey=01d800:0.1:0.2" -c:v libvpx -b:v 1M -auto-alt-ref 0 face.webm
Github: https://github.com/madjin/vr-conf
During post editing you'll be able to crop the window and the vtuber avatar into separate videos or UV map them in such a way to spatialize in a 3D scene.
This can work with real life conference videos too if the background behind the speaker can be keyed out. Either use AI removal or plan ahead with a plain solid color background behind the speaker area.
- VSeeFace setup
- Avatar + Screen share recording setup