Can this plugin remove the background of a game card instead of a human portrait?
Wells83 opened this issue · 18 comments
Hi everybdoy. Its possible to use this plugin for create a virtual background on a square, for exemple a card game ?
@Wells83 Firstly, could you please provide a more detailed description of your desired use case along with some screenshots? The information you provide will help us in providing a more accurate answer.
Yes, of course!
First of all, excuse my English, I'm French :)
Here's what I want to do, for my youtube channel, I want to inlay game cards into my main scene.
Initially, my camera will film my cards like this:
I then add green borders and chrominance overlay:
Unfortunately, in doing so, I lose the green in my cards :(
So I'd like to create a single-color virtual background that detects the outline of the cards, so that I can inlay them perfectly without losing the green they contain.
Importantly, when I don't have a card to show, the scene must be totally transparent.
Without card :
With card :
Last but not least, I obviously have several card sizes, orientations and formats:
To sum up, the objective is :
@Wells83 I understand your needs as you want to remove the background of game cards based on AI. Our matt model is based on human portraits so that it cannot recognize your game cards. Is my understanding correct?
It makes a lot.of sense
There are "segment anything" models and background removal model that are not for humans necessarily but for any object in the foreground, which I think would work for this pretty good.
Like Bria RMBG https://huggingface.co/briaai/RMBG-1.4
Which has onnx already and is quantized which means it may run close to realtime
Yes I don't think it's very hard to implement, honestly since we have everything set up in the plugin already, like onnxruntime and the image & models processing utility
The milestone is set to 1.1.11
Thinking about it, I wonder if the best thing wouldn't be for it to detect the green zone and replace it with a virtual background.
That way, if there's no game card, we're fine and if there is a card, of course it has to detect the outline of the card.
I don't know if it's the same on your side or if it's more complicated?
I suppose detecting the green zone and detecting the object are almost the same things. Just the matt is inverted and it won't make any difference in the implementaion.
I receive a notification when thé New version IS done ?
We will close this issue when we have done the implementation. We cannot promise that your suggestion will be implemented soon but stay tuned! 😄
implemented.
Ok i test that tomorow ans i give you feedback
If its the good parameter, its not working :(
@Wells83 You need the RMBG model
It is only available in the 1.1.12 version. You may need to upgrade
yes its the last version i use