BrokenSource/DepthFlow

(REQUEST) Ability to dump frames instead of a video

Closed this issue · 4 comments

agoralski commented

Would it be possible to dump frames to individual files instead of a video container?

Tremeschin commented

You can technically inherit your own class like here or on the DepthScene class's .update method, and use:

def update(self):
    frame = self.read_screen()
    image = PIL.Image.frombytes("RGB", self.resolution, frame)
    image = image.transpose(PIL.Image.FLIP_TOP_BOTTOM)

Similar mechanism for taking screenshots on shaderflow !

If you want a numpy array which will be faster than using PIL Image, something like the following could work:

numpy.frombuffer(frame, dtype=numpy.uint8).reshape((*self.resolution, -1))

I can try to do some trickery later for you if you want the raw frames being sent to ffmpeg to use the "render" command on the scene's main command, I'm curious what would be the use case for this though 🤔

Alternatively wait for some if self.time >= 10: self.quit(True) on the update method to stop writing frames this way

  • ❤️1
agoralski commented

Awesome, thanks for your help! I don't need the frames to be sent to ffmpeg in my case.

Using depthflow to generate different views on the object for training a model for object recognition. It helps me build a larger image dataset from slightly different views :)

  • 👍1
Tremeschin commented

Oh, interesting, yea, that way I've talked about should do it for your case, set random or custom/known parameters on a .update, grab the screen contents, move on infinitely until you reach a certain self.frame or self.time !

Be sure to use DepthAnythingV2 on the self.estimator property! it's much better :)

Tremeschin commented

Just adding to your use case, maybe a headless window can go handy for you with WINDOW_BACKEND=headless or initializing your inherited class with YourScene(backend="headless"), so there's not a chance to mess up window resizes / be limited by the monitor size

  • 👍1