zeruniverse/fast-artistic-videos

I've tested this repo with a faster optical flow and it works!

chrisuehlinger opened this issue · 15 comments

chrisuehlinger commented

Not sure if you're still following up on this project, but I figured I'd note this for you just in case.

Going off of the advice in your README (that we don't necessarily need deepflow, just optical flow), I found another project that is easy to build (it has 2 deps that can be installed with apt-get) and that produces .flo files that work with your Torch scripts. The project can be found at https://github.com/tikroeger/OF_DIS and the executable that works here is run_OF_RGB.

I used a 1 second 360p video to benchmark the new flow calculator (on an EC2 P2 instance). I saw an improvement from 15 minutes using deepflow to a couple seconds using run_OF_RGB. This is a real game changer for me in terms of being able to produce videos more easily, and I just wanted to share it with a project that has helped me out a lot. Cheers!

zeruniverse commented
  • 👍1
chrisuehlinger commented

I'm going to try a couple videos first just to make sure it meets a similar standard of quality to deepflow, but if it does, I'll make a PR sometime in the next week with installation instructions.

zeruniverse commented
zeruniverse commented

Any updates on this?

luismqueral commented

+1 @chrisuehlinger would love to connect if you already have this working!

zeruniverse commented
zeruniverse commented

No reply for a long time. If anyone figured it out, please submit a PR

schwittlick commented

Hey @zeruniverse

I've had a go at this and it indeed works! And the other optical flow is by magnitudes faster. On the downside, it's not as stable as deepflow. I'll have a look whether I can parametrize it a bit, but wanted to share some progress here. Compiling and using run_OF_RGB (https://github.com/tikroeger/OF_DIS) is actually very simple.

Here's the diff for run-deepflow.sh: schwittlick@1a81b07

Can you give me a hint what the deepmatching-static is exactly doing in this workflow? (I've removed it, cause it's very slow, too)

Here's two samples (better download the file and watch, vimeo compression artifacts might make the difference hard to spot):
deepflow: https://vimeo.com/246746437
run_OF_RGB: https://vimeo.com/246746400

My run_OF_RGB binary is dynamically linked at the moment, so I'll see to build it statically, so we could eventually add it to this repo?

Thanks for this, by the way, I'm using this for some video works and love it!

Marcel

schwittlick commented

PS: I've added some parameters to run_OF_RGB: schwittlick@32d0f81

This results in much better optical flow quality and looks the same like deepflow. Amazing!

zeruniverse commented
schwittlick commented

Unfortunately I'm not able to compile OF_DIS as a static executable, so it will not find the shared libraries on other people's systems...

For example

libopencv_highgui.so.2.4 => /usr/local/lib/libopencv_highgui.so.2.4 (0x00007fb06ff09000)
libopencv_imgproc.so.2.4 => /usr/local/lib/libopencv_imgproc.so.2.4 (0x00007fb06f9fa000)
libopencv_core.so.2.4 => /usr/local/lib/libopencv_core.so.2.4 (0x00007fb06f40b000)
libcudart.so.8.0 => /usr/local/cuda/lib64/libcudart.so.8.0 (0x00007fb06951a000)
libz.so.1 => /mnt/drive1/tools/anaconda2/lib/libz.so.1 (0x00007fb069304000)

Not sure it's a good idea to merge this.. Any suggestion?

zeruniverse commented
schwittlick commented

Hmm.. I doubt that's a good idea. But I can't say for sure. I wonder why a cuda dependency is in there anyhow.. I will do some tests and report back. As an alternative we could write up a little tutorial in the readme and that's it.-

zeruniverse commented
schwittlick commented

Interestingly, the dynamic executable worked out of the box on my other computer. pr'ing ;)