Arbitrary timestep interpolation
lassefschmidt opened this issue · 1 comments
Hello there !
First of all, thank you for all the work you have put into this!
I am currently trying out some algorithms for arbitrary timestep frame interpolation and came across your repository, which says in the very first paragraph
It supports arbitrary-timestep interpolation between a pair of images.
Since then, I am trying to get this to work. However, all the recent published models (e.g. the HD_v3 stuff, including what is published under Practical-RIFE) do not support this -- it seems this is only implemented for the RIFE model as published in ArXiV.
But even when I use the ArXiV RIFE model, I cannot load the RIFE_m weights you provide under this link due to shape mismatches. This is a bit unfortunate as a lot of computation power went into the pre-training of those weights. Quote of your paper, pg 8:
We use the Vimeo90K-Septuplet [62] dataset to extend RIFE to support arbitrary-timestep frame interpolation [9,24]. This dataset has 91, 701 sequence with a resolution of 448 × 256, each of which contains 7 consecutive frames. [...] We keep other training setting unchanged and denote the model trained on Vimeo90K-Septuplet as RIFEm.
Is there any chance you can tell me how I can load these weights correctly / provide an updated flownet.pkl file ? Or give me some tips on how to run arbitrary-timestep interpolation with your repo in general?
Thank you!
Best,
Lasse
You can refer to 'benchmark/HD_multi_4X.py' to run that model.