Longer sequences for validation / testing?
JHLew opened this issue · 6 comments
Hi, thanks to the authors for their impressive work.
I have a question on the data for validation / testing.
The currently available version for the public seems to be aimed for testing environments which take two frames as the input, namely the 0-th and 32-th frame of each scene.
However in that case, I'm afraid methods such as QVI (NeurIPS '19) which require more than two input frames cannot be compared fairly.
We could use more intermediate frames as the input (e.g. 0-th, 16-th, 32-th frame for a framework which requires 3 input frames),
but this may lead to a slightly different scenario (different fps settings), considering that the intended testing environment is interpolating from 30fps to 240fps.
More importantly, I've tried experimenting on interpolating 120fps to 960(or 1000)fps, using the intermediate frames as input, but seems like the task gets too easy and all methods that I've tried perform very well, making it hard to compare which is better.
For these reasons I think it would rather be better with a longer sequence...
According to the example videos on the very first figure of this repository, it seems like the original video sequence for validation / testing seems to be longer than the public version.
Would it be possible for you to share a version of a longer sequence to the public?
@JHLew
Thank you for your interest.
Do you mean that you need longer input frames whose temporal distances are of 32 frames (30fps) setting?
ex)
- number of input frames 2 => 1000.png, -- evaluation interval --,1032.png (original XVFI setting)
- number of input frames 4 => 0968.png, 1000.png, -- evaluation interval --,1032.png, 1064.png (For QVI)
- number of input frames 6 => 0936.png, 0968.png, 1000.png, -- evaluation interval --,1032.png, 1064.png, 1096.png (For All-at-once, Chi et al., ECCV 2020)
(Due to the frame indexing issue, we added 1000 to each frame index)
Yes, exactly.
Would it be possible to share those frames for validation & testing?
@JHLew
Following your suggestion, we will provide longer input sequences for the testsets.
Thanks a lot, looking forward to it!
@JHLew
We uploaded the extended version of X-TEST. Please refer to X4K1000FPS.
Thank you so much for the quick update!