sniklaus/softmax-splatting

Comparison with other work

rsjjdesj opened this issue · 5 comments

rsjjdesj commented

Hello, Nice work. Have you compared your work with other recent methods such as RIFE, XVFI etc., especially on high motion content.

sniklaus commented

Thank you for the kind words. SoftSplat was published well before RIFE and XVFI, so you will have to look into the papers of RIFE and XVFI for comparisons. You can find a comparison of RIFE and SoftSplat in Table 2 of the RIFE paper. It shows that RIFE-Large is similar to SoftSplat on Vimeo90k but is outperformed by SoftSplat on UCF101 and Middlebury.

image

Unfortunately, XVFI does not compare to SoftSplat in their paper but you can do the comparison yourself quite easily. In the XVFI repository, they quote a PSNR of 35.07 on Vimeo90k [1] whereas SoftSplat achieves 36.10 in this benchmark (a big difference).

If you want to compare SoftSplat/RIFE/XVFI on footage with large motion, you could run RIFE/XVFI on the following benchmark and compare it to the SoftSpalt results (Table 2 of our paper): https://github.com/sniklaus/softmax-splatting/blob/master/benchmark.py

[1] https://github.com/JihyongOh/XVFI#quick-start-for-vimeo90k-as-in-fig-8

rsjjdesj commented

Thanks for the details. Yes, I understand about the objective metrics. However, in XVFI demo, the subjective quality for the two clips they show look amazing as compared to DAIN and other works. So, wanted to know if you have also tested on similar sequences for SoftSplat and how does those sequences look.

sniklaus commented

Every paper has a demo where the proposed result works better than the others. 🙂

If you have a specific sample in mind, I am happy to run SoftSplat on it and post the result here.

  • 👍1
rsjjdesj commented

Every paper has a demo where the proposed result works better than the others. 🙂

If you have a specific sample in mind, I am happy to run SoftSplat on it and post the result here.

Sure. I understand. I will see if I can get some sample images.

There is also a separate thread going on JihyongOh/XVFI#5. @sniklaus : please check that thread, and those images and let us know your opinion.

sniklaus commented

I shared my thoughts in the reference thread/issue. 👍