Full model implementation of CVPR 2020 paper, "Softmax Splatting for Video Frame Interpolation"
UPDATE NOTE: the pretrained weights below should be used with the version commited on June 25, 2022. The current version can be used as a reference, but may not lead to the exact performance reported, since some details may be missing. Will be updated soon.
Pre-trained weights for pre-defined Z using alpha.
Pre-trained weights for fine-tuned Z using a small neural network v.
Both weights approximately lead to a PSNR of 35.8 in Vimeo90k. (reported score from the paper is 36.10)
The model assumes to take a tensor in the shape of [B, C, T, H, W] where B is the batch size, C is the number of channels (3 RGB channels), T is the number of frames (2 frames: frame 0 & frame 1), H and W is the height and width of the frames.
from SoftSplatModel import SoftSplatBaseline
frame0frame1 = torch.randn([1, 3, 2, 448, 256]).cuda() # batch size 1, 3 RGB channels, 2 frame input, H x W of 448 x 256
target_t = torch.tensor([0.5]).cuda() # target t=0.5
model = SoftSplatBaseline().cuda()
model.load_state_dict(torch.load('./SoftSplat_predefinedZ.pth')['model'])
framet = model(frame0frame1, target_t)