Baselining the scenes

[cpeng93]

Hi, thanks for the great work!

I came across your project and found it very interesting and useful. I've done some baselining to make sure i understand the code correctly, here are some results in my reproduced PSNR and (reported PSNR in the paper).

Everything looks good and within the range of errors other than the Factory scene. Do you have any idea why this may be?

Again, great work!
Best,

[dogyoonlee]

@cpeng93
I agree. It seems to be little bit weird on Factory scene only.
Did you solved?

[limacv]

I've uploaded the pretrained weights on the factory scene on the dataset link. Maybe you could evaluate the pretrained weights first. Suppose the pretrained model also has poor PSNR. In that case, that may be because I accidentally change some code relevant to the evaluation when doing the experiments (since I debug my code using the factory scene). You can just use your newly evaluated performance on the pretrained model. If not, then I guess it is probably because the model is sensitive to initialization.

[cpeng93]

I've uploaded the pretrained weights on the factory scene on the dataset link. Maybe you could evaluate the pretrained weights first. Suppose the pretrained model also has poor PSNR. In that case, that may be because I accidentally change some code relevant to the evaluation when doing the experiments (since I debug my code using the factory scene). You can just use your newly evaluated performance on the pretrained model. If not, then I guess it is probably because the model is sensitive to initialization.

Thanks for the reply! From just a glance there are two differences in the config page for blurfactory:
In the public repo, the following config is included but not in the config file from the uploaded checkpoint.
render_rmnearplane = 5
render_focuspoint_scale = 0.25
render_radius_scale = 0.5

Also in the public repo:
kernel_align_weight = 0.1
align_start_iter = 0
align_end_iter = 180000
instead of:
kernel_smooth_weight = 0.1
smooth_start_iter = 0
smooth_end_iter = 180000

I will run some benchmark in the mean time.
Cheers,

[cpeng93]

In fact it seems like none of them have the render_rmnearplane etc. hyper-parameters anymore.

[limacv]

Oh then I think render_rmnearplane configs may be the actual reason that the PSNR is different. It is just a hyper-parameter set manually to remove some floating artifacts when rendering novel views. When doing the evaluation, these configurations should better be removed. (render_focuspoint_scale and render_radius_scale are unused for evaluation and only affect the generated video)

[cpeng93]

Thanks! looks like this addressed it.