confusing about the occlusion maps

[lelelexxx]

Thanks for your sharing~!

After read your paper, I noticed that, the occlusion estimation of your work is mapped to (0,1), and the occlusion estimation is used as the weight of photometric loss.
While in your paper, the occlusion estimation in Fig.3 is something like a binary map, and in the last of second paragraph of section 4.2, you said 'This clearly shows the benefit of ignoring misleading information in accordance with the occlusion estimates'. It is a little confusing to me. Is the occlusion estimation transformed into binary map? Or you just transform occlusion maps for visualizing?

Any reply will be appreciated~
Thanks again.

[JJanai]

Hi,
sorry for the late reply but I was traveling the last weeks.
Anurag's answer from the pytorch implementation is correct:

The predicted occlusions from the network are indeed real numbers between 0 and 1. We threshold them for visualization.
Originally posted by @anuragranj in anuragranj/back2future.pytorch#1 (comment)

We chose real values to make the formulation fully differentiable and back propagate gradients also through the occlusions maps.

If you have more questions please let me know.
Best,
Joel

[JJanai]

For further clarification, here my answer to anuragranj/back2future.pytorch#4 (comment):

Our two-dimensional occlusion variables lie between [0,1] and sum up to one. The first dimension represents past occlusions while the second dimension represents future occlusions. Since we have continuous variables, we can have states that are in between our three occlusion states (partly visible).

However, you can visualize the three occlusion states by thresholding the occlusion variables in the following way: We have a past occlusion / future occlusion if the first dimension / second dimension is larger than 2/3 otherwise the pixel is visible.

Best,
Joel