Weights not attaining maximum at surface (SDF=0)

Hi, @Totoro97!
Thanks for your great work. I was impressed by the results of your work and am trying to understand the derivation of the weights. I agree that the weights should satisfy unbiased and occlusion-aware properties.

However, I empirically find that the actual discretized weights do not satisfy the unbiased property.
In the following, I plot the SDF values and weights w.r.t. t, the distance from sampled points to the ray origin.
We can see the weights attain maximum before the ray intersects the surface, which is supposed to be the naive solution one's property (i.e., NeRF).

I wonder whether this is due to the derivation for the denominator in eqn. (6) does not hold for general surfaces. To derive eqn. (6), the surface is assumed to be a plane. While we can make such assumptions for points sufficiently close to the surface, it seems not true for points far away from the surface. Since we are integrating all points along the ray, integrating the S-density of the points far away from the surface is inevitable, and this causes bias.

Do you have any thoughts on this phenomenon?

To reproduce the figure, the following code can be inserted right after

NeuS/models/renderer.py

Line 262 in 6f96f96

weights_sum = weights.sum(dim=-1, keepdim=True)

sdf_temp = sdf.reshape(batch_size, n_samples)
 import matplotlib.pyplot as plt
 for i in range(batch_size):
    plt.figure() 
    plt.plot(mid_z_vals[i].detach().cpu().numpy(), sdf_temp[i].detach().cpu().numpy(), c='r', label='sdf')
    plt.plot(mid_z_vals[i].detach().cpu().numpy(), weights[i].detach().cpu().numpy(), c='b', label='weights') 
    plt.legend() 
    plt.show()
    plt.close()

Hi, thanks so much for your interest in this project! Regarding the visualization result, I guess the reason is that the section lengths of the sampled section are not uniform. You can plot weight[i] / dists[i] (dists[i] is the length of i-th section) and see the visualization results.

On the other side, yes, the unbiased property is based on the assumption that the surface is a plane. So there can be some errors for general surfaces. But for the above visualization, I think it is mainly due to the non-uniform section length. Plotting the "weight per unit length" should be more reasonable.

You're right! The weights look more reasonable now!

Thanks for your kind reply.