Totoro97/NeuS

Why approximate alpha value instead of compute it as in paper?

alvaro-budria opened this issue · 3 comments

Hi, I am not sure why the $\alpha_i$ values are computed differently in the code and in the paper.

In the paper, we find this expression

Screenshot from 2023-07-24 14-54-13

where consecutive samples in a ray are utilized to obtain the $\alpha$.

In the code

NeuS/models/renderer.py

Lines 220 to 247 in 629b9cf

sdf_nn_output = sdf_network(pts)
sdf = sdf_nn_output[:, :1]
feature_vector = sdf_nn_output[:, 1:]
gradients = sdf_network.gradient(pts).squeeze()
sampled_color = color_network(pts, gradients, dirs, feature_vector).reshape(batch_size, n_samples, 3)
inv_s = deviation_network(torch.zeros([1, 3]))[:, :1].clip(1e-6, 1e6) # Single parameter
inv_s = inv_s.expand(batch_size * n_samples, 1)
true_cos = (dirs * gradients).sum(-1, keepdim=True)
# "cos_anneal_ratio" grows from 0 to 1 in the beginning training iterations. The anneal strategy below makes
# the cos value "not dead" at the beginning training iterations, for better convergence.
iter_cos = -(F.relu(-true_cos * 0.5 + 0.5) * (1.0 - cos_anneal_ratio) +
F.relu(-true_cos) * cos_anneal_ratio) # always non-positive
# Estimate signed distances at section points
estimated_next_sdf = sdf + iter_cos * dists.reshape(-1, 1) * 0.5
estimated_prev_sdf = sdf - iter_cos * dists.reshape(-1, 1) * 0.5
prev_cdf = torch.sigmoid(estimated_prev_sdf * inv_s)
next_cdf = torch.sigmoid(estimated_next_sdf * inv_s)
p = prev_cdf - next_cdf
c = prev_cdf
alpha = ((p + 1e-5) / (c + 1e-5)).reshape(batch_size, n_samples).clip(0.0, 1.0)

we see that instead of the expression above, we approximate the section points at each computed SDF value, which requires additional inputs: the directions and the intersample distances in the ray.

I wonder why this is done in this way. It seems to reduce the amount of queries to the SDF function by half, at the expense of introducing some noise. Could you elaborate on this choice?

Same question

I've attempted to answer this question in a previous issue:

#107 (comment)

Let me know what you guys think, because I'm not entirely sure myself why they are doing this approximation.

Yes, I agree with your comment there in that issue. It's for geometric consistency.