Evaluation results not matching

[indhu26]

Hi @yamhoresh,

When evaluated the keras model, I got the results as follows

abs_rel, sq_rel, rms, log_rms, a1, a2, a3
0.1314, 1.0760, 5.1840, 0.2088, 0.8496, 0.9490, 0.9773

But in the official repo numbers are

abs_rel,     sq_rel,        rms,    log_rms,          a1,         a2,         a3
    0.115 	0.903       4.863   0.193 	       0.877    0.959 	0.982

Why there is a difference in value ?
Or is there anything am i missing ?

[yamhoresh]

Hey,

That's a good a question. I have never actually checked the similarity within the numerical results between both models.
My guess would be that the difference is due to the definition of custom layers under the different platforms (specifically the ReflectionPadding, which does not exist natively in Tensorflow). Sadly, I see no solution for this.

As far as the final visual result (the depth image itself), the results seem par with what a human would expect the ground truth to be. So I'd advise treating the model as a approximation to the original, and not an exact one.