Model training not converging to a good result and producing grey images at iteration 531000

[matheushms]

Hello,

I have been following your project and I must say the team has done an excellent job so far. However, I have encountered an issue with the model training. At iteration 531000, the model is not converging to a good result and is generating grey images.

An image representing the output generated by the model at the current iteration is attached to this issue. As you can see, the image is very grey and the depth map does not represent well the scene.

config file:
tblock4_train.txt

Dockerfile(I modify to run in my GPU):
Dockerfile.txt

I trained the model with the SWORD dataset and generated the images with iPad images provided in the additional scene folder.

I hope that we can work together to find a solution to this issue and get the model to converge. Any suggestions or feedback on how to resolve this issue would be greatly appreciated.

Thank you for your hard work on this project.

[palsol]

Hello! Try to train the version with 8 layers.

[matheushms]

Thanks! I will try this.