May I ask how you initialized the network weights before training the network?

[kennyorn1]

I believe that different initialization methods are good for exploration of fitness landscape, and that always using similar or even the same initialization methods is not good for the exploration.