/leaps-lossSurfaceVisualize

lossSurfaceVisualize

Primary LanguagePython

typora-copy-images-to
./

surface visualization

Visualize the loss surface. This project is a shallow implementation of "Visualizing the Loss Landscape of Neural Nets."

image-20210313182606596

direction vector is a random vector sampled from the normal distribution. The direction vector D is obtained as follows.

direction = torch.randn(param.size()) * param

Bias and normalization were excluded from direction calculation (Set to 0)

image-20210313182546417

image-20210313182531243

image-20210313182500231

image-20210313182358058