PotFit

A PyTorch Example of Potential Fitting

Problem

Provide there exists a potential V(x). Given some observations { V(x_i), dV/dx(x_i) }_i=1,...,N, obtain the approximate potential P(x) which satisfies L(V,P) < δ, where

L(V,P) = (1/N) ∑_i ( |V(x_i)-P(x_i)|² + λ |dV/dx(x_i)-dP/dx(x_i)|² ).

δ > 0 and λ > 0, and they are given constants.

The values at N=100 random points are shown as solid cricles.

Models

This code uses the neural networks as follows:

z⁰ = σ( x W_in ),

z^l = σ( z^l-1 W^l ) (l=1,...,L),

P(x) = z^L W_out + b_out,

where σ(x) = SSP(x) = Softplus(x) - Softplus(0). W_in, W^l, and W_out are the 1xH, HxH, and Hx1 matrices, respectively.

The default setting: L=5 and H=500.

Enviroment

Python 3.7
PyTorch 1.0.0.dev20181107

How to run

python potfit.py --num_epochs 20000

This code uses λ = 1/25.

You will eventually get outputs like as follows:

epoch 19995 lr 0.0003774 mse0 0.00228 mse1 2.07881 loss 0.08543
epoch 19996 lr 0.0003774 mse0 0.00557 mse1 2.85378 loss 0.11972
epoch 19997 lr 0.0003774 mse0 0.00285 mse1 4.02198 loss 0.16373
epoch 19998 lr 0.0003774 mse0 0.00476 mse1 4.44875 loss 0.18271
epoch 19999 lr 0.0003774 mse0 0.01051 mse1 2.68157 loss 0.11777
Best score: 0.007930990790831857

loss = mse0 + λ mse1.

You can plot predictions of the best model as the following:

python plot_predictions.py best

References

Deep Neural Networks

Shiyu Liang and R. Srikant, "Why Deep Neural Networks for Function Approximation?", ICLR 2017 link

Softplus

Vinod Nair and Geoffrey E. Hinton, "Rectified Linear Units Improve Restricted Boltzmann Machines", ICML 2010 link

Shifted Softplus (SSP)

Kristof T. Schütt, Huziel E. Sauceda, Pieter-Jan Kindermans, Alexandre Tkatchenko, and Klaus-Robert Müller, "SchNet - a deep learning architecture for molecules and materials", J. Chem. Phys. 148, 241722 (2018) link

License

Apache License 2.0

Tony-Y/PotFit