DeepPipe
Deep Learning with elixir. This project is called Deep Pipe(DP). I implemented backpropagation and numerical-gradient. Now I'm testing small data set.
Network example (See test.ex)
defnetwork init_network2(_x) do
_x |> f(5,5) |> flatten
|> w(576,100) |> b(100) |> sigmoid
|> w(100,10) |> b(10) |> sigmoid
endTODO:
Organize test case
Improve testing
Create a library and register to hex
Installation
sudo apt install build-essential
sudo apt-get install build-essential erlang-dev libatlas-base-dev
mix deps.get
invoking
make clone or download and enter "iex -S mix" on terminal
modules
module DP is Deep Pipe(DP) module module CTensor is code for CNN data operation module Cmatrix is code for Matrix module MNIST is code for MNIST data set module Time is time/1 for measure execution time
running expample
iex(1)> require(Time)
Time
iex(2)> Time.time(Test.momentum(30,100))
preparing data
ready
2.866287227629866
2.5600212240059506
...
0.04318082027257467
0.029026173275906994
0.03131037967594155
0.06550367669302301
accuracy rate = 0.879
"time: 55248299 micro second"
"-------------"
:okspecification:
data structure
matrix
data
structure of Matrex(CBLAS) e.g. m[2*3]
row_vector
data structure of Matrex(CBLAS) e.g. m[1*3]
tensor
e.g. [m[23],n[23]]
network
e.g. [{:weight,w,lr,v},{:bias,b,lr},{:function,f,g}]
weight
{:weight,w,lr,v} w is matrix, lr is learning rate, v is for momentum,adagrad,adam
bias
{:bias,b,lr,v} b is row vector
filter
{:filter,w,lr,st,v} st is stradd for convolution
pad
{:pad,n} n is size of padding
pool
{:pool,st} st is stradd
function
{:function,f,g,h} f is function, g is differential function, h is function name softmax {:softmax,f,_ } f is function, only output layer softmax is set with cross_entropy
module macros
defnetwork is macros to describe network argument must have under bar to avoid warning message
w(m,n)
weight matrix size(m,n). elements are Gaussian distribution random float
w(m,n,lr)
lr is learning rate (default is 0.1)
w(m,n,lr,z)
z is multiple for Gaussian distribution random float. (default is 0.1)
b(n)
bias row_vector size(n). elements are all zero b(n,lr) lr is learning rate (default is 0.1)
function
sigmoid,relu,ident,softmax
f(m,n)
filter matrix size(m,n). elements are Gaussian distribution random float
f(m,n,lr)
lr is learning rate
f(m,n,lr,z)
z is multiple for Gaussian distribution random float.(default is 0.1)
f(m,n,lr,z,st)
st is stradd
pad(n)
padding n is size of padding
pool(st)
pooling stride size is st
module DP
forward/2
forward calculation for batch data forward(x,network) x is data(matrix or tensor) , network
numerical_gradient/3
calculate gradient by numerical differentiation
numerical_gradient(x,network,t)
x is data, t is train data, loss function is mean_square
numerical_gradient(x,network,t,:cross)
loss function is cross_entropy
numerical_gradient(x,network,t,:square)
loss function is mean_square
gradient/3
caluculate gradient by backpropagation
gradient(x,network,t)
x is data, t is train data
learning/2, /3
update network with gradient
learning(network,gradient)
update with sgd
learning(network,gradient,:sgd)
update with sgd
learning(network,gradient,:momentum)
update with momentum method
learning(network,gradient,:adagrad)
update with adagrad method
learning(network,gradient,:adam)
update with adam method
print/1
print data
newline/0
print LF
save/2
save network data to file
save(filename,network)
load/1
load network data from file
load(filename)
mean_square/2
loss function
mean_square(y,t)
y is row vector of result and t is row vector of train
cross_entropy/2
loss function
cross_entropy(y,t)
y is row vector of result and t is row vector of train
module MNIST
train_image(n)
get train image data size of n. Each data is 28*28 matrix
train_image(n,:flatten)
get train image data size of n. Each data is 784 row_vector
train_label_onehot(n)
get train label data aas onehot row_vector
test_image(n)
get test image data size of n. Each data is 28*28 matrix
test_image(n,:flatten)
get test image data size of n. Each data is 784 row_vector
test_label_onehot(n)
get test label data aas onehot row_vector