Minimal examples for basic artificial neural network architectures, problems, layers and general techniques.
These examples are targeted at coders without experience in using artificial neural networks.
I will try and expand the number of examples to cover a wide spectrum of different areas and use cases.
- Clone this repository.
- Download python (https://www.python.org/downloads/) if you dont have it already (careful: 64bit is needed)
- Be sure
pip
as well aspython
are in yourPATH
- Navigate to where you cloned this repository and enter the folder FridAI
- Use
pip install -r ./requirements.txt
to install all required packages - Check everything is working by running a arbitrary example
(
python -m digits
) It should load a few seconds, after which it will print the neural network's training progress. - Wait for the process to finish without errors
You already ran the digits example, while the network trained, it produced logs, which can be plotted.
Run tensorboard --logdir=digits/logs --host=localhost
and use your
favourite browser to navigate to http://localhost:6006, where you can
view the training progress. For more infos check
this section!
After you correctly set up everything, have a look at the
digits/solution.py
. It contains a working example of a simple yet
interesting problem solved with an artificial neural network.
The (more simple) artificial neural network is created in
digits/models/mlp.py
. Performance of this model is heavily dependent
on so called hyper-parameters, which are for example the units of a
layer, the number of hidden layers, activation function in hidden/output
layer and many more. Try and experiment with the value of these a little!
Keras is very well documented, you can find it here: https://keras.io/
Layers are documented here: https://keras.io/layers/core/
Activation functions are here: https://keras.io/activations/
The model (fit/predict functions) is documented here: https://keras.io/models/model/
Once you played around with different layer hyper parameters, you can get your feet wet with alternative loss functions or optimizers.
Loss functions you can find here: https://keras.io/losses/
Optimizers are documented here: https://keras.io/optimizers/
There are plenty of use cases, if you feel like you want to practice with prepared datasets a little bit more though, check here: https://keras.io/datasets/
There are (most commonly) four different plots:
- loss: error (difference between prediction and actual value) the network produces while training
- accuracy: accuracy in predicting the correct label/class while training
- val-loss: error the network produces while testing it, which means: error it produces while presenting samples, the network has never seen
- val-accuracy: accuracy in predicting the correct label/class while testing it, which means: accuracy in predicting the correct label/class while presenting samples, the network has never seen
The x-axis of each plot is the training progress, measured in epochs. One epoch is finished, when all training data has been presented to the neural net once.
The y-axis is the loss(error)/accuracy (or whatever metric is measured). Low loss and high accuracy are desirable.
A falling loss graph means, that the model is learning! Careful: Even though the loss graph falls the validation loss graph maybe rising! This means your neural net is over-fitting (see here) and needs tweaks in its architecture or more data.