/NeuroFox

a neural network written from scratch and accelerated with cuda

Primary LanguageC++

Development on this Project is beeing continued in a private repository

A Convolutional Neural Network

by Elias Kramer

The goal ist to make a library for building a convolutional neural network.

Implementation

  • Layer
    • Fully Connected
      • Forward Propagation
      • Back Propagation
      • Fully GPU supported
    • Convolutional
      • Foward Propagation
      • Backprop is still a TODO
      • Fully supported GPU for every implemented function
    • Pooling
      • Does only work as a standalone
        • integrating it into the nnet is still a TODO
      • Forward Propagation
      • Backprop is still a TODO
      • Fully supported GPU for every implemented function
  • Data Space
    • Allocates a huge block of memory for all data you need to train/test a nnet
    • Is implemented as an easy way to store your matrix collection
    • Can be used directly with a Neural Network
    • Pros
      • By allocating one big block of memory for to save a fixed amount of data, it is very space efficient
      • This can be achieved by saving the metadata of a matrix (like format and if it is on the gpu or not) only once
    • Cons
      • A big, contiguous block of memory is often hard to allocate, since you need a lot of free space on your RAM or GPU
  • Neural Network
    • Can be trained and tested on
      • a Data Space
      • individual matrices
    • Implemented Optimizer
      • Momentum
    • Implemented Initialization
      • Random
      • Xavier