theolepage/tensorflood

Tiny automatic differentiation (autodiff) engine for NumPy tensors implemented in Python.

Python

TensorFlood

Tiny automatic differentiation (autodiff) engine implemented in Python.

The following resources were used while creating this project:

Build Your Own Automatic Differentiation Program by Jonathan Kernes
Yaae by Ferdinand Mom

Example

The notebook examples/classification.ipynb demonstrates how to use the engine and how it can be used to train a small neural network.

Usage

from tensorflood.engine import Graph, Variable, Constant

with Graph() as g:
    x = Variable(2, name='x')
    y = Variable(10, name='y')
    c = Constant(7, name='c')
    
    z = (x * y) + c
    # z.data = 27
    
    z.backward()
    # x.grad = 10 (dz/dx)
    # y.grad = 2  (dz/dy)
    # c.grad = 1  (dz/dc)

To-Do

NN: Adam optimizer, more layers
Merge backward_ and backward
Refactor hasattr(node, 'input_nodes')
Unit tests with pytorch