/tensor-compress

Primary LanguagePython

TensorCompress

Codebase for neural network model compression using tensors. We show that tensor can capture higher order interactions with much fewer parameters, thus significantly reduce the memory overhead. We demonstrate the advantages using three type of neural network layers

  1. Fully-connected (dense) layer

  2. Convolutional layer

  3. Recurrent layer

tensornet

Core tensor compression layer. Implemented with different tensor models

  1. layers: tensor reformuation of neural network layer

  • CP_dense.py: dense layer with CP compression

  • Tucker_dense.py: dense layer with Tucker compression

  • Mf_dense.py: dense layer with Matrix factorization

  1. tt_decomp: deprecated tensor train operations

experiments

Example applications demonstrating the tensor compression

  1. mnist: hand-written digit recognition, demo for dense layer
  • 2-layer-cp: 2-layer dense layer with CP compress model
  1. ptb: text annotation, demo for recurrent layer
  • LSTM network: higher order temporal correlation $$x(t)$$

  1. cifar-10: image classification, demo for convolutional layer

  2. data: raw as well as processed data