tfzip
An experimental compression pipeline for TensorFlow.
Goal
This project aims to implement a TensorFlow version of the Deep Compression pipeline outlined by Han et al.
Current Progress
This project is still in the experimental stage, it is not currently ready for production use
Pruning
- Initial results:
- Running
compression_test.py
can generate uncompressed_model and compressed_model protobuf files from a simple MNIST model for comparison. - Note that there is no difference in the protobuf file sizes until you apply
gzip
or some other compression tool. - The provided protobuf files were generated using a LeNet-300-100 model trained for MNIST digit classification.
- Running
Parameters | Parameter Compression | Protobuf Size | Protobuf Size (gzipped) | Protobuf Compression | Accuracy | |
---|---|---|---|---|---|---|
Uncompressed | ~267k | 3.2MB | ~2.8MB | 98.19% | ||
Compressed | ~27k | ~10x | 3.2MB | ~490kB | ~6x | 97.31% |
- Next steps:
- Improve accuracy preservation and increase compression ratio.
- Learning both Weights and Connections for Efficient Neural Networks suggests ~12x parameter compression without loss of accuracy can be obtained for this architecture.
- Refactor compression logic so that it can be more easily applied to other networks.
- Improve accuracy preservation and increase compression ratio.
Trained Quantization
- Not started.
Huffman Coding
- Not started.