mstiehr-dev/data_dedup

designed to store a lot of (likely redundant) data for backup purpose. Restore specific files as well.

#Status - Ready to use!

##description

• files are cut in blocks of 512 byte
• unique blocks are stored in a huge dump file and organized in a central index journal
• deduplicated file is represented by a binary file containing indices of the original blocks

##dependencies

• GNU/Linux
• gcc
• libssl-dev
• cuda-sdk

##usage

• deduplicate{Host,GPU} -i <filename>

• reassemble <metafile>

• detailed description

##cuda command line switches

• -b n: use n kernel copies (blocks)
• -t n: use n threads per block