Alternative Backup Solution to Rsync
schnipdip opened this issue · 4 comments
Basically, rebuild functionality of Rsync, however the function will allow a percentage display on the backups files. This will allow for multicore copying. Differential backup will have to be supported to prevent duplicates - checksum.
Issue with checksum - cpu intensive
Major Benefit, more information can be displayed to the user.
Can create a local local cache/database to store checksums to reduce checksum time
includes name of picture and checksum of the picture = photo rainbow table.
Do i need to perform a checksum on the files located in the destination drive? If I am creating a rainbow table then the values should exist in the cache. I would just have to do a comparison the the cache and the source checksum to add the files a dict object and then copy those new files.
added dump_it.py.
Contains the following classes:
class backup -> performs differential backup - spawns new process to copy each object to the destination
class hashing - > hashes each image object. It spawns a new process for each object to perform a checksum
class cacheTable -> creates a caching database of filename + checksum
Now that I am thinking about this more it doesn't make much sense to create a tailored backup solution. Instead we can continue to work with Rsync and if need be create a wrapper for Rsync to do what we need it to do. We are going to have to tailor the Raspberry Pi for performance. We can probably get rsync to multithread to increase performance and integrity checking can be done in rsync to ensure the files were transferred properly (advanced setting configuration for advanced users)