/sparsecat

CLI tool that reduces bandwidth usage when transmitting sparse files.

Primary LanguageGoMIT LicenseMIT

SparseCat

Goal

Transmitting sparse files using minimal amount of network bandwidth. Sparsecat uses the SEEK_HOLE and SEEK_DATA capabilities of unix filesystems to find holes in sparse files and only transmits sections containing data. The wire format is simple and compatible with the ceph rbd diff v1 format as described here

asciicast

Support

Because the tool relies on the lseek syscall with SEEK_HOLE and SEEK_DATA only unix systems with the correct filesystems are supported. See the man pages for more information. sparsecat does work with unsupported filesystems, but it will simply transmit the entire file with a couple of bytes of overhead.