Add ability to simply, fully verify file checksum
nh2 opened this issue · 2 comments
For accepting E57 data in automated processing, it would be awesome if libE57Format could offer a simple way to verify (the checsums of) an entire file with a simple function that can be called ahead of time, vs noticing E57_ERROR_BAD_CHECKSUM
during (potentially more time consuming) stream processing.
If we wanted to add a .checkAll()
function, how should it best be designed to generically traverse all data while still being as fast as possible?
Are there some slower approaches that would do the trick until then?
Finally, after a file has been verified this way, what would be the best way to disable checksumming during subsequent stream processing?
As a related aside:
I am getting a bit confused by CHECKSUM_POLICY_ALL
docs saying:
Only verify 50% of the checksums.
and CHECKSUM_POLICY_HALF
saying
Only verify 25% of the checksums.
Are those copy-paste errors?
Yes - the checksums are very expensive which is why I added the checksum policy.
Offhand I don't think there's an easy way to skip processing of the data - i.e. just read the bytes and do the checksum. The format is very.... intricate. The checksum is calculated on each 1020 bytes of the file, so that alone makes reading these thing a mess. Speed-wise it might be fine for a 5MB files, but with 1.5GB it becomes unmanageable.
I haven't looked at this code in a very long time, but if I can see a way to do what you're asking I'll let you know.
Are those copy-paste errors?
Nope - just my mishandling of the doxygen comments. I've fixed it - thanks!