integrity check, sha256 hashes, checksums, chunklist, corrupt files
myspaghetti opened this issue · 3 comments
The script does not perform any integrity checks on downloaded files. Apple provides signed .chunklist files that list sha256 hashes of file chunks. These files can be used for integrity-checking. Another method is hosting a database of known-good file hashes.
Currently I have no plans to implement either solution. Using Apple's signed chunklists is the correct solution and I'm willing to accept pull requests implementing it. Creating a database requires trusting the database maintainer which is not unreasonable but should be avoided.
It's been pointed out a few times (#459 #486) that the script can report it has terminated "successfully" even when the downloaded files (and subsequent disk images) are corrupt.
The solution is out there but requires compiling a tool to check the .chunklist integrity data. I'm re-opening this issue open for exposure in case anyone wants to implement the solution in a shell script without any further dependencies.
How do you feel about an 'if possible/available, do it in Python' approach?
original source, actually already linked above though the comment was updated recently
→ incorporated into a python tool for grabbing the images, with AFAIK only stdlib dependencies for Python2/3
→ used in a bash4 script to setup QEMU, which btw. demonstrates grabbing hashes from webpages with wget and verifying in bash+various *sum utils
If you have any pointers to how dependency-free bash can work with binary files, feel free to share! It may be possible with some xxd wizardry.
https://unix.stackexchange.com/questions/10801/how-to-use-bash-script-to-read-binary-file-content
Thank you so much for pointing me at the Python code! I am definitely in favor of the correct solution (verifying hashes with the Apple .chunklist files) and I might take the Python code and implement it using the existing script dependencies. I haven't looked at it thoroughly but at a glance it looks doable (famous last words?)