GTNewHorizons/DreamAssemblerXXL

guard against file corruption during download

Opened this issue · 2 comments

Today i tried to assemble manually the 2.2.0.0 RC3 via DAXXL, without changing the locale, resulting in #47. So when i saw that, i terminated my python process, changed the local and rerun DAXXL. That caused the downloads to be killed, resulting in corrupted files.

Because DAXXL assumes that if a file is present, then it's valid, i managed to assemble """valid""" 2.2.0.0 RC3 with a weight of 355MB instead of 435MB.

This is a real issue as if we can assume the cache is corrupted if UnicodeDecodeError arise during download, we have no check against random packet corruption when files are downloaded.

Sounds like we need file hashes or size estimates

Another option is to start signing release jars.... Daxxl can verify those..... 🤔