Contributing
NWPlayer123 opened this issue · 4 comments
Hello, @hlixed sent me, I figured I'd help out since python and Nintendo is my expertise, I started by looking at the Yaz0 decompressor which could probably use some rework (I haven't tested it vs. my implementation), but I wanted to send this your way
https://gist.github.com/NWPlayer123/05dd55dd1cce6eb5457741ce14f48587
some things I learned/would suggest:
ord + bit-manipulation is far faster than using struct
could probably use BytesIO instead of an array (I use wb+ to seek back and forth, might be faster to write in one go instead of allocating all the zeroes )
I dunno what you're doing here, but I'd recommend doing the masking like I do.
I dunno how the cycles compare on what I do vs. what you're doing, but I'd also do this so we don't have to catch anything, one less compare (I made the same mistake before)
My fancy buffer trick would also save some time since you don't have to keep a track of the full amount back you can go (0x111, 0xFF on third byte + the 0x12 added)
Also probably a slight waste to re-read the first two bytes.
One fun fact: you have to read byte-by-byte cuz their algo lets stuff overlap (aka, data isn't already there if you just copy the whole size at once), but you don't have to every time, only if size is bigger than how far back you go.
Ok, thanks for your suggestions! I'll try to rework this (and if I can other decompressors too, the LZ11 one is one if the biggest bottlenecks of 3DSkit...)
I also slightly modified that gist to do LZ11, dunno how effective this one is tho, basically the same thing except 0 is copy and 1 is RLE and there's more modes
https://gist.github.com/NWPlayer123/eba6a75fac84bda0d263afc5b8eca755
I rewritten the Yaz0 decompressor, thanks for help! For the LZ11 decompressor, I'll try to optimize it too, and to make it a bit less horrible.
I optimized the LZ11 decompressor and rewritten the compressor. Now, to compress 500KB, it takes 15 seconds instead of 15 minutes :)