The CXI loader chokes on the CXI dumped from Alpha Sapphire v1.0
Closed this issue · 9 comments
Attempting to batch import from a Pokemon Alpha Sapphire v1.0 CXI produces the following error:
-971440936
java.lang.NegativeArraySizeException: -971440936
at com.martmists.ctr.ext.ByteArrayKt.lzss(ByteArray.kt:23)
at com.martmists.ctr.loader.filesystem.CXIFileSystem$getByteProvider$1$1$1.invoke(CXIFileSystem.kt:142)
at com.martmists.ctr.loader.filesystem.CXIFileSystem$getByteProvider$1$1$1.invoke(CXIFileSystem.kt:138)
at com.martmists.ctr.ext.InputStreamKt.reader(InputStream.kt:14)
at com.martmists.ctr.ext.InputStreamKt.reader$default(InputStream.kt:9)
at com.martmists.ctr.loader.filesystem.CXIFileSystem$getByteProvider$1.invoke$lambda-1(CXIFileSystem.kt:138)
at ghidra.formats.gfilesystem.FileSystemService.getDerivedByteProviderPush(FileSystemService.java:486)
at com.martmists.ctr.loader.filesystem.CXIFileSystem$getByteProvider$1.invoke(CXIFileSystem.kt:137)
at com.martmists.ctr.loader.filesystem.CXIFileSystem$getByteProvider$1.invoke(CXIFileSystem.kt:123)
at com.martmists.ctr.ext.InputStreamKt.reader(InputStream.kt:14)
at com.martmists.ctr.ext.InputStreamKt.reader$default(InputStream.kt:9)
at com.martmists.ctr.loader.filesystem.CXIFileSystem.getByteProvider(CXIFileSystem.kt:123)
at ghidra.formats.gfilesystem.FileSystemService.getFullyQualifiedFSRL(FileSystemService.java:887)
at ghidra.formats.gfilesystem.FileSystemService.getFullyQualifiedFSRL(FileSystemService.java:847)
at ghidra.plugins.importer.batch.BatchInfo.processFS(BatchInfo.java:344)
at ghidra.plugins.importer.batch.BatchInfo.processAsFS(BatchInfo.java:318)
at ghidra.plugins.importer.batch.BatchInfo.doAddFile(BatchInfo.java:229)
at ghidra.plugins.importer.batch.BatchInfo.addFile(BatchInfo.java:197)
at ghidra.plugins.importer.batch.BatchInfo.doAddFiles(BatchInfo.java:503)
at ghidra.plugins.importer.batch.BatchInfo$AddFilesRunnable.monitoredRun(BatchInfo.java:542)
at ghidra.util.task.TaskBuilder$TaskBuilderTask.run(TaskBuilder.java:306)
at ghidra.util.task.Task.monitoredRun(Task.java:134)
at ghidra.util.task.TaskRunner.lambda$startTaskThread$0(TaskRunner.java:106)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
---------------------------------------------------
Build Date: 2023-Dec-22 0936 EST
Ghidra Version: 11.0
Java Home: C:\Program Files\OpenLogic\jdk-17.0.9.9-hotspot
JVM Version: OpenLogic 17.0.9
OS: Windows 11 10.0 amd64
Workstation: TIM-PC
I am going to attempt to patch it in a fork and push the changes upstream if you no longer wish to work on this project. Your loader is currently the best one for ghidra, so I'm going to try and use it as my starting point.
I wonder if it'd be worth looking into creating a bunch of custom CXI files to have some unit tests, since this was only ever tested using X/Y.
That could be interesting! So far I can only confirm that it works with Pokemon Ultra Moon, and does not work with Alpha Sapphire, from cartridges I dumped myself. Do you intend to look into this, or should I take a whack at fixing it?
I'm a C++ developer and have never touched Java or Kotlin, so I'm not sure just how well I'd do at chasing this down.
Unfortunately my lzss implementation is mostly ported from elsewhere if I recall correctly, so I'm not sure if I'd be able to fix this myself.
I guess I'll start deep diving. Thanks!
I discovered the issue. It's in ByteArray.kt:
val offSizeComp = ByteBuffer.wrap(sliceArray(size-8 until size-4)).order(ByteOrder.LITTLE_ENDIAN).int
val addSize = ByteBuffer.wrap(sliceArray(size-4 until size)).order(ByteOrder.LITTLE_ENDIAN).intThese two expressions grab four bytes from the array, interpret them as a little-endian four-byte value, and then use that to calculate various sizes and offsets. Eventually they're used to calculate the offset of an array.
In Pokemon Alpha Sapphire, the four bytes for both of these values are the same: [24, -58, 24, -58]
When interpreted as a little endian value and then converted into an integer, this becomes 0xC618C618
That value in decimal is 3,323,512,344, which is far larger than an integer can hold. It then overflows to a negative value.
So either the bytearray is being read from the binary incorrectly initially, or the size is much larger than the plugin expects it to be.
Given that these are offsets into a 32-bit address space, I suspect it's the former -- The bytearray is being parsed incorrectly before lzss() is called.
Actually, nevermind. A 32-bit address space can reach up to the maximum for an unsigned 32-bit integer, so this is valid. ByteArrays have a fundamental limitation that prevent them from getting that large, though.
The CXIFilesystem object was attempting to LZSS decompress the icon element of the exeFS, which could not be decompressed. I added a quick check to guarantee that this does not occur.