BUG: mkpsxiso v1.06 crashes if any directory contains more than 32 files
Closed this issue · 18 comments
If any given directory/folder contains more than 32 files, mkpsxiso will crash just as it appears to be finalizing the disc.
As a result, the cue sheet is not generated and the BIN image appears to be useless.
Hmmm.... well, the current project that I'm working on is actually a modification to an already existing game - Resident Evil 2 Dual Shock. To be more specific, I am simply modifying the original executable with armips and recompling the disc image.
There is a folder titled "BSS" on the root directory which contains 127 files, by default. This is not a modification - the game is originally built that way.
Sony's BUILDCD application, dated Thursday, August 20, 1998, 8:49:10 PM, has absolutely no problem whatsoever compiling an image using the default layout of that specific game. Since I no longer have an x86 operating system, I am now dependent on mkpsxiso (which is utterly fantastic, btw).
I do believe that the library functionality that you speak of does contain the specific limitations that are mentioned.
-However-
CAPCOM never strictly adhered to those specific library functions and instead programmed their PSone games to load data by referencing an internal LBA table within the executable, via functionality like "DsIntToPos()", "DsSyncCallback()", "DsCommand()", etc. Doing so allowed them to drastically cut loading time by bypassing the bloated PSYQ functionality. It is also my understanding that many non-CAPCOM games implemented similar methods that do the exact same thing.
Okay, I've quickly put together this 1.07test release which should be able to create a directory with more than 32 entries without crashing:
http://lameguy64.github.io/mkpsxiso/mkpsxiso-1.07test.zip
Because I cannot test this new implementation myself extensively (I only tested it with around 45 directory entries) due to time constraints with personal stuff, I'm going to need it tested extensively with your project before I put it out as a main release.
I too, have written my own functionality which utilize an internal LBA index and parses file containers accordingly :) When I mention bloat, well, I really mean that in the sense of bypassing all the standard SDK functionality to get right to the purpose - loading files. In my years of research, I've noticed that games which do not implement their own LBA-loading techniques take longer to load data simply because of all the functionality that must be processed before the data is actually loaded. Obviously, bypassing that will free up precious processing cycles and doesn't play hell with the laser, not to mention that the previously aforementioned restrictions can be bypassed.
Okay, onto the subject at hand... I have tested the build that you have provided and it was able to properly compile the disc image, however, something seems to have gone wonky with the LBA.
Here is a partial build log provided by BUILDCD:
Lpath 00:24:18 1 18
OptLpath 00:24:19 1 19
Mpath 00:24:20 1 20
OptMpath 00:24:21 1 21
Dir() 00:24:22 1 22
Dir(BINARY) 00:24:23 1 23
File(BIO1.BIN) 00:24:24 8 24 00:24:23 07/20/1998 22:43:42 16384 Form1 C:\BIODS\DISC\BINARY\BIO1.BIN
File(BIO2.BIN) 00:24:32 8 32 00:24:23 07/20/1998 22:43:42 16384 Form1 C:\BIODS\DISC\BINARY\BIO2.BIN
File(BIO2P.BIN) 00:24:40 8 40 00:24:23 07/20/1998 22:43:42 16384 Form1 C:\BIODS\DISC\BINARY\BIO2P.BIN
File(BIO3.BIN) 00:24:48 8 48 00:24:23 07/20/1998 22:43:42 16384 Form1 C:\BIODS\DISC\BINARY\BIO3.BIN
File(BOOT.BIN) 00:24:56 12 56 00:24:23 07/20/1998 22:43:42 24576 Form1 C:\BIODS\DISC\BINARY\BOOT.BIN
File(CONFIG.BIN) 00:24:68 14 68 00:24:23 07/20/1998 22:43:42 27756 Form1 C:\BIODS\DISC\BINARY\CONFIG.BIN
File(DEBUG.BIN) 00:25:07 7 82 00:24:23 07/20/1998 22:43:42 14336 Form1 C:\BIODS\DISC\BINARY\DEBUG.BIN
File(DIEDEMO.BIN) 00:25:14 8 89 00:24:23 07/20/1998 22:43:42 14536 Form1 C:\BIODS\DISC\BINARY\DIEDEMO.BIN
File(ENDING.BIN) 00:25:22 24 97 00:24:23 07/20/1998 22:43:42 49152 Form1 C:\BIODS\DISC\BINARY\ENDING.BIN
File(MEM_CARD.BIN) 00:25:46 12 121 00:24:23 07/20/1998 22:43:42 23932 Form1 C:\BIODS\DISC\BINARY\MEM_CARD.BIN
File(OPENING.BIN) 00:25:58 7 133 00:24:23 07/20/1998 22:43:42 14336 Form1 C:\BIODS\DISC\BINARY\OPENING.BIN
File(RESULT.BIN) 00:25:65 9 140 00:24:23 07/20/1998 22:43:42 17360 Form1 C:\BIODS\DISC\BINARY\RESULT.BIN
File(SELECT.BIN) 00:25:74 10 149 00:24:23 07/20/1998 22:43:42 18760 Form1 C:\BIODS\DISC\BINARY\SELECT.BIN
Dir(BSS) 00:26:09 4 159
File(BH1TIT.TIM) 00:26:13 76 163 00:26:09 07/20/1998 22:43:42 153620 Form1 C:\BIODS\DISC\BSS\BH1TIT.TIM
....and here is a partial build log provided by mkpsxiso v1.07 test:
BIN LBA:23
+-BIO1.BIN;1 LBA:24
+-BIO2.BIN;1 LBA:32
+-BIO2P.BIN;1 LBA:40
+-BIO3.BIN;1 LBA:48
+-BOOT.BIN;1 LBA:56
+-CONFIG.BIN;1 LBA:68
+-DEBUG.BIN;1 LBA:82
+-DIEDEMO.BIN;1 LBA:89
+-ENDING.BIN;1 LBA:97
+-MEM_CARD.BIN;1 LBA:121
+-OPENING.BIN;1 LBA:133
+-RESULT.BIN;1 LBA:140
+-SELECT.BIN;1 LBA:149
BSS LBA:159
+-BH1TIT.TIM;1 LBA:255
NOTE: Everything seems to align perfectly, until a folder is reached/parsed and mkpsxiso somehow manages to go wrong. Specifically, notice that the "BSS\BH1TIT.TIM" file is supposed to be located at LBA sector 163, whereas mkpsxiso pushes it all the way back to 255.
On a side note, yes, this disc structure is obviously greatly different compared to the original RE2 ISO hahaha Remember, I did mention that I created my own file containers and such! A few simple modifications to the original executable here and there and voila. All of this is regardless to the matter at hand, however. Compiling with BUILDCD has never been a problem; I only didn't want to confuse you on that!
Ah, looks like the new LBA calculation method turned out to be a bit screwy as it calculates file LBAs a few sectors ahead of the directory record entry which also results in completely unwritten sectors that have no subheader information which wouldn't be good as CD drives needs it for tracking sectors during seeks.
Here's a new test build that fixes the LBA calculation bug:
http://lameguy64.github.io/mkpsxiso/mkpsxiso-1.08-test.zip
So... Problem solved and working great?
Forgive me for the much delayed response, as I haven't had any time to work with this at all.
I'm still experiencing a similar error, though it's not nearly as bad as before. Specifically, the LBA is still slightly off when a folder is encountered:
+-SELECT.BIN;1 LBA:149 BSS LBA:159 +-BH1TIT.TIM;1 LBA:161
The "BH1TIT.TIM" file should have an LBA of 163.
We're getting close!
In the "int iso::DirTreeClass::CalculateTreeLBA(int lba)" function, I've noticed this method will get the correct LBA:
// Set LBA of directory record of this class
recordLBA = lba;
// lba += (CalculateDirEntryLen(true)+2047)/2048;
int _Size = (numentries * 64);
int SectorSize = (_Size + 2048 - 1) / 2048 * 2048;
SectorSize /= 2048;
lba = (recordLBA + SectorSize);
I compiled the app with Code Blocks and yet the program is still buggy, outputting a corrupted ISO albeit with proper LBA entries. Playstation emulators refuse to boot it and ISO viewing apps such as CDmage spit out warnings about a corrupt file system.
Late response as I've been pretty busy on a bunch of things as of late, sorry about that.
I'm going to have to take a look at it more in-depth when I have the time. I think certain file directory records had to be padded a bit more by the looks of it.
Can you send me your test xml project and files?
No worries, I totally understand :)
Here ya go: https://drive.google.com/open?id=0B5megDVWOileRmJxVWJKSDZGV2s
Thank you for your time and patience with me!!
Okay, it looks like I have finally fixed the LBA bug! Turns out my LBA calculation routines we're a bit bugged which led to files being assigned inside a directory record that exceeds more than one record which then get overwritten by the directory record leading to data corruption.
Attached is a fresh new test build of mkpsxiso which contains the above fix as well as a few other bugfixes:
mkpsxiso-(1.14-test).zip
According to my tests, the LBA assignments are now more or less 1:1 to the buildcd logfile you provided. Your issue with mkpsxiso should be solved with this shiny new test build.
Excellent, thank you so very much! @Lameguy64 🥇
I've noticed that ISO applications still throw a fit when opening the output, but the game finally boots and plays fine now. The sizeof the output file doesn't match what BUILDCD creates, but regardless. It works now and I am very thankful for that! :)
You're welcome! I'm glad to hear that it is finally fixed. I'm still going to do some additional tweaking before pushing out an update.
What ISO programs were you using that threw a fit when you opened the images my tool generated though?
CDmage, specifically:
Well, CDmage still throws a fit when opening images generated with the latest release of MKPSXISO but it appears that it still works okay and displays all the files in the list despite the error. Looks like the ISO generation routines sill needs a bit more tweaking and will require analyzing ISO images further but for now, I need to do something else and I'll get to that in the future.
Just released version 1.15 of MKPSXISO which should fix issues like that when opening generated images with ISO tools. I've also done some tweaks to the file system encoding and at this point, ISOs generated by MKPSXISO should more or less be 100% ISO9660 compliant.
Excellent job well done, thank you :)