drhelius/Gearboy

Core not usable on PS3

crystalct opened this issue · 8 comments

Beetwen v3.0.4 and v3.1.0 was introduced an issue on PS3 that makes Gearboy non usable.
RPCS3 emulator send an not well identificated sys_memory_allocate' failed.
Further investigation will be required.....

I don't have the hardware to test this, any help is welcome.
Can you confirm that 3.0.4 works on PS3?
I had some endianness issues in the past, maybe something related.

Commit 9b90dce introduced the issue.....
21 files changed... many of these about memory.... let's see.

Memory.cpp
m_pDisassembledROMMap = new stDisassembleRecord[MAX_ROM_SIZE]; cause:

exception: bad allocation
abort() is called from 0x000000000072cf78
                  from 0x000000000072bfbc
                  from 0x000000000072bed0
                  from 0x000000000049dd9c
                  from 0x0000000000496e04
                  from 0x0000000000494f60
                  from 0x00.........................

It's the same problem as libretro/RetroArch#11537 and libretro/RetroArch#11527
Is it possible save some stack ram using dynamic allocation of memory like this?

#ifdef __CELLOS_LV2__
         char *msg = (char*)malloc(8192 * sizeof(char));
#else
         char msg[8192];
#endif

#ifdef __CELLOS_LV2__
         free(msg);
#endif

I see what's going on.

AFAIK m_pDisassembledROMMap = new stDisassembleRecord[MAX_ROM_SIZE]; is actually using heap memory, it's using new but it is too much for the PS3 (around 500MB). My bad.

Those changes are intended for a desktop debugger but are impacting the libretro core.

I have to rework this whole memory map.

Thanks for the catch!

I'll try to fix this asap as it is definitely impacting Gearboy on all RetroArch systems with low memory, not only PS3.

Thanks to keep Gearboy endian safe.... many (young) programmers don't even know about the existence of Big Endian - little endian problem.