Compiling on Mac, issues and workarounds
adriweb opened this issue · 33 comments
I guess we can centralize the Mac compiling issues and workarounds here:
Some changes to os/os-linux.c so that it works on Mac too :
#ifdef __unix__=>#if defined(__unix__) || defined(__APPLE__)- In addition to
#define _GNU_SOURCE, we need:#define _XOPEN_SOURCE
Changes in general :
- gcc and g++ have to be the GNU ones (for example
gcc-4.9when using brew or possibly other package managers), not the one Xcode provides/replaces (as they're actually clang), or else the FASTCALL things will error (see #2), and possibly other things.
That's a quick change in the qmake-resulting MakeFile, but I guess it could be done in the .pro somewhere when targeting macx ? - Compiling with the
-march=nativeflag results in assembly errors ({standard input}:244:no such instruction:vxorpd %xmm1, %xmm1,%xmm1'for example), which can be fixed by removing the flag, or, if supported, replacing it with-msse4.2` for instance (cf. this, as well as the other replies : http://stackoverflow.com/a/10894096/378298). The latter worked fine for me. #TODO: armsnippets.o generation
Well, yeah, that too :P
The problem is the "manual" binary segment handling by the GNU ld, not available on OS X.
Thearm-none-eabi-objcopypart works fine, and theobjcopytoo, once binutils is installed (brew...).
The farthest I got was with this trick from here : http://stackoverflow.com/a/13772389/378298.
It didn't complain anymore for the file, but later when some code was referring to the asm-defined things, it couldn't find them, so... not sure what to do from here.
(Note: that was before the switch to QT, although for this specific thing, it shouldn't matter)
Some changes to os/os-linux.c so that it works on Mac too:
#ifdef unix => #if defined(unix) || defined(APPLE)
In addition to #define _GNU_SOURCE , we need: #define _XOPEN_SOURCE
Ok, does the ucontext stuff work as well?
gcc and g++ have to be the GNU ones (for example gcc-4.9 when using brew or possibly other package managers), not the one Xcode provides/replaces (as they're actually clang), or else the FASTCALL things will error (see #2), and possibly other things.
That's a quick change in the qmake-resulting MakeFile, but I guess it could be done in the .pro somewhere when targeting macx ?
The toolchain has to be specified to qmake, it's not possible to override them. So to build on mac with Qt Creator, you'd have to create a custom compiler entry and combine it with a suitable Qt version.
Compiling with the -march=native flag results in assembly errors ( {standard input}:244:no such instruction: `vxorpd %xmm1, %xmm1,%xmm1' for example), which can be fixed by removing the flag, or, if supported, replacing it with -msse4.2 for instance (cf. this, as well as the other replies : http://stackoverflow.com/a/10894096/378298). The latter worked fine for me.
It was only for testing anyway and didn't perform better in any way, so it's not an issue to remove it.
#TODO: armsnippets.o generation
Well, yeah, that too :P
The problem is the "manual" binary segment handling by the GNU ld, not available on OS X.
The arm-none-eabi-objcopy part works fine, and the objcopy too, once binutils is installed (brew...).
The farthest I got was with this trick from here : http://stackoverflow.com/a/13772389/378298.
It didn't complain anymore for the file, but later when some code was referring to the asm-defined things, it couldn't find them, so... not sure what to do from here.
(Note: that was before the switch to QT, although for this specific thing, it shouldn't matter)
I guess we can just provide the ARM asm and a header file containing the snippets.bin code generated by xxd.
Ok, does the ucontext stuff work as well?
Ah, no, sorry, forgot to mention that - I had to comment out the middle part of the function.
The toolchain has to be specified to qmake, it's not possible to override them. So to build on mac with Qt Creator, you'd have to create a custom compiler entry and combine it with a suitable Qt version.
Hmmm not sure what you mean, I didn't use Qt Creator, just the terminal. But I'll look around. I have version 5.3.2 btw.
Edit: oh wait, I think I know what you mean - I'll report back later
It was only for testing anyway and didn't perform better in any way, so it's not an issue to remove it.
Oh well, ok then.
I guess we can just provide the ARM asm and a header file containing the snippets.bin code generated by xxd.
Ah, if that works, it's all good. Let me know when you want me to test it once you figure it out.
You can test it right now, should work out of the box (except ucontext) for mac, if you configured it right.
You can try "qmake -spec macx-g++" for example.
I got the notification of your reply just after I pushed.
Almost !
Got this at the end :
Undefined symbols for architecture x86_64:
"arm_shift_proc", referenced from:
_translate in translate.o
(maybe you meant: _arm_shift_proc)
"translation_next", referenced from:
_translate in translate.o
(maybe you meant: _translation_next, _translation_next_bx )
"translation_next_bx", referenced from:
_translate in translate.o
(maybe you meant: _translation_next_bx)
ld: symbol(s) not found for architecture x86_64
So I assumed something went wrong with the asm file ?
(nothing suspicious in the build log ( http://pastebin.com/U9PPJNRJ ), though ?)
Can you try deleting your build dir and starting from the beginning? Those symbols are defined in asmcode.c
It's actually already a clean build.
In asmcode.c, I see the functions defined, but just prototypes, though, no body.
And in translate.c, they're extern.
Oh, there you go, in translate.c, it needed the underscores (as suggested :P)
extern void translation_next() __asm__("_translation_next");
extern void translation_next_bx() __asm__("_translation_next_bx");
extern uint32_t arm_shift_proc[2][4] __asm__("_arm_shift_proc");
These three are defined as void* at the top of asmcode.c. Can you do "nm asmcode.o" in the build dir and verify that it's correct? Or maybe it tries to assemble asmcode.S, could to look at the build log?
nm asmcode.o:
U addr_cache
U addr_cache_miss
0000000000000008 C arm_shift_proc
U data_abort
U _GLOBAL_OFFSET_TABLE_
U mmio_read_byte
U mmio_read_half
U mmio_read_word
U mmio_write_byte
U mmio_write_half
U mmio_write_word
0000000000000090 T read_byte
0000000000000120 T read_half
00000000000001b0 T read_word
0000000000000000 T read_word_ldr
0000000000000008 C translation_next
0000000000000008 C translation_next_bx
0000000000000240 T write_byte
00000000000002d0 T write_half
0000000000000370 T write_word
And yup, it gave me this :
...
00000000000003c0 s EH_frame1
U _addr_cache
0000000000000008 C _arm_shift_proc
0000000000000008 C _translation_next
0000000000000008 C _translation_next_bx
U addr_cache_miss
U data_abort
...
which explains why it works when adding the underscores to the source file.
Another platform-specific thing ?
Oh, there you go, in translate.c, it needed the underscores (as suggested :P)
extern void translation_next() asm("_translation_next");
extern void translation_next_bx() asm("_translation_next_bx");
extern uint32_t arm_shift_proc[2][4] asm("_arm_shift_proc");
Ninja'd - I didn't type fast enough >:-(
But yeah, that's how it looks like. If you fix the hardcoded paths in emuthread.cpp, it should start.
There is some console output in case of errors.
Hum, it "starts", but immediately quits, when given a cx boot1 and a cx cas flash.
There is absolutely no log, though.
Trying with the debug flag, the only thing I got was 00000000: e59ff018 ldr pc,[00000020] = 00003b70, which might be normal (?), but then it exists.
I'll try different args for emulate();
"Good news", thanks to http://stackoverflow.com/a/7093839/378298, this seems to work for Mac : u->uc_mcontext->__ss.__rip
Now, I get a log before it crashes (better than nothing :D).
[nspire_emu] Got SIGSEGV trying to access 0x19301020 (RIP=0x376f)
[nspire_emu] Got SIGSEGV trying to access 0x30 (RIP=0x18795d0)
So, with some more #if trickery, addr_cache_exception should be good on linux + mac now.
(Edit : while trying some more, I now only get the first error, no more the one with 0x30, which looked surprising anyway)
Trying with the debug flag, the only thing I got was 00000000: e59ff018 ldr pc,[00000020] = 00003b70 , which might be normal (?), but then it exists.
That is normal, the debugger output is redirected to stdout for now and you can enter debug commands like "s" on the line input at the bottom of the debug tab.
"Good news", thanks to http://stackoverflow.com/a/7093839/378298, this seems to work for Mac : u->uc_mcontext->__ss.__rip
This shouldn't be needed anyway, as the segv handler must not be called :-/
Can you do the following: -step through boot1 with the debugger and tell me at which address it crashes
-run nspire_emu with gdb and give me source line and some more info about the crash cause?
(Edit : while trying some more, I now only get the first error, no more the one with 0x30, which looked surprising anyway)
Heisenbug? Please not.
"Good news", thanks to http://stackoverflow.com/a/7093839/378298, this seems to work for Mac : u->uc_mcontext->__ss.__rip
This shouldn't be needed anyway, as the segv handler must not be called :-/
Well, it's comforting to finally have some output ^^ And anyway at least it's on par with linux now.
Can you do the following: -step through boot1 with the debugger and tell me at which address it crashes
Well, it crashes at the very first instruction decoding, so there is not much to step through... (see below)
-run nspire_emu with gdb and give me source line and some more info about the crash cause?
I configured Qt Creator to debug and run it correctly, so I'm working on it from there now.
Here's what I see : (I put the values in hex, at the top right)
Again, this is for the very first decoded instruction etc., so first time it goes through all that.
Tell me if you need more tests or specific configuration.
Ok, doesn't look that good. What is mem_and_flags in this case?
Can you access mem_and_flags[0]?
mem_and_flags is at 0x10041FA00 and shows a value of "0x18301000", but I can't access element 0, nope. (Also, "data variable, no debug info", on the array)
The memory editor shows this, though : https://i.imgur.com/HUMQiII.png
Well, that's the address of the pointer.. It's returned by mmap and as it's not -1, mmap succeeded. I don't know why it segv'd.
What happens if you disable ASLR and replace mmap in os_reserve with
void *ptr = mmap((void*)0x70000000, size, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_SHARED|MAP_ANON, -1, 0);
Hmm, with the modification and -Wl,-no_pie, it's now crashing giving me this :
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x176b of process 22186]
0x00007fff91d02c69 in _platform_bzero$VARIANT$Unknown () from /usr/lib/system/libsystem_platform.dylib
Not much better :/
(the flag was taken from here : http://stackoverflow.com/a/24219734/378298, and seems to be what's necessary to disabled ASLR)
This looks like it crashes before executing main...
That might explain why :
fprintf(stderr, "ptr = %p", ptr); gives : ptr = 0xffffffffffffffff
So mmap failed. But why? What does perror report?
No more than mmap: Cannot allocate memory
I tried the MAP_NORESERVE mask as http://stackoverflow.com/a/4804448/378298 suggests, but still same thing. I have plenty of RAM available though.
What happens if you change some of the flags and change the pointer to 0x0?
It will crash later, though, if mem_and_flags has bit 31 set.
No luck changing to 0 and messing around with the flags...
The only way mmap didn't fail is when I don't set MAP_FIXED
I also tried malloc, but I get the same error as before : [nspire_emu] Got SIGSEGV trying to access 0x18301020 (RIP=0x3634)
No luck changing to 0 and messing around with the flags...
The only way mmap didn't fail is when I don't set MAP_FIXED
That's a good thing, it's not necessary ;-) It should work if you remove it.
I also tried malloc, but I get the same error as before : [nspire_emu] Got SIGSEGV trying to access 0x18301020 (RIP=0x3634)
Yeah, the value returned by malloc doesn't fit into the 31 bits, so addr_cache overflows.
Well, when I remove it, it's like before :
mem size to reserve : 0x8200000
mmap: Undefined error: 0
ptr = 0x118b01000
[nspire_emu] Got SIGSEGV trying to access 0x18b01020 (RIP=0x3633)
[nspire_emu] Got SIGSEGV trying to access 0x0 (RIP=0x15e9622)
(hey look, the 0x0 heisenbug is back, it's been a while...)
(hey look, the 0x0 heisenbug is back, it's been a while...)
Great :-/
Well, when I remove it, it's like before :
At least mmap doesn't fail. The issue is, if size is 0x8200000 and the pointer returned by mmap is 0x118b01000, why is access to 0x18b01020 forbidden? Can you do some independant tests to allocate 0x8200000 bytes with mmap? It may be an issue that mmap doesn't run in the main thread or isn't reentrant or whatever.
BTW: The first instruction is "ldr pc, [20]", so that works :-)
Edit: Just saw it, theres a '1' missing. mmap doesn't like to allocate memory below 32bits apparently.
Edit: Just saw it, theres a '1' missing. mmap doesn't like to allocate memory below 32bits apparently.
Oh, yeah, good catch...
Now, how to workaround/fix that... :P
Edit : I can try some mmap testing "sometime later".
I guess addr_cache needs to be more flexible... That has to be fixed anyway as the current way with fixed adresses for everything is just horrible.
OK.
In case that helps, here's what valgrind has to say about it :
http://pastebin.com/fLRgU6Gr
(Edit : that's with the original mmap)
Doesn't really help, just a longer segv trace.
Anyway, I think I managed to get a 64bit addr_cache working. It isn't 100% reliable, though, you should try more than once.
Hey, would you look at that.... :)
Thanks !