beehive-lab/mambo

elf_loader fails when trying to execute MAMBO with qemu

umarcor opened this issue · 3 comments

umarcor commented

When trying to execute MAMBO (compiled for aarch64) in a x86_64 machine with qemu-user and qemu-user-binfmt, I get the following error with any of the test programs in subdir test:

$ ./dbm test/mmap_munmap
dbm: elf_loader/elf_loader.c:127: load_elf: Assertion `tmpmem != MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

$ ./dbm test/mprotect_exec
dbm: elf_loader/elf_loader.c:127: load_elf: Assertion `tmpmem != MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

$ ./dbm test/self_modifying
dbm: elf_loader/elf_loader.c:127: load_elf: Assertion `tmpmem != MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

$ ./dbm test/signals
dbm: elf_loader/elf_loader.c:127: load_elf: Assertion `tmpmem != MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

$ ./dbm test/load_store
dbm: elf_loader/elf_loader.c:127: load_elf: Assertion `tmpmem != MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)
$ file dbm
dbm: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=6fdc5668b8be289035afb34d6b94f8a2073a6f89, with debug_info, not stripped

$ file test/mmap_munmap
test/mmap_munmap: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=f455bb2d4e1611bda4d4208be72e372f78a9fabd, not stripped

$ file test/mprotect_exec
test/mprotect_exec: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=9e04f900f9c66b5a1d44ae05654c9ba28deda6ff, not stripped

$ file test/self_modifying
test/self_modifying: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=38e0550f60883f8c8629968321fb86f935a0ac76, with debug_info, not stripped

$ file test/signals
test/signals: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=26880b03cee6e95beb2ad95c3956b523a51a0a84, with debug_info, not stripped

$ file test/load_store
test/load_store: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=ec50e8372d12854f55cd56989062d16de12a295e, with debug_info, not stripped

If I copy the same binaries to an ARM machine, all of them work as expected.

I tried compiling MAMBO and the tests both natively in an ARM machine and cross-compiling them for aarch64 in a x86_64 host. The results are the same.

Moreover, when trying to run the tests without MAMBO, mmap_munmap produces the same error, but not the others:

$ ./test/mmap_munmap 
mmap_munmap: mmap_munmap.c:33: main: Assertion `alloc == MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

$ ./test/mprotect_exec 
main()

$ ./test/self_modifying 
This should be printed
This should also be printed

$ ./test/signals 
Simple signal handler: success
Signal after flushing the code cache: success
Test sigsuspend: success
Test against race conditions between code generation and signals: success
Test for missed signals: success
Test signal handling in fragments containing CB(N)Z: success
Test signal handling in fragments containing TB(N)Z: success
Test handling of a synchronous SIGTRAP signal: success
Test handling of a synchronous SIGILL signal: success
Test receiving SIGILL when no handler is installed
qemu: uncaught target signal 4 (Illegal instruction) - core dumped
Illegal instruction (core dumped)

$ ./test/load_store 
start: a64
end: a64

@lgeek, any hint about how to debug this?

lgeek commented

Last time I've tried to run MAMBO using qemu user mode emulation, I've ran into numerous qemu bugs. Use native hardware or system mode emulation. The assertion at elf_loader/elf_loader.c:127 should never fail when running natively. I'm curious to see what's causing it to fail in qemu, but I don't have a debug build available at the moment.

$ ./test/mmap_munmap 
mmap_munmap: mmap_munmap.c:33: main: Assertion `alloc == MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

That's a qemu bug. Please report it to them. According to the mmap manual:

If len is zero, mmap() shall fail and no mapping shall be established.

umarcor commented

Indeed, I stripped down mmap_munmap to:

#include <stdlib.h>
#include <assert.h>
#include <sys/mman.h>
#include <errno.h>
#include <stdio.h>

int main() {
  // Failing allocation
  void *alloc = mmap(NULL, 0, PROT_EXEC | PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
  
  printf("alloc: %d\n", alloc);
  printf("MAP_FAILED: %d\n", MAP_FAILED);
  printf("errno: %d\n", errno);
  
  assert(alloc == MAP_FAILED);

  return 0;
}

I cross compiled in x86_64:

aarch64-linux-gnu-gcc -static -std=gnu99 -lpthread test/mmap_qemu.c -o mmap_qemu

This is the output in x86_64:

$ ./mmap_qemu 
alloc: 0
MAP_FAILED: -1
errno: 0
mmap_qemu: test/mmap_qemu.c:15: main: Assertion `alloc == MAP_FAILED' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

This is the output in a Jetson TX1:

$ ./mmap_qemu 
alloc: -1
MAP_FAILED: -1
errno: 22

So, as you said, qemu is not handling mmap properly. It is returning a success (0) when it should be returning a failure (-1, MAP_FAILED). I reported it to them: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1783362

I'm closing this for now. Thanks!