icepic/mdump

mdump

This is a tool to interpet and show information generated b y my experimental changes to OpenBSD's malloc.

Please note that this will only work on a current OpenBSD source tree. At the moment of writing this that means newer than Oct 12 2020.

First step is to apply the diff to src/lib/libc/stdlib/malloc.c, recompile and install libc:

$ cd /usr/src/lib/libc
$ patch < malloc.diff
$ make obj
$ make
$ doas make install

After this, malloc will generate utrace(2) records when instructed to do so. These ulog records are written to a file on regular exit of your program

The trace file is generated by running your program with

MALLOC_OPTIONS=DT ktrace -tu program

The MALLOC_OPTIONS instruct malloc to write the infomration and the ktrace invocation takes care of catching the utrace records and writing themn to a file, by default called ktrace.out. After that you can use the mdump program to display the information.

Remeber to build your program and libraries with debug information. On OpenBSD system libraries are installed with debug info, so that's convenient. To get deeper stack trace information on leaks use more T's. Note that the mechanism used (__builtin_return_address) is not guaranteed to work for more than one T, some executables on some platforms will crash when too many T's are used.

To compile mdump, you'll need to elftoolchain package.

# doas pkg_add elftoolchain
$ make obj
$ make
$ doas make install

This wil install the tool and man page into /usr/local.

Basic usage is

  mdump

to show leak info and

  mdump -D

to show a dump of malloc's internal state at program exit.

To produce readable stack traces, the program and its libraries should be compiled with debug information, typically -g. Statically linked programs must be compiled with the -nopie option. High optimization levels can produce debug information that mdump cannot interpret. Setuid or setgid processes can only be traced by the superuser.

Forking programs can be traced by using the -i flag to ktrace(1), but selecting a single process is needed when running mdump on the trace file generated.

An example run

The program:

#include <stdio.h>
#include <stdlib.h>

char *p;

void f(void)
{
        int i;
        void *s[64];

        p = realloc(p, 4096);
        for (i = 0; i < 64; i++)
                s[i] = malloc(4096);
        p = realloc(p, 8192);
        for (i = 1; i < 64; i++)
                free(s[i]);

}

int main()
{
        int i;

        p = malloc(100);
        for (i = 0; i < 10; i++)
                f();

        printf("done\n");
}

Compile, run and show leaks:

$ cc -g x.c 
$ MALLOC_OPTIONS=DTTTT ktrace -tu ./a.out 
done
$ mdump
Leak sum=40960 count=10 avg=4096
 f at /home/otto/x.c:13
 main at /home/otto/x.c:25
 ?? at ??:0

Leak sum=8192 count=1 avg=8192
 f at /home/otto/x.c:14
 main at /home/otto/x.c:25
 ?? at ??:0

Leak sum=65536 count=1 avg=65536
 __smakebuf at /usr/src/lib/libc/stdio/makebuf.c:62
 __swsetup at /usr/src/lib/libc/stdio/wsetup.c:75
 __vfprintf at /usr/src/lib/libc/stdio/vfprintf.c:460
 vfprintf at /usr/src/lib/libc/stdio/vfprintf.c:264

$

How does it work?

While running (and if enabled by MALLOC_OPTIONS) malloc stores backtrace information for allocations. Per stackframe the return address is recorded. Any allocation equal or larger than a page is tracked. For smaller allocations, only a sample is recorded. This means that not all leaks will be shown for smaller allocations.

At program exit, malloc will check which allocations are not freed, translate the addresses into library/executable plus offset information using dladdr(3) and construct utrace records to send out.

The mdump program then takes this information and translates the library plus offset information into function name + file + linenumber information using the debug information embedded in the program and its libraries.