Galera Load Balancer segment fault
smartdoc opened this issue · 0 comments
Hi there!
I was trying to shot GLB for Galera cluster. In normal mode, it works correctly (mostly).
During verbose mode, it always take me to segfault.
I test it with CFLAGS -O2, result the same.
I test it on VirtualBox, Debian Wheezy 32-bit, 512Mb ram.
When i start it, i take a segfault:
root@galera1:~# glbd -w exec:"/root/glb-1.0.1/files/mysql.sh -uroot -proot" -t 3 0.0.0.0:3306 galera1:3308 galera2:3308 galera3:3308
glb v1.0.1 (epoll)
Incoming address: 0.0.0.0:3306, control FIFO: /tmp/glbd.fifo
Control address: none
Number of threads: 3, max conn: 493, nodelay: ON, keepalive: ON, defer accept: OFF, linger: OFF, daemon: NO, lat.count: 0, policy: 'least connected', top: NO, verbose: NO
Destinations: 3
0: 127.0.1.1:3308 , w: 1.000
1: 192.168.55.114:3308 , w: 1.000
2: 192.168.55.115:3308 , w: 1.000
Watchdog:
Address : exp setw state lat curw
127.0.1.1:3308 : + 1.000 READY 0.21315 1.000
192.168.55.114:3308 : + 1.000 READY 0.21868 1.000
192.168.55.115:3308 : + 1.000 NOTFOUND 0.00000 -1.000
Destinations: 3
Router:
Address : weight usage map conns
192.168.55.114:3308 : 1.000 0.000 N/A 0
127.0.1.1:3308 : 1.000 0.000 N/A 0
Destinations: 2, total connections: 0 of 493 max
Pool: connections per thread: 0 0 0
*** glibc detected *** glbd: realloc(): invalid pointer: 0x09b69230 ***
======= Backtrace: =========
/lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xb760ff01]
/lib/i386-linux-gnu/i686/cmov/libc.so.6(realloc+0x2bb)[0xb7615acb]
glbd[0x804d269]
======= Memory map: ========
08048000-0805d000 r-xp 00000000 08:01 13804 /usr/sbin/glbd
0805d000-0805e000 rw-p 00014000 08:01 13804 /usr/sbin/glbd
09b5b000-09b7c000 rw-p 00000000 00:00 0 [heap]
b2ca4000-b2cc0000 r-xp 00000000 08:01 129301 /lib/i386-linux-gnu/libgcc_s.so.1
b2cc0000-b2cc1000 rw-p 0001b000 08:01 129301 /lib/i386-linux-gnu/libgcc_s.so.1
b2cc6000-b2cc7000 ---p 00000000 00:00 0
b2cc7000-b34c7000 rw-p 00000000 00:00 0
b34c7000-b34c8000 ---p 00000000 00:00 0
b34c8000-b3cc8000 rw-p 00000000 00:00 0
b3cc8000-b3cc9000 ---p 00000000 00:00 0
b3cc9000-b44cb000 rw-p 00000000 00:00 0
b44cb000-b44cc000 ---p 00000000 00:00 0
b44cc000-b4ccc000 rw-p 00000000 00:00 0
b4ccc000-b4ccd000 ---p 00000000 00:00 0
b4ccd000-b54cd000 rw-p 00000000 00:00 0
b54cd000-b54ce000 ---p 00000000 00:00 0
b54ce000-b5cce000 rw-p 00000000 00:00 0
b5cce000-b5ccf000 ---p 00000000 00:00 0
b5ccf000-b64cf000 rw-p 00000000 00:00 0
b64cf000-b64d0000 ---p 00000000 00:00 0
b64d0000-b6cd0000 rw-p 00000000 00:00 0
b6cd0000-b6cd1000 ---p 00000000 00:00 0
b6cd1000-b7592000 rw-p 00000000 00:00 0
b7592000-b759c000 r-xp 00000000 08:01 134570 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so
b759c000-b759d000 r--p 00009000 08:01 134570 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so
b759d000-b759e000 rw-p 0000a000 08:01 134570 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so
b759e000-b759f000 rw-p 00000000 00:00 0
b759f000-b76fb000 r-xp 00000000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so
b76fb000-b76fc000 ---p 0015c000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so
b76fc000-b76fe000 r--p 0015c000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so
b76fe000-b76ff000 rw-p 0015e000 08:01 134576 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so
b76ff000-b7703000 rw-p 00000000 00:00 0
b7703000-b7718000 r-xp 00000000 08:01 134565 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so
b7718000-b7719000 r--p 00014000 08:01 134565 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so
b7719000-b771a000 rw-p 00015000 08:01 134565 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so
b771a000-b771c000 rw-p 00000000 00:00 0
b771c000-b771e000 r-xp 00000000 08:01 134569 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so
b771e000-b771f000 r--p 00001000 08:01 134569 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so
b771f000-b7720000 rw-p 00002000 08:01 134569 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so
b7720000-b7727000 rw-p 00000000 00:00 0
b7727000-b7728000 r-xp 00000000 00:00 0 [vdso]
b7728000-b7744000 r-xp 00000000 08:01 129352 /lib/i386-linux-gnu/ld-2.13.so
b7744000-b7745000 r--p 0001b000 08:01 129352 /lib/i386-linux-gnu/ld-2.13.so
b7745000-b7746000 rw-p 0001c000 08:01 129352 /lib/i386-linux-gnu/ld-2.13.so
bfd7d000-bfd9e000 rw-p 00000000 00:00 0 [stack]
INFO: glb_signal.c:42: Received signal 6. Terminating.
Aborted
I looked up to messages:
root@galera1:~# cat /var/log/messages
Dec 20 15:40:06 galera1 kernel: [ 5426.947387] glbd[4956]: segfault at 135d ip b76452f4 sp b5d2cbdc error 4 in libc-2.13.so[b7601000+15c000
root@galera1:~# glbd -v -w exec:"/root/glb-1.0.1/files/mysql.sh -uroot -proot" -t 3 3306 galera1:3308 galera2:3308 galera3:3308
glb v1.0.1 (epoll)
Incoming address: 0.0.0.0:3306, control FIFO: /tmp/glbd.fifo
Control address: none
Number of threads: 3, max conn: 493, nodelay: ON, keepalive: ON, defer accept: OFF, linger: OFF, daemon: NO, lat.count: 0, policy: 'least connected', top: NO, verbose: YES
Destinations: 3
0: 127.0.1.1:3308 , w: 1.000
1: 192.168.55.114:3308 , w: 1.000
2: 192.168.55.115:3308 , w: 1.000
DEBUG: glb_wdog.c:156: Adding ' 127.0.1.1:3308 , w: 1.000' at pos. 0
DEBUG: glb_wdog.c:97: Created context for 127.0.1.1:3308
Segmentation fault
Trying to find in code, testing with debug output, clear out the segfault place:
File glb_wdog.c
line 186: pthread_create (&ctx->id, NULL, wdog->backend.thread, ctx);
int status;
status = pthread_create (&ctx->id, NULL, wdog->backend.thread, ctx);
if(status != 0)
printf("%i\n", status); // Not Execute !!!
Please, fix it.
Take my best wishes.
From Russia, with love.