shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed on small reconnect intervals
filipecosta90 opened this issue · 0 comments
filipecosta90 commented
Similar to #232
Steps to reproduce:
setup env
./tests/gen-test-certs.sh
redis-server --tls-port 12000 --port 0 --tls-cert-file ./tests/tls/redis.crt --tls-key-file ./tests/tls/redis.key --tls-ca-cert-file ./tests/tls/ca.crt --tls-auth-clients no
run benchmark and trigger failed assert
memtier_benchmark -p 12000 --key-maximum=1000000 -c 100 -t 8 --ratio=1:10 --key-pattern=P:P --pipeline=8 --test-time=300 --tls --cacert=./tests/tls/ca.crt --reconnect-interval=1 --tls-skip-verify
run with debugger
gdb -- `pwd`/memtier_benchmark -p 12000 --key-maximum=1000000 -c 100 -t 8 --ratio=1:10 --key-pattern=P:P --pipeline=8 --test-time=300 --tls --cacert=`pwd`/tests/tls/ca.crt --reconnect-interval=1 --tls-skip-verify
sample log:
$ gdb --args `pwd`/memtier_benchmark -p 12000 --key-maximum=1000000 -c 100 -t 8 --ratio=1:10 --key-pattern=P:P --pipeline=8 --test-time=300 --tls --cacert=`pwd`/tests/tls/ca.crt --reconnect-interval=1 --tls-skip-verify
GNU gdb (Ubuntu 12.1-3ubuntu2) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/fco/redislabs/memtier_benchmark/memtier_benchmark...
(gdb) c
The program is not being run.
(gdb) r
Starting program: /home/fco/redislabs/memtier_benchmark/memtier_benchmark -p 12000 --key-maximum=1000000 -c 100 -t 8 --ratio=1:10 --key-pattern=P:P --pipeline=8 --test-time=300 --tls --cacert=/home/fco/redislabs/memtier_benchmark/tests/tls/ca.crt --reconnect-interval=1 --tls-skip-verify
This GDB supports auto-downloading debuginfo from the following URLs:
https://debuginfod.ubuntu.com
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Writing results to stdout
[RUN #1] Preparing benchmark client...
[RUN #1] Launching threads now...
[New Thread 0x7ffff71ff6c0 (LWP 215302)]
[New Thread 0x7fffeffff6c0 (LWP 215303)]
[New Thread 0x7ffff69fe6c0 (LWP 215304)]
[New Thread 0x7ffff61fd6c0 (LWP 215305)]
[New Thread 0x7ffff59fc6c0 (LWP 215306)]
[New Thread 0x7ffff51fb6c0 (LWP 215307)]
[New Thread 0x7ffff49fa6c0 (LWP 215308)]
[New Thread 0x7fffef7fe6c0 (LWP 215309)]
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
connect: resolve error: System error
memtier_benchmark: shard_connection.cpp:481: void shard_connection::process_response(): Assertion `ret == 0' failed.
Thread 9 "memtier_benchma" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffef7fe6c0 (LWP 215309)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
Download failed: Invalid argument. Continuing without source file ./nptl/./nptl/pthread_kill.c.
44 ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3 0x00007ffff723bc46 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4 0x00007ffff72227fc in __GI_abort () at ./stdlib/abort.c:79
#5 0x00007ffff722271b in __assert_fail_base (fmt=0x7ffff73bac30 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55555558cdde "ret == 0", file=0x55555558aefc "shard_connection.cpp", line=481,
function=<optimized out>) at ./assert/assert.c:92
#6 0x00007ffff7233596 in __GI___assert_fail (assertion=assertion@entry=0x55555558cdde "ret == 0", file=file@entry=0x55555558aefc "shard_connection.cpp", line=line@entry=481,
function=function@entry=0x55555558ad18 "void shard_connection::process_response()") at ./assert/assert.c:101
#7 0x0000555555567b60 in shard_connection::process_response (this=0x55555a9b8110) at shard_connection.cpp:481
#8 0x00007ffff7f614bc in bufferevent_run_readcb_ () from /lib/x86_64-linux-gnu/libevent-2.1.so.7
#9 0x00007ffff7fa2854 in ?? () from /lib/x86_64-linux-gnu/libevent_openssl-2.1.so.7
#10 0x00007ffff7fa2895 in ?? () from /lib/x86_64-linux-gnu/libevent_openssl-2.1.so.7
#11 0x00007ffff7f6bfd5 in ?? () from /lib/x86_64-linux-gnu/libevent-2.1.so.7
#12 0x00007ffff7f6d87f in event_base_loop () from /lib/x86_64-linux-gnu/libevent-2.1.so.7
#13 0x000055555555e822 in cg_thread_start (t=0x55555a389be0) at memtier_benchmark.cpp:1115
#14 0x00007ffff7290402 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#15 0x00007ffff731f590 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) q
A debugging session is active.
Inferior 1 [process 215272] will be killed.
Quit anyway? (y or n) y