samsk/log-malloc2

Invalid log in multithreaded app on Linux

Opened this issue · 0 comments

I am trying to run log-malloc-2 on the CentOS 7 with my highly-multithreaded applications (runs tens of threads) and getting strange, invalid output in the log (executable name and most function names in stack traces are intentionally obfuscated):

+ malloc 24 0x1e65f40 [895997:990912] #84410 3015 2364 2746 0 34282 0
/lib64/libstdc++.so.6(_Znwm+0x1d)[0x7f54d1cde18d]
+ malloc 43 0x7f54c00008e0 [895973:990856] #84410 3015 2364 2746 0 34282 0
+ malloc 56 0x7f54c0000940 [896053:991000] #84410 3015 2364 2746 0 34282 0
+ malloc 31 0x7f54c00009a0 [896084:991072] #84410 3015 2364 2746 0 34282 0
+ malloc 31 0x7f54c00009f0 [896115:991144] #84410 3015 2364 2746 0 34282 0
+ malloc 37 0x7f54c0000a40 [896152:991216] #84410 3015 2364 2746 0 34282 0
+ free -31 0x7f54c00009f0 [896121:991144] #84410 3015 2364 2746 0 34282 0
+ malloc 49 0x7f54c0000a90 [896170:991232] #84410 3015 2364 2746 0 34282 0
+ free -37 0x7f54c0000a40 [896133:991160] #84410 3015 2364 2746 0 34282 0
+ free -31 0x7f54c00009a0 [896102:991088] #84410 3015 2364 2746 0 34282 0
+ free -49 0x7f54c0000a90 [896053:991000] #84410 3015 2364 2746 0 34282 0
+ malloc 376 0x7f54c0000af0 [896429:991408] #84410 3015 2364 2746 0 34282 0
+ malloc 40 0x7f54c00009a0 [896469:991480] #84410 3015 2364 2746 0 34282 0
./my_application(_myf2+0x33a)[0x68500a]
+ malloc 537 0x7f54c0000c90 [897006:992064] #84410 3015 2364 2746 0 34282 0
+ malloc 70 0x7f54c0000ee0 [897076:992168] #84410 3015 2364 2746 0 34282 0
+ free -70 0x7f54c0000ee0 [897006:992064] #84410 3015 2364 2746 0 34282 0
+ malloc 68 0x7f54c0000ee0 [897074:992168] #84410 3015 2364 2746 0 34282 0
+ free -68 0x7f54c0000ee0 [897006:992064] #84410 3015 2364 2746 0 34282 0
./my_application(_myf1+0xbf)[0x6858af]
./my_application(_myf00+0x59b)[0x6841fb]
./my_application(_myf000+0x33)[0x690ed3]
./my_application(_myf0000+0xb7)[0xac683b]
[0x0]

i.e. multiple malloc appear w/o stack trace at all, first malloc has only one frame of the stack trace and later on, more frames appear after free. It looks to me as output synchronization issue, but I've tried to look at log-malloc sources and seems like synchronization is present and properly done. So what could be the reason?

Configuration: log-malloc compiled with GCC 4.8.5 on CentOS 7. I have pthreads, libunwind but don't have "unwind details". Application under log-malloc compiled with GCC 7.3.1 (from the RedHat Devtoolset 7) on CentOS 7 and actively uses pthreads.