Running libfuzzer with /MD on Windows & MSVC is using incontrollable memory consumption?
rajeemm opened this issue · 0 comments
rajeemm commented
I'm trying to fuzz test my library which compiled on Windows using /MD (C Runtime) with MSVC compiler.
I get incontrollable memory consumption while using libfuzzer
Repro steps:
- I cloned LLVM repo
- To allow MD compilation of LLVM : at https://github.com/llvm/llvm-project/blob/master/compiler-rt/CMakeLists.txt#L337, I replaced the block of
if(COMPILER_RT_HAS_MT_FLAG)
block withinclude(ChooseMSVCCRT)
(I know it's not 100% supported, but I think it will be and I'm not sure if it's the cause?) - Compile LLVM with LLVM_USE_CRT_RELEASE=MD
- Compile my own code using Windows 10, MSVC compiler (v14.28.29333) while linking libfuzzer libs:
'clang_rt.fuzzer-x86_64.lib','libsancov.lib', 'clang_rt.asan_dynamic-x86_64.lib', 'clang_rt.asan_dynamic_runtime_thunk-x86_64.lib'
(some from MSVC libs dir, some from LLVM compilation) and using CXXFLAGS:/fsanitize=address', '/fsanitize-coverage=inline-8bit-counters', '/fsanitize-coverage=edge', '/fsanitize-coverage=trace-cmp', '/fsanitize-coverage=trace-div'
. - Libfuzzer crashes because of too much memory usage (even with -rss_limit_mb=4000), so I did a sanity check to make sure no memory leak on my code:
- Run 1 (with libfuzzer libs & CXXFLAGS as explained above):
FUZZ_EXPORT int __cdecl LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
const char* tt = "66";
RunFuzzingTest((size_t)2,(const char*)tt);
return 0;
}
- Run 2 (without libfuzzer libs & CXXFLAGS as above):
int main() {
while (true) {
const char* tt = "66";
RunFuzzingTest((size_t)2,(const char*)tt);
}
return 0;
}
- Results:
Run of (1 - libfuzzer with same input) resulted in huge and increasing memory usage (2.6GB after 5 mins)
Run of (2 - my own main with same input) resulted in pretty constant memory usage (10MB after more than 20 mins).
Thanks