"Stack Trace Support" requirement breaks simple use cases
Closed this issue · 8 comments
Hey there,
This is more of a general inquiry than a code issue but it appears that this is the most active channel of communication so I figured it would be best to ask here.
I help run an Operating Systems course at Columbia University and we encourage students to use KEDR to check for memory leaks in their linux kernel module assignments.
However, we found that a95e849 breaks simple use cases and requires that students recompile their kernels with CONFIG_STACKTRACE enabled.
While this is feasible for the later assignments in the class where students are given a linux kernel to compile to begin, our first assignment is meant to be straightforward and have students insert modules into the kernel they are already running in their VM (we have students run Debian Buster). This is before we introduce kernel compilation in the semester.
We got around that issue this semester by simply asking students to checkout to the commit before a95e849 for the first assignment. While this does alleviate the issue of requiring kernel recompilation for KEDR to work, I'm curious as to how feasible this solution is going forward as we continue to upstream the linux kernels we use (currently on 4.19.50). I'd also appreciate more background information on the motivation behind this commit so we can convey it to students if they're curious.
Thanks!
Hi,
Interesting. I thought KEDR is very rarely used these days, now that all flavours of K*San and other tools are available. Most of these require rebuilding the kernel though, so, I see the benefit of using KEDR in your case.
I added the requirement of CONFIG_STACKTRACE because it makes it much easier to adapt KEDR to e9b98e162aa5 "stacktrace: Provide helpers for common stack trace operations" from mainline kernel 5.2 (31e086a).
Missed the fact that Debian still refuses to enable CONFIG_STACKTRACE by default, even now when kernel live patching, which needs reliable stack traces, becomes more and more common. Well, Debian has always been ... special ;-)
Before a95e849, I kept the implementation of save_stack_trace() in KEDR itself, for the systems where it is not available. Without it, the memory leak detector would not output meaningful stack traces, making it pretty much useless.
I think, for now, it is OK for your students to use that earlier revision of KEDR.
In the meantime, I'll see what I can do to restore support for the systems without CONFIG_STACKTRACE in KEDR.
Hm. I have just installed Debian 10.4 in a VM, and its default kernel 4.19.0-9-amd64 does have CONFIG_STACKTRACE=y.
Perhaps, some other kernels used in Debian might not have it.
You said, you were using version 4.19.50 - is it a stock kernel provided by Debian or a custom-built one?
If it is a custom-built kernel, I'd like to see its config: there could be other pitfalls there.
Thank you so much for your fast response! We really appreciate the support :^)
I might have been wrong about my initial guess that CONFIG_STACKTRACE is what is causing issues. For more context, the students have been running debian 10.2 (the stable release at the time in January). The config file I found at /boot/config-4.19.0-6-amd64 does have CONFIG_STACKTRACE enabled. Yet, I get the following error when I build from source:
hans@porygon:~/kedr/sources/build $ cmake .. && sudo make install
CMake Deprecation Warning at CMakeLists.txt:6 (cmake_policy):
The OLD behavior for policy CMP0054 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
-- Build type is "Release"
-- The C compiler identification is GNU 8.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 8.3.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Global installation
-- Found Kbuild: /lib/modules/4.19.0-6-amd64/build (Required is at least version "3.10.0")
-- Checking if kernel modules can be built on this system
-- Checking if kernel modules can be built on this system - yes
-- Checking which memory allocator is used by the kernel
-- Checking which memory allocator is used by the kernel - slub
-- Checking if stack trace information is reliable
-- Checking if stack trace information is reliable - no
CMake Error at cmake/modules/kmodule.cmake:206 (message):
It looks like reliable stack traces cannot be obtained on this system.
Please rebuild the kernel with CONFIG_STACKTRACE=y and
CONFIG_FRAME_POINTER=y or CONFIG_STACK_UNWIND=y (if available) and then
reconfigure and rebuild KEDR.
Call Stack (most recent call first):
CMakeLists.txt:305 (check_stack_trace)
-- Configuring incomplete, errors occurred!
See also "/home/hans/kedr/sources/build/CMakeFiles/CMakeOutput.log".
I also just installed a Debian 10.4 VM and encountered the same build error. That kernel also has CONFIG_STACKTRACE=y, as you observed.
Any idea what could be happening here? If it helps, I've dumped the contents of CMakeOutput.log here (built from my Debian 10.2 VM): https://pastebin.com/Lh27XGLj
For what it's worth, neither Debian 10.2 nor Debian 10.4's stock kernels even have CONFIG_FRAME_POINTER or CONFIG_STACK_UNWIND as build configurations. Could this be causing the "reliable stack trace" check to fail?
Ah, makes sense. Debian is one of the rare distros that do not like to enable CONFIG_FRAME_POINTER, hoping to get some performance gains, in theory.
They do enable CONFIG_UNWINDER_ORC=y, it seems, which, in theory, should be enough. I'll re-check if it is.
Yes, both "frame pointer" and "ORC" unwinders produce reliable stack traces.
I have pushed the fix: cc51514, seems to work OK.
Please check if it works for you as well.
Just built KEDR on Debian 10.2 and 10.4. LGTM
Thank you so much for looking into this and I hope you’re staying safe and healthy!
I hope you’re staying safe and healthy!
I am, thanks! Wish the same to you.
Just built KEDR on Debian 10.2 and 10.4. LGTM
Glad it helped!
And - good to know that KEDR is still useful to someone besides myself ;-)