Static analysis logging

[vChavezB]

I am trying to do a static analysis of a C++ project which uses Tup. In general, some analyzers as Coverity and Codechecker, log the compiler calls that are done through the build system. I built the project with Tup but for example Codechecker could not log any calls to the c++ compiler (arm-none-eabi-gcc).

From this attempt I did, I would like to ask if Tup has been used with any static analysis tools succesfully (e.g. Klocwork, PVS Studio , Codechecker,Coverity) ?

[vChavezB]

Update: I can confirm that PVS Studio is able to detect the compiler calls in Windows.

When I have time I will probably check if Tup works with coverity.

[gittup]

Thanks for the report! I haven't used Codechecker before, but I gave it a try on Linux.

When you say it wasn't able to log any of the compiler calls, you mean for the CodeChecker log --build "tup" --output ./compile_commands.json step, right? I was seeing an empty file on this step, so presumably that's what you hit as well. I don't have a great solution at the moment, but there are a few avenues to pursue:

1. Try to get the logging working.
   Presumably there's some instrumentation that it's doing that conflicts with tup. This could be from environment variables that Codechecker needs, but tup is clearing out since presumably they are not in any 'export' lines in the Tupfiles. Another possibility is maybe Codechecker or clang uses process groups, which tup also uses (via setpgid() / getpgid()) to allow access to the FUSE filesystem. (I tried running 'CodeChecker check ...' as a rule in tup, and saw some pgid warnings when using --debug-fuse to watch file accesses, but removing that warning didn't seem to help). The FUSE filesystem itself could be causing some issue, though I was able to run CodeChecker fine when using the example passthrough filesystem in FUSE, which is very similar to the one tup uses. Still, there could be some difference there that is causing a conflict.

2. Use a tup generate script as the build for logging.
   Instead of calling CodeChecker on tup directly, you can have tup generate a one-off build script, like tup generate build-script.sh, and then run CodeChecker log --build "sh build-script.sh" --output ./compile_commands.json. There's no FUSE instrumentation or environment variable scrubbing when running the build script, so that works fine. It does feel a little clunky this way, but if you are doing it as part of a build server somewhere it might not matter. Using 'tup generate' inside a project that you also plan to build with 'tup' directly is likely going to be frustrating though, since 'tup generate' expects a clean tree.

3. Use tup compiledb to create compile_commands.json instead of CodeChecker log
   Although it's not part of a released version of tup yet, on the master branch you can run tup compiledb to generate a compile_commands.json directly (one for each variant, if you're using variants). You just need to annotate which commands you want exported into the json file with the ^j flag, but that should be straightforward to stick into your compilation rule. Once you have the ^j flags in place and run tup compiledb, you can skip straight to the CodeChecker analyze step.

4. Use tup to run clang --analyze directly.
   I only have a shallow understanding of what CodeChecker does, but it looks to me like clang is doing the actual code analysis, and then CodeChecker groups the individual analysis files into reports and has support for doing incremental analysis independent of the underlying build system (I think?). You could instead bypass CodeChecker and just add clang --analyze ... rules to the Tupfiles, so they can get run alongside the actual compilation commands. Some caveats: CodeChecker passes in a bunch of specific -Xclang flags to turn on individual analyzers, while clang --analyze seems to have a different default set. Also clang --analyze doesn't fail with an error code if it finds any issues, so to make the build fail you'd have to check if the output is non-empty (unless there's a flag similar to -Werror that works in this case?). A wrapper script like this would be helpful:

output=$(clang --analyze "$@" 2>&1)
if [ "$output" ]; then
        echo "$output"
        exit 1
fi   

Then you can have a rule that does:

: foreach *.c |> sh check-helper.sh -c %f -o %o $(CFLAGS) ... |> %B.plist

Wish I had a better answer for getting the whole thing working. It might be a very easy fix but I'm just missing it.

[vChavezB]

Thanks for the detailed answer. I have tested your second option as follows:

tup init
tup generate tup-build.sh
CodeChecker log --build "sh tup-build.sh >tup.out 2>&1" -q -o codechecker.log >logger.out 2>&1

And it works :)

Not good for local testing, as you mentioned it needs a clean build but for the gitlab server I have running it's fine.