updated code quality benchmark results for swarm 3.0
Closed this issue · 11 comments
hi @adrianzap would you consider updating swarm's results in your Code Quality Benchmark?
We've released swarm 3.0 a few days ago. The software works the same, but way faster thanks to @lczech and @torognes, so you should be able to run your pipeline with the new swarm without any modification. @torognes also did a fantastic job fixing warnings and modernizing our code base, so we expect an overall better quality value for swarm.
Thanks,
I updated the benchmark. Nice work!
Thanks @adrianzap
Your softwipe run seems to report more clang-tidy warnings than mine. It could be because I am using a 18.04 LTS Ubuntu. While I install a more recent Ubuntu VM, would it be possible for you to share the report files for swarm with me?
Here is what I obtain with a fresh Ubuntu 19.10 (zero clang-tidy warnings!):
Lines of pure code (excluding blank and comment lines): 7091
--- Running: COMPILER ---
Weighted compiler warning rate: 0.0 (0/7091)
Number of level 3 warnings (must be fixed): 0/7091
Number of level 2 warnings (should be fixed): 0/7091
Number of level 1 warnings (could be fixed): 0/7091
Detailled results have been written into softwipe_compilation_warnings_must_be_fixed.txt
Detailled results have been written into softwipe_compilation_warnings_should_be_fixed.txt
Detailled results have been written into softwipe_compilation_warnings_could_be_fixed.txt
--- EXECUTING the program with clang sanitizers ---
AddressSanitizer error rate: 0.0 (0/7091)
UndefinedBehaviorSanitizer error rate: 0.0 (0/7091)
Detailled results have been written into softwipe_sanitizer_output.txt
Compiler + Sanitizer Score: 10.0/10
--- Running: ASSERTION CHECK ---
Assertion rate: 0.0007051191651389084 (5/7091)
Detailled results have been written into softwipe_assertion_check.txt
Assertion Score: 0.5/10
--- Running: CPPCHECK ---
Total weighted Cppcheck warning rate: 0.0 (0/7091)
Detailled results have been written into softwipe_cppcheck_results.txt
Cppcheck Score: 10.0/10
--- Running: CLANG-TIDY ---
Weighted Clang-tidy warning rate: 0.0 (0/7091)
Detailled results have been written into softwipe_clang_tidy_results.txt
Clang-tidy Score: 10.0/10
--- Running: LIZARD ---
Average cyclomatic complexity: 4.6
Cyclomatic complexity Score: 8.3/10
Lizard warning rate (~= rate of functions that are too complex): 0.04716981132075472 (10/212)
Lizard warning Score: 8.5/10
Unique code rate: 0.8945000000000001
Unique (code duplication) Score: 4.7/10
Detailled results have been written into softwipe_lizard_results.txt
--- Running: KWSTYLE ---
KWStyle warning rate: 0.0001410238330277817 (1/7091)
Detailled results have been written into softwipe_kwstyle_results.txt
KWStyle Score: 10.0/10
Overall program Score: 7.7/10
(I've just fixed a few KWstyle warnings, but that's the only code difference with the swarm release you've tested)
Clang-tidy seems to behave extremely strange here. Try removing the bin/swarm file in the swarm directory, and rerun softwipe after that. Seems like only then clang-tidy yields any warnings (?!)
Also, your cppcheck test doesn't run properly, it should find a few (very few :) ) warnings. Try updating to the latest commit of softwipe, it's probably due to an issue I have fixed yesterday (see 2851292).
Also, your cppcheck test doesn't run properly, it should find a few (very few :) ) warnings. Try updating to the latest commit of softwipe, it's probably due to an issue I have fixed yesterday (see 2851292).
I am already running the latest version of softwipe, so this is not what's causing the issue.
Removing the bin/swarm file in the swarm directory did the trick for clang-tidy. I've also installed bear on my machine and I was able to run run-clang-tidy (clang-tidy10) and to compare the results. There are still quite a lot of warnings we could fix in swarm :-)
Hm, weird. I've attached the results I get from cppcheck so you can check them if you like! :)
softwipe_cppcheck_results.txt
thanks @adrianzap the attached file is empty though.
Hm works for me, but here's the content:
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/algod1.cc:572]: (style) Variable 'hits_count' is assigned a value that is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:470]: (information) Skipping configuration 'HAVE_BUILTIN_EXPECT;LIKELY' since the value of 'LIKELY' is unknown. Use -D if you want to check it. You can use -U to skip it explicitly.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:470]: (information) Skipping configuration 'LIKELY' since the value of 'LIKELY' is unknown. Use -D if you want to check it. You can use -U to skip it explicitly.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:567]: (warning) Assignment of function parameter has no effect outside the function. Did you forget dereferencing it?
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:567]: (style) Variable 's' is assigned a value that is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search16.cc:314]: (style) Variable 'h4' is assigned a value that is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search16.cc:377]: (style) Variable 'h4' is assigned a value that is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search8.cc:551]: (style) Variable 'h4' is assigned a value that is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search8.cc:614]: (style) Variable 'h4' is assigned a value that is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search16.cc:668] -> [/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search16.cc:339]: (error) Array index out of bounds; buffer 'Sm' is accessed at offset 0.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search16.cc:800] -> [/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search16.cc:416]: (error) Array index out of bounds; buffer 'Sm' is accessed at offset 0.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search8.cc:907] -> [/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search8.cc:576]: (error) Array index out of bounds; buffer 'Sm' is accessed at offset 0.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search8.cc:1036] -> [/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/search8.cc:653]: (error) Array index out of bounds; buffer 'Sm' is accessed at offset 0.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:175]: (style) The function 'CityHash32' is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:392]: (style) The function 'CityHash64WithSeed' is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:624]: (style) The function 'CityHashCrc128' is never used.
[/Users/adrian/Documents/Studium/Hiwi/swarm-3.0.0/src/city.cc:611]: (style) The function 'CityHashCrc128WithSeed' is never used.
Just checked in an Ubuntu VM, this is indeed the issue. For some reason apt-get (on Ubuntu) installs cppcheck version 1.82, however the latest is 1.89, which brew (on Mac) does install correctly. Manually installing cppcheck 1.89 on Ubuntu works, and outputs the above warnings.
I confirm, it works when installing a dev version of cppcheck