Lcov2.0 filters normal branched when --filter branch and no_exception_branch=1 both works

[Allen19930926]

Hi Henry,
Sorry to bother you! However I have another trouble in branch filtering. Due to the restriction to some 3rd party libraries, my project could not use compilation parameter - "-fno-exceptions". I filter the bogus branch by lcov. Use no_exception_branch=1 when lcov 1.15, and --filter branch when lcov 2.0-1.
Now with cross compilation and remote gcda data collection, the "--filter branch" does not make a satisfying result, while I know this is most possily caused by gcov version mismatch which is a hot potato. According to this, I use both "no_exception_branch=1" and "--filter branch", and get a strange branch filtering results which filteded many normal branches. Please pay some time to analyze this issue.
Thanks a lot. I'm looking forward to your reply soon.

only --filter branch

--filter branch and no_exception_branch=1 both works

[henry2cox]

Your first snapshot looks like you applied exception filtering; it tends to leave single orphan branches in place.
If that isn't what you did/is not what you expected: then this looks like a bug - or at least a case I don't recall having seen.

Your second snapshot looks wrong. It appears as if the actual branches on your if/else expressions were removed - which is definitely not what is supposed to happen.

Three things to do next:

• verify the parameters you are passing to lcov --capture and genhtml.
  Exception filtering should be removing exception branches (possibly leaving orphans). Branch filtering should be removing branches that aren't associated with conditional expressions.
  If this is not what is happening: please send a testcase which illustrates the issue, so I can see what is happening.

• Download/clone the current top-of-tree and see if the most recent code illustrates the same issues.
  If so...then I probably need a testcase.

• Update to newer compiler version (...which is likely not possible for your project, and may not help resolve case in any event).

Branch filtering will remove exception branches on lines where the tool doesn't think it found a conditional expression...but you may need exception filtering as well if those exception branches are associated with lines which do contain conidtional expressions. (Without data from gcov/llvm-cov, the tool can't distinguish the exception handlers from the conditionals.)

As with your other case: you can turn up verbosity and/or debug logging so the tool will tell you a bit more about what it is filtering/why it is filtering something - but that probably won't be useful to you. Debugging aid more than user feature.

[Allen19930926]

Hi Henry,
It seems you should have a try. I downloaded the lcov repository, and installed the 2.0beta. When "no_exception_branch=1" and "--filter branch"both works, the issue occurs again. Moreover, the test code is work in my WSL and gcc version is 7.5.0, not the cross compiler.

issue snapshot : I highlight the lcov version.

test code

 void LcovTest::TestBranch(const std::map<std::string, int>& ref_map)
{
    for (const auto& iter : ref_map)
    {
        if (iter.first == "11111")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else if (iter.first == "22222")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else if (iter.first == "33333")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else if (iter.first == "44444")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else if (iter.first == "55555")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else if (iter.first == "66666")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else if (iter.first == "77777")
        {
            std::cout << iter.first << " executed" << std::endl;
        }
        else
        {
            std::cout << iter.first << " error type" << std::endl;
        }
    }
} 

TEST_F(TestLcov, lcov_test)
{
    std::map<std::string, int> test_data;
    for (int i=1; i<9; i++)
    {
        test_data[std::to_string(i*11111)]++;
    }

    LcovTest::TestBranch(test_data);
}

lcov command: I turn on the branch coverage in lcovrc and other switch keeps the same as code repository

    mkdir -p $SCRIPT_DIR/coverage && cd $SCRIPT_DIR/coverage
    # lcov -q -c -d $SCRIPT_DIR/compile/CMakeFiles/ball-demo.dir/app -o app.info
    # lcov -q -e  app.info "*lcov*" -o app.info
    # genhtml -q --filter branch app.info
    lcov -q -c --rc no_exception_branch=1 -d $SCRIPT_DIR/compile/CMakeFiles/ball-demo.dir/app -o app.info
    lcov -q -e --rc no_exception_branch=1 app.info "*lcov*" -o app.info
    genhtml -q --rc no_exception_branch=1 --filter branch app.info
    cd $SCRIPT_DIR
    tar -cf coverage.tar coverage/
    rm -rf coverage/

[henry2cox]

Your example is incomplete.
No definition for your LcovTest class. No definition for your TEST_F macro. No makefile or compile/link command line.
Example needs to be complete, correct, and runnable in order to be useful.

[Allen19930926]

Hi Henry，I have uploaded my demo project, please download to analyse this issue. It was compiled by cmake and sh build.sh would complie and execute the demo. build.sh contains lcov command, and I'm sure the project will reproduce the issue.

gtest_demo.tar.gz

[henry2cox]

Ran your example - and cannot reproduce the issue you describe.

There are (at least?) 2 bugs/issues in your example:

• the lcov --capture command you are using did not turn on --branch-coverage.
  Branch coverage is not enabled, by default - so I guess that is why you didn't see branches in your output.
  I don't know why you did see branches in your earlier example - but my suspicion is that you simply dropped a flag and didn't notice.
• your cmake environment seems to use llvm - or at least, uses llvm when I build locally (possibly because it is in my path?)
  But - despite using LLVM to generate your coverage data (*.gcno, *.gcda), you are using the default gcov to extract the data.
  You wanted to use llvm-cov gcov ....
  This probably worked for you because you have gcc/4.9 in your path - so we picked up gcov from the gcc package. Te llvm-generated gcov data is (mostly) in gcc/4 format - so it didn't die. Despite that it appeared to work, it is recommended to use the same toolchain to extract the data as you used to build it. Discussion page #234 talks about llvm.

In addition, your coverage report is rather noisy and shows lower coverage than you probably wanted because it includes a bunch of gtest and stl code that you probably don't care about. You might want to use --include 'app/*' on your capture or genhtml command line - to report only your application code. You might also want a --flat or --hierarchical report - as the default format is a bit of overkill for a 2-file project.

There may still be a bug here - but this example does not appear to demonstrate it.

[Allen19930926]

Henry, please set branch_coverage=1 in lcovrc or add --rc branch_coverage=1, the issue may be reproduced.

there are the more detailed descriptions

• please treat this issue as a new and independent one, and could be reproduced on a PC with ubuntu 18.04. The gcc version is gcc version 7.5.0

• sorry for my vague descriptions about lcov configuration. Actually, as you know, the lcov configuration is consist of two parts, lcovrc and lcov command --rc. I turn the branch coverage on in lcovrc and keep other configurations the same as lcov repository, meanwhile remain the differential options in lcov command, such as the --rc no_exception_branch=0/1, in order to change it quickly. At the base of this, I conduct the coverage report with branch coverage. If the parameter branch_coverage was 0, the report would not show branches row in summary table header like this.
  

• Maybe, there is nothing about LLVM In my CMakeLists.txt. I haven't tried llvm-cov so far...

• As for the report noise, I use -e "test_code" to only extract the code I care about. If there is any optimal point in my lcov command, please tell me straightly, after all you are the expert in lcov.

happy weekend~

[henry2cox]

I fear that you need to submit a different test case or a simpler driver.
When I run this one locally, it doesn't show branches because branch coverage is not enabled.
When I enable branch coverage, I see branches where we expect them, and none where we don't - so everything looks fine.

It is certainly possible that things look different in your environment. Your goal is to capture that difference in as simple a test as possible - so the issue can be reproduced.
I'm afraid that hacking your cmake environment, gtest, or whatever - to see if I can make something fail - is not something I'm particularly interested in doing.

It should be sufficient if you just send the "capture" result you see on your machine - when you don't filter or exclude any cover points in your source directory.
The captured '.info' file should contain all the line, branch, and function points, and should contain markers for exceptions, etc.
Then I can just look at the filter result.
(You might want to check the data - e.g., look at the html report), to verify that everything is there - including the bogus points we want to remove.)

[Allen19930926]

I understand you that to analyse the difference between enviroments is quite a disappointing thing. So I add -v to collect lcov running log, could you find something wrong in the logs?

As is shown below, the branch data differentiated after extracting . Branch data decreased to 0 at extracting step executed by lcov.

Some enviroment version info is at the end of picture such as cmake, lcov, gcc and etc. As now I am back home , the version in my home laptop is a little different than my office laptop, but the issue also occured.

branch exist

branch disappear

[henry2cox]

I don't really need the log.
Far easier is if you simply attach the app.info file that is generated by the lcov --capture command in your testcase script.

Never mind. No longer need the log.
Found the issue, I believe. Testing now.

[Allen19930926]

Good news. Here is the app.info generated by my script. Hope to be helpful.
appinfo.tar.gz

[Allen19930926]

make latest lcov, bugfixed. commit-id:70a44f841bd99826d6f9e6df89897d2a6b9b2fe0

thanks!