Code coverage generation

[luncliff]

Note

This issue is about generating code coverage of the library & test executable

Visual Studio Testing Tools

• vstest, CodeCoverage

Sonar code analysis failes for latest C++ syntax. Seems like this is the reason for low coverage in SonarCloud. Coverage report in Azure Pipelines is about 78%

Issue: coveragexml generation

• Commit 42a7235
• Build: Azure Pipelines Build 191

Hotfix applied with Azure Pipelines build 204

# ...
sonar.sourceEncoding=UTF-8 
sonar.cfamily.build-wrapper-output=bw-output 

# this line was originally **/*.coveragexml to collect all related files
sonar.cfamily.vscoveragexml.reportsPath=**/luncliff-coroutine-visual-studio.coveragexml
# ...

And coverage to xml generation script is like the following

# Path to Codecoverage.exe
$env:Path += ";C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools"

# Acquire coverage file and generate temporary coverage file
$coverage_file = Get-ChildItem -Recurse *.coverage;
Write-Host $coverage_file
$temp_coverage_xml_filepath  = "./TestResults/coverage-report.xml"
CodeCoverage.exe analyze /output:$temp_coverage_xml_filepath $coverage_file

# Display result files of CodeCoverage.exe
Tree ./TestResults /F

# Filter lines with invalid line number 
#   and Create a new coverage xml
$final_coverage_xml_filepath = "./TestResults/luncliff-coroutine-visual-studio.coveragexml"
$xml_lines = Get-Content $temp_coverage_xml_filepath
foreach($text in $xml_lines){
    if($text -match 15732480){
        continue;
    }
    else {
       Add-Content $final_coverage_xml_filepath $text;
    }
}

# Check if the coveragexml is in right place
Tree ./TestResults /F

# Display information of a new coverage xml
Get-ChildItem $final_coverage_xml_filepath
Get-Content   $final_coverage_xml_filepath

LLVM Coverage

Unfortunately, Clang makes a crash for the build with --coverage option. Preparing for detailed reproduction of the crash and reporting to LLVM Bugzilla. The crash both occurs for clang-6 and AppleClang

TBA

[luncliff]

Closing the issue since there is no update until now.
It will be reopened when there is a LLVM coverage issue.

[luncliff]

Note

LLVM Coverage

clang crashed with "Instruction does not dominate all uses!" error for channel code. Seems like lambda's capture is the source of it.

• Related Commit: 0117482

    void crashed_test_code()
    {
        using element_t = uint64_t;
        using channel_without_lock_t = channel<element_t, bypass_lock>;

        auto ch = make_unique<channel_without_lock_t>();
        bool ok = true;

        {
            auto coro_write = [&ok](auto& ch, auto value) -> return_frame {
                ok = co_await ch.write(value);
            };

            // coroutine will suspend and wait in the channel
            auto h = coro_write(*ch, element_t{});
            {
                auto truncator = std::move(ch); // if channel is destroyed ...
            }
            test_require_true(ch.get() == nullptr);

            auto coro = h.get();
            test_require_true(coro.done()); // coroutine is in done state
            coro.destroy();                 // destroy to prevent leak
        }
    }

However, when the captured argument is changed to explicit parameter, the build works.

    void success_test_code()
    {
        using element_t = uint64_t;
        using channel_without_lock_t = channel<element_t, bypass_lock>;

        auto ch = make_unique<channel_without_lock_t>();
        bool ok = true;

        {
            auto coro_write
                = [](auto& ch, auto& ok, auto value) -> return_frame {
                ok = co_await ch.write(value);
            };

            // coroutine will suspend and wait in the channel
            auto h = coro_write(*ch, ok, element_t{});
            {
                auto truncator = std::move(ch); // if channel is destroyed ...
            }
            test_require_true(ch.get() == nullptr);

            auto coro = h.get();
            test_require_true(coro.done()); // coroutine is in done state
            coro.destroy();                 // destroy to prevent leak
        }
    }

Going to write more code after branch setup

[luncliff]

Now the build became quite stable. I'm going to work on this issue again.

To Do

• coverage from Linux build (use some service? or print in build log?)
• coverage from Mac OS

[luncliff]

Disabling JMC(Just My Code) build option removed the wrong line numbers in the coverage report.

https://developercommunity.visualstudio.com/content/problem/411331/vstest-coveragexml-with-wrongout-of-range-line-num.html?childToView=703832#comment-703832

I disabled the feature in the project properties window

<SupportJustMyCode>false</SupportJustMyCode>

Build log: before & after