microsoft/coyote

fix compatibility issue with VS Code coverage and assembly rewriting

pdeligia opened this issue · 3 comments

It seems that the code coverage tool is misreading the rewritten binaries as uncovered code blocks, causing % coverage to be reduced. The issue arises when you run code coverage on multiple projects that use the same project references. Since they maintain their own separate copies of assemblies in the build directories, one of them is rewritten, and the other one is not, resulting in duplicate assemblies in code coverage.

+@jhwj9617

@pdeligia
Hi. Any update on this?
This is still number one prio issue for us. The workaround we have is not very elegant, and drastically increases build times (since we're building twice, without rewrite and with rewrite)

Hi @jhwj9617, sadly no update that I am aware of from the VS tooling side. I keep this issue open here and I totally understand your pain and would love this resolved ASAP, but as we discussed before I don't consider this as a coyote bug that I am able to fix from my side, since it should really be the code coverage tool that handles combining rewritten + non-rewritten DLL stats when CI mixes tests that use both DLLs, and not something that coyote itself can actually deal with on its own (the ideal setup for coyote is to run it separately as you are doing now, but I know it can be a pain when you want to run a single configuration with all types of tests).

Have you checked recently with the folks that you introduced me back then? We could get on another call/chat with them if that helps speed up things (and I am happy to join the discussion)?

@jhwj9617, regarding performance hit from building twice -- I was thinking more about this, but I am a bit confused why that is necessary. Maybe I am missing something here, but couldn't you build once and copy the bin directory and rewrite the copy?

Also coyote rewrite supports an output directory destination other than source, which allows out-putting all rewritten DLLs in another directory, but still copying would be better, since you need to manually copy all non-rewritten DLLs today.