ilyachur/cmake4vim

Quickfix results for :Ctest failures unable to locate source

KnBrckr opened this issue · 7 comments

Describe the bug
When using :CTest --output-on-failure the paths in failure text are relative to the build directory, not the parent/root of the project where vim launched. As a result, vim is not finding files that are listed in the ctest output.

The error lines are correctly identified in the quickfix list as targets.

I have no issue navigating to errors identified as a result of running :CMakeBuild
To Reproduce
Steps to reproduce the behavior:

  1. Run :CTest --output-on-failure with a test that fails
  2. Use quickfix list to navigate to source file identified in failing test case
  3. Observe that an empty file is opened, not the project file containing the error

Expected behavior
Files identified in ctest --output-on-failure output that are included in quickfix list should be navigable to the project source using quickfix commands.

Desktop (please complete the following information):

  • OS: Ubuntu 20.04.1 LTS
  • Editor vim
  • Version 8.1 (2018 May 18, compiled Apr 15 2020 06:40:31)

Additional context
Plugin is current to commit hash 54c2338

Example ctest output:

[==========] Running 4 test(s).
[ RUN      ] read_fail_test
**undefined**: Failed retrieving read buffer size for fd=25; error (9) Bad file descriptor
[       OK ] read_fail_test
[ RUN      ] write_fail_test
**undefined**: Failed retrieving write buffer size for fd=25; error (9) Bad file descriptor
[       OK ] write_fail_test
[ RUN      ] read_ok_test
[  ERROR   ] --- [   LINE   ] --- ../Sources/tests/lib/libNetUtils_test/test_get_buf_size.c:33: error: Failure!
[  FAILED  ] read_ok_test
[ RUN      ] write_ok_test
[  ERROR   ] --- [   LINE   ] --- ../Sources/tests/lib/libNetUtils_test/test_get_buf_size.c:38: error: Failure!
[  FAILED  ] write_ok_test
[==========] 4 test(s) run.
[  PASSED  ] 2 test(s).
[  FAILED  ] 2 test(s), listed below:
[  FAILED  ] read_ok_test
[  FAILED  ] write_ok_test

 2 FAILED TEST(S)

It seems to be a limitation of ctest itself; it's not possible to run ctest from a different folder.

Source: https://stackoverflow.com/questions/38644741/run-ctest-from-different-directory-than-build-directory-used-by-cmake

@mark2185 Thank you for the investigation!

Maybe the problem can be solved if you change the local work directory to cmake build folder for quickfix list. In this case the paths should be correct, but it needs to be checked (I am not sure that this WA can resolve the issue).

@KnBrckr I tried to reproduce your issue on the test project: https://github.com/ilyachur/cmake4vim/tree/master/test/test%20proj

I modified tests in order to have fails, but looks like all works as expected:

|| Test project /tmp/cmake4vim/test/test proj/cmake-build-Release
||     Start 1: unit
|| 1/1 Test #1: unit .............................***Failed    0.00 sec
|| [==========] Running 2 tests from 1 test suite.
|| [----------] Global test environment set-up.
|| [----------] 2 tests from TestClassTests
|| [ RUN      ] TestClassTests.TestMethodF1
|| [       OK ] TestClassTests.TestMethodF1 (0 ms)
|| [ RUN      ] TestClassTests.TestMethodF2
tests/src/test_class.cpp|15| Failure
|| Expected: (m.f2()) != ("F2 default"), actual: "F2 default" vs "F2 default"
|| [  FAILED  ] TestClassTests.TestMethodF2 (0 ms)
|| [----------] 2 tests from TestClassTests (0 ms total)
|| 
|| [----------] Global test environment tear-down
|| [==========] 2 tests from 1 test suite ran. (0 ms total)
|| [  PASSED  ] 1 test.
|| [  FAILED  ] 1 test, listed below:
|| [  FAILED  ] TestClassTests.TestMethodF2
|| 
||  1 FAILED TEST
|| 
|| 
|| 0% tests passed, 1 tests failed out of 1
|| 
|| Total Test time (real) =   0.01 sec
|| 
|| The following tests FAILED:
|| 	  1 - unit (Failed)
|| Errors while running CTest

Jump to error also works as expected... In the test project I am using the gtest, maybe it is why I didn't get the same problems as you.

I tried your test project and ctest is reporting a fully qualified path to the offending source file where in my environment it's using a relative path. I'll explore from that angle. If I can get ctest to report the full path this is no longer a problem.

I was partially wrong about the fully qualified paths in my last update. In some cases the path is fully qualified, in others ctest will report a relative path. The difference is in either running the build manually or using :CMake and :CMakeBuild from inside vim. If I manually perform the following from the project root:

% mkdir build
% cd build
% cmake -DENABLE_TESTS=ON .. 
% cmake --build .

Then ctest errors will include the full path.

Using :CMake -DENABLE_TESTS=ON followed by :CMakeBuild and then running ctest from within the build directory at shell prompt, the paths are relative.

But, even though the paths are relative, the quickfix list works correctly.

There is a difference in the output from the gtest environment in how it formats the error line, vs the environment I'm using that is a factor in how the line is being interpreted by the quickfix parsing.

If I add a fprintf(stdout) to my sources to emit a message in the same format as that used by gtest that line is correctly interpreted and I'm able to jump to the line using the quickfix list. The error produced by my test library however still does not get parsed correctly so this is not an issue with your plugin but in the quickfix list parsing.

Here's output from ctest (at shell prompt) for the offending area:

[ RUN      ] has_trailing_slash_test
../Sources/tests/lib/libDT2_test/test_paths.c:121: Failure
[  ERROR   ] --- [   LINE   ] --- ../Sources/tests/lib/libDT2_test/test_paths.c:122: error: Failure!
[  FAILED  ] has_trailing_slash_test

Within vim this ends up looking like:

|| [ RUN      ] has_trailing_slash_test
Sources/tests/lib/libDT2_test/test_paths.c|121| Failure
[  ERROR   ] --- [   LINE   ] --- ../Sources/tests/lib/libDT2_test/test_paths.c|122| error: Failure!
|| [  FAILED  ] has_trailing_slash_test

The line start with ../Sources works as expected in the quickfix list. The list starting with [ ERROR ] is the one that the quickfix list fails to parse to point to the proper place.

After a bit more research about how the quickfix list works and a closer examination of the error, the problem is in the errorformat strings used to parse the output in my environment. After adding the following to my vimrc the faulty source file is correctly identified:

set efm^=[\ \ ERROR\ \ \ ]\ ---\ [\ \ \ LINE\ \ \ ]\ ---\ %f:%l:\ error:\ %m

Thank you for the quick responses and the nudge I needed to get me to the actual problem.

I'm closing the issue since it's not a problem with the plugin.

HII @KnBrckr,

Thank you for the responce.
If the issue is related with errorformat I think we can change the errorformat for :CTest command.
Here in we can pass custom errorFormat as a second argument.
In this case the quickfix list should work as expected without any manipulations from your side.

Not sure you want to get into the business of modifying for each possible error format.

If you do want to make changes, so far I have two strings defined to capture the CMocka style failures that I've observed:

" Quickfix list error format for ctest errors using CMocka environment
"
" General assertion failure:
"   [  ERROR   ] --- <text>
"   [   LINE   ] --- <path>:<line>: error: Failure!
set errorformat^=%E[\ \ %trror\ \ \ ]\ ---\ %m,%Z[\ \ \ line\ \ \ ]\ ---\ %f:%l:\ %.%#
" fail() Error format: 
"   [ ERROR ] --- [   LINE   ] --- <path>|<line>| error: <text>
set errorformat^=[\ \ %trror\ \ \ ]\ ---\ [\ \ \ line\ \ \ ]\ ---\ %f:%l:\ error:\ %m