ros/catkin

catkin_make run_tests - can't import Python code

Closed this issue ยท 31 comments

Not sure if this is my fault or a bug. I prepared a test repository with minimal example: https://github.com/ZdenekM/rostest-test-repo. The thing is that there are two packages - one with a custom message and the other with rostest where I would like to use the message. When I run test like this rostest test_package test.test it succeeds - but it fails when I run catkin_make run_tests (see whole output below - it seems that messages are generated) with ImportError: No module named my_msgs.msg. What might be wrong?

Scanning dependencies of target tests
Scanning dependencies of target clean_test_results_test_package
Scanning dependencies of target _my_msgs_generate_messages_check_deps_MyMessage
[  0%] Built target tests
Removing test result files from '/home/zdenal/test_workspace/build/test_results/test_package'
[  0%] Built target clean_test_results_test_package
[  0%] Built target _my_msgs_generate_messages_check_deps_MyMessage
Scanning dependencies of target my_msgs_generate_messages_cpp
Scanning dependencies of target my_msgs_generate_messages_py
Scanning dependencies of target my_msgs_generate_messages_lisp
[ 25%] [ 50%] [ 75%] Generating C++ code from my_msgs/MyMessage.msg
Generating Lisp code from my_msgs/MyMessage.msg
Generating Python from MSG my_msgs/MyMessage
[ 75%] Built target my_msgs_generate_messages_lisp
[100%] Generating Python msg __init__.py for my_msgs
[100%] Built target my_msgs_generate_messages_py
[100%] Built target my_msgs_generate_messages_cpp
Scanning dependencies of target _run_tests_test_package_rostest_test_test.test
-- run_tests.py: execute commands
  /opt/ros/indigo/share/rostest/cmake/../../../bin/rostest --pkgdir=/home/zdenal/test_workspace/src/rostest-test-repo/test_package --package=test_package --results-filename test_test.xml --results-base-dir /home/zdenal/test_workspace/build/test_results /home/zdenal/test_workspace/src/rostest-test-repo/test_package/test/test.test 
... logging to /home/zdenal/.ros/log/rostest-ZMThinkpad-30945.log
[ROSUNIT] Outputting test results to /home/zdenal/test_workspace/build/test_results/test_package/rostest-test_test.xml
Traceback (most recent call last):
  File "/home/zdenal/test_workspace/src/rostest-test-repo/test_package/test/test.py", line 8, in <module>
    from my_msgs.msg import MyMessage
ImportError: No module named my_msgs.msg
testtest_package_test ... FAILURE!
FAILURE: test [test_package_test] did not generate test results
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
    testMethod()
  File "/opt/ros/indigo/lib/python2.7/dist-packages/rostest/runner.py", line 164, in fn
    self.assert_(os.path.isfile(test_file), "test [%s] did not generate test results"%test_name)
  File "/usr/lib/python2.7/unittest/case.py", line 424, in assertTrue
    raise self.failureException(msg)
--------------------------------------------------------------------------------

[ROSTEST]-----------------------------------------------------------------------

[testtest_package_test][failed]

SUMMARY
 * RESULT: FAIL
 * TESTS: 0
 * ERRORS: 0
 * FAILURES: 1

ERROR: The following tests failed to run:
 * testtest_package_test

Please provide a reproducible example to get feedback on the problem.

From the output I would just guess that the test_package package lacks a dependency on the package containing the message my_msgs.

Please check this example with two packages: https://github.com/ZdenekM/rostest-test-repo. Dependency is there.

Thank for the example. I can reproduce the problem now. This can even be reduced to a single package generating the Python message and having a test trying to import the Python module. This fails since rostest is not aware that it needs an extended PYTHONPATH.

There are a couple of workarounds possible:

  • The non-isolated case can't handle this since all packages share the same environment variables. The second package doesn't get custom environment variable the first package defines. Therefore you need to update the environment manually before running the tests:

    • catkin_make
    • . devel/setup.bash
    • catkin_make run_tests
  • In the isolated case you need to first build the workspace (or at least my_msgs to generate the Python code) and then run the tests. The reason is that run_tests for the first package doesn't build anything:

    • catkin_make_isolated
    • catkin_make_isolated --make-args run_tests

I don't think there is a way to fix this problem other then the workarounds described above.

Thank you for your detailed answer. The first approach does not work for me but the second does - great! Maybe, it would be nice to document this somewhere.

Oh I was hit by this problem as well, and just wanted to confirm what ZdenekM experienced: the first solution did not work, the second did. A work-around yes, but a really annoying one.

Ran into this same issue and narrowed it down to build/catkin_generated/setup_cached.sh which overwrites $PYTHONPATH to /opt/ros/kinetic/lib/python2.7/dist-packages for me. Commenting that line out lets my tests run successfully.

I should add that commenting that line out and sourcing devel/setup.sh got the PYTHONPATH into the right state to run the tests successfully.

I could make a new issue for this or perhaps this issue could be reopened to explore a solution?

I could make a new issue for this or perhaps this issue could be reopened to explore a solution?

Please provide the complete steps to reproduce the problem (what are you sourcing before building the workspace, what exactly is in your workspace, etc.). If the problem is reproducible with that information it is probably best to create a new ticket.

Sure, using @ZdenekM's repo https://github.com/ZdenekM/rostest-test-repo, here are my steps to reproduce the failure:

Open a new shell without any ROS scripts sourced or environment variables set and run the following commands (or paste to a script and run it all at once)

mkdir -p test_ws/src
cd test_ws/src
git clone https://github.com/ZdenekM/rostest-test-repo
cd ../
. /opt/ros/kinetic/setup.sh
catkin_make
. devel/setup.sh
catkin_make run_tests

Expected Result: The test.py script errors with ImportError: No module named my_msgs.msg

And to reproduce the success, do the following after the above:

  1. Comment out the PYTHONPATH= line in build/catkin_generated/setup_cached.sh
  2. Run catkin_make run_tests

Expected result: Test passes without errors.

Hmm, a new twist, however. If I run . /opt/ros/kinetic/setup.sh before I run the script above, the tests run successfully because there is no PYTHONPATH=... line in setup_cached.sh

Doing the following from a new terminal leads to the tests succeeding. The only difference is sourcing the system setup.sh twice,

. /opt/ros/kinetic/setup.sh
mkdir -p test_ws/src
cd test_ws/src
git clone https://github.com/ZdenekM/rostest-test-repo
cd ../
. /opt/ros/kinetic/setup.sh
catkin_make
. devel/setup.sh
catkin_make run_tests

Addition: it doesn't seem to matter where I source kinetic/setup.sh as long as I source it twice before running catkin_make. This causes something to change in the generated files such that PYTHONPATH is no longer overwritten.

I tried your first block of commands (with a single source of the Kinetic setup) and I works fine for me (tests passing). Please try the instructions again starting with a clean directory and environment.

The additional source command can't make any difference since neither of the commands in between even uses ROS.

Hmm, I'll look into this more but it was very reproducible between after deleting test_ws and starting a fresh shell.

Hmm, I'll look into this more but it was very reproducible between after deleting test_ws and starting a fresh shell.

Did you use a "fresh" shell?

Sorry, is that question asking, "fresh" vs. "bash" or "sh"? I was using bash, I just meant starting a new terminal with a new shell.

Didn't realize there was a "fresh" shell, my bad.

Didn't realize there was a "fresh" shell, my bad.

I meant opening a new terminal to ensure that the environment has not been modified in the meantime.

Ah, yes. I did just that each time. It's strange, I don't get this error in a fresh kinetic Docker container but I can repeatedly on my host, so there must be something odd there which causes the PYTHONPATH to change during setup in a way which embeds it into setup_cached.sh

Wild guessed:

  • do you source a setup.bash in your bashrc
  • is your pythonpath set otherwise in your bashrc

Maybe look at the output of env in your clean shell.

@NikolausDemmel for the win! Just excluding the one line in my .bashrc where I set (prepend) my PYTHONPATH fixes it. Do catkin/ROS rely on having an unset PYTHONPATH before setup.sh is sourced?

What is it set to and does the issue occur with any other value?

It seems that having a trailing colon on my PYTHONPATH causes the issue.

The line in my .bashrc is export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH which when the PYTHONPATH is empty (seems to be at the start), this leaves it with a trailing colon.

Setting it to the same path but without the trailing colon causes the tests to pass again.

Here's the difference in my PYTHONPATH when sourcing setup.sh twice when having a trailing colon in the PYTHONPATH

/usr/local/lib/python2.7/site-packages: (before sourcing)
/opt/ros/kinetic/lib/python2.7/dist-packages:/usr/local/lib/python2.7/site-packages: (after sourcing once)
/opt/ros/kinetic/lib/python2.7/dist-packages:/usr/local/lib/python2.7/site-packages (after sourcing twice)

Note the trailing colon disappears after the second source

Was hit by this again. Seems to me there are several versions of this problem.

My most confusing version of the problem has been whenever a wrong PYTHONPATH ends up being cached in build/catkin_generated/setup_cached.sh. In this case it is not solved by sourcing devel/setup.bash between build and test.

When running "rostest ..." directly this is not a problem, but whenever cmake is controlling the test execution, the cached PYTHONPATH seems to be used. And the testing fails. I solve it by touching a CMakeLists.txt, this seems to rebuild the cache.

I have tried to reproduce this reliably, but in vain.
I don't really understand when environment is cached to build/catkin_generated/setup_cached.sh.

I've fixed this and posed inside #974 but also posting here for reference.

I discovered that the issue comes from a combination of what users are sourcing (e.g source devel/setup.bash vs source devel/setup.sh) and how the environment variable CATKIN_SETUP_UTIL_ARGS is used.

The entire story is explained. Hold tight.

If you want a tl;dr solution, simply wipe your devel/ build/ install/ directories and have this set in your ~/.bashrc OR if you want just a one off do 'export CATKIN_SETUP_UTIL_ARGS=--extend'. Then, run catkin_make run_tests and you will see that everything is working fine and dandy. But why?

Users most of the time use setup.bash, which doesn't pass down the --extend parameter anyway, so running source devel/setup.bash --extend, so that won't work as it is just a passthrough script which will call setup.sh. That's where the magic happens. setup.sh however does respect the --extend parameter and it also respects the env var CATKIN_SETUP_UTIL_ARGS which is why the above solution works. But what does --extend do and why does it cause things to work?

Looking inside _setup_util.py inside your devel directory, you will see that it can take an --extend param (discussed above). If its set, args.extend will be set to true, and so it will avoid the routine _rollback_env_variables. This function fucks everything up as it explicitly states (and does)

" For each catkin workspace in CMAKE_PREFIX_PATH remove the first entry from env[NAME] matching workspace + subfolder.
"

Well, if you look in the file you will see that CMAKE_PREFIX_PATH is most likely of the sort "$yourworkspace/devel/:$opt/ros/kinetic/" or something. The problem is that your workspace is first, and unless extend is set then it trims it off of the CMAKE_PREFIX_PATH, thus preventing your libraries from being found. This file generates an environment cache file which sticks around unless you add/remove new catkin packages and so if --extend isn't set, your path will continuously just not contain the proper $PYTHONPATH which includes your custom modules.

Now, this can all be completely avoided by doing @dirk-thomas solution in the linked issue because catkin_make_isolated will just take the env var from your existing shell, which is most likely correct because it wouldn't trim off your custom module path in its generated env cache.

I can't be fucked to make a PR to fix this workflow because the bashrc work around is fine for me, but if someone would do that I would be grateful that the default behaviour is changed.

Proof of it working when the --extend is passed in via env vars:

dash@dash:~/99robotics$ sudo rm -rf build devel
dash@dash:~/99robotics$ echo $CATKIN_S
$CATKIN_SETUP_UTIL_ARGS  $CATKIN_SHELL            
dash@dash:~/99robotics$ echo $CATKIN_SETUP_UTIL_ARGS 
--extend
dash@dash:~/99robotics$ catkin_make run_tests
Base path: /home/dash/99robotics
Source space: /home/dash/99robotics/src
Build space: /home/dash/99robotics/build
Devel space: /home/dash/99robotics/devel
Install space: /home/dash/99robotics/install
####
#### Running command: "cmake /home/dash/99robotics/src -DCATKIN_DEVEL_PREFIX=/home/dash/99robotics/devel -DCMAKE_INSTALL_PREFIX=/home/dash/99robotics/install -G Unix Makefiles" in "/home/dash/99robotics/build"
####
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/dash/99robotics/devel
-- Using CMAKE_PREFIX_PATH: /home/dash/99robotics/devel;/home/dash/contracting/shield/code/pikachu/workspace/devel;/opt/ros/kinetic
-- This workspace overlays: /home/dash/99robotics/devel;/home/dash/contracting/shield/code/pikachu/workspace/devel;/opt/ros/kinetic
-- Found PythonInterp: /usr/bin/python (found version "2.7.12") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/dash/99robotics/build/test_results
-- Found gtest: gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.14
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 2 packages in topological order:
-- ~~  - robot_library
-- ~~  - rosdesigner
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'robot_library'
-- ==> add_subdirectory(ros_easy/robot_library)
-- +++ processing catkin package: 'rosdesigner'
-- ==> add_subdirectory(ros_easy/rosdesigner)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dash/99robotics/build
####
#### Running command: "make run_tests -j16 -l16" in "/home/dash/99robotics/build"
####
Scanning dependencies of target clean_test_results_rosdesigner
Scanning dependencies of target tests
Built target tests
Removing test result files from '/home/dash/99robotics/build/test_results/rosdesigner'
Built target clean_test_results_rosdesigner
Scanning dependencies of target _run_tests_rosdesigner_nosetests_tests.test_insert_button.py
-- run_tests.py: execute commands
  /usr/bin/cmake -E make_directory /home/dash/99robotics/build/test_results/rosdesigner
  /usr/bin/nosetests-2.7 -P --process-timeout=60 /home/dash/99robotics/src/ros_easy/rosdesigner/tests/test_insert_button.py --with-xunit --xunit-file=/home/dash/99robotics/build/test_results/rosdesigner/nosetests-tests.test_insert_button.py.xml
.E..QWidget: Must construct a QApplication before a QWidget
Aborted (core dumped)
-- run_tests.py: verify result "/home/dash/99robotics/build/test_results/rosdesigner/nosetests-tests.test_insert_button.py.xml"
Cannot find results, writing failure results to '/home/dash/99robotics/build/test_results/rosdesigner/MISSING-nosetests-tests.test_insert_button.py.xml'
Built target _run_tests_rosdesigner_nosetests_tests.test_insert_button.py
Scanning dependencies of target _run_tests_rosdesigner_nosetests
Built target _run_tests_rosdesigner_nosetests
Scanning dependencies of target _run_tests_rosdesigner
Built target _run_tests_rosdesigner
Scanning dependencies of target run_tests
Built target run_tests

Proof of it not working when the env var is not set:

dash@dash:~/99robotics$ echo $CATKIN_SETUP_UTIL_ARGS 
--extend
dash@dash:~/99robotics$ echo $CATKIN_SETUP_UTIL_ARGS ^C
dash@dash:~/99robotics$ export ^C
dash@dash:~/99robotics$ echo $CATKIN_SETUP_UTIL_ARGS 
--extend
dash@dash:~/99robotics$ export CATKIN_SETUP_UTIL_ARGS=''
dash@dash:~/99robotics$ catkin_make run_^C
dash@dash:~/99robotics$ sudo rm -rf build devel
dash@dash:~/99robotics$ catkin_make run_tests
Base path: /home/dash/99robotics
Source space: /home/dash/99robotics/src
Build space: /home/dash/99robotics/build
Devel space: /home/dash/99robotics/devel
Install space: /home/dash/99robotics/install
####
#### Running command: "cmake /home/dash/99robotics/src -DCATKIN_DEVEL_PREFIX=/home/dash/99robotics/devel -DCMAKE_INSTALL_PREFIX=/home/dash/99robotics/install -G Unix Makefiles" in "/home/dash/99robotics/build"
####
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/dash/99robotics/devel
-- Using CMAKE_PREFIX_PATH: /home/dash/99robotics/devel;/home/dash/contracting/shield/code/pikachu/workspace/devel;/opt/ros/kinetic
-- This workspace overlays: /home/dash/99robotics/devel;/home/dash/contracting/shield/code/pikachu/workspace/devel;/opt/ros/kinetic
-- Found PythonInterp: /usr/bin/python (found version "2.7.12") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/dash/99robotics/build/test_results
-- Found gtest: gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.14
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~  traversing 2 packages in topological order:
-- ~~  - robot_library
-- ~~  - rosdesigner
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'robot_library'
-- ==> add_subdirectory(ros_easy/robot_library)
-- +++ processing catkin package: 'rosdesigner'
-- ==> add_subdirectory(ros_easy/rosdesigner)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dash/99robotics/build
####
#### Running command: "make run_tests -j16 -l16" in "/home/dash/99robotics/build"
####
Scanning dependencies of target tests
Scanning dependencies of target clean_test_results_rosdesigner
Built target tests
Removing test result files from '/home/dash/99robotics/build/test_results/rosdesigner'
Built target clean_test_results_rosdesigner
Scanning dependencies of target _run_tests_rosdesigner_nosetests_tests.test_insert_button.py
-- run_tests.py: execute commands
  /usr/bin/cmake -E make_directory /home/dash/99robotics/build/test_results/rosdesigner
  /usr/bin/nosetests-2.7 -P --process-timeout=60 /home/dash/99robotics/src/ros_easy/rosdesigner/tests/test_insert_button.py --with-xunit --xunit-file=/home/dash/99robotics/build/test_results/rosdesigner/nosetests-tests.test_insert_button.py.xml
E
======================================================================
ERROR: Failure: ImportError (No module named rosdesigner.widgets.toolbarbuttons)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/dash/99robotics/src/ros_easy/rosdesigner/tests/test_insert_button.py", line 11, in <module>
    from rosdesigner.widgets.toolbarbuttons import *
ImportError: No module named rosdesigner.widgets.toolbarbuttons
-------------------- >> begin captured logging << --------------------
rospy.topics: INFO: topicmanager initialized
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)
-- run_tests.py: verify result "/home/dash/99robotics/build/test_results/rosdesigner/nosetests-tests.test_insert_button.py.xml"
Built target _run_tests_rosdesigner_nosetests_tests.test_insert_button.py
Scanning dependencies of target _run_tests_rosdesigner_nosetests
Built target _run_tests_rosdesigner_nosetests
Scanning dependencies of target _run_tests_rosdesigner
Built target _run_tests_rosdesigner
Scanning dependencies of target run_tests
Built target run_tests

@dirk-thomas @jschleicher

I ran into this issue (rostest works, catkin_make run_tests fails) as well. The first solution proposed by @dirk-thomas here also didn't work for me and since I prefer not to use the second option, I tried removing the if-statement from

if (CATKIN_ENABLE_TESTING)
  find_package(rostest REQUIRED)
  add_rostest(launch/mytest.launch)
endif()

in my CMakeLists.txt. This solved the problem running catkin_make run_tests but is not recommended in the rostest documentation. Does this provide any clues for anyone?

EDIT I couldn't get this to work reliably. @TheDash 's solution doesn't work for me either. I don't understand why this issue is closed.

I don't understand why this issue is closed.

@Achllle The ticket is closed since the originally reported problem was resolved. That there are other comments on thread with different cases isn't a reason to keep the ticket open. For any different case where the workarounds don't work consider to open a separate ticket with complete reproducible steps.

@dirk-thomas The original problem which is the exact same problem I have got resolved using a partial solution (running catkin_make_isolated). I'd argue that isn't a good solution and so the problem still stands. I can make a new issue but it would be a duplicate of this one.

Please see #986 for a proposed patch which avoid the need for the mentioned workarounds.

I can make a new issue but it would be a duplicate of this one.

That already exists: #974