google/clusterfuzz

Fix issue where global variables are causing tests to pass when they should fail

jonathanmetzman opened this issue · 1 comments

jonathanmetzman commented

This happens when you run python butler.py py_unittest -t core -m -p build_manager*

======================================================================
ERROR: test_setup_fuzz_0_True (tests.core.build_management.build_manager_test.RegularLibFuzzerBuildTest)
Tests setting up a build during fuzzing [with unpack_all='True'].
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/user/home.local/share/virtualenvs/clusterfuzz-H0A0yd4x/lib/python3.7/site-packages/parameterized/parameterized.py", line 620, in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
  File "/user/homeprojects/clusterfuzz/src/clusterfuzz/_internal/tests/core/build_management/build_manager_test.py", line 483, in test_setup_fuzz
    self.assertEqual('target2', os.environ['FUZZ_TARGET'])
  File "/usr/local/lib/python3.7/os.py", line 681, in __getitem__
    raise KeyError(key) from None
KeyError: 'FUZZ_TARGET'

======================================================================
ERROR: test_setup_fuzz_with_extra_0_True (tests.core.build_management.build_manager_test.RegularLibFuzzerBuildTest)
Tests setting up a build during fuzzing with an extra build [with unpack_all='True'].
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/user/home.local/share/virtualenvs/clusterfuzz-H0A0yd4x/lib/python3.7/site-packages/parameterized/parameterized.py", line 620, in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
  File "/user/homeprojects/clusterfuzz/src/clusterfuzz/_internal/tests/core/build_management/build_manager_test.py", line 527, in test_setup_fuzz_with_extra
    self.assertEqual('target2', os.environ['FUZZ_TARGET'])
  File "/usr/local/lib/python3.7/os.py", line 681, in __getitem__
    raise KeyError(key) from None
KeyError: 'FUZZ_TARGET'

ClusterFuzz's use of global variables is very bad and not defensible (fine for a constant config, not for something that is set and reset many times throughout the life of a program). We should prioritize getting rid of them.

jonathanmetzman commented

This failure was caused by removing the line: from clusterfuzz._internal.bot.tasks.utasks import fuzz_task from build_manager_test

  • 😄1