parameterized_class doesn't remove test methods from the base class's superclasses

[leifwalsh]

If I have a parameterized test class whose test methods are inherited:

class FooTestMethods:
    def test_foo(self):
        ...


@parameterized_class(...)
class FooTests(FooTestMethods, unittest.TestCase):
    pass

Then it appears that while #73 attempts to remove all the test* methods from the base class FooTests that it leaves in the module, it doesn't remove them from the whole MRO hierarchy. Therefore, I'm left with a FooTests object with test methods that will be discovered and run, but they might assume the parameterized values will be set, when they actually aren't.

I'm not totally sure what to do about this, maybe the base class's test methods should all be replaced with a noop method?

[JohnnyDeuss]

I've run into this exact issue as well. I've currently worked around it by adding a setUpClass method to skip the unparameterized test case.

@parameterized_class(...)
class FooTests(FooTestMethods, unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        if cls == FooTests:
            raise unittest.SkipTest("`parameterized_class` bug")
        super().setUpClass()

[zachlowry]

Currently struggling with this as well. The workaround from @JohnnyDeuss has been helpful but I'd prefer a more generic solution.

class FooTestMethods
    @classmethod
    def setUpClass(cls, parameterized_cls):
        if cls is parameterized_cls:
            raise SkipTest("skipping test for non-parameterized class")

@parameterized_class(...)
class FooTests(FooTestMethods, unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        super().setUpClass(FooTests)

[eltoder]

@wolever I think the cleanest way to solve this is to change the API of @parameterized_class and ask users to not inherit the decorated class from unittest.TestCase. parameterized_class will add this inheritance to the classes that it generates. For backward compatibility, if the class is already inherited from TestCase, we can either keep your logic of deleting "test_" methods¹ or instead rebuild the class and exclude TestCase from bases. The former has the downside that cases reported in this bug will not work. The latter has the downside that super() calls with implicit arguments will not work, but cases from this bug and super with explicit arguments will work. Either way, we'll only have issues in the "backward compatibility" mode. With the new approach of not inheriting the class from TestCase, everything will work.

What do you think?

Footnotes

1. Note that this code is not strictly correct. The default prefix is "test" rather than "test_" and it is configurable at runtime: source code. ↩

[sshishov]

I feel to fix the issue all-together, we have to duplicate (create with indices) not only the decorated class, but also all custom MRO classes. In this case we can easily remove the base classes and not have that patch with removing test_ prefixes.

Currently: unittest.TestCase -> BaseClass -> ChildClass

Transformed into:

• unittest.TestCase -> BaseClass -> ChildClass_0
• unittest.TestCase -> BaseClass -> ChildClass_1

Instead it should be transformed into:

• unittest.TestCase -> BaseClass_0 -> ChildClass_0
• unittest.TestCase -> BaseClass_1 -> ChildClass_1

In this case the chain is not common and we can manipulate with it.

What do you think guys? I just do not know if it is possible to achieve this

@eltoder @zachlowry @JohnnyDeuss @wolever