BenWoodworth/Parameterize

Add support for shared "before each" and "after each" logic

BenWoodworth opened this issue · 2 comments

Useful for testing, so each parameterize test case also hooks into the testing framework's before/after each code.

With kotlin.test, for example:

class MyTestClass : ParameterizeContext {
    override val parameterizeConfiguration = ParameterizeConfiguration {
        beforeEach = { beforeTest() }
        afterEach = { afterTest() }
    }

    @BeforeTest
    fun beforeTest() {
        println("Before test")
    }

    @AfterTest
    fun afterTest() {
        println("After test")
    }
    
    @Test
    fun ordinary_test() {
        println("- ordinary test")    
    }
    
    @Test
    fun parameterized_test() = parameterize {
        val iteration by parameter(1..3)
        
        println("- parameterized test: $iteration")
    }
}

This should print: (empty lines added for readability)

Before test
- ordinary test
After test

Before test
Before test
- parameterized test: 1
After test

Before test
- parameterized test: 2
After test

Before test
- parameterized test: 3
After test
After test

Note the duplicated Before test/After test lines. This would occur because the test framework AND parameterize would be calling the before/after functions. This could maybe be avoided by providing e.g. isFirstIteration/isLastIteration properties in the beforeEach/afterEach blocks' scopes:

ParameterizeConfiguration {
    beforeEach = {
        if (!isFirstIteration) beforeTest()
    }
    afterEach = {
        if (!isLastIteration) afterTest()
    }
}

Other options

I was also considering a more general decorator option that allows the library user to insert additional logic around the parameterize block, which could look something like this:

ParameterizeConfiguration {
    decorator = { block ->
        beforeTest()
        block()
        afterTest()
    }
}

I do like this approach, but it gets awkward when the block throws with a failure. Including the onFailure as well might be troublesome, unless that's all bundled into the block passed into the decorator. Also providing the isFirst/LastIteration properties (which, could be done, but isLastIteration would not be usable until after block is run). I do still like this approach, and have not entirely ruled it out. The before/afterEach just gives more control to the library and may be more familiar to those using it for testing.

It may also be possible to introduce both, and see which is more favored by library users after it's released.

Started implementing the decorator approach in the config-decorator branch

Decided on a single decorator option, since it's more flexible than separate "before each" and "after each" options, and it was easy enough to implement. The decorator validates that the iteration function is invoked exactly once, and the isLastIteration provided in scope will fail if accessed before iteration (since it can't be known until after).

It looks like Kotlin will probably be getting decorators in the future, so this should be right in line with that, and be a familiar pattern. See here.