Tests in multiple config groups should run more than once in node.js

[GCheung55]

Hey guys,

I don't know if this is expected or an issue. Tests clearly defined in multiple groups in the config file are only run once.

Something like:

var config = module.exports;

config["Group 1"] = {
    rootPath: "../",
    environment: "node",
    tests: [
        "test/test1.js",
        "test/test2.js"
    ]
};

config["Group 2"] = {
    extends: "Group 1",
    tests: [
        "test/test1.js",
        "test/test2.js"
    ]
};

I've identified why it's only running once. buster-test-cli/lib/runners/node.js has a descriptiveRequire function that require files. As expected, require'ing a file loads it into node.js' cache so every call to the same module would return the same module without executing the code in that module more than once.

Is this testing to be expected? It looks like it should actually be run for every time it's defined in a group. An example of why this should be the case is when I have an extension that has different options for each group.

@cjohansen Said this was a bug.

[GCheung55]

I was thinking of solutions for resolving this issue. So I'll throw out some ideas.

1. What if we changed the tests to return a function, so the test could easily be executed again?

module.exports = function(){
    buster.testCase('test' {...});
}

I don't know how the browser env would respond to this though.

1. Blowing away node require cache is possible but I think it'll be hard to control. Do we blow away all the cache for every module required by the one test file? If require('buster') occurs in the test, then it could cause some problems, right?
2. What if we mapped out test file paths with testCase objects to reuse the testCase objects?

[cjohansen]

1. We'll blow the entire require cache between each configuration group. I don't think this will cause any problems, and it will maintain the expected behavior of running each individual test configuration in isolation.

[dkl-ppi]

Fixed by 52412ca.