lucono/karma-test-explorer

Can't run tests in multiple browsers. Why overwrite "browsers" from karma.conf.js?

lana-k opened this issue · 1 comments

I want my karma tests to be run in two browsers: Chromium and Firefox with custom settings which are described as a custom launcher. But karma-test-explorer doesn't allow to do that. It takes browser from the extension settings and overwrites browser settings from my karma.conf.js:

 public applyConfigOverrides(config: KarmaConfig) {
    config.port = this.karmaPort;
    config.logLevel = (config as any)[`LOG_${this.karmaLogLevel.toUpperCase()}`];
    config.singleRun = false;
    config.autoWatch = this.autoWatchEnabled;
    config.autoWatchBatchDelay = this.autoWatchBatchDelay ?? config.autoWatchBatchDelay;
    config.restartOnFileChange = false;
    config.browsers = [this.browser]; // <--- Here. Why overwrite browsers from `karma.conf.js` ?
    config.customLaunchers = this.customLauncher ? { [this.browser]: this.customLauncher } : config.customLaunchers;
    config.browserNoActivityTimeout = 1000 * 60 * 60 * 24;
    config.browserDisconnectTimeout = Math.max(config.browserDisconnectTimeout || 0, 30_000);
    config.pingTimeout = 1000 * 60 * 60 * 24;
    config.captureTimeout = Math.max(config.captureTimeout || 0, KARMA_BROWSER_CAPTURE_MIN_TIMEOUT);
    config.browserSocketTimeout = 30_000;
    config.processKillTimeout = 2000;
    config.retryLimit = Math.max(config.retryLimit || 0, 3);
    (config.client ??= {}).clearContext = false;
  }

Expected Behavior

Don't overwrite browsers from karma.conf.js

The extension overrides the browser setting of the karma config in order to configure it for running and debugging tests in a way that properly interfaces with Karma Test Explorer. It also has the karmaTestExplorer.browser setting which allows you to specify a browser of your choice while leaving you responsible for also providing a corresponding value for the karmaTestExplorer.debuggerConfig or karmaTestExplorer.debuggerConfigName setting that will ensure that test debugging will work with your chosen browser.

But regarding testing concurrently with multiple browsers, the extension currently supports running, listening, and reporting tests and their results from only a single browser. It could be enhanced to support testing with multiple browsers concurrently, and would require enhancing it to listen, recognize and distinguish test results from multiple browsers running tests concurrently, as well as be able to report them in the UI in some form - probably either as separate test trees (for instance, one test tree in the UI for Chrome showing all the tests and their results, and a separate one for Firefox showing the same), or it could have each test in the UI have a child node for each of the browsers, which can independently pass or fail.

If you'd like to work on this enhancement, I'd be willing to review and accept it.