Karma-UI5 cannot find testpage occasionally

[markus-schwarz87]

I use karma-ui5 in a CAP project to run frontend-integration-tests (OPA5). Everything is configured and the tests are executed and work. But occasionally the tests fail and I cannot find the solution for that.

Error:
31 07 2023 10:46:48.342:WARN [Chrome Headless 114.0.5735.199 (Windows 10)]: Adapter did not report total number of specs.
Chrome Headless 114.0.5735.199 (Windows 10) /base/webapp/test/integration/opaTests.qunit.html FAILED
Testpage did not define any tests.
Chrome Headless 114.0.5735.199 (Windows 10): Executed 1 of 1 (1 FAILED) (0.004 secs / 0 secs)

Maybe in 1 out 50 times the testpage(which starts the OPA5 tests) cannot be found and the log states that no tests are defined.
I already tried nearly all config options in the karma.conf which could be somehow related to that problem but the error still occurs occasionally.

Could this be a bug or do you have any suggestions?

Thanks in advance!

---

Relevant Files:

package.json

"karma-chrome-launcher": "^3",
"karma-ui5": "^3",

Karma.conf.js

process.env.CHROME_BIN = require('puppeteer').executablePath();
module.exports = function (config) {
'use strict';
config.set({
frameworks: ['ui5'],
reporters: ['progress'],
browsers: ['Chrome_custom'],
customLaunchers: {
Chrome_custom: {
base: 'ChromeHeadless',
flags: ['--window-size=1920,1080'],
},
},
singleRun: true,
ui5: {
configPath: 'ui5-mock.yaml',
testpage: 'webapp/test/testsuite.qunit.html',
failOnEmptyTestPage: true,
},
});
};

ui5-mock.yaml
specVersion: "2.6"
metadata:
name: xxxx
type: application
server:
customMiddleware:

• name: fiori-tools-proxy
  afterMiddleware: compression
  configuration:
  ignoreCertError: false # If set to true, certificate errors will be ignored. E.g. self-signed certificates will be accepted
  ui5:
  path:
  - /resources
  - /test-resources
  url: https://ui5.sap.com
• name: sap-fe-mockserver
  beforeMiddleware: csp
  configuration:
  service:
  urlPath: /xxx
  metadataPath: ./webapp/localService/metadata.xml
  mockdataPath: ./webapp/localService/data/
  generateMockData: false

[matz3]

Can you also provide the contents of webapp/test/integration/opaTests.qunit.html and the scripts loaded from it? This sounds like a race-condition where QUnit starts but no tests have been defined, yet. QUnit might be able to recover from this and wait for tests to be defined, but the adapter certainly can't. It should be made sure that either all tests are defined before the browser "load" event is fired, or QUnit.start should be called explicitly after all tests have been defined.