salesforce/sa11y

test(wdio): migrate from ts-node for wdio tests

mohanraj-r opened this issue · 1 comments

mohanraj-r commented

Currently ts-node is used in wdio conf as described in TypeScript Setup · WebdriverIO

But babel with '@babel/preset-typescript' is already used for jest tests as described in Getting Started · Jest

Can we reuse babel instead as is it much more popular and actively maintained

Also ts-node seems to be resulting in some issues as described below

mohanraj-r commented

When there is a TS warning/error, wdio runner currently skips running the tests but still reports passed with an exit status of 0

[0-0] RUNNING in chrome - /__tests__/browser-lib.test.ts
[1603406914.670][WARNING]: This version of ChromeDriver has not been tested with Chrome version 86.
[0-0] PASSED in chrome - /__tests__/browser-lib.test.ts

Spec Files:      1 passed, 1 total (100% completed) in 00:00:03

✨  Done in 4.55s.

Changing reporters (e.g. from spec to dot or concise) doesn't help (e.g. concise reports "All went well !!")

This needs to be fixed as it is misleading and could lead to issues passing undetected (e.g. in CI).
Currently to make sure that wdio tests are being run, have to closely watch the test output for signs e.g. tests names, count etc

[chrome 86.0.4240.111 mac os x #0-0] Running: chrome (v86.0.4240.111) on mac os x
[chrome 86.0.4240.111 mac os x #0-0] Session ID: dada5295279bd39d727176976a9a3b9f
[chrome 86.0.4240.111 mac os x #0-0]
[chrome 86.0.4240.111 mac os x #0-0] @sa11y/browser-lib
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should not have axe or sa11y loaded to start with
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should inject minified js
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should inject un-minified js
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should invoke functions on axe e.g. getRules
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should run a11y checks using axe
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should run a11y checks using sa11y
[chrome 86.0.4240.111 mac os x #0-0]    ✓ should filter a11y violations using sa11y
[chrome 86.0.4240.111 mac os x #0-0]
[chrome 86.0.4240.111 mac os x #0-0] 7 passing (3.1s)


Spec Files:      1 passed, 1 total (100% completed) in 00:00:06

✨  Done in 8.12s.

Can this issue be fixed with a ts-node config or migrating away from ts-node (to babel e.g.)