Either disable user interactions during test execution or headless mode for NVDA

[DeanNeal]

Is there a way how to run tests in headless mode or how can I use my computer during test execution.
Example of my code:

await nvda.press('Tab');
await nvda.act();   
const log = await nvda.spokenPhraseLog();
await nvda.stop();
expect(log).toEqual(['some text']);```

[cmorten]

Hi @DeanNeal there is no way currently to operate screen readers (that I’m aware of) that allows for a non-invasive / headless operation. Screen readers haven’t been designed with this capability in mind.

I can understand that this is frustrating and a friction point for developing locally because running tests requires you to step away from you device until they have finished - something of a naff developer experience by modern standards!

A way forward here would be for us to develop a solution where the tests can be run remotely in a VM, a bit like who one might run tests with BrowserStack or SauceLabs say. There is an alternative lib that already implements this idea with the web driver protocol, see https://github.com/AmadeusITGroup/Assistive-Webdriver (can’t vouch as haven’t used!).

In potentially promising news for the long term, there is a W3C initiative https://aria-at.w3.org/ looking into interoperability, testing, and standardised APIs for screen readers which may eventually provide the kind of headless capability you’re after.

R.E. blocking user input, there are security implications to consider here and I’m not keen to extend the scope of this library to include that kind of capability. This doesn’t stop you using/building out said functionality separately and using it in a before like hook in your projects.

[DeanNeal]

@cmorten Thanks for fast response and your clarifications. Is it possible to run this test in Docker?

[cmorten]

@cmorten Thanks for fast response and your clarifications. Is it possible to run this test in Docker?

It’s an interesting thought - for VoiceOver no unfortunately, there isn’t a macOS docker image afaik. For NVDA it could be possible with a windows docker image 🤔

[DeanNeal]

@cmorten
Just tried to run tests in Docker.
Unfortunately await nvda.spokenPhraseLog() returns empty values

I tried to use mcr.microsoft.com/windows/nanoserver:1809 but there was a problem with Microsoft Visual C++ Redistributable so switched to mcr.microsoft.com/windows:1809

Dockerfile

FROM mcr.microsoft.com/windows:1809

COPY nodejs /windows/system32

WORKDIR /app
COPY . .

RUN cd /app

RUN rmdir /s /q nodejs

RUN npm install
RUN npx playwright install
RUN npx @guidepup/setup --ci

CMD ["npm", "run", "test"]

package.json

{
  "name": "dockerada",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@guidepup/guidepup": "0.17.1",
    "@playwright/test": "1.32.0",
    "@types/jest": "27.5.1",
    "babel-jest": "27.5.1",
    "jest": "27.5.1",
    "playwright": "1.32.0",
    "ts-jest": "27.1.5"
  },
  "jest": {
    "testEnvironment": "jsdom",
    "testPathIgnorePatterns": [
      "<rootDir>/node_modules/"
    ],
    "transform": {
      "\\.(js|jsx)$": "babel-jest",
      "\\.(ts|tsx)$": "ts-jest"
    }
  },
  "dependencies": {
    "@guidepup/setup": "^0.11.1"
  }
}

app.test.tsx

import { chromium, Browser } from 'playwright';
import { nvda } from "@guidepup/guidepup";

declare const window: any;

window.setImmediate = window.setTimeout;

describe('description', () => {
    let browser: Browser;
    beforeAll(async () => {
        browser = await chromium.launch({ headless: false });
    });

    test('test', async () => {
        const page = await browser.newPage();
        await page.goto('https://playwright.dev/');
        await nvda.start();
        await nvda.press('Tab');
        await nvda.press('Tab');
        await nvda.press('Tab');
        await nvda.press('Tab');
        await nvda.press('Tab');
        const log = await nvda.spokenPhraseLog();
        await nvda.stop();
        expect(log).toEqual('Playwright')
    }, 20000)

    afterAll(async () => {
        browser.close();
    });
})

[DeanNeal]

@cmorten Hi! Do you know why it happens?

[cmorten]

Hey - unfortunately I don't have a windows machine capable of supporting docker for windows containers so not able to repro (GitHub actions also doesn't support them). Haven't got around to exploring setting up a VM to try this.

Does this just happen when using the docker image, or is on your machine as well?

[DeanNeal]

It works locally using Windows 11. The issue occurs in Docker container.

[DeanNeal]

I need this to run test as a part of CI/CD