/testcafe-browser-provider-browserstack

This is the BrowserStack browser provider plugin for TestCafe. https://devexpress.github.io/testcafe/

Primary LanguageJavaScriptMIT LicenseMIT

testcafe-browser-provider-browserstack

Build Status

This plugin integrates TestCafe with the BrowserStack Testing Cloud.

Install

Step 1: Clone the repo

git clone https://github.com/browserstack/testcafe-browser-provider-browserstack.git

Step 2: Go into the directory

cd testcafe-browser-provider-browserstack

Step 3: Install the dependencies

npm install (use lts version to avoid breaking changes)

Step 4: Gulp build the module

./node_modules/.bin/gulp build

Step 5: Link the package globally, for consumption by testcafe

npm link

Usage

Before using this plugin, save the BrowserStack username and access key to environment variables BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY.

Project name and build name will be displayed in BrowserStack if you set the environment variables BROWSERSTACK_PROJECT_NAME and BROWSERSTACK_BUILD_ID.

If you have troubles starting multiple browsers at once, or get browserstack-local related errors like #27, try setting the BROWSERSTACK_PARALLEL_RUNS environment variable to the number of browsers you want to run simultaneously, or to 1 if you want to run just one browser.

You can determine the available browser aliases by running

testcafe -b browserstack

If you run tests from the command line, use the alias when specifying browsers:

testcafe "browserstack:Chrome@53.0:Windows 10" "path/to/test/file.js"

When you use API, pass the alias to the browsers() method:

testCafe
    .createRunner()
    .src('path/to/test/file.js')
    .browsers('browserstack:Chrome@53.0:Windows 10')
    .run();

Tip: you can skip version (@53.0) or/and OS name (:Windows 10).

BrowserStack Proxy Options

Proxy options can be passed via environment variables.

  • BROWSERSTACK_PROXY - a string that specifies a proxy for the BrowserStack local binary. It should have the following structure: user:pass@proxyHostName:port,
  • BROWSERSTACK_LOCAL_PROXY - a string that specifies a proxy for the local web server. It should have the following structure: user:pass@proxyHostName:port,
  • BROWSERSTACK_FORCE_PROXY - if it's not empty, forces all traffic of BrowserStack local binary to go through the proxy,
  • BROWSERSTACK_FORCE_LOCAL - if it's not empty, forces all traffic of BrowserStack local binary to go through the local machine
  • BROWSERSTACK_NO_LOCAL - If it's not empty, forces all traffic of BrowserStack to go over public internet

Example:

export BROWSERSTACK_PROXY="user:p@ssw0rd@proxy.com:8080"
export BROWSERSTACK_LOCAL_PROXY="admin:12345678@192.168.0.2:8080"
export BROWSERSTACK_FORCE_PROXY="1"
export BROWSERSTACK_FORCE_LOCAL="1"
testcafe browserstack:chrome test.js

Other BrowserStackLocal Options

This plugin also allows you to specify the following BrowserStackLocal options via environment variables:

Option Environment Variable
binarypath BROWSERSTACK_BINARY_PATH
logFile BROWSERSTACK_LOGFILE
verbose BROWSERSTACK_VERBOSE

Example:

export BROWSERSTACK_BINARY_PATH="~/BrowserStack/BrowserStackLocal"
export BROWSERSTACK_LOGFILE="~/BrowserStack/logs.txt"
export BROWSERSTACK_VERBOSE="1"
testcafe browserstack:chrome test.js

BrowserStack JS Testing and BrowserStack Automate

BrowserStack offers two APIs for browser testing:

JS testing supports more types of devices (compare: JS Testing Devices vs Automate Devices), while Automate allows for much longer tests (2 hours vs 30 minutes) and provides some additional features (like the window resizing functionality).

TestCafe uses the JS Testing API by default. In order to use BrowserStack Automate, set the BROWSERSTACK_USE_AUTOMATE environment variable to 1.

Example:

export BROWSERSTACK_USE_AUTOMATE="1"
testcafe browserstack:chrome test.js

Setting Display Resolution

To set the display resolution, use the BROWSERSTACK_DISPLAY_RESOLUTION environment variable. Valid resolutions can be found here.

Remember that this only sets the display resolution and does not resize the browser window. You'll still need to use TestCafe's window resizing API to do so.

Example:

export BROWSERSTACK_DISPLAY_RESOLUTION="1024x768"
testcafe browserstack:chrome test.js

Specifying Chrome Command Line Arguments

To set Chrome command line arguments, use the BROWSERSTACK_CHROME_ARGS environment variable. You can specify multiple arguments by joining them with the space symbol. This option works only if the BrowserStack Automate API is enabled.

Examples:

export BROWSERSTACK_USE_AUTOMATE="1"
export BROWSERSTACK_CHROME_ARGS="--autoplay-policy=no-user-gesture-required"
testcafe browserstack:chrome test.js
export BROWSERSTACK_USE_AUTOMATE="1"
export BROWSERSTACK_CHROME_ARGS="--start-maximized --autoplay-policy=no-user-gesture-required"
testcafe browserstack:chrome test.js

Other BrowserStack Options

BrowserStack Automate allows you to provide options for its internal Selenium Grid in the form of key-value pairs called capabilities.

To specify BrowserStack capabilities via the TestCafe BrowserStack provider, use environment variables. This provider supports the following capabilities:

Capability Environment Variable
browserstack.debug BROWSERSTACK_DEBUG
browserstack.console BROWSERSTACK_CONSOLE
browserstack.networkLogs BROWSERSTACK_NETWORK_LOGS
browserstack.video BROWSERSTACK_VIDEO
browserstack.timezone BROWSERSTACK_TIMEZONE

Refer to the BrowserStack documentation for information about the values you can specify.

Example

export BROWSERSTACK_DEBUG="true"
export BROWSERSTACK_TIMEZONE="UTC"
testcafe browserstack:chrome test.js

Exceeding the Parallel Test Limit

When you run tests in multiple browsers or concurrently, you may exceed the maximum number of parallel tests available for your account.

Assume your plan allows 2 parallel tests, and you run one of the following commands:

testcafe 'browserstack:ie@11.0:Windows 10' 'browserstack:chrome@59.0:Windows 10' 'browserstack:safari@9.1:OS X El Capitan' tests/acceptance/
testcafe browserstack:ie@11.0:Windows 10 -c3 tests/acceptance/

In this instance, BrowserStack will refuse to provide all the required machines and TestCafe will throw an error:

Unable to establish one or more of the specified browser connections.

To keep within your account limitations, you can run tests sequentially (or in batches), like in the following bash script (credits to @maoberlehner for this example):

browsers=( "browserstack:ie@10.0:Windows 8" "browserstack:ie@11.0:Windows 10" "browserstack:edge@15.0:Windows 10" "browserstack:edge@14.0:Windows 10" "browserstack:firefox@54.0:Windows 10" "browserstack:firefox@55.0:Windows 10" "browserstack:chrome@59.0:Windows 10" "browserstack:chrome@60.0:Windows 10" "browserstack:opera@46.0:Windows 10" "browserstack:opera@47.0:Windows 10" "browserstack:safari@9.1:OS X El Capitan" "browserstack:safari@10.1:OS X Sierra" )

for i in "${browsers[@]}"
do
	./node_modules/.bin/testcafe "${i}" tests/acceptance/
done

Author

Developer Express Inc. (https://devexpress.com)