⚠️ Archived

This repository is archived and is no longer maintained.

For the latest Keystone release please visit the Keystone website.

Keystone End-2-End Functional Testing Framework

The Keystone Nightwatch E2E framework (or simply KNE) is an end-2-end UI/functional testing framework for KeystoneJS and keystoneJS applications.

Installation

npm install keystone-nightwatch-e2e --save-dev

Running

Before running KNE ensure that your keystone application is up and running. To run KNE do the following in your test application.

    var keystoneNightwatchE2e = require('keystone-nightwatch-e2e');
    keystoneNightwatchE2e.startE2E(options, done);

    where options is:

    {
        keystone: <the-keystone-instance>
    }

Command Line Options

KNE may also be passed additional options that control its behavior:

Option Name	Description
--selenium-in-background	This option tells KNE to start the selenium server manually, instead of letting Nightwatch start it. Some users in some unix-based platforms have had issues with Nightwatch starting selenium.
--browser-name	Locally on "firefox" and "chrome" are supported. Please see SauceLab's Platform Configurator for supported browser names.
--browser-version	Applies to SauceLabs runs only. Locally it will run whatever browser version is installed. Please see SauceLab's Platform Configurator for supported versions. You may also specify "latest".
--env	The nightwatch context to use. Should be one of: `default`, `saucelabs-local`, `saucelabs-travis`. If not specified, `default` is used. For local dev testing either `default` or `saucelabs-local` should be used. `saucelabs-travis` should be reserved for travis builds.
--sauce-username	The SauceLabs user name to create a secured tunnel with SauceLabs. Should be used in combination with `--env saucelabs-local` or `--env saucelabs-travis` to perform testing against SauceLabs. You need a SauceLabs account to use this option.
--sauce-access-key	The SauceLabs access key to create a secured tunnel with SauceLabs. Should be used in combination with `--env saucelabs-local` or `--env saucelabs-travis` to perform testing against SauceLabs. You need a SauceLabs account to use this option.
--group	The test group to run. This is nightwatch terminology. Please see NightwatchJS Test Runner for on this and other options.
--test	The test to run. This is nightwatch terminology. Please see NightwatchJS Test Runner for on this and other options.

Examples:

node test/e2e/testApp.js --browser-name chrome --group test/e2e/group000
node test/e2e/testApp.js --browser-name firefox --test test/e2e/group000/test1.js
node test/e2e/testApp.js --env saucelabs-local --browser-name chrome --sauce-username MYUSERNAME --sauce-access-key MYACCESSKEY --group test/e2e/group000
node test/e2e/testApp.js --env saucelabs-local --browser-name firefox --browser-version 50 --sauce-username MYUSERNAME --sauce-access-key MYACCESSKEY

NOTES on examples:

The first example, runs the group000 tests in chrome in your localhost
The second example, runs the group000/test1 test in firefox in your localhost
The third example, runs the group000/test1 test in the latest version of chrome in SauceLabs
The fourth example, runs all the e2e tests in version 50 of firefox in SauceLabs

Overview

UI/functional end-to-end tests ensure regression coverage of all aspects of a KeystoneJS application. They also ensure that KeystoneJS itself has not regressed on the application functionality. The tests use a real keystone application and should do so with as much available configuration as possible. Please note that this is not a replacement for component-based unit testing. Unit testing attempts to do full regression coverage of all the operations a particular application component is responsible for without regards to its dependencies, which are usually mocked. This framework uses the nightwatchjs.org functional test framework. Thus, if you will be contributing updates to this framework it is a good idea to have some familiarity with its concepts. If all you are interested in is writing your own e2e tests for your own KeystoneJS application then you do not really need to become an expert in NightwatchJS but some familiarity with it is still greatly recommended since you will need to control the nightwatch configuration for your tests. Below we outline all you need to do to get started.

Application Test Environment Setup

For a sample e2e test setup, please refer to the one in KeystoneJS that is used for testing the KeystoneJS AdminUI functionality. Using that structure as a guide in your own application and updating it per your application requirements is considered a reasonable approach to get started. The following is an overview of a simple KNE test application setup:

test/e2e
    testApp.js                              => your main test app server (requires and starts KNE)

    adminUI                                 => adminUI e2e test suite
        tests                               => directory to group all e2e tests
            group000                        => a test group (000 is the group number)
                    test1.js                => first test in test group 000
                    test2.js                => second test in test group 000
                    test3.js                => third test in test group 000
                    ...
                    testMMM.js
            ...
            groupNNN
    modelTestConfig                         => describe your application models to the test framework
       ...                                  => use the KeystoneJS e2e test ones as a guide

    models                                  => test application models
       ...