Write, test and repeat using YAML language
- Write automated tests in plain Yaml
- Design visual tests in a matter of minutes
- Use OpenCV and/or Bitmap comparison to perform visual tests
- Scale up browser compatibility checks
- Of course, hassle-free installation
IMPORTANT: Given this package is a node project, let's ensure to install node
and npm
as pre-requisites before setting up the project. For further info.
npx setup-kanstructor demo-project
Running the above command automatically sets up project structure, along with example files. This section provides the details on how the project resources are organised. Note that running this command also requires you to have git
installed , if not install from here.
In order to run the tests, the following command does the job:
cd demo-project
npm run test
video-example.mp4
-
Step 1 : The
src
folder under the project is going to be a root directory for all testing stuff, and the most of the project contents will be insideresources
directory. -
Step 2 : Under
resources
folder, let's createtests
folder which will contain test files in a plain YAML format. Note that, this packagekanstructor
identifies a file as a test file only if it ends in*test.yaml
. More on how to write tests? -
Step 3 : Many a times,
CSS
andXPath
values used to identify html elements will be used in several places across test files. To keep all such values in centralized place, the folderelements
needs to be created withinresources
. Just like test files, the element files need to be ending in*element.yaml
The following snippet shows some example element yaml.### Login page elements' xpath and css login_email: "[name='email']" login_password: "[name='password']" login_button: "//button[normalize-space()='Login']" home_logo: "a[href*='home']"
-
Step 4 : The next step is, the folders
extracted-contents
andsnapshots
need to be created to save all contents extracted during testing to external files and to keep baseline screenshots that will be verified against app under tests during testing respectively. -
Step 5 : All common configurations such as browser, env etc will have to be in the file:
config.yaml
underconfig
folder. The following snippet shows some examples.browser: chrome headless: false device: Desktop Chrome url: https://github.com/5v1988/kanstructor
In addition, there is also a way to configure options required to perform visual comparison of snapshots through the file
visual.tests.config.yaml
underconfig
folder, and the following snippet shows the basic configurations that are generally used.output: errorColor: red: 255 green: 0 blue: 0 errorType: flat #"flat" or "movement" or "flatDifferenceIntensity" or "movementDifferenceIntensity" or "diffOnly" transparency: 0.3 largeImageThreshold: 1200 useCrossOrigin: false boundingBoxes: - left: 0 top: 0 right: 1300 bottom: 800 returnEarlyThreshold: 0 scaleToSameSize: false ignore: antialiasing # "nothing" or "less" or "antialiasing" or "colors" or "alpha";
-
Step 6 : Lastly, to run all tests, the test runner
runMe.js
needs to be created in the project as follows:import runMe from 'kanstructor' runMe();
Now execute tests using node src/runMe.js
( or npm run test
) from command line. Note that, not necessarily that the runner method must always be named as runMe
; Once all setup is complete, the below is the expected project structure
.
├── README.md
├── node_modules
├── package-lock.json
├── package.json
└── src
├── resources
│ ├── config
│ │ └── config.yaml
| │ └── visual.tests.config.yaml
│ ├── elements
│ │ └── todo-element.yaml
│ ├── reports
│ │ ├── results.html
│ │ └── results.json
│ ├── snapshots
│ │ ├── original-screenshot-1.png
│ │ └── reference-screenshot-1.png
│ └── tests
│ └── todo-test.yaml
└── runMe.js
— Tests are expected to be written in Yaml files, otherwise known as test files while using this package. Each of these tests should have to be written using 3 A's of testing: Arrange-Act-Assert
description: Some tests on cypress todo demo site
tests:
- name: Set and delete todo lists
exclude: false
arrange:
- name: Open the url for the app under test
action: openUrl
url: url
- name: Set value for the first item in storage
action: setValue
key: firstItem
value: Schedule doctor appointment
- name: Set value for the second item in storage
action: setValue
key: secondItem
value: Prepare a blog content
act:
- name: Add the first item
id: 10001
role: textbox
text: What needs to be done?
action: type
value: ${firstItem}
- name: Press Enter
pause: 1
action: press
value: Enter
- name: Add the second item
locator: .new-todo
action: type
value: ${secondItem}
- name: Press Enter
pause: 1
action: press
value: Enter
- name: Add the third item
refId: 10001
value: Fix the air conditioner
- name: Press Enter
pause: 1
action: press
value: Enter
- name: Screenshot after adding all items
pause: 1
action: snapshot
path: "src/example/resources/snapshots/original-screenshot-1.png"
- name: Hover to the first item
text: ${firstItem}
action: hover
- name: Delete the first item
pause: 2
locator: "//div[normalize-space()='Schedule doctor appointment']//button"
action: click
- name: Hover to the second item
text: ${secondItem}
action: hover
- name: Delete the second item
pause: 2
locator: "//div[normalize-space()='Prepare a blog content']//button"
action: click
- name: Hover to the third item
text: Fix the air conditioner
action: hover
- name: Delete the third item
pause: 2
locator: "//div[normalize-space()='Fix the air conditioner']//button"
action: click
- name: Screenshot after deleting all items
pause: 1
action: snapshot
path: "src/example/resources/snapshots/original-screenshot-2.png"
assert:
- name: Verify if the first item deleted
pause: 2
type: standard
text: ${firstItem}
state: invisible
- name: Verify if the second item deleted
pause: 2
type: standard
text: ${secondItem}
state: invisible
- name: Verify if the third item deleted
pause: 2
type: standard
text: Fix the air conditioner
state: invisible
- name: Compare screenshot after all items added
type: snapshot
original: "src/example/resources/snapshots/original-screenshot-1.png"
reference: "src/example/resources/snapshots/reference-screenshot-1.png"
tolerance: 1
# Visual comparison using OpenCV
- name: Compare screenshot after all items deleted by OpenCV
type: glancing
original: "src/example/resources/snapshots/original-screenshot-2.png"
reference: "src/example/resources/snapshots/reference-screenshot-2.png"
tolerance: 3
— A test file can have more than one test, however, our recommendation is to have a few of them, organized by some commonalities
— A test folder tests
can contain several test files; No limits
— The high-level blocks — Arrange, Act and Assert, contain a sequence of steps to perform certain actions during testing.
This package allows to locate page elements using their accessible name and implicit role. For instance, in the below example, the textbox is located using its placeholder text What needs to be done?
.
- name: Add the first item
id: 10001
role: textbox
text: What needs to be done?
action: type
value: ${firstItem}
It's also important to that role
must always go with text
, and at the moment, the other supported roles are as follows:
- textbox
- checkbox
- radio
- link
- option
- button
- slider
- switch
Alternatively, If you are well versed in writing Xpath
or CSS
, you can simply use locator
attribute in the blocks.
By setting id
for a test block, it will become reusable and can be used again within the same test or even in the test under the different yaml file. This is done using refId
when needed.
- name: Add the first item
id: 10001
role: textbox
text: What needs to be done?
action: type
value: ${firstItem}
In the above example, the id
is set as 10001
. So, by using this id as refId
, this test block can be re-used as follows:
- name: Add the third item
refId: 10001
value: Fix the air conditioner
While doing so, all attributes except name
and value
will be taken up from the original test block.
Sometimes, the test data either static or dynamic will have to be shared among blocks. This can be achieved by setting such values with an action named setValue
so they can be accessed later part of the tests using ${variableName}
or $variableName
.
- name: Set value for the first item in storage
action: setValue
key: firstItem
value: Schedule doctor appointment
As you can see from this example, the value for the key firstItem
is set-up once and it can be accessed anywhere else later with text
(or value
) later as shown below.
- name: Verify if the first item deleted
pause: 2
type: standard
text: ${firstItem}
state: invisible
Action | Description | Keys | Example |
---|---|---|---|
openUrl |
Open an app url in browser | Required — name, url Optional — pause |
- name: openUrl
url: https://github.com/5v1988
|
setValue |
Set test data in value storage | Required — key, value |
- name: Set value for the username
action: setValue
key: firstItem
value: 5v1988
|
Action | Description | Keys | Example |
---|---|---|---|
type |
Enter characters into textboxes | Required — name, action, locator, value Optional — pause |
- name: Type in username
action: type
locator: "input[name='email']"
value: 5v1988@gmail.com
|
check, uncheck | Check (or Uncheck) radio button/checkbox | Required — name, action, locator, Optional — pause |
- name: Choose a gender
action: check
locator: "input[type='checkbox']"
|
click, doubleclick |
Click (or Doubleclick) button/link | Required — name, action, locator, Optional — pause |
- name: Type in username
action: click
locator: '#file-submit'
|
select | Select a dropdown value | Required — name, action, locator, value Optional — pause |
- name: choose_dropdown
locator: "#dropdown"
action: select
value: Option 2
|
press | Simulate a key press | Required — name, action, value Optional — pause |
- name: Press enter
action: press
value: Enter
|
clear, focus, hover |
Clear (or focus or hover) on html element | Required — name, action, locator Optional — pause |
- name: hover on the login link
locator: "//button[@id='login']"
action: hover
|
snapshot | Take a screenshot of a current window | Required — name, action, path Optional — pause |
- name: Screenshot the login failure
pause: 1
action: snapshot
path: "path/to/save.png"
|
upload | Upload a file to the app | Required — name, action, locator path Optional — pause |
- name: Upload an image
action: upload
locator: '#file-upload'
path: src/example/innerText.txt
|
extract |
Extract text contents from the current window. Page source of the current window will be extracted by default and locator attribute is mandatory if `extractType` is given |
Required — name, action, path Optional — locator extractType — textContents, innerText, innerHTML pause |
- name: Extract form contents
action: extract
path: "path/to/save.txt"
locator: "form#customer"
extractType: innerText
|
setValue |
Set test data in value storage | Required — key, value( or locator) |
- name: Set value for the username
action: setValue
key: firstItem
value: 5v1988
|
Type | Description | Keys | Example |
---|---|---|---|
standard | Assert the expectation by using page element(s) | Required — name, type, locator, or (role or/and text) role (always used with 'text'), text, state (accepted values: visible, invisible, enabled, disabled, checked, unchecked, containText), Optional — pause |
- name: Verify dropdown selected
type: element
locator: "//option[@selected]"
state: containText
text: Option 2
|
compare | Assert if text matches with value storage | Required — key, locator (or value or text), Optional — pause |
- name: Compare username
key: ${username}
text: 5v1988
|
snapshot | Compare the expected screenshot with the actual one on the current screen | Required — name, type, original, reference, tolerance (lies between 0 (to be exact) and 100 (ignore comparison)) Optional — pause |
- name: Verify failure screen
type: snapshot
original: "path/to/screenshot.png"
reference: "path/to/reference.png"
tolerance: 1
|
— [X] Browser
— [X] Summary report
— [O] Parameterized tests
— [O] Mobile
We really appreciate and value the work that you do for this project. Needless to say, contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make will benefit everybody else and are greatly appreciated.
Please read our contribution guidelines, and thank you for being involved!
See the Code of Conduct for details. Basically it comes down to:
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
This project is licensed under the GPLv3 license.
See LICENSE for more information.
Happy Testing!