/kanstructor

Write, test and repeat using YAML syntax

Primary LanguageTypeScriptGNU General Public License v3.0GPL-3.0


Kanstructor

Write, test and repeat using YAML language

Highlights

  • Write automated tests in plain Yaml
  • Design visual tests in a matter of minutes
  • Use OpenCV and/or Bitmap comparison to perform visual tests
  • Scale up browser compatibility checks
  • Of course, hassle-free installation

Installation

IMPORTANT: Given this package is a node project, let's ensure to install node and npm as pre-requisites before setting up the project. For further info.

npx setup-kanstructor demo-project

Running the above command automatically sets up project structure, along with example files. This section provides the details on how the project resources are organised. Note that running this command also requires you to have git installed , if not install from here.

In order to run the tests, the following command does the job:

cd demo-project
npm run test
video-example.mp4

Quick Start

  • Step 1 : The src folder under the project is going to be a root directory for all testing stuff, and the most of the project contents will be inside resources directory.

  • Step 2 : Under resources folder, let's create tests folder which will contain test files in a plain YAML format. Note that, this package kanstructor identifies a file as a test file only if it ends in *test.yaml. More on how to write tests?

  • Step 3 : Many a times, CSS and XPath values used to identify html elements will be used in several places across test files. To keep all such values in centralized place, the folder elements needs to be created within resources. Just like test files, the element files need to be ending in *element.yaml The following snippet shows some example element yaml.

     ### Login page elements' xpath and css
    
     login_email: "[name='email']"
     login_password: "[name='password']"
     login_button: "//button[normalize-space()='Login']"
     home_logo: "a[href*='home']"
  • Step 4 : The next step is, the folders extracted-contents and snapshots need to be created to save all contents extracted during testing to external files and to keep baseline screenshots that will be verified against app under tests during testing respectively.

  • Step 5 : All common configurations such as browser, env etc will have to be in the file: config.yaml under config folder. The following snippet shows some examples.

      browser: chrome
      headless: false
      device: Desktop Chrome
      url: https://github.com/5v1988/kanstructor

    In addition, there is also a way to configure options required to perform visual comparison of snapshots through the file visual.tests.config.yaml under config folder, and the following snippet shows the basic configurations that are generally used.

    output:
      errorColor:
        red: 255
        green: 0
        blue: 0
      errorType: flat #"flat" or "movement" or "flatDifferenceIntensity" or "movementDifferenceIntensity" or "diffOnly"
      transparency: 0.3
      largeImageThreshold: 1200
      useCrossOrigin: false
      boundingBoxes:
        - left: 0
          top: 0
          right: 1300
          bottom: 800
    returnEarlyThreshold: 0
    scaleToSameSize: false
    ignore: antialiasing # "nothing" or "less" or "antialiasing" or "colors" or "alpha";
  • Step 6 : Lastly, to run all tests, the test runner runMe.js needs to be created in the project as follows:

      import runMe from 'kanstructor'
      runMe();

Now execute tests using node src/runMe.js ( or npm run test) from command line. Note that, not necessarily that the runner method must always be named as runMe; Once all setup is complete, the below is the expected project structure

.
├── README.md
├── node_modules
├── package-lock.json
├── package.json
└── src
    ├── resources
    │   ├── config
    │   │   └── config.yaml
    |   │   └── visual.tests.config.yaml
    │   ├── elements
    │   │   └── todo-element.yaml
    │   ├── reports
    │   │   ├── results.html
    │   │   └── results.json
    │   ├── snapshots
    │   │   ├── original-screenshot-1.png
    │   │   └── reference-screenshot-1.png
    │   └── tests
    │       └── todo-test.yaml
    └── runMe.js

Write Tests

— Tests are expected to be written in Yaml files, otherwise known as test files while using this package. Each of these tests should have to be written using 3 A's of testing: Arrange-Act-Assert

description: Some tests on cypress todo demo site
tests:
 - name: Set and delete todo lists
   exclude: false

   arrange:
     - name: Open the url for the app under test
       action: openUrl
       url: url

     - name: Set value for the first item in storage
       action: setValue
       key: firstItem
       value: Schedule doctor appointment

     - name: Set value for the second item in storage
       action: setValue
       key: secondItem
       value: Prepare a blog content

   act:
     - name: Add the first item
       id: 10001
       role: textbox
       text: What needs to be done?
       action: type
       value: ${firstItem}

     - name: Press Enter
       pause: 1
       action: press
       value: Enter

     - name: Add the second item
       locator: .new-todo
       action: type
       value: ${secondItem}

     - name: Press Enter
       pause: 1
       action: press
       value: Enter

     - name: Add the third item
       refId: 10001
       value: Fix the air conditioner

     - name: Press Enter
       pause: 1
       action: press
       value: Enter

     - name: Screenshot after adding all items
       pause: 1
       action: snapshot
       path: "src/example/resources/snapshots/original-screenshot-1.png"    

     - name: Hover to the first item
       text: ${firstItem}
       action: hover

     - name: Delete the first item
       pause: 2
       locator: "//div[normalize-space()='Schedule doctor appointment']//button"
       action: click

     - name: Hover to the second item
       text: ${secondItem}
       action: hover

     - name: Delete the second item
       pause: 2
       locator: "//div[normalize-space()='Prepare a blog content']//button"
       action: click

     - name: Hover to the third item
       text: Fix the air conditioner
       action: hover

     - name: Delete the third item
       pause: 2
       locator: "//div[normalize-space()='Fix the air conditioner']//button"
       action: click

     - name: Screenshot after deleting all items
       pause: 1
       action: snapshot
       path: "src/example/resources/snapshots/original-screenshot-2.png"

   assert:
     - name: Verify if the first item deleted
       pause: 2
       type: standard
       text: ${firstItem} 
       state: invisible

     - name: Verify if the second item deleted
       pause: 2
       type: standard
       text: ${secondItem}
       state: invisible

     - name: Verify if the third item deleted
       pause: 2
       type: standard
       text: Fix the air conditioner
       state: invisible

     - name: Compare screenshot after all items added
       type: snapshot
       original: "src/example/resources/snapshots/original-screenshot-1.png"
       reference: "src/example/resources/snapshots/reference-screenshot-1.png"
       tolerance: 1

 # Visual comparison using OpenCV
     - name: Compare screenshot after all items deleted by OpenCV
       type: glancing
       original: "src/example/resources/snapshots/original-screenshot-2.png"
       reference: "src/example/resources/snapshots/reference-screenshot-2.png"
       tolerance: 3

Guidelines

— A test file can have more than one test, however, our recommendation is to have a few of them, organized by some commonalities

— A test folder tests can contain several test files; No limits

— The high-level blocks — Arrange, Act and Assert, contain a sequence of steps to perform certain actions during testing.

Locating Strategy

This package allows to locate page elements using their accessible name and implicit role. For instance, in the below example, the textbox is located using its placeholder text What needs to be done?.

- name: Add the first item
  id: 10001
  role: textbox
  text: What needs to be done?
  action: type
  value: ${firstItem}

It's also important to that role must always go with text, and at the moment, the other supported roles are as follows:

  • textbox
  • checkbox
  • radio
  • link
  • option
  • button
  • slider
  • switch

Alternatively, If you are well versed in writing Xpath or CSS, you can simply use locator attribute in the blocks.

Reusing Blocks

By setting id for a test block, it will become reusable and can be used again within the same test or even in the test under the different yaml file. This is done using refId when needed.

- name: Add the first item
  id: 10001
  role: textbox
  text: What needs to be done?
  action: type
  value: ${firstItem}

In the above example, the id is set as 10001. So, by using this id as refId, this test block can be re-used as follows:

- name: Add the third item
  refId: 10001
  value: Fix the air conditioner

While doing so, all attributes except name and value will be taken up from the original test block.

State management

Sometimes, the test data either static or dynamic will have to be shared among blocks. This can be achieved by setting such values with an action named setValue so they can be accessed later part of the tests using ${variableName} or $variableName.

- name: Set value for the first item in storage
  action: setValue
  key: firstItem
  value: Schedule doctor appointment

As you can see from this example, the value for the key firstItem is set-up once and it can be accessed anywhere else later with text(or value) later as shown below.

- name: Verify if the first item deleted
  pause: 2
  type: standard
  text: ${firstItem} 
  state: invisible

Block Reference

Arrange

Action Description Keys Example
openUrl
Open an app url in browser Required —
name,
url
Optional —
pause
        - name: openUrl 
          url: https://github.com/5v1988
      
setValue
Set test data in value storage Required —
key,
value
        - name: Set value for the username
          action: setValue
          key: firstItem
          value: 5v1988
      

Act

Action Description Keys Example
type
Enter characters into textboxes Required —
name,
action,
locator,
value
Optional —
pause
          - name: Type in username
            action: type
            locator: "input[name='email']"
            value: 5v1988@gmail.com
        
check, uncheck Check (or Uncheck) radio button/checkbox Required —
name,
action,
locator,
Optional —
pause
        - name: Choose a gender
          action: check
          locator: "input[type='checkbox']"
          
    
click,
doubleclick
Click (or Doubleclick) button/link Required —
name,
action,
locator,
Optional —
pause
        - name: Type in username
          action: click
          locator: '#file-submit'
      
select Select a dropdown value Required —
name,
action,
locator,
value
Optional —
pause
        - name: choose_dropdown
          locator: "#dropdown"
          action: select
          value: Option 2
      
press Simulate a key press Required —
name,
action,
value
Optional —
pause
        - name: Press enter
          action: press
          value: Enter
      
clear,
focus,
hover
Clear (or focus or hover) on html element Required —
name,
action,
locator
Optional —
pause
        - name: hover on the login link
          locator: "//button[@id='login']"
          action: hover
      
snapshot Take a screenshot of a current window Required —
name,
action,
path
Optional —
pause
        - name: Screenshot the login failure
          pause: 1
          action: snapshot
          path: "path/to/save.png"
      
upload Upload a file to the app Required —
name,
action,
locator
path
Optional —
pause
        - name: Upload an image
          action: upload
          locator: '#file-upload'
          path: src/example/innerText.txt
      
extract Extract text contents from the current window.
Page source of the current window will be extracted by default
and locator attribute is mandatory if `extractType` is given
Required —
name,
action,
path
Optional —
locator
extractType — textContents, innerText, innerHTML
pause
        - name: Extract form contents
          action: extract
          path: "path/to/save.txt"
          locator: "form#customer"
          extractType: innerText
      
setValue
Set test data in value storage Required —
key,
value( or locator)
        - name: Set value for the username
          action: setValue
          key: firstItem
          value: 5v1988
      

Assert

Type Description Keys Example
standard Assert the expectation by using page element(s) Required —
name,
type,
locator, or (role or/and text)
role (always used with 'text'),
text,
state (accepted values: visible, invisible,
enabled, disabled, checked, unchecked, containText),
Optional —
pause
        - name: Verify dropdown selected
          type: element
          locator: "//option[@selected]"
          state: containText
          text: Option 2
      
compare Assert if text matches with value storage Required —
key,
locator (or value or text),
Optional —
pause
        - name: Compare username
          key: ${username}
          text: 5v1988
      
snapshot Compare the expected screenshot with the actual one on the current screen Required —
name,
type,
original,
reference,
tolerance (lies between 0 (to be exact) and 100 (ignore comparison))
Optional —
pause
        - name: Verify failure screen
          type: snapshot
          original: "path/to/screenshot.png"
          reference: "path/to/reference.png"
          tolerance: 1
      

Roadmap

— [X] Browser
— [X] Summary report
— [O] Parameterized tests
— [O] Mobile

Contributing

We really appreciate and value the work that you do for this project. Needless to say, contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make will benefit everybody else and are greatly appreciated.

Please read our contribution guidelines, and thank you for being involved!

Code of Conduct

See the Code of Conduct for details. Basically it comes down to:

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

Support

https://discord.gg/GWfMu5Cwq6

License

This project is licensed under the GPLv3 license.

See LICENSE for more information.

Happy Testing!