/wp-mobile-e2e-tests

Automated end-to-end tests for WordPress mobile apps https://github.com/wordpress-mobile/WordPress-Android and https://github.com/wordpress-mobile/WordPress-iOS

Primary LanguageRubyGNU General Public License v3.0GPL-3.0

Wordpress Mobile Automation

CircleCI

This project demonstrates mobile app automated testing using the parallel_appium gem to do a lot of the heavy lifting. The project demos the basics of testing with Appium and Ruby, shows some advanced techniques and through the gem uses SeleniumGrid to distribute tests across devices and execute tests in parallel.

Automation Gif

Automation Gif

What does this project do

  • Run UI tests on the Wordpress mobile apps
  • Demonstrate a project structure for running automated e2e tests with Appium, RSpec and Selenium
  • Demonstrate the use of the parallel_appium gem for running rspec appium tests in a single or multi-threaded environment across platforms
  • Demonstrate setting up a continuous integration flow with CircleCi, Appium and Rspec for Mobile testing
  • Demonstrate the use of Allure for generating reports of test runs
  • Demonstrate techniques for locating and interacting with mobile app elements through Appium

View Report

A report aside from the one generated by CircleCi is provided in the artifacts of the build. You can check out the latest one by clicking the link under the title at the top of the README. Here's a link to the report for one of the builds to give you an idea of what's provided.

Getting setup

The app builds tested with this project I've made publicly available to download below,

If you'd like to build the apps yourself you can visit the wordpress-mobile github pages and build the app from source via the following commands for Android and iOS.

If you'd like to use your own version follow the instructions here and generate an apk file.

If you'd like to use your own version follow the instructions here and generate an .app file, compress it to .app.zip file.

Once you've got your apk and .app.zip file you'll need to move them into the ./apps folder or update the appium text in the root folder of the project files to point to their correct location. For CI purposes I would recommend the apps folder to keep things simple.

System dependencies

  • Xcode command line tools should be installed
  • Ruby should be installed, preferably using the version in the .ruby-version file

Install bundler

gem install bundler

Install project dependencies

After installing bundler run bundle install --path vendor to install the dependencies.

Parallel Appium depenency

One of the things about this project is that it handles firing up the appium and selenium grid server, connecting to emulator/simulator, coordinating environment variables and executing tests in parallel. All of this code is bundled into the parallel_appium gem For this project to work well ensure you've followed the instructions in that repository for setting up the gem. These requirements basically ensure appium and all the required commands are available.

Appium text files

Appium text files are provided in the repository as per the requirement of the parallel_appium gem but when running this locally please remember to adjust these to match out with the paths you might be using on your machine. These files are used to specify the desired capabilities for appium.

Running the Project

The task for testing is defined as test within the wordpress namespace and should be fired up via rake. Recall as per the requirement of the parallel_appium gem an environment variable with the platform will need to be set prior to running. The command also accepts a parameter which can be the path to the file or folder of the specs to be executed, this will default to spec/ if left blank.... Now here's a few examples:

Running all iOS tests

platform=ios bundle exec rake wordpress:test

Running all android tests

platform=android bundle exec rake wordpress:test

Running all tests on both platforms simultaneously

platform=all bundle exec rake wordpress:test

Running a single iOS test

platform=ios bundle exec rake wordpress:test[relative/path/to/spec]

Viewing report locally

Install allure

Instructions for installing Allure can be found here.

Reports are generated with allure and the data for the report is stored in the output folder after the tests are finished, this includes any screenshots along with the test results and meta data. It's then compiled and stored in a wordpress-report folder in the project, you can open the report via Allure or you can simply execute,

bundle exec rake wordpress:report

and the report will open up in a browser for you.

Contributing and Bug reporting

Not going to say much here right now, just be polite, open issues and send in PRs if there's any problems or improvements you'd like to make.

This software is provided "as is" and without any express or implied warranties, including, without limitation, the implied warranties of merchantibility and fitness for a particular purpose.