gustafnk/h-include

Move PhantomJS tests to cross browser testing tool

Closed this issue · 8 comments

gustafnk commented

It would be awesome to be able to run the tests in as many browsers as possible. What is the best free tool today for that? And we need to rip out the tight PhantomJS dependency then, replacing it with some browser instrumentation library.

shogun70 commented

Travis suggests and uses
SauceLabs which is free for open-source and it looks like you can sign-up with your github account (just like Travis).

I've created a branch to experiment with this and finally got something working.

I found some write-ups
where others have already done this but some info seems out-of-date.

To use SauceLabs you need to install the Selenium WebDriver. Travis should automatically install this if you add it to the package.json devDependencies.

You also need a SAUCE_USERNAME - which is your SauceLabs account id - and a SAUCE_ACCESS_KEY - which is NOT your SauceLabs password - you'll have to generate one from your SauceLabs dashboard (after you get an account).
You may want to encrypt these before adding them to your .travis.yml file.

The Selenium web-driver can also be used locally. It supports Firefox out-of-the-box and other browsers when the appropriate plugins are preinstalled.

gustafnk commented

Golden! :D

shogun70 commented

I've updated my saucelabs branch so that on Travis it runs the tests on several browsers.

TODO:

  • The test directory needs a README.
  • The test code needs more comments, especially to relevant Selenium documentation.
  • The test-code might need a few delays added to allow time for content to be included.
gustafnk commented

I have checked your branch now and overall I think it looks really nice 👍

My thoughts:

  • This branch will add a lot of value to the project, well done! :)
  • It would be good to support a separate browsers.json for local development, like browsers-local.json (which could be in .gitignore after initial commit). That way, a developer with only FF installed can run the tests without modifying browsers.json.
  • The current test/README.md is wrong now anyway, no worries (I'll fix that later)
  • If we find that the tests are unstable we can add delays later
  • The approach with having test expectations in an index.json is not exactly what I expected. I was planning a move in the other direction: more local control in each test case (think tests written in Jasmine or similar). But it's ok to merge anyway.
  • There are (of course) many "debug commits" for travis and saucelabs. It would be nice if these were removed from the git history somehow.

So, please squash your commits (to one commit or many, doesn't matter) and send me a PR and I'll be happy to merge it.

shogun70 commented

Ok. I squashed it in a different branch: saucelabs-squashed.

I also updated "test/README.md".

You will need to get a Saucelabs Open-source registration for the "gustafnk/h-include" repo.
Then you can update the SAUCE_USERNAME and SAUCE_ACCESS_KEY env variables in ".travis.yml". Currently they are encrypted.

gustafnk commented

Changed and encrypted the saucelabs env variables. Pushed branch here: https://github.com/gustafnk/h-include/tree/saucelabs-squashed

gustafnk commented

Merged PR #34