iamchrismiller/grunt-casper

Grunt should fail when casperjs fails

Closed this issue · 10 comments

iamchrismiller commented

Hmm.. I have a test that shows a different result.
grunt casper:fail aborts with warnings.

What version of casperjs and this plugin are you using?

schickling commented

1.1.0-beta3

iamchrismiller commented

Does this still happen with 0.3.0?
If so can you give me a bit more context?

What is your casperjs task config look like? How are you encountering this error?

bboyle commented

I just faced a similar issue, on Windows 8. I switched to grunt-casper v0.2.2 and grunt now fails when the test fails.

zikifer commented

Seeing the same issue (Windows 8.1) with version 0.3.10 and CasperJS 1.1.0-beta3. I downloaded the source and tried the failure unit test and it has the same problem, namely Grunt is reporting the tests ran without error.

C:\Users\sk\Downloads\grunt-casper-master>grunt casper:fail
Running "casper:fail" (casper) task
Test file: C:\Users\sk\Downloads\grunt-casper-master\test\fixtures\testFail.js
# Basic Site Testing Fail Tests
PASS Page title is: "Test Title"
PASS Header Exists
PASS P Tag Exists
FAIL Should Fail - Span Tag Does Not Exist
#    type: assertExists
#    file: C:\Users\sk\Downloads\grunt-casper-master\test\fixtures\testFail.js:8
#    code: test.assertExists('span', 'Should Fail - Span Tag Does Not Exist');
#    subject: false
#    selector: "span"
FAIL 4 tests executed in 0.097s, 3 passed, 1 failed, 0 dubious, 0 skipped.

Details for the 1 failed test:

In C:\Users\sk\Downloads\grunt-casper-master\test\fixtures\testFail.js:8
  Basic Site Testing Fail Tests
    assertExists: Should Fail - Span Tag Does Not Exist
Result log stored in tmp/casper/testFail-results.xml
Casper Task 'casper:fail' took ~2682ms to run

Done, without errors.

It's the Done, without errors that's the issue. Grunt is returning success when it should be returning failure, so the testing harness doesn't know something failed. In my case I'm kicking things off using ANT (from Maven via the maven-antrun-plugin) and looking at ${cmdresult} - it should contain a failure code but instead is 0 (success).

zikifer commented

I tracked it down - it's a bug in CasperJS 1.1-beta3. The good news is the bug is fixed in the HEAD revision on GIT, so the next version of CasperJS will have it fixed (be that 1.1-beta4 or 1.1 (final)). Who knows when that might be though...

iamchrismiller commented

I have fixed this issue in 0.4.0. Please let me know if it resolves your issues, @schickling @zikifer @bboyle

icd2k3 commented

Hey Chris, I still see the issue using SlimerJS as my engine in grunt-casper 0.4.1 with CasperJS 1.1-beta3 - details here: #58

xzer commented

I got this issue for grunt-casper 0.4.2 and phantomjs 1.9.8 with casperjs 1.1-beta3