maxgalbu/nightwatch-custom-commands-assertions

SaveElementScreenshot not working correctly

iabw opened this issue · 13 comments

iabw commented

Using the completely unedited version of the custom command, it's not working for specified elements on my site, google.com, bing.com. The size is correct, but the location is not.

There's several issues in EasyImage about crop and gravity not working. I went into the Easy Image code and removed all the passing of gravity options, but my images still don't crop to the correct place.

If this isn't some weird local issue for me and a bunch of other people, I would assume this is a recent issue in Easy Image, maybe caused by this commit - hacksparrow/node-easyimage@3ca8c42 where they modified the arguments heavily.

I don't need this feature from Nightwatch or have time to look into it specifically, but there ya go.

Thanks for the report, I'm not using this feature too so unfortunately I didn't test it thoroughly.

Anyway, does reverting to easyimage 1.2 solves the issue? You can change it in the package.json:

"dependencies": {
    "easyimage": "~1.2.2"
  }

and then do a npm install --production again.

I'm not able to try it right now, I will test it on this weekend.

iabw commented

Nope. Didn't fix it.

All the test i've done are working (steampowered.com, github.com, news site), with node-easyimage 1.2 and 1.3. What version of imagemagick are you using? Can you share the code you're using?

iabw commented

I just tried it a bunch more times and got a successful result a couple times, but not reliably...

Here's a pared down version of the code I was running in nightwatch. I ran this exact code and got both successes and failures:

module.exports = {
  tags: ['sometag'],
  'Screenshot "with some text in quotes"': function (client) {
    client.url( "http://www.google.com" )
      .saveElementScreenshot(".content img", "./results/screenshot-name.jpg")
  }
};

A failure has this output:
screenshot-name

This is the correct output, which I managed to get sometimes, less than 20% and with no apparent pattern to the successes.
screenshot-name

You can see the size is correct, it's just cropping to the wrong location. When it crops to the wrong location, it's always to the upper left of the target area, but not coordinates 0,0.

The size and location arguments being passed are correct

size: { height: 207,
  width: 269,
  class: 'org.openqa.selenium.Dimension',
  hCode: 1744830464 }
location: { class: 'org.openqa.selenium.Point',
  hCode: 1610612736,
  y: 80,
  x: 390 }

This is a screenshot of the page with an absolutely positioned element at those coordinates:
screen shot 2015-04-10 at 12 07 07 pm 2

I tried adding a setTimeout of 2 seconds around the easyimage.crop function in saveElementScreenshot, but that didn't change the result.

I installed Imagemagick with homebrew on osx Yosemite 10.10.1. Imagemagick was 6.9.0-3 when I first encountered the error. I just upgraded it to 6.9.1-0 before running these tests, in case that information is relevant. I saw no successful crops using 6.9.0-3, but I only attempted a few tries.

Edit: This was using easyimage 1.2.2, I didn't upgrade it back up to 1.3 after downgrading yesterday.

Hope this helps!

Well I always get the right image :-/ and I'm testing with your code, adding just /ncr to the url so it doesn't get redirected to the .it domain.
I have ImageMagick 6.9.0-3, so that's not it. node-easyimage is working as well, with any version.. There are so many thing involved in this that it could be any of them 💯
Do you see the image in the browser that selenium launches or are you testing headless?
What version of nightwatch, selenium and browser are you using?

iabw commented

I'm not running it headless, and the page is definitely loading fine ( I even tried adding a pause into the test before taking the shot ).

These are my dependencies

  "dependencies": {
    "chromedriver": "~2.10.0-1",
    "selenium-server": "2.38.0",
    "dalek-browser-ie": "0.0.4",
    "nightwatch": "*",
    "phantomjs": "~1.9.7-10"
  }

I tried upgrading selenium server and chromedriver just now, but then Selenium threw an error.

Oh i see you're using chrome, can you test on firefox driver? I'll try with chromedriver myself

Thinking about it, that question about headless is nonsense if you look at the screenshot :D

Also I didn't know chromedriver and selenium were on npm, good to know :) I'm guessing i could put this repo on npm too

Can you also share your nightwatch.json? I've tried everything to use the chromedriver but it just won't run. Here's the relevant part in my nightwatch.json:

"selenium": {
        "start_process": false,
        "server_path": "",
        "log_path": false,
        "host": "127.0.0.1",
        "port": 4444,
        "cli_args": {
            "webdriver.chrome.driver": "../node_modules/.bin/chromedriver",
            "webdriver.ie.driver": ""
        }
    },

    "test_settings": {
        "default": {
            "launch_url": "http://localhost",
            "selenium_port" : 4444,
            "selenium_host" : "localhost",
            "silent": true,
            "screenshots": {
                "enabled": false,
                "path": ""
            },
            "desiredCapabilities": {
                "browserName": "firefox",
                "javascriptEnabled": true,
                "acceptSslCerts": true
            }
        },

        "chrome": {
            "desiredCapabilities": {
                "browserName": "chrome"
            }
        }
    }

Then i run nightwatch --test --env chrome

But selenium answers:

Error retrieving a new session from the selenium server:
 { state: 'unhandled error',
  sessionId: null,
  hCode: 587925288,
  value: 
   { localizedMessage: 'The path to the driver executable must be set by the webdriver.chrome.driver system property; for more information, see http://code.google.com/p/selenium/wiki/ChromeDriver. The latest version can be downloaded from http://chromedriver.storage.googleapis.com/index.html',
     cause: null,
     suppressed: [],
     message: 'The path to the driver executable must be set by the webdriver.chrome.driver system property; for more information, see http://code.google.com/p/selenium/wiki/ChromeDriver. The latest version can be downloaded from http://chromedriver.storage.googleapis.com/index.html',
     hCode: 1281695058,
     class: 'java.lang.IllegalStateException',
     screen: null },
  class: 'org.openqa.selenium.remote.Response',
  status: 13 }

F**k.

iabw commented

My nightwatch.json looks like this -

{

  "selenium": {
    "start_process": true,
    "server_path": "./node_modules/selenium-server/lib/runner/selenium-server-standalone-2.38.0.jar",
    "log_path": "./results",
    "host": "127.0.0.1",
    "port": 4444,
    "cli_args": {
      "webdriver.chrome.driver": "./node_modules/.bin/chromedriver",
      "webdriver.ie.driver": "./node_modules/dalek-browser-ie/lib/bin/IEDriverServer.exe"
    }
  },

  "test_settings": {
    "default": {
      "launch_url": "http://localhost",
      "selenium_host": "127.0.0.1",
      "selenium_port": 4444,
      "silent": true,
      "disable_colors": false,
      "screenshots": {
        "enabled": true,
        "path": "./results/screenshots"
      },
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    },

    "prod": {
      "globals": {
        "domain": "http://www.domain.com"
      }
    },

    "local": {
      "globals": {
        "domain": "http://dev.domain.com"
      }
    },

    "chrome": {
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true
      }
    },

    "ie": {
      "desiredCapabilities": {
        "browserName": "internet explorer",
        "javascriptEnabled": true
      }
    },

    "phantom": {
      "desiredCapabilities": {
        "browserName": "phantomjs",
        "phantomjs.binary.path": "./node_modules/phantomjs/bin",
        "javascriptEnabled": true
      }
    }
  }
}

This is probably a bug on chromedriver: https://code.google.com/p/chromedriver/issues/detail?id=294.

I've set travis to test on chrome, and it fails on there too: https://travis-ci.org/maxgalbu/nightwatch-custom-commands-assertions/builds/58434918.

For now I can only recommend using firefox and not chrome if you want screenshots 😩

iabw commented

I got it reliably working and not working for me. I have 2 screens - laptop and attached monitor.

When I leave it opening on the laptop screen, the screenshot is the wrong size. I hadn't noticed, because the ratio is basically the same as the window that opens, but the size it's saving the screenshot at is actually the same as the Chrome window I already have open, NOT the size of the Chrome window opened by the test.

If I focus onto the second monitor before the test opens, the Chrome window opens at a different size, and for some reason that screenshot is taken at that accurate size, and then crops to the correct location.

If I initiate the test from the second monitor, it also crops correctly, so it's not a focusing issue, just something about the laptop screen and/or how chromedriver responds to it.

The laptop screen is a retina display, but otherwise I can't think of any reason this would be happening. And I don't think it explains why the tests would fail on Travis, as I would assume Travis operates headlessly, right?

iabw commented

I had an added issue in Chrome where if the element to screenshot was below the fold, I needed to scroll to the element, then subtract the scrollTop from the y location passed to the crop method of saveElementScreenshot. I think this is the chromedriver bug you linked to? Is that not a problem with Firefox?

There's some issues about blank screenshots on the chromedriver's page (this is another), when I run this repo's test on my mac I get a screenshot that seemingly has the correct width but wrong height, and looks like the page was zoomed in before taking the screenshot. This could be linked to the retina screen, and i share your thoughts on travis' failure. I probably need to see the screenshot from travis

This is not a problem in firefox, which takes a screenshot of the whole page and has no zoom.