Browserstack: execute / executeAsync fails with Android Samsung Galaxy S9

[klipstein]

Environment:

• WebdriverIO version: 5.2.2
• Mode: wdio testrunner
• If WDIO Testrunner, running sync/async: sync
• Node.js version: 10.11.0
• NPM version: 6.4.1
• Browser name and version: Android Chrome
• Platform name and version: Android 8.0, Samsung Galaxy S9
• Additional wdio packages used (if applicable): minimal set (dot-reporter, cli, local-runner, mocha-framework)

Config of WebdriverIO

exports.config = {
    user: process.env.BROWSERSTACK_USERNAME,
    key: process.env.BROWSERSTACK_ACCESS_KEY,
    specs: [
        './test/**/*.js'
    ],
    capabilities: [{
        os_version: '8.0',
        device: 'Samsung Galaxy S9',
        real_mobile: true
    }],
    sync: true,
    logLevel: 'trace',
    waitforTimeout: 10000,
    connectionRetryTimeout: 90000,
    connectionRetryCount: 3,
    framework: 'mocha',
    reporters: ['dot'],
    mochaOpts: {
        ui: 'bdd',
        require: ['babel-core/register']
    }
};

Describe the bug
When browser.execute or browser.executeAsync is used it fails with the following console messages:

Request failed due to The URL '/wd/hub/session/.../execute/sync' did not map to a valid resource
Request failed due to The URL '/wd/hub/session/.../execute/async' did not map to a valid resource

To Reproduce
Steps to reproduce the behavior:

The following mocha test was executed with the above mentioned configuration:

import assert from 'assert';

describe('Bug Android Samsung Galaxy S9', () => {
  it('execute/executeAsync', () => {
    browser.url('https://www.google.de');
    browser.setTimeout({
      script: 10000
    });
    const result1 = browser.execute(() => {
      return document.title;
    });
    const result2 = browser.executeAsync((done) => {
      done(document.title);
    });
    assert.equal(result1, result2);
  });
});

Expected behavior
It should execute the given JS code sync or async on Android in Browserstack.

Additional context
I'm not too deep into the different varying webdriver protocols, but it seems that the Android devices on Browserstack follow the JSONWP protocol instead of the W3C webdriver protocol. The above test works with webdriver v4 and there the following URLs are used: /wd/hub/session/.../execute and /wd/hub/session/.../execute_async. I've also found the following issue #3120 where the W3C and the JSONWP protocols should've been somehow overlapped, but it seems that they slightly differ around execute and executeAsync.

[christian-bromann]

@klipstein

but it seems that the Android devices on Browserstack follow the JSONWP protocol instead of the W3C webdriver protocol.

it seems (I can't prove it because you didn't provide any logs) that Browserstack responses with a W3C valid response resulting the webdriver package to apply the W3C WebDriver protocol to the session. It seems that Browserstack is inconsistent here. Please reach out to their support team.

[klipstein]

After writing all the above I've found out that this can be solved by passing a newer browserstack.appium_version to capabilities (when nothing is given Browserstack defaults to Appium 1.6.5):

capabilities: [{
  os_version: '8.0',
  device: 'Samsung Galaxy S9',
  real_mobile: true,
  'browserstack.appium_version': '1.9.1'
}]

Maybe this will be useful for someone else having the same issue.

[christian-bromann]

Browserstack defaults to Appium 1.6.5

uff, that is bad.

Thanks for sharing!

[klipstein]

I just can assume that they use 1.6.5 because when you select Android here it shows appium version 1.6.5 below.

[lo1tuma]

I get the same issue with the latest webdriverio version but using saucelabs instead of browserstack. For some devices I could fix this issue by using appium 1.9.1 but this appium version is not available for the iPhome 5 simulator with iOS 9.3 on saucelabs, so I have to use appium 1.7.1.
With webdriverio 4.x it worked without problems.

[fiducioso-karthikayan]

Thanks @klipstein . You saved my day! I've been going nuts facing this problem without any clue.

[Vadikenaladiken]

After writing all the above I've found out that this can be solved by passing a newer browserstack.appium_version to capabilities (when nothing is given Browserstack defaults to Appium 1.6.5):

capabilities: [{
  os_version: '8.0',
  device: 'Samsung Galaxy S9',
  real_mobile: true,
  'browserstack.appium_version': '1.9.1'
}]

Maybe this will be useful for someone else having the same issue.

Thanks! you've saved my life!