webdriverio-boneyard/wdio-junit-reporter

RFC: New capability `wdio:capabilityName` that takes precedence in lib/helpers/sanitize.caps

christian-bromann opened this issue · 2 comments

christian-bromann commented

From @spencerwilson-optimizely on October 20, 2017 22:7

Motivation

tl;dr sanitize.caps can alias several capabilities objects to the same sanitizedCapabilityName, which can cause problems downstream in a reporter like wdio-junit-reporter.

Neither the spec (admittedly, in Candidate Recommendation stage) nor BrowserStack recognize a deviceName capability, which is what webdriverio consults during sanitization. Rather, BrowserStack has an extension capability device that is semantically the same as a deviceName (the fact that it should have been called browserstack:device per the spec is another issue).

As a result, as a BrowserStack customer, different versions of mobile devices can be aliased to the same string after passing through sanitize.caps. This becomes the first segment of the classname in wdio-junit-reporter (src), and so the fact that the tests originated from different capabilities is erased.

Ideas for solution

  • Change references to caps.deviceName to caps.deviceName || caps.device. I think that might resolve the one failure case I've identified (mobile devices).
  • Support a wdio:capabilityName capability. If defined, then sanitizeString(caps['wdio:capabilityName']) (or wdio:configName, wdio:browserName, etc.) would be the return value of sanitize.caps.

If the maintainers (by the way, thank you!) agree this is a problem, I'm happy to submit a PR implementing the desired resolution. Cheers.

Copied from original issue: webdriverio/webdriverio#2360

christian-bromann commented

This should be handled in the reporter. So more or less solution 1 would work.

spencerwilson-optimizely commented

Thanks for the reply @christian-bromann !

I'm not sure I understand why this should be resolved in reporters when the aliasing is occurring upstream. It seems like a bug that sanitize.caps can alias multiple inputs to the same output. It also seems like an oversight that the sanitization implementation refers to Appium's deviceName extension capability and not also to BrowserStack's device extension capability, when the two are semantically equal.

Would you mind pointing out what I'm missing? It seems like making the change upstream would fix it for all consumers of sanitizedCapabilities, but I'm very new to this code so I don't doubt there's something I'm not considering. Thanks in advance.