Feature request: pre-run hook
ChrisBAshton opened this issue · 15 comments
Summary
We would like to be able to run arbitrary code before running the accessibility suite, as a lot of what we build isn't actually rendered until the user has interacted in some way.
We've made similar improvements to Wraith visual regression testing by providing hook functionality.
Possible Solution
It should be fairly simple to extend the API we have, ie
page('http://www.bbc.co.uk', {
skip: [
'Headings: Content must follow headings'
]
})...becomes:
page('http://www.bbc.co.uk', {
skip: [
'Headings: Content must follow headings'
],
before: (webdriver) => {
require('./before.js')(webdriver);
}
})And in before.js:
module.exports = (webdriver) => {
webdriver.select('button').click(); // or whatever the API is for the headless browser we use
};Context & Motivation
We want to check the accessibility of our content, but a lot of what we do is highly interactive, so the initial page may just contain an input and a button, so not much to scan. When the form is filled in, we dynamically render complex charts, various sections of results, etc.
Currently, BBC-a11y is just not very useful to Visual Journalism without being able to perform an interaction prior to the scan.
Thanks!
Is it possible to achieve the same effect using the URL / history API as the user moves through states?
This is how I have been using a11y to test various in page states on other projects.
Not without re-engineering our projects to use the history API. And our results are typically very custom, e.g. based on user's height & postcode or something, so we wouldn't just visit mypage.html#results because results will vary.
How does the user share thier result?
They often don't.
Here's an example of the kind of thing I'm talking about: https://www.bbc.co.uk/news/uk-england-45559619
I’m confused. I thought you said users can’t share a URL?
Anyway, I think this comes down too:
-
Do we pave the cow paths to ease testing for things which have been built without ways to test by state
-
Do we require state as part of the URL and keep the tool simpler, easier to support and more maintainable.
My personal view would be that states which can only be reached by driving the UI is a test design issue.
I’d recommend approaching future projects in such a way that you can test states based on the URL. This tool would then be able to assist you.
That’s just my take. I’d be interested in how other people approach this issue.
There's no custom share on that page, so indeed you can't share a custom URL that preloads the "My name is Ama, and there were 5 girls with that name in 2017" experience. Sometimes we do have a custom share like that but we usually just link to the page's 'vanilla' state, as there's little benefit in pre-loading another person's experience; it's all about getting your own personal experience by interacting with the thing.
Outside of our immediate use case, there are other good uses for a 'hook' mechanism, e.g. reloading a page with a specific HTTP header, or with a specific cookie, or without JavaScript, or with adverts. We found it very useful in Wraith!
it's all about getting your own personal experience by interacting with the thing.
I think its good to be able to share the results, but perhaps this is wondering off topic.
Outside of our immediate use case, there are other good uses for a 'hook' mechanism, e.g. reloading a page with a specific HTTP header, or with a specific cookie, or without JavaScript, or with adverts. We found it very useful in Wraith!
The tool already supports setting a cookie. I would be supportive of headers being include in a similar way via a property in the config.
My concern for support, maintainability etc is that the proposed hook is executable. That introduces more complexity and consideration. Especially around documentation, security, robustness and encapsulation. What would happen if we changed driver etc.
Perhaps, there is a middle ground. A way to provide to the config object via a property a set of actions to perform. From that we can then do those tasks much the same was as we set headings and cookies.
Something like (very very draft!):
page('http://www.bbc.co.uk', {
skip: [
'Headings: Content must follow headings'
],
before: [
'click': "button",
'click': "#submit"
]
})
(PS: included as an array as the steps are ordered, but this is just some play code rather than specific proposal)
This way we could commit to an API and maintain the config as static. We could also cover the feature with BDD and TDD tests to ensure quality for the actions we have agreed to support.
I'm still not sure we should pave the cow paths this way, but an approach like this would reduce the risk.
Cheers,
J&L
Is there not already a means to do something like this? It's all javascript based, so it should be possible.
Could you use this?
Looks very similar.
I would suggest the same as @EmmaJP. visit option sounds like what you need, @ChrisBAshton.
Oh it exists! Thanks. Didn't see anything in the main README so I never knew about this feature.
This should be all I need! 👍 Should we document it elsewhere or just close this issue?
This approach is nicer. It doesn’t suffer from the same issues around webdriver as it appears to be executed inside the page by the browser not within the a11y tool itself.
Not quite the same, but possible to achieve the same effects.
It isn't in the main README, but it is in the docs for experienced devs.
I would agree that the documentation should probably be revisited a little.
I've made a small change to the main README, to indicate that the second link includes information about more advanced features. As it is such a minor change and simple to revert, I committed it directly to master.
@ChrisBAshton if visit meets your requirement, could you please close this issue.
Sure, thanks