dequelabs/axe-core-nuget

Excludes/Includes don't stack

thisaja opened this issue · 2 comments

Which package were you using when you encountered the issue?**

  • Deque.AxeCore.Selenium
  • Deque.AxeCore.Playwright
  • Deque.AxeCore.Commons

Describe the bug

When stacking excludes/includes only one of each of the excludes/includes actually work.

How to reproduce

Steps to reproduce the behavior:

  1. Create an AxeBuilder object: AxeBuilder axe = new AxeBuilder(driver);
  2. Stack includes/excludes: axe.Exclude("#abc").Exclude("#def").Exclude("#ghi");
  3. Then analyze the page: axe.Analyze();

Expected behavior

Elements with css selectors "abc", "def" and "ghi" and all their children should not be scanned.

Actual behaviour

Only elements with css selectors "abc" and their children are actually excluded.

Device Information

  • OS: Windows 11
  • Browser: Google Chrome
  • "Deque.AxeCore.Selenium" Package Version "4.4.0-alpha.c9651f2c2284967c64d2f28e460b5181d0af5c6d"

Additional context

A work-around to this bug is instead of chaining the excludes/includes, you only have one of them but seperate them with commas.
Ex: axe.Exclude("#abc, #def, #ghi");

AdnoC commented

Hi! Thanks for the bug report. Can you verify that the problem still persists? I was unable to replicate it and have added a test for this exact situation. We recently released version 4.8.0 after resolving some publishing issues so there should be some bug fixes if you upgrade.

Verified with the latest build - Deque.AxeCore.Selenium_v4.8.0

Here On the Test page is http://qateam.dequecloud.com/attest/api/test.html , with a complete scan we will get the below voilations:

code for full scan:

namespace AxeTest {
    public class SeleniumTests {
        public static void Main() {
           var options = new ChromeOptions();
            IWebDriver webDriver = new ChromeDriver(options);
            webDriver.Navigate().GoToUrl("http://qateam.dequecloud.com/attest/api/test.html");
            AxeResult axeResult = new AxeBuilder(webDriver).Analyze();
            Console.WriteLine("Axe Results:");
            Console.WriteLine(axeResult);
           Console.WriteLine($"Number of Violations: {axeResult.Violations.Length}");
        }
    }
}

1. aria-allowed-attr (one occurance with target id `"#wcag2a-fail ` )
2. autocomplete-valid - (one occurance with target id '#name` )
3. color-contrast - (one occurance with target id '#wcag2aa-fail` )
4. dlitem - (4 occurances - `#testFrame`)
5. heading-order (one occurance with target id '#h6` )
6. image-alt(two occurances with target id '#testFrame` )
7. label((one occurance with target id '#lastname` )
8. link-in-text-block"(two occurances with target id '#experimental-fail-1&#experimental-fail-2` )

Recoding for the violations and occurrences list

Screen.Recording.2023-11-07.at.10.08.52.AM.mov

Scenarios for multiple include/excludes

  1. Excludes a single element
    code:
webDriver.Navigate().GoToUrl("http://qateam.dequecloud.com/attest/api/test.html");
            AxeResult axeResult = new AxeBuilder(webDriver).Exclude("#wcag2aa").Analyze();
            Console.WriteLine("Axe Results:");
            Console.WriteLine(axeResult);

It excludes - #wcag2aa element - which can exclude violation for color-contrast , so that the test page can give 7 violations

1. aria-allowed-attr (one occurance with target id `"#wcag2a-fail ` )
2. autocomplete-valid - (one occurance with target id '#name` )
3. dlitem - (4 occurances - `#testFrame`)
4. heading-order (one occurance with target id '#h6` )
5. image-alt(two occurances with target id '#testFrame` )
6. label((one occurance with target id '#lastname` )
7. link-in-text-block"(two occurances with target id '#experimental-fail-1&#experimental-fail-2` )

Screen.Recording.2023-11-07.at.10.14.53.AM.mov
  1. Excludes multiple elements
    code:
webDriver.Navigate().GoToUrl("http://qateam.dequecloud.com/attest/api/test.html");
            AxeResult axeResult = new AxeBuilder(webDriver).Exclude("#wcag2aa").Exclude("#testFrame").Analyze();
            Console.WriteLine("Axe Results:");
            Console.WriteLine(axeResult);

It excludes - #wcag2aa element an d #testFrame element- which can exludes violations for color-contrast , dl-item and image-alt , so that the test page can give 5 violations

1. aria-allowed-attr (one occurance with target id `"#wcag2a-fail ` )
2. autocomplete-valid - (one occurance with target id '#name` )
3. heading-order (one occurance with target id '#h6` )
4. label((one occurance with target id '#lastname` )
5. link-in-text-block(two occurances with target id '#experimental-fail-1 , `#experimental-fail-2`  & `#testFrame`)

Screen.Recording.2023-11-07.at.10.20.58.AM.mov
  1. Include a single element

code:

 webDriver.Navigate().GoToUrl("http://qateam.dequecloud.com/attest/api/test.html");
            AxeResult axeResult = new AxeBuilder(webDriver).Include("#wcag2aa").Analyze();
            Console.WriteLine("Axe Results:");
            Console.WriteLine(axeResult);

It includes - #wcag2aa element which can inludes violation for color-contrast, so that the test page can give 1 violation

color-contrast - (one occurance with target id '#wcag2aa-fail )`

Screen.Recording.2023-11-07.at.10.28.58.AM.mov
  1. Includes multiple elements

code:

webDriver.Navigate().GoToUrl("http://qateam.dequecloud.com/attest/api/test.html");
            AxeResult axeResult = new AxeBuilder(webDriver).Include("#wcag2aa").Include("#testFrame").Analyze();
            Console.WriteLine("Axe Results:");
            Console.WriteLine(axeResult);

It includes - #wcag2aa element and #testFrame element- which can inludes violations for color-contrast , dl-item image-alt, link-in-text-block and page-has-heading-one , so that the test page can give 5 violations

1. color-contrast - (one occurance with target id '#wcag2aa-fail` )
2. dlitem - (4 occurances - `#testFrame`)
3. image-alt(two occurances with target id '#testFrame` )
4. landmark-one-main(one occurances with target #testFrame )
5. page-has-heading-one(one occurance with target id `#testFrame`)

Screen.Recording.2023-11-07.at.10.44.24.AM.mov
  1. Exclude&Include both
    code:
 IWebDriver webDriver = new ChromeDriver(options);
            webDriver.Navigate().GoToUrl("http://qateam.dequecloud.com/attest/api/test.html");
            AxeResult axeResult = new AxeBuilder(webDriver).EXclude("#wcag2aa").Include("#testFrame").Analyze();
            Console.WriteLine("Axe Results:");
            Console.WriteLine(axeResult);

It Excludes - #wcag2aa element so that color-contrast rule excludes and #testFrame element- which can inludes violations for dl-item image-alt, landmark-one-main and page-has-heading-one , so that the test page can give 4 violations

1. dlitem - (4 occurances - `#testFrame`)
2. image-alt(two occurances with target id '#testFrame` )
3. landmark-one-main(one occurances with target #testFrame )
4. page-has-heading-one(one occurance with target id `#testFrame`)
Screen.Recording.2023-11-07.at.10.50.16.AM.mov

Environment:

Label Value
Product axe-core-selenium-nuget
Version 4.8.0
OS-Details _MAC- Intel Core i7 - 11.6.8 _
BrowserDetails Chrome Version 119.0.6045.105 (Official Build) (64-bit)