How do I use the saveAboveTolerance option?

[ayau1]

Hi 👋
My tests are failing due to a minor difference which at the moment we want to ignore.

AssertionError
    + expected - actual

    -0.18
    +0

I saw the saveAboveTolerance option and have tried to implement it in my wdio test runner configuration file like this ...

exports.imageComparison = [
  'image-comparison',
  {
    baselineFolder: join(process.cwd(), './visualScreenshots/baselines/'),
    formatImageName: '{tag}-{logName}-{width}x{height}',
    screenshotPath: join(process.cwd(), 'visualScreenshots/'),
    savePerInstance: true,
    autoSaveBaseline: true,
    blockOutStatusBar: true, // mobile only
    blockOutToolBar: true, // mobile only
    scaleImagesToSameSize: true,
    ignoreAntialiasing: true,
    saveAboveTolerance: 1,
  },
];

But the test still fails with the same difference. I have also tried passing the option within checkElement, but that doesn't work either. I have tried to search for some example implementations and I can't see this in FAQ or any other open issue. Am I passing this incorrectly?

Environment (please complete the following information):

• Node.js version: 18.1.0
• NPM version: 8.8.0
• Browser name and version: - unable to specify with sauce labs
• Platform name and version: Android 12
• WebdriverIO version: 7.20.5
• wdio-image-comparison-service version: 3.1.1

Additional context
Add any other context about the problem here.

[wswebcreation]

Hi @ayau1
The saveAboveTolerance option only saves the diffs above a tolerance, but does not have an impact on the assertions.

If you want to assert and allow a tolerance, then you need to change your assertions. You can doe something like this

await expect(
            await browser.checkElement(
                await $("#element-id"),
                "firstButtonElement",
                {
                    /* some options */
                }
            )
        ).toBeLessThan(1)

NOTE: Please prevent using tolerances like this, you might allow bugs you don't want to slip. Try to figure out why the difference is created

[ayau1]

Ah ok, thanks for the clarification @wswebcreation