Change threshold of mugshot comparison

[makl-gulk]

I'm currently using mugshot for visual regression testing with webdriverio.
However, I could not find a working way to change the threshold of the comparison.

I know that the PixelDifferOptions include the threshold, however, there is no documentation about how to use it. I cannot inject it into the mugshot constructor, I cannot change the mugshots threshold via PNGDiffer since it doesn't have the required attribute.

So I am asking for a constructor, that takes the PixelDiffer insteaf of a constructor that doesn't seem to use the PixelDiffers threshold.
If it does already do so, then a documentation about how to use it or where to use it would be beneficial in order to use mugshot properly.

Here is the little test I'm running with webdriverio and chai.

it('visual regression tests', async () => {
await browser.setWindowSize(1280, 1024);
await browser.url('/?action=setTitle');
await browser.pause(1000);
const differ = new PixelDiffer({threshold: 1});
const mugshot = new Mugshot(
new BrowserScreenshotter(
new WebdriverIOAdapter(browser)
),
new FsStorage('./screenshots/title-tests'),
{ createMissingBaselines: true},
);
const result = await mugshot.check('title-test');
assert.isTrue(result.matches, 'title matches');
})

[NiGhTTraX]

Hi, @makl-gulk. You can pass the differ instance through the options argument in Mugshot's constructor. You can check the docs here. In your example you just need to pass pngDiffer: differ next to createMissingBaselines: true.

Let me know if anything is unclear or if you find something's that's not working correctly.

[makl-gulk]

Hi, @NiGhTTraX,
thanks for your reply!

Now I feel kinda stupid.. The thing is I did like this: (for some reason I pasted a different version above)
const mugshot = new Mugshot(
new BrowserScreenshotter(
new WebdriverIOAdapter(browser)
),
new FsStorage('./screenshots/title-tests'),
{ differ, createMissingBaselines: true},
);

and wondered why it doesn't work since it also didn't throw an error I assumed I was correct. But I forgot the key to the value...

Now that I use it the correct way like this

const mugshot = new Mugshot(
new BrowserScreenshotter(
new WebdriverIOAdapter(browser)
),
new FsStorage('./screenshots/title-tests'),
{ pngDiffer: differ, createMissingBaselines: true},
);

it works fine! Thank you!

But I want to dd something: the docs says if you use 1 as the threshold, it says always true even if the images are completely different. That seems to be incorrect in regards to picture size.

I use webdriverio and opened for my baseline the chrome browser. The test however runs in headless mode what makes the size of the images different. Therefore, even with 1 as my parameter it fails the test.
I assume this is maybe intended. If it is intended, maybe you could add it to the docs.

Anyways thanks again for your helps!

[NiGhTTraX]

@makl-gulk no worries, glad it helped!

and wondered why it doesn't work since it also didn't throw an error I assumed I was correct

There's no runtime checking that you pass in the correct thing, but if you use Mugshot in a TypeScript project then you would get compile errors.

the docs says if you use 1 as the threshold, it says always true even if the images are completely different. That seems to be incorrect in regards to picture size.

Correct. If the images have different sizes they can't really be compared so the test will fail.

If it is intended, maybe you could add it to the docs.

Thank you for the feedback, I'll make sure to update the docs to mention this.

I use webdriverio and opened for my baseline the chrome browser. The test however runs in headless mode what makes the size of the images different.

You should use the same environment when creating the baselines and when comparing them. If you run the tests once and let Mugshot create the baselines and then run the test again it should pass. If it's not, look at the *.diff.png image that was created to see what's different. If you use different environments for creating baselines and for running the tests e.g. you create baselines locally on Mac OSX but run the tests in CI on Linux, then you could use a Docker container to create a consistent browser environment.

[makl-gulk]

You should use the same environment when creating the baselines and when comparing them. If you run the tests once and let Mugshot create the baselines and then run the test again it should pass.

Yea I figured that. Actually the baseline was created with mugshot, just the wdio config was changed from normal to headless chrome. As a result the image sizes changed and therefore, the tests fail.
Thanks for your replies!