[Proposal] a matches file snapshot assertion
Closed this issue · 2 comments
I'm working on two different projects where i either generate or manipulate images. I'm unit testing this using the the assertMatchesFileHashSnapshot assertion that i added to this package a while ago. This works well, but its not ideal: there is no way to see what the image is supposed to look like, and if the test fails there is no easy way to compare the two images manually.
If i want to see an image now my workflow is to put an exit statement in my unit test to prevent it from emptying the temporary directory i save the image to, and then manually checking the image.
A few days ago i made a custom assertion method that makes testing images (but also pdfs or any kind of binary file) way easier. I was wondering if it would be accepted if i made a PR.
The new assertMatchesFileSnapshot($filePath) would work like this:
If no file snapshot exists yet, it puts the file in the __snapshots__/files/ directory with getSnapshotId() . $fileExtension as a file name (so for example: /tests/__snapshots__/files/ImageTest__it_can_transform_an_image__0.jpg)
When running the test again, it compares the hash of the file passed to assertMatchesFileSnapshot to the hash of the snapshot file. If the hashes don't match, the test fails.
If the test fails, it puts the failed test file in the files directory with a suffix, for example: /tests/__snapshots__/files/ImageTest__it_can_transform_an_image__0_failed.jpg. Now you can easily compare the two files, and see why the test failed. The assertMatchesFileSnapshot will automatically clean up failed files before asserting, so if your tests are green the files directory only contains valid snapshot files.
I've been using this method for a couple of days, and its a huge improvement over what i used to do. I was wondering what you guys think about this idea, and if it would be accepted if i made a PR?
Hi Sjors,
An assertMatchesFileSnapshot method sounds good to have! I'm not sure if we should deal with keeping failing tests files. Can't we assume you still have access to the file you generated during the test when it fails?
I think there's quite some complexity that comes up when we need to add another sort of file besides snapshot, it also should be ignored, and we'd need to teach users that.
So assertMatchesFileSnapshot -> yay! Persisting failed test files -> not sure yet...
Yea, we can assume failed file names, so we can ensure failed files are only persisted when the tests fail. When the tests pass all failed files are deleted. The failed files will never end up in version control, unless you push code with failing tests
I'll make a PR tomorrow, i'm pretty sure you'll like the way it works