Screenshot path does not corresponds to the spec path
odai-alali opened this issue · 12 comments
Current behavior
I have two spec files, each contains a simple test case where I take a screenshot with cy.screenshot(). Both screenshots will be saved directly under scressnshots directory.
Desired behavior
Screenshot from cypress/e2e/deep/deep-example.cy.js to be saved under cypress/screenshots/deep/deep example --passes.png
Test code to reproduce
https://github.com/odai-alali/cypress-screenshot-issue
Cypress Version
10.0.3
Other
No response
Hi, thanks for reporting this. I believe this is an unavoidable side effect of the change in Cypress 10 from setting an integrationFolder for specs to setting a specPattern . From the changelog for 10.0.0:
Generated screenshots and videos will still be created inside their respective folders (screenshotsFolder, videosFolder). However, the paths of generated files inside those folders will be stripped of any common ancestor paths shared between spec files found by the specPattern option and may no longer be deterministic. Addressed in #19319.
may no longer be deterministic
Sorry for the late answer, but wouldn't such a behavior be bad if cypress was used for Visual Regression tests?
If my Visual Regression tests are distributed into folders, running the tests under one of these folders will have unexpected results
Update
I think I can use onAfterScreenshot to move the screenshot to the desired location
@odai-alali onAfterScreenshot is a good idea, hopefully that works for your case. I think we're going to take a closer look at this overall, appreciate the feedback.
Hello, I understand the idea to get rid of the ancestry folders.
But it comes with a big disadvantage, that you can't be sure the screenshots are in the same place. It depends on which e2e tests you are executing. You can run ALL tests but also only ONE specific test. The directory for the generated files can be different then.
This is annoying, especially when you want to test with visual regression and compare snapshots.
I will try the 'after:screenshot' solution, but I would prefer a configuration to KEEP the directory structure that is defined in the specs directory.
Hi, is there any news how it will be resolved?
Current implementation is no longer deterministic. Screenshots generated via cypress open (single file testing) are generated in different location than running all tests. After migration cypress became unusable in those cases.
There is possibility to introduce some option to ignore stripping the common ancestor paths?
+1 for adding an option to ignore stripping the common ancestor paths
I have the exact same problem now after upgrading to v10+.
I am working on a very big project and with this breaking change, the only way I see is to move the snapshots in a different directory. It is a very big task provided we have 100s of snapshots that will be required to move now.
I would like to understand if anyone was able to work around this issue using onAfterScreenshot ? @odai-alali @shtief
An example would be great if possible.
@VaibhavP17 you can use https://github.com/jaredpalmer/cypress-image-snapshot as an example how to setup screenshot behaviour onAfterScreenshot. We are using modified version of this plugin to use old style folder structure.
@mateuszwrobel : In fact, I am using the same plugin in my project. But I am encountering the folder structure issue when I migrate to Cypress 10+ version. Can you please guide me to the modified version of cypress-image-snapshot plugin that you are using to use the old style structure ?
@VaibhavP17 jaredpalmer/cypress-image-snapshot@f621ff6 or just can use their package: https://github.com/simonsmith/cypress-image-snapshot
In our project we decided that we don't want to be dependant on those packages so we have in our source code modified version of simonssmith's version. jaredpalmer's isn't maintained anymore. when we were tring resolve problem simons' wan't published properly to npm and we couldn't wait.
Thanks for the help @mateuszwrobel :)
Duplicate of #24052