Proof of concept to demo a visual regression solution using storybook-chrome-screenshot
and reg-suit
integrated into TeamCity with reg-suit github app. Example report here
Clone this repository, and install the required dependencies:
git clone git@github.com:omonk/vr-poc.git
cd vr-poc
npm i
Start storybook to get familiar with the current components, also check the code
in src/Hello.jsx
As you can see, we only have a single component that either shows a string, if a prop message was given, or it displays a default 'Hello World' message that receives through react-intl.
- New feature/fix/etc branches from master
- New commits
- Open PR
- TeamCity CI triggers script to run
npm run full-visual-regression-test
- reg-suit adds comment to PR approving or blocking
- Changes are made or PR is approved by a reviewer
- PR is merged
- After merge CI triggers another build step to update snapshots file with the most up to date versions of stories
- Rinse and repeat
package.json scripts includes screenshot
, regression
and a combo of the two full-visual-regression-test
.
screenshot
-
- Use
storybook-chrome-screenshot
to takes screenshots of all our stories and stores them in__snapshots__/
. - Options for viewports can be passed in
regression
-
- Runs
reg-suit run
, reg-suit has access to an S3 bucket (env variables) where it stores tests based on a git commit SHA. When this command is run it picks up the SHA based off where this branch has been created (and thus where the most up to date tests have been stored), downloads these files and compares them with the recently updated snapshots in__snapshots__
. After comparison it creates a report and uploads the tests to a new folder based off the current SHA of HEAD. - reg-suit comes with a github notify plugin, allowing failing tests to block PR. PR becomes unblocked when PR is approved by a reviewer.
- Ideally we would run these in TeamCity on the creation of a PR, not locally on commit or push.
- Reports are concise and provide a clear display of whats changed
- Provides a detailed audit within the Github repo and AWS S3 bucket
- Concerns about running in TeamCity with such a large number of stories.
- How long will it take?
- I've been told concurrency is a limiting factor in our CI. However
reg-suit
has a concurrency flag we can make use of.
- Storing such a large number of image files in the repo may cause some issues, maybe a case for https://git-lfs.github.com/
- With such a large number of stories in the project, PRs may be polluted by reg-suit automated comment.