Marketplace homepage: rainforestapp/github-action
This is the Rainforest QA GitHub Action, it allows you to easily kick off a Rainforest run from your GitHub workflows, to make sure that every release passes your Rainforest integration tests.
If you don't already have one, you can talk to us about setting up an account here.
You can find yours on the Integrations setting page.
Do not expose this token in your .github/workflows/<workflow>.yml file. Instead, use a GitHub encrypted secret. You may name this secret as you wish (as long as it's a name allowed by GitHub), e.g. RAINFOREST_API_TOKEN.
Run groups are a way to group tests that should be run together (for example, a smoke suite you might want to run on every deploy). For more information on run groups, see this help article.
Once you have a run group which contains at least one enabled test, you can run it in GitHub workflows using this Action. You will need its ID (visible at the end of the run group URL: https://app.rainforestqa.com/run_groups/<ID>).
This is a simple workflow file that will start a Rainforest run using run group #1234 every time a commit is pushed to the main branch. To use it, change the run group ID to the one you wish to run, ensure your Rainforest API token is properly configured in your GitHub secrets, and commit this file in your repo at .github/workflows/rainforest.yml.
on:
push:
branches:
- main
jobs:
rainforest:
runs-on: ubuntu-latest
name: Run Rainforest
steps:
- name: Rainforest
uses: rainforestapp/github-action@master
with:
token: ${{ secrets.RAINFOREST_API_TOKEN }}
run_group_id: 1234An arbitrary string to associate with the run.
The default value :
"${GITHUB_REPOSITORY} - ${GITHUB_REF_NAME} ${GITHUB_JOB} $(date -u +'%FT%TZ')"
This means that if no description parameter is passed in and your repository is named my_org/my_repo, the GitHub job is #42 on the my_feature_branch branch, and the current time (in UTC) is noon on December 31st, 2021; then the created run's description will be:
my_org/my_repo - my_feature_branch 42 2021-12-31T12:00:00Z
Use a specific environment for this run. This parameter will be ignored if the custom_url parameter is also passed in.
If no environment_id parameter is passed in, the created run will use the Run Group's default environment.
Use a specific URL (via a temporary environment) for this run.
If no custom_url parameter is passed in, the created run will use the Run Group's default environment.
How we should handle currently active runs.
| Value | Behavior |
|---|---|
cancel |
Cancel all other runs in the same environment. |
cancel-all |
Cancel all other runs, regardless of environment. |
If no conflict parameter is passed in, then no active runs will be canceled.
The execution method to use for this run.
| Value | Behavior | Requirements |
|---|---|---|
automation |
Run against our automation agent. | - All tests in the run group are written with the Visual Editor. - No tests use a Tester Instruction/Confirmation action. |
crowd |
Run against our global crowd of testers. | |
automation_and_crowd |
Run against our automation agent where possible, fall back to the crowd of testers. | |
on_premise |
Run against your internal testers. | - On-premise is enabled for your account. |
If no execution_parameter parameter is passed in, the created run will run against the run group's default execution method.
A string used to link a run to a release (for example, a git SHA or tag, a version number, a code name)
If no release parameter is passed in, the SHA1 hash of the latest commit of the current workflow (obtained via the GITHUB_SHA environment variable) will be used.
Set to a value larger than 0 to retry failed tests excuted by our automation agent in the same run up to that number of times.
If no automation_max_retries parameter is passed in, the default from your account or run group is used.
Use a specific Rainforest branch for this run.
If no branch parameter is passed in, the main branch will be used.
Set to true to immediately complete the GitHub workflow job without waiting for the Rainforest run to complete
By default we wait for the run to complete in order to pass or fail the workflow job based on the run's result.
Set to true to run parameter validations without actually starting a run in Rainforest.
If no dry_run parameter is passed in, the run will be started in Rainforest.
If your Rainforest run fails due to a "non-bug" (your testing environment might have had a hiccup, or a test might have needed to be tweaked, etc), then rather than make code changes and then run your full testing suite once more, you'll instead want to rerun just the tests that failed. The Rainforest QA GitHub Action uses GitHub caching to know when a workflow is being rerun. It will then automatically rerun only the tests which failed in the previous run.
The Rainforest QA GitHub Action cannot detect on its own when it is being called multiple times in a single workflow through an intermediary reusable workflow. This means that the second invocation will attempt to rerun the run created by the first invocation, which will fail if that run has no failed tests to rerun. In order to avoid this situation, set the cache_key parameter to a distinct value in each invocation:
# reusable workflow
on:
workflow_call:
inputs:
cache_key:
type: string
jobs:
rainforest:
runs-on: ubuntu-latest
name: Run Rainforest
steps:
- name: Rainforest
uses: rainforestapp/github-action@master
with:
token: ${{ secrets.RAINFOREST_API_TOKEN }}
run_group_id: 1234
cache_key: ${{ inputs.cache_key }}# calling workflow
jobs:
first-invocation:
uses: ./.github/workflows/reusable.yml
with:
cache_key: first-invocation
second-invocation:
uses: ./.github/workflows/reusable.yml
with:
cache_key: second-invocationThis section describes the release process for the Action itself:
- Create a feature branch and do your work.
- Update the version in
action.yml. - Get review and merge to master.
- Create a GitHub Release with the proper
v-prefixed version tag (i.e.v0.0.1). List Bugfixes, Breaking changes, and New features (if present), with links to the PRs. See previous releases for an idea of the format we're using. - The
release.ymlworkflow will then run to update the major release tag. E.g. if your release was forv1.2.3, then it will automatically update thev1tag.
If you want to run an integration test, create a new branch in a repo of your choice and add a new workflow to.github/workflows/ with the action pointing to your commit.