Marketplace homepage: rainforestapp/github-action

This is the Rainforest QA GitHub Action, it allows you to easily kick off a Rainforest run from your GitHub workflows, to make sure that every release passes your Rainforest integration tests.

• Prerequisites
• Base Usage
• Optional Parameters
• Release Process

If you don't already have one, you can talk to us about setting up an account here.

You can find yours on the Integrations setting page. Do not expose this token in your .github/workflows/<workflow>.yml file. Instead, use a GitHub encrypted secret. You may name this secret as you wish (as long as it's a name allowed by GitHub), e.g. RAINFOREST_API_TOKEN.

Run groups are a way to group tests that should be run together (for example, a smoke suite you might want to run on every deploy). For more information on run groups, see this help article.

Once you have a run group which contains at least one enabled test, you can run it in GitHub workflows using this Action. You will need its ID (visible at the end of the run group URL: https://app.rainforestqa.com/run_groups/<ID>).

This is a simple workflow file that will start a Rainforest run using run group #1234 every time a commit is pushed to the main branch. To use it, change the run group ID to the one you wish to run, ensure your Rainforest API token is properly configured in your GitHub secrets, and commit this file in your repo at .github/workflows/rainforest.yml.

on:
  push:
    branches:
      - main

jobs:
  rainforest:
    runs-on: ubuntu-latest
    name: Run Rainforest
    steps:
      - name: Rainforest
        uses: rainforestapp/github-action@master
        with:
          token: ${{ secrets.RAINFOREST_API_TOKEN }}
          run_group_id: 1234

An arbitrary string to associate with the run.

The default value :

"${GITHUB_REPOSITORY} - ${GITHUB_REF_NAME} ${GITHUB_JOB} $(date -u +'%FT%TZ')"

This means that if no description parameter is passed in and your repository is named my_org/my_repo, the GitHub job is #42 on the my_feature_branch branch, and the current time (in UTC) is noon on December 31st, 2021; then the created run's description will be:

my_org/my_repo - my_feature_branch 42 2021-12-31T12:00:00Z

Use a specific environment for this run. This parameter will be ignored if the custom_url parameter is also passed in.

If no environment_id parameter is passed in, the created run will use the Run Group's default environment.

Use a specific URL (via a temporary environment) for this run.

If no custom_url parameter is passed in, the created run will use the Run Group's default environment.

How we should handle currently active runs.

If no conflict parameter is passed in, then no active runs will be canceled.

The execution method to use for this run.

If no execution_parameter parameter is passed in, the created run will run against the run group's default execution method.

A string used to link a run to a release (for example, a git SHA or tag, a version number, a code name)

If no release parameter is passed in, the SHA1 hash of the latest commit of the current workflow (obtained via the GITHUB_SHA environment variable) will be used.

Set to a value larger than 0 to retry failed tests excuted by our automation agent in the same run up to that number of times.

If no automation_max_retries parameter is passed in, the default from your account or run group is used.

Use a specific Rainforest branch for this run.

If no branch parameter is passed in, the main branch will be used.

Set to true to immediately complete the GitHub workflow job without waiting for the Rainforest run to complete

By default we wait for the run to complete in order to pass or fail the workflow job based on the run's result.

Set to true to run parameter validations without actually starting a run in Rainforest.

If no dry_run parameter is passed in, the run will be started in Rainforest.

If your Rainforest run fails due to a "non-bug" (your testing environment might have had a hiccup, or a test might have needed to be tweaked, etc), then rather than make code changes and then run your full testing suite once more, you'll instead want to rerun just the tests that failed. The Rainforest QA GitHub Action uses GitHub caching to know when a workflow is being rerun. It will then automatically rerun only the tests which failed in the previous run.

The Rainforest QA GitHub Action cannot detect on its own when it is being called multiple times in a single workflow through an intermediary reusable workflow. This means that the second invocation will attempt to rerun the run created by the first invocation, which will fail if that run has no failed tests to rerun. In order to avoid this situation, set the cache_key parameter to a distinct value in each invocation:

# reusable workflow
on:
  workflow_call:
    inputs:
      cache_key:
        type: string

jobs:
  rainforest:
    runs-on: ubuntu-latest
    name: Run Rainforest
    steps:
      - name: Rainforest
        uses: rainforestapp/github-action@master
        with:
          token: ${{ secrets.RAINFOREST_API_TOKEN }}
          run_group_id: 1234
          cache_key: ${{ inputs.cache_key }}

# calling workflow
jobs:
  first-invocation:
    uses: ./.github/workflows/reusable.yml
    with:
      cache_key: first-invocation

  second-invocation:
    uses: ./.github/workflows/reusable.yml
    with:
      cache_key: second-invocation

This section describes the release process for the Action itself:

1. Create a feature branch and do your work.
2. Update the version in action.yml.
3. Get review and merge to master.
4. Create a GitHub Release with the proper v-prefixed version tag (i.e. v0.0.1). List Bugfixes, Breaking changes, and New features (if present), with links to the PRs. See previous releases for an idea of the format we're using.
5. The release.yml workflow will then run to update the major release tag. E.g. if your release was for v1.2.3, then it will automatically update the v1 tag.

If you want to run an integration test, create a new branch in a repo of your choice and add a new workflow to.github/workflows/ with the action pointing to your commit.