davidwessman/blinka_action

Github Action for reporting test results to Blinka or Github Checks

Reporting test results with Blinka

What is Blinka?

• A test result format that can be reported to blinka.app or in a more limited form directly in a Github Action.

Setup without blinka.app

This is a simpler version running directly in Github Action with no access to external service.

Limitations

• No access to test history
• No ability to show screenshots in test reports

Steps

1. Generate test results by using one of the reporters:

• For Ruby on Rails see blinka-reporter
• For Jest see Generate test report for Jest

2. Add the action davidwessman/blinka_action@v1 in a step after the tests run with parameters:

• github_token: ${{ secrets.GITHUB_TOKEN }} after tests.
• filename - the path of your formatted test results from step 1.

  # Rails Github Action seetup

  - name: Run tests
    env:
      BLINKA_JSON: true
    run: bundle exec rails test:system test

  - name: Report to Github PR
    uses: davidwessman/blinka_action@v1
      with:
        filename: ./test/blinka_results.json
        github_token: ${{ secrets.GITHUB_TOKEN }}

See full example

name: Main
on: [push]

jobs:
  tests:
    name: Tests
    runs-on: ubuntu-20.04
    services:
      postgres:
        image: postgres:13
        env:
          POSTGRES_USER: postgres
          POSTGRES_DB: synka_test
          POSTGRES_PASSWORD: 'password'
        ports: ['5432:5432']

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup Node
        uses: actions/setup-node@v2
        with:
          node-version: 14.x
          cache: 'yarn'

      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Setup test database
        env:
          RAILS_ENV: test
          PGHOST: localhost
          PGUSER: myapp
        run: |
          bin/rails db:setup

      - name: Run tests
        env:
          BLINKA_JSON: true
        run: bundle exec rails test:system test

      - name: Report to Github PR
        uses: davidwessman/blinka_action@v1
        with:
          filename: ./test/blinka_results.json
          github_token: ${{ secrets.GITHUB_TOKEN }}

3. Start building your tests and see the results reported!

Setup with blinka.app

• Add the Github App to your account and give access to repositories Blinka Bot.
• Register an account on Blinka
• When your account is confirmed:
  • Add your repositories in the Blinka UI.
  • Create an access token and store the token_id and token_secret in your Github Secrets, I use the names BLINKA_TOKEN_ID and BLINKA_TOKEN_SECRET in the actions below.
• Setup reporting with this action.

Workflow

1. Generate test results by using one of the reporters:
   • For Ruby on Rails see blinka-reporter
   • For Jest see Generate test report for Jest
2. If your CI builds are always running with access to Secrets (not from forks or for example Dependabot), continue with Example 1.
3. If you sometimes build without secrets, continue with Example 2
   • Store blinka_results.json as an artifact along with any screenshots.
   • In a reporting job running with access to Secrets, download the report and screenshots.
   • Run the action

Example 1 - Use blinka.app with full access to secrets

  # Rails Github Action seetup

  - name: Run tests
    env:
      BLINKA_JSON: true
    run: bundle exec rails test:system test

  - name: Report to Blinka
    uses: davidwessman/blinka_action@v1
      with:
        token_id: ${{ secrets.BLINKA_TOKEN_ID }}
        token_secret: ${{ secrets.BLINKA_TOKEN_SECRET }}

See full example

name: Main
on: [push, pull_request]

jobs:
  tests:
    name: Tests
    runs-on: ubuntu-20.04
    services:
      postgres:
        image: postgres:13
        env:
          POSTGRES_USER: postgres
          POSTGRES_DB: synka_test
          POSTGRES_PASSWORD: 'password'
        ports: ['5432:5432']

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      - name: Setup Node
        uses: actions/setup-node@v2
        with:
          node-version: 14.x
          cache: 'yarn'

      - name: Install packages
        run: |
          yarn install --pure-lockfile

      - name: Setup test database
        env:
          RAILS_ENV: test
          PGHOST: localhost
          PGUSER: myapp
        run: |
          bin/rails db:setup

      - name: Run tests
        env:
          BLINKA_JSON: true
        run: bundle exec rails test:system test

      - name: Report to Blinka
        uses: davidwessman/blinka_action@v1
        with:
          token_id: ${{ secrets.BLINKA_TOKEN_ID }}
          token_secret: ${{ secrets.BLINKA_TOKEN_SECRET }}

Example 2 - Separate reporting job

Running tests

  # Rails Github Action seetup

  - name: Run tests
    env:
      BLINKA_JSON: true
    run: bundle exec rails test:system test

    - if: always()
      run: |
        mkdir -p ./report/tmp # -p allows creating multiple folders at once
        [ ! -f ./blinka_results.json ] || mv blinka_results.json ./report
        [ ! -f ./tmp/screenshots ] || mv tmp/screenshots ./report/tmp/screenshots

    - uses: actions/upload-artifact@v2
      if: always()
      with:
        name: report
        path: ./report

Reporting

Inspired by https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

name: report-results

on:
  workflow_run:
    workflows: ['build-test']
    types:
      - completed

jobs:
  upload:
    runs-on: ubuntu-latest
    if: >
      ${{ github.event.workflow_run.event == 'pull_request' &&
      github.event.workflow_run.conclusion == 'success' }}
    steps:
      - uses: actions/checkout@v2

      - name: 'Download artifact'
        uses: actions/github-script@v3.1.0
        with:
          script: |
            var artifacts = await github.actions.listWorkflowRunArtifacts({
               owner: context.repo.owner,
               repo: context.repo.repo,
               run_id: ${{github.event.workflow_run.id }},
            });
            var matchArtifact = artifacts.data.artifacts.filter((artifact) => {
              return artifact.name == "report"
            })[0];
            var download = await github.actions.downloadArtifact({
               owner: context.repo.owner,
               repo: context.repo.repo,
               artifact_id: matchArtifact.id,
               archive_format: 'zip',
            });
            var fs = require('fs');
            fs.writeFileSync('${{github.workspace}}/report.zip', Buffer.from(download.data));
      - run: unzip report.zip
      - name: 'Upload reports'
        uses: ./
        with:
          filename: './blinka_results.json'
          token_id: ${{ secrets.BLINKA_TOKEN_ID }}
          token_secret: ${{ secrets.BLINKA_TOKEN_SECRET }}

Laravel junit

Output the tests results with --log-junit and pass the filename to the action.

name: Main
on: [push, pull_request]
jobs:
  laravel:
    runs-on: ubuntu-latest
    services:
      mysql-service:
        # Docker Hub image (also with version)
        image: mysql:8.0
        env:
          MYSQL_ROOT_PASSWORD: db_test_password
          MYSQL_DATABASE: db_test
        ## map the "external" 33306 port with the "internal" 3306
        ports:
          - 33306:3306
        options: >-
          --health-cmd="mysqladmin ping"
          --health-interval=10s
          --health-timeout=5s
          --health-retries=3
    steps:
      - uses: actions/checkout@v2
      - name: Prepare The Environment
        run: cp .env.example .env

      - name: Install Composer Dependencies
        run: composer install --no-progress --no-suggest --prefer-dist --optimize-autoloader

      - name: Generate Application Key
        run: php artisan key:generate

      - name: Upgrade Chrome Driver
        run: php artisan dusk:chrome-driver `/opt/google/chrome/chrome --version | cut -d " " -f3 | cut -d "." -f1`

      - name: Start Chrome Driver
        run: ./vendor/laravel/dusk/bin/chromedriver-linux &

      - name: Run Laravel Server
        run: php artisan serve &

      - name: Run Tests
        continue-on-error: true
        env:
          APP_URL: 'http://127.0.0.1:8000'
        run: php artisan dusk tests --log-junit ./blinka_results.xml

      - name: Report test results to pull request via Github API
        uses: davidwessman/blinka_action@main
        with:
          filename: ./blinka_results.xml
          github_token: ${{ secrets.GITHUB_TOKEN }}

      - name: Report test results to pull request via Blinka
        uses: davidwessman/blinka_action@main
        with:
          filename: ./blinka_results.xml
          token_id: ${{ secrets.BLINKA_TOKEN_ID }}
          token_secret: ${{ secrets.BLINKA_TOKEN_SECRET }}

Generate test report for Jest

• Copy the blinka-json-reporter.ts to your own project.
• Configure Jest to use the reporter, for example in jest.config.js or using the flag --reporters (see Documentation)
• Make sure to run Jest using --testLocationInResults to include reporting of line numbers.

Development

• npm run all to prepare dist.
• Generate a new release with a new tag to update the action.