karlhorky/github-tricks

A collection of useful GitHub tricks

GitHub Tricks

A collection of useful GitHub tricks

GitHub Actions: Configure actions/cache to Skip Cache Restoration on Changes in Directory

To configure actions/cache to skip cache restoration on modification of any files or directories inside a Git-tracked directory, configure actions/checkout to fetch all commits in all branches and tags (warning: may be expensive) and use a key based on the last Git commit hash which modified anything contained in the directory:

name: Skip cache restoration on changes in directory
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          # Fetch all commits in all branches and tags
          fetch-depth: 0

      - name: Get last Git commit hash modifying packages/abc
        run: |
          echo "ABC_HASH=$(git log -1 --pretty=format:%H -- packages/abc)" >> $GITHUB_ENV

      - name: Cache packages/abc
        uses: actions/cache@v4
        with:
          path: packages/abc
          key: abc-build-cache-${{ env.ABC_HASH }}

      - name: Build packages/abc
        run: |
          pnpm --filter=abc build

GitHub Actions: Configure actions/cache to Skip Cache Restoration on Re-runs

To configure actions/cache to skip cache restoration on any re-runs of the workflow (to avoid having to manually delete flaky caches), use an if conditional on the workflow step to check that github.run_attempt is set to 1, indicating that it is the first attempt to run the workflow:

name: Skip cache restoration on re-runs
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Cache packages/abc
        # Only restore cache on first run attempt
        if: ${{ github.run_attempt == 1 }}
        uses: actions/cache@v4
        with:
          path: packages/abc
          key: abc-build-cache

      - name: Build packages/abc
        run: |
          pnpm --filter=abc build

GitHub Actions: Correct Broken actions/setup-node Version Resolution

Version resolution of Node.js aliases like lts/* in actions/setup-node is broken as of Aug 2024 (and will probably continue to be broken).

To resolve this, switch off actions/setup-node and instead use the preinstalled nvm to install the correct Node.js version based on the alias:

      # Use nvm because actions/setup-node does not install latest versions
      # https://github.com/actions/setup-node/issues/940
      - name: Install latest LTS with nvm
        run: |
          nvm install 'lts/*'
          echo "$(dirname $(nvm which node))" >> $GITHUB_PATH
        shell: bash -l {0}

If you also need caching for pnpm (replacement for the cache setting of actions/setup-node), follow with this config of actions/cache:

      - name: Get pnpm store directory
        shell: bash
        run: |
          echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
      - uses: actions/cache@v4
        name: Setup pnpm cache
        with:
          path: ${{ env.STORE_PATH }}
          key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
          restore-keys: |
            ${{ runner.os }}-pnpm-store-

GitHub Actions: Create Release from CHANGELOG.md on New Tag

Create a new GitHub Release with contents from CHANGELOG.md every time a new tag is pushed.

.github/workflows/release.yml

name: Release
on:
  push:
    tags:
      - '*'
permissions:
  contents: write
jobs:
  release:
    name: Release On Tag
    if: startsWith(github.ref, 'refs/tags/')
    runs-on: ubuntu-latest
    steps:
      - name: Checkout the repository
        uses: actions/checkout@v4
      - name: Extract the changelog
        id: changelog
        run: |
          TAG_NAME=${GITHUB_REF/refs\/tags\//}
          READ_SECTION=false
          CHANGELOG_CONTENT=""
          while IFS= read -r line; do
            if [[ "$line" =~ ^#+\ +(.*) ]]; then
              if [[ "${BASH_REMATCH[1]}" == "$TAG_NAME" ]]; then
                READ_SECTION=true
              elif [[ "$READ_SECTION" == true ]]; then
                break
              fi
            elif [[ "$READ_SECTION" == true ]]; then
              CHANGELOG_CONTENT+="$line"$'\n'
            fi
          done < "CHANGELOG.md"
          CHANGELOG_CONTENT=$(echo "$CHANGELOG_CONTENT" | awk '/./ {$1=$1;print}')
          echo "changelog_content<<EOF" >> $GITHUB_OUTPUT
          echo "$CHANGELOG_CONTENT" >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT
      - name: Create the release
        if: steps.changelog.outputs.changelog_content != ''
        uses: softprops/action-gh-release@v1
        with:
          name: ${{ github.ref_name }}
          body: '${{ steps.changelog.outputs.changelog_content }}'
          draft: false
          prerelease: false

Credit: @edloidas in nanostores/nanostores#267

GitHub Actions: Edit .json, .yml and .csv Files Without Installing Anything

yq (similar to jq) is preinstalled on GitHub Actions runners, which means you can edit a .json, .yml or .csv file very easily without installing any software.

For example, the following workflow file would use yq to copy all "resolutions" to "overrides" in a package.json file (and then commit the result using stefanzweifel/git-auto-commit-action.

.github/workflows/copy-resolutions-to-overrides.yml

name: Copy Yarn Resolutions to npm Overrides

on:
  push:
    branches:
      # Match every branch except for main
      - '**'
      - '!main'

jobs:
  build:
    name: Copy Yarn Resolutions to npm Overrides
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        # To trigger further `on: [push]` workflow runs
        # Ref: https://github.com/peter-evans/create-pull-request/blob/main/docs/concepts-guidelines.md#triggering-further-workflow-runs
        # Ref: https://github.com/peter-evans/create-pull-request/blob/main/docs/concepts-guidelines.md#push-using-ssh-deploy-keys
        with:
          ssh-key: ${{ secrets.SSH_PRIVATE_KEY }}

      - name: Use Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 'lts/*'

      - name: Copy "resolutions" object to "overrides" in package.json
        run: yq --inplace --output-format=json '.overrides = .resolutions' package.json

      - name: Install any updated dependencies
        run: npm install

      - uses: stefanzweifel/git-auto-commit-action@v4
        with:
          commit_message: Update Overrides from Resolutions

Or, to copy all @types/* and typescript packages from devDependencies to dependencies (eg. for a production build):

yq --inplace --output-format=json '.dependencies = .dependencies * (.devDependencies | to_entries | map(select(.key | test("^(typescript|@types/*)"))) | from_entries)' package.json

GitHub Actions: Free Disk Space

On GitHub Actions, runners are only guaranteed 14GB of storage space (disk space) (docs), which can lead to the following errors if your workflow uses more than that:

System.IO.IOException: No space left on device

or

You are running out of disk space. The runner will stop working when the machine runs out of disk space. Free space left: 72 MB

OR

ENOSPC: no space left on device, write

To free up disk space for your workflow, use the Free Disk Space (Ubuntu) action (GitHub repo):

      - name: Free Disk Space (Ubuntu)
        uses: jlumbroso/free-disk-space@v1.3.1
        with:
          # Avoid slow clearing of large packages
          large-packages: false

You may need to disable some of the clearing options, if your workflow relies upon features or programs which are being removed:

      - name: Free Disk Space (Ubuntu)
        uses: jlumbroso/free-disk-space@v1.3.1
        with:
          # Re-enable swap storage for processes which use more memory
          # than available and start using swap
          swap-storage: false

          # Avoid slow clearing of large packages
          large-packages: false

GitHub Actions: Only Run When Certain Files Changed

Only run a GitHub Actions workflow when files matching a pattern have been changed, for example on an update to a pull request:

name: Fix Excalidraw SVG Fonts
on:
  pull_request:
    paths:
      # All .excalidraw.svg files in any folder at any level inside `packages/content-items`
      - 'packages/content-items/**/*.excalidraw.svg'
      # All .excalidraw.svg files directly inside `packages/database/.readme/`
      - 'packages/database/.readme/*.excalidraw.svg'

For example, the following workflow uses sed to add default fonts to Excalidraw diagrams (no longer needed):

# Workaround to fix fonts in Excalidraw SVGs
# https://github.com/excalidraw/excalidraw/issues/4855#issuecomment-1513014554
#
# Temporary workaround until the following PR is merged:
# https://github.com/excalidraw/excalidraw/pull/6479
#
# TODO: If the PR above is merged, this file can be removed
name: Fix Excalidraw SVG Fonts
on:
  pull_request:
    # Run only when Excalidraw SVG files are added or changed
    paths:
      - 'packages/content-items/contentItems/documentation/*.excalidraw.svg'
      - 'packages/database/.readme/*.excalidraw.svg'

jobs:
  build:
    # Only run on Pull Requests within the same repository, and not from forks
    if: github.event.pull_request.head.repo.full_name == github.repository
    name: Fix Excalidraw SVG Fonts
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - name: Checkout Repo
        uses: actions/checkout@v4
        with:
          ref: ${{ github.head_ref }}

      - name: Fix fonts in Excalidraw SVGs
        run: |
          find packages/content-items/contentItems/documentation packages/database/.readme -type f -iname '*.excalidraw.svg' | while read file; do
            echo "Fixing fonts in $file"
            sed -i.bak 's/Virgil, Segoe UI Emoji/Virgil, '"'"'Comic Sans MS'"'"', '"'"'Segoe Print'"'"', '"'"'Bradley Hand'"'"', '"'"'Lucida Handwriting'"'"', '"'"'Marker Felt'"'"', cursive/g' "$file"
            sed -i.bak 's/Helvetica, Segoe UI Emoji/Helvetica, -apple-system,BlinkMacSystemFont, '"'"'Segoe UI'"'"', '"'"'Noto Sans'"'"', Helvetica, Arial, sans-serif, '"'"'Apple Color Emoji'"'"', '"'"'Segoe UI Emoji'"'"'/g' "$file"
            sed -i.bak 's/Cascadia, Segoe UI Emoji/Cascadia, ui-monospace, SFMono-Regular, '"'"'SF Mono'"'"', Menlo, Consolas, '"'"'Liberation Mono'"'"', monospace/g' "$file"
            rm "${file}.bak"
          done
      - name: Commit files
        run: |
          git config user.email github-actions[bot]@users.noreply.github.com
          git config user.name github-actions[bot]
          git add packages/content-items/contentItems/documentation/*.excalidraw.svg
          git add packages/database/.readme/*.excalidraw.svg
          if [ -z "$(git status --porcelain)" ]; then
            exit 0
          fi
          git commit -m "Fix fonts in Excalidraw SVGs"
          git push origin HEAD:${{ github.head_ref }}
        env:
          GITHUB_TOKEN: ${{ secrets.EXCALIDRAW_FONT_FIX_GITHUB_TOKEN }}

GitHub Actions: Push to Pull Request and Re-Run Workflows

It can be useful to commit and push to a pull request in a GitHub Actions workflow, eg. an automated script that fixes something like upgrading pnpm patch versions on automatic Renovate dependency upgrades.

Once your script makes the commit and pushes to the PR, it can also be useful to re-run the workflows on that commit, eg. linting the new commit, so that GitHub auto-merge can run also on protected branches with required status checks.

‼️ WARNING: Make sure that you do not create a loop! Once your script creates a commit, the workflow will run your script again on the new commit. On the 2nd run, it should not create a new commit, or you will have an endless loop of GitHub Actions workflow runs. 😬 We will revisit this in the script below.

1. First, create a GitHub fine-grained personal access token:

   1. Visit https://github.com/settings/personal-access-tokens/new
   2. Fill out Token name (repository name + a short reminder), Expiration (longest available, 1 year), Description (a reminder of the purpose)
      
      
      
      
   3. Under Repository access, choose Only select repositories and then choose the repository where the workflow will run
      
      
      
      
   4. Under Permissions, expand Repository permissions, locate Contents and choose Access: Read and write
      
      
      
      This will also by default set Metadata to Access: Read-only
      
      
      
      
   5. Review your settings under Overview - it should be set to "2 permissions for 1 of your repositories" and "0 Account permissions"
      
      
      
      
   6. If you are satisfied with this, click on Generate token.
   7. This will show you the token on your screen only once, so be careful to copy the token.

2. Add a repository secret

   1. In the repository where the workflow will run, visit Settings -> Secrets and variables -> Actions and click on New repository secret
      
      
      
      
   2. For Name, enter a SCREAMING_SNAKE_CASE name that makes it clear that it's a GitHub token (eg. PNPM_PATCH_UPDATE_GITHUB_TOKEN) and for Secret paste in the token that you copied at the end of step 1 above. Click Add secret.
      
      
      
      

3. Adjust your workflow for the token

   1. Under uses: actions/checkout, add a with: block including persist-credentials: false

      - uses: actions/checkout@v4
        with:
          # Disable configuring $GITHUB_TOKEN in local git config
          persist-credentials: false

   2. ‼️ IMPORTANT: As mentioned above, make sure that you don't create a loop! Make sure that your script which alters files includes some kind of way to exit early, for example checking git status --porcelain and running exit 0 to exit the script early without errors or by skipping steps based on the last commits

      - name: Fix `pnpm patch` not upgrading patch versions automatically
        run: |
          # <your script makes changes here>
      
          git add package.json pnpm-lock.yaml patches
          if [ -z "$(git status --porcelain)" ]; then
            echo "No changes to commit, exiting"
            exit 0
          fi
      
          # <your script sets Git credentials and commits here>
          # <your script pushes here>

   3. Set your Git user.email and user.name credentials to the GitHub Actions bot and commit:

      - name: Fix `pnpm patch` not upgrading patch versions automatically
        run: |
          # <your script makes changes here>
          # <your script exits early here>
      
          git config user.email github-actions[bot]@users.noreply.github.com
          git config user.name github-actions[bot]
      
          git commit -m "Upgrade versions for \`pnpm patch\`"
      
          # <your script pushes here>

   4. Use the token via secrets.<name> in your Git origin in your workflow (credit: ad-m/github-push-action)

      - name: Fix `pnpm patch` not upgrading patch versions automatically
        run: |
          # <your script makes changes here>
          # <your script exits early here>
          # <your script sets Git credentials and commits here>
      
          # Credit for oauth2 syntax is the ad-m/github-push-action GitHub Action:
          # https://github.com/ad-m/github-push-action/blob/d91a481090679876dfc4178fef17f286781251df/start.sh#L43-L5
          git push https://oauth2:${{ secrets.PNPM_PATCH_UPDATE_GITHUB_TOKEN }}@github.com/${{ github.repository }}.git HEAD:${{ github.ref }}

Example

name: Fix pnpm patches
on: push

jobs:
  fix-pnpm-patches:
    name: Fix pnpm patches
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          # Disable configuring $GITHUB_TOKEN in local git config
          persist-credentials: false
      - uses: pnpm/action-setup@v3
        with:
          version: 'latest'
      - uses: actions/setup-node@v4
        with:
          node-version: 'lts/*'
          cache: 'pnpm'

      # Fix `pnpm patch` not upgrading patch versions automatically
      # https://github.com/pnpm/pnpm/issues/5686#issuecomment-1669538653
      - name: Fix `pnpm patch` not upgrading patch versions automatically
        run: |
          # Exit if no patches/ directory in root
          if [ ! -d patches ]; then
            echo "No patches/ directory found in root"
            exit 0
          fi

          ./scripts/fix-pnpm-patches.sh

          git add package.json pnpm-lock.yaml patches
          if [ -z "$(git status --porcelain)" ]; then
            echo "No changes to commit, exiting"
            exit 0
          fi

          git config user.email github-actions[bot]@users.noreply.github.com
          git config user.name github-actions[bot]

          git commit -m "Upgrade versions for \`pnpm patch\`"

          # Credit for oauth2 syntax is the ad-m/github-push-action GitHub Action:
          # https://github.com/ad-m/github-push-action/blob/d91a481090679876dfc4178fef17f286781251df/start.sh#L43-L55
          git push https://oauth2:${{ secrets.PNPM_PATCH_UPDATE_GITHUB_TOKEN }}@github.com/${{ github.repository }}.git HEAD:${{ github.ref }}

Pull request with automatic PR commit including workflow checks: karlhorky/archive-webpage-browser-extension#47

Screenshot of a GitHub PR showing two commits, the first by Renovate bot upgrading dependencies and the second by the automated script shown above. Both commits have status icons to the right of them (one red X and one green checkmark), indicating that the workflows have run on both of them. At the bottom, the PR auto-merge added by the Renovate bot is carried out because the last commit has a green checkmark.

GitHub Actions: Create PostgreSQL databases on Windows, macOS and Linux

PostgreSQL databases can be created and used cross-platform on GitHub Actions, either by using the preinstalled PostgreSQL installation or installing PostgreSQL:

• windows-latest and ubuntu-latest runners have PostgreSQL preinstalled
• macos-latest runners don't have PostgreSQL preinstalled (as of May 2024)

To conditionally install PostgreSQL, initialize a cluster, create a user and database and start PostgreSQL cross-platform, use the following GitHub Actions workflow steps (change database_name, username and password to whatever you want):

name: CI
on: push

jobs:
  ci:
    name: CI
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [windows-latest, macos-latest, ubuntu-latest]
    timeout-minutes: 15
    env:
      PGHOST: localhost
      PGDATABASE: database_name
      PGUSERNAME: username
      PGPASSWORD: password
    steps:
      - name: Install PostgreSQL on macOS
        if: runner.os == 'macOS'
        run: |
          brew install postgresql@16
          # --overwrite: Overwrite pre-installed GitHub Actions PostgreSQL binaries
          brew link --overwrite postgresql@16
      - name: Add PostgreSQL binaries to PATH
        shell: bash
        run: |
          if [ "$RUNNER_OS" == "Windows" ]; then
            echo "$PGBIN" >> $GITHUB_PATH
          elif [ "$RUNNER_OS" == "Linux" ]; then
            echo "$(pg_config --bindir)" >> $GITHUB_PATH
          fi
      - name: Start preinstalled PostgreSQL
        shell: bash
        run: |
          echo "Initializing database cluster..."

          # Convert backslashes to forward slashes in RUNNER_TEMP for Windows Git Bash
          export PGHOST="${RUNNER_TEMP//\\//}/postgres"
          export PGDATA="$PGHOST/pgdata"
          mkdir -p "$PGDATA"

          # initdb requires file for password in non-interactive mode
          export PWFILE="$RUNNER_TEMP/pwfile"
          echo "postgres" > "$PWFILE"
          initdb --pgdata="$PGDATA" --username="postgres" --pwfile="$PWFILE"

          echo "Starting PostgreSQL..."
          echo "unix_socket_directories = '$PGHOST'" >> "$PGDATA/postgresql.conf"
          pg_ctl start

          echo "Creating user..."
          psql --host "$PGHOST" --username="postgres" --dbname="postgres" --command="CREATE USER $PGUSERNAME PASSWORD '$PGPASSWORD'" --command="\du"

          echo "Creating database..."
          createdb --owner="$PGUSERNAME" --username="postgres" "$PGDATABASE"

Example PR: upleveled/preflight-test-project-next-js-passing#152

GitHub Flavored Markdown Formatted Table Width

Use   entities to give a table column a width:

| property                                                            | description                           |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `border-bottom-right-radius`                                                                                                                                                                                                                                                                                                                                               | Defines the shape of the bottom-right |

Demo:

property	description
border-bottom-right-radius	Defines the shape of the bottom-right

GitHub Flavored Markdown Linking to Anchors in Other Markdown Files

Linking to an anchor in a relative Markdown file path in the same repo (eg. ./windows.md#user-content-xxx) doesn't currently work on GitHub (Mar 2023). Probably another bug in GitHub's client-side router, maybe fixed sometime.

A workaround is to link to the full GitHub URL with a www. subdomain - this will cause a redirect to the non-www. version, and scroll to the anchor:

-[Expo + React Native](./windows.md#user-content-expo-react-native)
+[Expo + React Native](https://www.github.com/upleveled/system-setup/blob/main/windows.md#user-content-expo-react-native)

README Symlinks

When in a particular folder (such as the root directory), GitHub displays content from README files underneath the files in that folder:

However, these README files need to be named README.md, readme.md, README.mdx or readme.mdx in order to be recognized. GitHub doesn't display the content of certain common Markdown index filenames such as index.md or index.mdx (❓MDX file extension) (as of 18 June 2019).

GitHub does however follow symlinks named README.md, readme.md, README.mdx and readme.mdx. See example here: mdx-deck root folder, mdx-deck symlink README.md

So if you want to use another file (also in a subdirectory) as the contents displayed within a directory, create a symlink pointing at it:

ln -s index.md README.md

Shell Script: Create README Symlinks

If you have many directories with index.mdx files that you want to display as the readme content when you enter those directories on the web version of GitHub, you can run this script in the containing directory.

# Create symlinks for all directories within the current directory that
# do not yet have README.mdx files.
find . -mindepth 1 -maxdepth 1 -type d '!' -exec test -e "{}/README.mdx" ';' -print0 | while IFS= read -r -d $'\0' line; do
    cd $line && ln -s index.mdx README.mdx && cd ..
done