Snowflake-Labs/schemachange

Create a custom GitHub Action

SPSCS-Simon opened this issue · 4 comments

Is your feature request related to a problem? Please describe.
To ease integration with GitHub Workflows provide a custom GitHub Action, this will make deploying schema change into sorrow faster and more efficient.

Describe the solution you'd like
Create an Action

Describe alternatives you've considered
N/A

Additional context
It's all about Dev X - reduce the friction!

I build a simple process to integrate schemachange with github actions via terraform as i am using snowflake. I have multiple environment and I tied each database env to a branch name, so the developer just write the script one time and when he deploy, the changes would apply to the database that is tied to the working branch. as a next step, i export all the variables in the github environment so I can use them into the schemachange command. I used terraform variables to store SF credentials as well as github secrets ( found it much straightforward )
hope this helps!

here is the yml file i am using to deploy to production database (prod_db)


on:
  pull_request:
    branches:
      - main  

jobs:
  
  snowflake-terraform-production:
    name: Plan & Deploy Terraform plan on main(production)
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request' && github.event.action == 'opened'&& success()  # Check if the staging build is a success

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2
        with:
          cli_config_credentials_token: ${{ secrets.PROD_SNOWFLAKE_TOKEN }}

      - name: Run Python 3.8.x
        uses: actions/setup-python@v2.2.1
        with:
          python-version: 3.8.x

      - name: Terraform Init
        id: init
        run: |
            cd scripts-prod  
            terraform init -no-color

      - name: Terraform Validate
        id: validate
        run: |
            cd scripts-prod
            terraform validate -no-color 
  
      - name: Terraform Apply
        id: apply
        run: |
            cd scripts-prod
            terraform apply -auto-approve 
  

      - name: schemachange install
        id: sc-install
        run: pip install schemachange

      - name: Determine Branch and Set Snowflake Database (main)
        id: set-snowflake-db-prod
        run: |
          current_branch=$(echo "${{ github.ref }}" | sed 's/refs\/pull\/[0-9]*\/merge/main/' | xargs)
          echo "Current Branch: $current_branch"  # Add this line to print the branch name
          if [[ "${current_branch}" == "stage" ]]; then
              SF_DATABASE="STAGE_DB"
            elif [[ "${current_branch}" == "main" ]]; then
              SF_DATABASE="PROD_DB"
            elif [[ "${current_branch}" == "util" ]]; then
              SF_DATABASE="UTIL_DB"
            else
            # For any other branches not explicitly handled, print a message and exit
            echo "Please contact admin for setting up a database for this branch: $current_branch"
            exit 1
            fi
            echo "SF_DATABASE=$SF_DATABASE" >> $GITHUB_ENV


      - name: Set Snowflake Variables (Production)
        id: set-snowflake-vars-prod
        run: |
          if [[ "${{ github.actor }}" == "your_github_handle" ]]; then
            SF_ACCOUNT="${{ secrets.SF_ACCOUNT }}"
            SF_USERNAME="${{ secrets.SF_USR }}"
            SF_ROLE="${{ secrets.SF_ROLE_USER }}"
            SNOWFLAKE_PASSWORD="${{ secrets.SF_PASSWORD }}"
            SF_WAREHOUSE="${{ secrets.SF_WAREHOUSE }}"

          elif [[ "${{ github.actor }}" == "somebody_else" ]]; then
            SF_ACCOUNT="${{ secrets.SF_ACCOUNT }}"
            SF_USERNAME="${{ secrets.SF_USR }}"
            SF_ROLE="${{ secrets.SF_ROLE }}"
            SNOWFLAKE_PASSWORD="${{ secrets.SF_PASSWORD}}"
            SF_WAREHOUSE="${{ secrets.SF_WAREHOUSE }}"

          else
            echo "Unknown user triggering the Production workflow."
            exit 1
          fi
          echo "SF_ACCOUNT=$SF_ACCOUNT" >> $GITHUB_ENV
          echo "SF_USERNAME=$SF_USERNAME" >> $GITHUB_ENV
          echo "SF_ROLE=$SF_ROLE" >> $GITHUB_ENV
          echo "SNOWFLAKE_PASSWORD=$SNOWFLAKE_PASSWORD" >> $GITHUB_ENV
          echo "SF_WAREHOUSE=$SF_WAREHOUSE" >> $GITHUB_ENV
          echo "SF_DATABASE=$SF_DATABASE" >> $GITHUB_ENV

      - name: schemachange deploy (Prod)
        id: sc-deploy-Prod
        working-directory: ./scripts-prod
        run: | 
          schemachange -f $GITHUB_WORKSPACE/scripts-prod -a $SF_ACCOUNT \
          -u $SF_USERNAME \
          -r $SF_ROLE \
          -w $SF_WAREHOUSE \
          -d $SF_DATABASE \
          -c $SF_DATABASE.SCHEMACHANGE.CHANGE_HISTORY --create-change-history-table```

@SPSCS-Simon Thank you for the idea for building a custom action for schemachange. I assume you are suggesting to create an actions/schemachange-deploy@v1 and actions/schemachange-render@v1 in the github actions marketplace.

Where should we place the custom GitHub Action?

Thoughts ?

@sfc-gh-tmathew - precisely!

not for me to say where they would reside, but assume a Snowflake-Labs repo would be best!?

The significance is that it be identifiable as maintained by Snowflake and not a third party. I’d go so far as to suggest it should sit logically side by side with the (what was a third party) Terraform provider, and schemachange!

Thank you for the suggestion. Will explore it and see if we can build something out. For now, there are other aspects of the repo we are focusing on. Will keep this open until there is progress on this subject.