Snowflake-Labs/schemachange

Auth Authentication Fails in GitHub Action Pipeline

Opened this issue · 2 comments

Masoud-Ghodrati commented

Issue: OAuth Authentication Fails in GitHub Action Pipeline

Description:

I am trying to use OAuth Authentication for schemachange in my GitHub Action pipeline but encounter the following error message:

Traceback (most recent call last):
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/bin/schemachange", line 8, in <module>
    sys.exit(main())
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 1309, in main
    deploy_command(config)
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 601, in deploy_command
    session = SnowflakeSchemachangeSession(config)
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 283, in __init__
    if self.set_connection_args():
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 372, in set_connection_args
    oauth_token = self.get_oauth_token()
  File "/opt/actions-runner/_work/_tool/Python/3.8.18/x64/lib/python3.8/site-packages/schemachange/cli.py", line 329, in get_oauth_token
    "url": self.oauth_config["token-provider-url"],
TypeError: 'NoneType' object is not subscriptable

This suggests that the oauth_config is not being correctly passed or is None. However, I have verified that the required environment variables are correctly set and passed in the GitHub Actions pipeline.

Steps to Reproduce:

  1. I have the following GitHub Actions pipeline configured:
name: testing

on:
  push:
    branches:
      - feat/*

jobs:
  Deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    environment: 
      name: 'Dev-SF-DB' 
    env:
      SF_ACCOUNT: ${{ secrets.SF_ACCOUNT }}
      SF_USERNAME: ${{ secrets.SF_USERNAME }}
      SF_ROLE: ${{ secrets.SF_ROLE_MULESOFT }}
      SF_WAREHOUSE: ${{ vars.SF_WAREHOUSE_MULESOFT }}
      SF_DATABASE: ${{ vars.SF_DATABASE }}
      CLIENT_ID: ${{ vars.CLIENT_ID }}
      CLIENT_SECRET: ${{ vars.CLIENT_SECRET }}
      SCOPE_URL_TEST: ${{ vars.SCOPE_URL_TEST }}
      AZURE_ORG_GUID: ${{ vars.AZURE_ORG_GUID }}
      SNOWFLAKE_AUTHENTICATOR: oauth
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      
      - name: Use Python 3.8.x
        uses: actions/setup-python@v5
        with:
          python-version: 3.8.x
      
      - name: Install Python dependencies
        run: |
          pip install --upgrade pip
          pip install -r code/python/requirements.txt

      - name: Run python
              run: |
                python code/python/src/snowflake_conn_tocken.py

      - name: Run schemachange to deploy the snowflake Stored Procedures
        run: |
          schemachange -f ${{ github.workspace }}/code/schemachange/stored_procedure \
          -a $SF_ACCOUNT \
          -r $SF_ROLE \
          -u $SF_USERNAME \
          -w $SF_WAREHOUSE \
          -d $SF_DATABASE \
          --create-change-history-table \
  1. I have also confirmed that a separate Python script using the same credentials works to authenticate and connect to Snowflake (in the pipeline). Here’s the relevant Python code that works:
import snowflake.connector
import requests
import os

AUTH_CLIENT_ID = os.getenv("CLIENT_ID")
AUTH_CLIENT_SECRET = os.getenv("CLIENT_SECRET")
AUTH_GRANT_TYPE = "client_credentials"
SCOPE_URL = os.getenv("SCOPE_URL_TEST")
AZURE_ORG_GUID = os.getenv("AZURE_ORG_GUID")
url = f"https://login.microsoftonline.com/{AZURE_ORG_GUID}/oauth2/v2.0/token"

req_info = {
            "url": url,
            "headers": {"Content-Type": "application/x-www-form-urlencoded", "User-Agent": "python/schemachange"},
            "data": {"client_id": AUTH_CLIENT_ID, "client_secret": AUTH_CLIENT_SECRET, "grant_type": "client_credentials", "scope": SCOPE_URL},
        }
response = requests.post(**req_info)
TOKEN = response.json()['access_token']

ctx = snowflake.connector.connect(
    account=os.getenv("SF_ACCOUNT"),
    authenticator="OAUTH",
    token=TOKEN,
    warehouse=os.getenv("SF_WAREHOUSE"),
    database=os.getenv("SF_DATABASE"),
    role=os.getenv("SF_ROLE")
)

Potential Cause:

It appears that the schemachange OAuth config is not parsing the environment variables correctly. I suspect the issue might be related to how the OAuth config is being passed within schemachange.

Additional Information:

Here's the config file I’m using with schemachange:

config-version: 1
snowflake-account: ''
snowflake-user: 'user'
snowflake-role: 'role'
snowflake-warehouse: 'warehouse'
oauthconfig:
  token-provider-url: 'https://login.microsoftonline.com/{{ env_var('AZURE_ORG_GUID', 'default') }}/oauth2/v2.0/token'
  token-response-name: 'access_token'
  token-request-headers:
    Content-Type: "application/x-www-form-urlencoded"
    User-Agent: "python/schemachange"
  token-request-payload:
    client_id: '{{ env_var('CLIENT_ID', 'default') }}'
    client_secret: '{{ env_var('CLIENT_SECRET', 'default') }}'
    grant_type: 'client_credentials'
    scope: '{{ env_var('SCOPE_URL_TEST', 'default') }}'

Request for Help:

Can someone please provide guidance on what might be causing the issue? Specifically, it seems like schemachange is unable to parse the OAuth configuration properly in the GitHub Actions pipeline.

herakilla commented

@Masoud-Ghodrati I ran into the same problem myself. When I looked at the code for how the cli is grabbing the config I found that the property in the yml file needs to be oauth_config instead of oauthconfig The CLI line in question

Once I made that change in my schemachange-config.yml the github action started working 😄

I opened PR #286 to fix this so hopefully a maintainer will merge it

  • 👍1
Masoud-Ghodrati commented

@Masoud-Ghodrati I ran into the same problem myself. When I looked at the code for how the cli is grabbing the config I found that the property in the yml file needs to be oauth_config instead of oauthconfig The CLI line in question

Once I made that change in my schemachange-config.yml the github action started working 😄

I opened PR #286 to fix this so hopefully a maintainer will merge it

Thanks @herakilla for this, I can't believe my eyes didn't caught this differece.

It seems it's getting the config now, however I have another issue, which is not related to this but it's related to schemachange. I get this error now:
image

I don't know why I need -u $SF_USERNAME when I use oauth. It should work without username as far as I understand.

Schemachange uses python connector

ctx = snowflake.connector.connect(
    account=os.getenv("SF_ACCOUNT"),
    authenticator="OAUTH",
    token=TOKEN,
    warehouse=os.getenv("SF_WAREHOUSE"),
    database=os.getenv("SF_DATABASE"),
    role=os.getenv("SF_ROLE")
)

and it doesn't need username.