/openai-at-scale

Simple ChatGPT UI application

Primary LanguageTypeScriptMIT LicenseMIT

OpenAI at Scale

Open in GitHub Codespaces Open in Remote - Containers

🌏 English | 日本語

✨ OpenAI at Scale is a workshop by FastTrack for Azure in Microsoft team that helps customers to build and deploy simple ChatGPT UI application on Azure.

🎯 Features

  • Chat UI
  • Configure system prompts and hyperparameters
  • Authenticate with Azure Active Directory and get user information from Microsoft Graph
  • Collect application logs with Azure Log Analytics
  • Store prompt log data to Azure Cosmos DB

🚀 Getting Started

⚒️ Prerequisites

To run locally

  • OS - Windows 11, MacOS or Linux

⚠ For Windows client user, please use Ubuntu 20.04 LTS (Windows subsystem for Linux) to run this application.
⚠ GitHub Codespaces is supported as Linux envionment.

To run on Azure

  • Azure subscription
    • Resources
      • Azure OpenAI Service
      • Azure Active Directory application
      • Azure Log Analytics
      • (Optional) Azure Cosmos DB

⚠ Free account is not supported

  • Role
    • Contributor role or higher for Azure subscription
    • Permission to create Azure Active Directory application
    • Permission to create Azure OpenAI Service

1. Creating Azure OpenAI Service 🧠

There are some ways to create Azure OpenAI Service, we recommend to use Azure Portal if you are not familiar with Azure CLI.

Before started you need to choose a location for your service. You can find the list of available locations here -> supported location is here : Supported locations

  • Azure Portal : You can follow the official document to create Azure OpenAI Service.
  • Azure CLI : You can use the following command to create Azure OpenAI Service.
command example
# Set environment variables
export SUBSCRIPTION_ID=<your subscription id>
export RESOURCE_GROUP=<your resource group name>
export LOCATION=eastus #hard coded to avoid confusion, you can change any avaiable location.
export AZURE_OPENAI_SERVICE=<your openai service name>
export AZURE_OPENAI_CHATGPT_DEPLOYMENT=<deployment name of your gpt-35-turbo model>
az login #check your subscription id
az account set --subscription $SUBSCRIPTION_ID
az group create --name $RESOURCE_GROUP  --location $LOCATION
az cognitiveservices account create \
    --name $AZURE_OPENAI_SERVICE \
    --kind OpenAI \
    --sku S0 \
    --resource-group $RESOURCE_GROUP \
    --location $LOCATION \
    --yes
az cognitiveservices account deployment create \
   -g $RESOURCE_GROUP  \
   -n $AZURE_OPENAI_SERVICE \
   --deployment-name $AZURE_OPENAI_CHATGPT_DEPLOYMENT \
   --model-name gpt-35-turbo \
   --model-version "0301"  \
   --model-format OpenAI \
   --scale-settings-scale-type "Standard"

2. Creating Azure Active Directory application 🔑

Follow the steps in register your application to register your application.

  • Select Single-page application (SPA) as platform type
  • The Redirect URI will be http://localhost:5000 and http://localhost:5173 for local development
  • Keep the Application (client) ID and Directory (tenant) ID for later use

(Optional) 3. Creating Azure Cosmos DB 🪐

You can create a Azure Cosmos DB account by following instructions on the Azure docs here and please make sure you enable Analytical Store, more details can be found here.

  • Select Core (SQL) as API
  • Container name will be chat_log and partition key will be /chat_session_id

4. Deploying to local environment 💻

Environment variables

You need to create .env files from .env.sample to set up your environment variables before you spin up your application.

  • app/frontend/.env
    • This file will be used for authentication function by Azure Active Directory SDK.
# Azure Active Directory application
VITE_CLIENTID="<your client id>"
VITE_TENANTID="<your tenant id>"
  • app/backend/.env
    • This file will be used for accessing to Azure OpenAI Service and Azure Cosmos DB.
# Azure OpenAI Service
AZURE_OPENAI_SERVICE="<your Azure OpenAI Service endpoint>"
OPENAI_API_KEY="<your Azure OpenAI Service key>"
AZURE_OPENAI_CHATGPT_DEPLOYMENT="<your model deployment>"


# (Optional) Azure Cosmos DB
AZURE_COSMOSDB_ENDPOINT="https://<account_name>.documents.azure.com:443/"
AZURE_COSMOSDB_KEY="<your Azure Cosmos DB access Key>"
AZURE_COSMOSDB_DB="< your Azure Cosmos DB database name>"

⚠ Please use Azure Key Vault to configure environment variables in production environments.

command examples to get environment variables from Azure CLI.
export RESOURCE_GROUP=<your resource group name>
export AZURE_OPENAI_SERVICE=<your openai service name>
export AZURE_OPENAI_CHATGPT_DEPLOYMENT=<deployment name of your gpt-35-turbo model>
export OPENAI_API_KEY=`az cognitiveservices account keys list \
-n $AZURE_OPENAI_SERVICE \
-g $RESOURCE_GROUP \
-o json \
| jq -r .key1`

Python environment

Python is required to run the backend Flask application.

Install Python libraries
cd app/backend
python -m venv ./backend_env
source .backend_env/bin/activate  #bash
pip install -r requirements.txt
Start Backend (Flask)
cd app/backend
flask run --debug #hot reload
#python ./app.py

Node.js environment

Node.js is required to run the frontend React application.

Install Node.js packages
cd app/frontend
npm install
Start Frontend (React)

For development

npm run dev

For production

npm run build

It is used to optimize and reduce the size of all application files which are deployed in app/backend/static folder.


5. Deploying to Azure ☁️

Deploy to Azure App Service

⚠ Before you run following command, you must run npm run build on app/frontend to set frontend files to backend static dir.

  • Example of Azure App Service

    • Deploy app to Azure App Service with easist way.

      cd app/backend
az webapp up --runtime "python:3.10" --sku B1 -g <Resource Group Name>

    • Deploy Azure App Service Plan and Web App separately.

      • You can deploy an app with above command but the command doesn't allow to change detailed App Service Plan and Web App settings. So if you want to change these settings you can deploy it separately with following command.

      • Create Azure App Service Plan resources

        az appservice plan create -g <Resource Group Name> --is-linux -n <App Service Plan Name> --sku <SKU Name> --location eastus

      • Create WebApp Resource on above App Service Plan

        az webapp create -g <Resource Group Name> -n <WebApp Name> -p <App Service Plan Name> -r "python:3.10"

        ⚡️completely optional: if your system needs to add private endpoint and/or VNET integration, you can add it here with following options.

        • VNET Integration

          # you need create vnet/subnet before execute this command
az webapp create -g <Resource Group Name> -n <WebApp Name> -p <App Service Plan Name> -r "python:3.10" --vnet <VNET Name> --subnet <Subnet Name>

        • Private Endpoint

          # you need create vnet/subnet webapp before execute this command
az network private-endpoint create \
  -n <PE Name> \
  -g <Resource Group Name> \
  --vnet-name <VNET Name> \
  --subnet <Subnet Name> \
  --connection-name <Private Endpoint Connection Name> \
  --private-connection-resource-id /subscriptions/SubscriptionID/resourceGroups/myResourceGroup/providers/Microsoft.Web/sites/<WebApp Name> \
  --group-id sites

      • Update redirectURI at aadConfig.ts and rebuild frontend app

        • Update redirectURI with following FQDN, which is webapp endpoint.

          az webapp config hostname list -g <Resource Group Name> --webapp-name <WebApp Name> -o json | jq '.[0].name'

        • Rebuild frontend

          cd app/frontend
npm run build

      • Before deployed webapp, you must change the environment variable with application settings of Azure App Service.

        az webapp config appsettings set --name <Web App Name> -g <Resource Group Name> --settings SCM_DO_BUILD_DURING_DEPLOYMENT="true"

      • Deploy demo app to WebApp

        cd app/backend
zip -r deploy.zip .
az webapp deploy -g <Resource Group Name> -n <Webapp Name> --src-path deploy.zip --type zip

      • After deployed webapp, you must change the environment variables with application settings of Azure App Service.

        az webapp config appsettings set --name <Web App Name> -g <Resource Group Name> --settings OPENAI_API_KEY=<KEY> AZURE_OPENAI_CHATGPT_DEPLOYMENT=<Deployment Model Name> AZURE_OPENAI_SERVICE=<OpenAI Service Name>

6. Configuration ⚙️

Collect application logs with Azure Log Analytics

  • Example of Log collection

    • Deploy Azure Log Analytics workspace
    export APP_SERIVCE=<your app service name>
export LOCATION=<azure datacenter region - eastus, japaneast, etc...>
export RESOURCE_GROUP=<your resource group name>
export WORKSPACE=<your log analytics workspace name>
export DIAGSETTINNG_NAME=<your diagnistics setting name (arbitary)>

az monitor log-analytics workspace create --name $WORKSPACE  --resource-group $RESOURCE_GROUP --location $LOCATION
    • Enable diagnostics setting
    export RESOURCE_ID=`az webapp show -g $RESOURCE_GROUP -n $APP_SERIVCE --query id --output tsv | tr -d '\r'`
export WORKSPACE_ID=`az monitor log-analytics workspace show -g $RESOURCE_GROUP --workspace-name $WORKSPACE --query id --output tsv | tr -d '\r'`

az monitor diagnostic-settings create \
  --resource $RESOURCE_ID \
  --workspace $WORKSPACE_ID \
-n $DIAGSETTINNG_NAME \
--logs '[{"category": "AppServiceAppLogs", "enabled": true},{"category": "AppServicePlatformLogs", "enabled": true},{"category": "AppServiceConsoleLogs", "enabled": true},{"category": "AppServiceAuditLogs", "enabled": true},{"category": "AppServiceHTTPLogs", "enabled": true}]'

(Optional) Store prompt log data to Azure Cosmos DB

The logging chat on Azure Cosmos DB section explains in detail on how chat messages can be logged into Azure Cosmos DB and used in deriving insights further downstream.

🙋🏾‍♂️Question and Feedback

You can ask question and feedback about this repo on GitHub Issues.

📚 Resources

🤝 Contributing

We are welcome your contribution from customers and internal Microsoft employees. Please see CONTRIBUTING. We appreciate all contributors to make this repo thrive!