This is not an officially supported Google product
-
You have an existing Dialogflow CX Agent/Bot in one language and would like to quickly have this same instance support secondary language/s
-
While Dialogflow CX can quickly duplicate your conversation flow and design to a supported secondary language, you have to provide the equivalent training phrases, entity type synonyms and bot responses for the secondary language/s. Thus, adding a new language to a fully developed agent/bot can take time, specifically in these areas:
- Intents training phrases
- Entity Types synonyms
- Bot responses in:
- Flows
- Pages
- Transition Routes
- Event Handlers and
- Transition Route Groups
-
You would like to automate and leverage Google Cloud Translate API to bulk translate from your default language to the supported secondary language/s to jumpstart the process. This will reduce the initial load of linguists/analysts in language translation and allow them to focus on reviewing and correcting the translated output.
The Jupyter Notebooks in this repository uses the following APIs:
- Dialogflow CX v3beta1 API
- Google Sheets (v4) API
- Google Cloud Translate v3beta1 API
The Jupyter Notebooks (Python) performs the following:
- Query your Dialogflow CX Agent/Bot
- Populate your Google Sheets with values that require translation from the default language
- Use Google Cloud Translate for bulk translation and output to Google Sheets
- Update your Dialogflow CX Agent/Bot with translations from Google Sheets
Each sequential step above is within your control as you execute different cells within the Jupyter Notebook.
Assumption: You have a working Dialogflow CX Agent/Bot.
Implication: You have a working:
- Google Cloud Project with Billing enabled
- Google Account with Owner/Editor access
- https://sheets.google.com OR enter sheets.new in your browser
- Suggested naming convention (optional - for human readability): "CX Bot - [Name] - Multilingual Config"
- Note the Sheets Link/URL. Example:
https://docs.google.com/spreadsheets/d/<Your Sheets ID HERE>/edit#gid=0- you will need to provide this in the Jupyter Notebook later.
-
In your CX Agent instance, add the secondary language/s in the [Agent Settings | Languages] section. This will enable the code to determine which languages to translate from your default language.
-
Note your CX Agent Link (see screenshot below on how to get it)
- you will need to provide this in the Jupyter Notebook later.
- you will need to provide this in the Jupyter Notebook later.
- Enable Google Cloud Translate API in your Google Cloud Console
- Menu - APIs and services - Library OR https://console.cloud.google.com/apis/library
- Search for "Cloud Translate" and enable the Cloud Translation API
- Note the Google Cloud Project ID where you have enabled Cloud Translation API
- The easiest path is to enable the Cloud Translation API in the same Google Cloud Project as your Dialogflow Agent
- It is not mandatory as you may have another Project with Translation API enabled and billing setup
- you will need to provide the GCP Project ID for the enabled Cloud Translation API in the Jupyter Notebook later.
- Basic Jupyter Notebook install - see https://jupyter.org/install
- Additional packages to install (
pip or pip3 install --upgrade):- Google Sheets:
google-api-python-client google-auth-httplib2 google-auth-oauthlib - Dialogflow CX:
google-cloud-dialogflow-cx - Google Cloud Translate:
google-cloud-translate - Pandas:
pandas - BeautifulSoup:
beautifulsoup4
- Google Sheets:
- From Jupyter Notebook, open CX-Bot-Translate__MAIN.ipynb and follow along, entering the values noted above in steps 1 through 3.
If you are familiar with Jupyter Notebooks, Python and Google Cloud Platform, you should be good to go from this point. We would still highlight reading the section below on Google Cloud Authentication & Authorization options. As Jupyter Notebooks can be executed from a variety of environments, from local compute to online platforms, your mileage may vary as you balance ease of deployment and potential security exposure.
For a more detailed walkthrough, from setup to translating your first Dialogflow CX bot, all actionable from your web browser with resources deployed in Google Cloud Platform, please see the longer Setup & Demo Walkthrough section below.
- Accessing Google Cloud APIs and Services requires Authentication and Authorization
- It is very common to see the use of Services Accounts and Service Account Keys (in JSON files) for programs to act on your behalf
- The code in this repository is flexible enough to support Service Accounts and Service Account Keys but...
-
before you go ahead and create Service Accounts and Keys, consider this excellent blog Choose the best way to use and authenticate service accounts on Google Cloud
-
For Google Cloud services (eg Dialogflow, Cloud Translate etc), consider downloading Google Cloud SDK. With Google Cloud SDK, you can setup OAuth locally with your credentials without the use of Service Accounts. Running the Jupyter Notebooks will automatically inherit the OAuth already setup on the system via Google Cloud SDK.
-
For Google Sheets, consider creating an OAuth credentials instead of Service Accounts. Details can be found:
-
This walkthrough focuses on ease of getting started and will use resources in Google Cloud Platform, which means you will only need a laptop/PC/Mac with a web browser to get up and running.
There are 9 steps in this walkthrough.
Steps 1 through 7 are for setting up the Demo. This is a "soup to nuts" walkthrough.
In Step 8, we walkthrough the sequential execution of the Notebook, with language translation and updates to the design-side of Dialogflow CX Healthcare Bot.
In Step 9, we illustrate the testing of the Dialogflow CX Healthcare bot.
- Create a Google Cloud Project
- Create a Dialogflow CX agent from pre-built agent template
- Enable Cloud Translate API
- Enable Google Sheets API
- Create a Vertex Workbench
- Clone this repo into the Notebook
- One-time Setup in the
CX-Bot-Translate__MAIN.ipynbnotebook - Running
CX-Bot-Translate__MAIN.ipynbnotebook - Testing the Healthcare Agent in the new languages
-
Create a new Google Cloud Project and for free* via https://cloud.google.com/free with $300 in free credits (offer as at the time of this writing)
free* - eligibility details at https://cloud.google.com/free/docs/gcp-free-tier/#free-trial
-
Name your project as desired. More importantly, note the project's ID. This is a unique identifier globally for your GCP project.
It is important to ensure you are in the right GCP project when executing your setup via the Cloud Console. You can verify this by looking at the top-left of the Cloud Console (screenshot below)
-
From your Cloud Console, navigate to Billing and ensure you've got that sorted (credit card and/or free credits). For more details on billing, see https://cloud.google.com/billing/docs
-
Open a new browser tab and navigate to https://dialogflow.cloud.google.com/cx
-
Select the right GCP project for Dialogflow CX (see screenshot below)
-
As this is a brand new GCP project, you will need to enable the Dialogflow API before you proceed.
-
In the Dialogflow CX Console, click Use pre-built agents.
-
Select the Healthcare pre-built agent template and click Import as agent.
-
Click Create, accepting the defaults
-
Interact with this pre-built agent in English via Test Agent (top-right). A sample of text chat below.
-
Add 2 new secondary languages to this agent (Simplified Chinese and Spanish) via Agent Settings. Remember to click Save.
-
Switch to a new Language (Simplified Chinese) and try a chat.
Note the conversation flow and structure of the Healtcare agent is available in the new language. However, the agent lacks the language content (training phrases, responses etc) to be useful. That is what this project will fix... let's get cracking!
-
In the GCP Cloud Console, navigate to APIs and services
Click + ENABLE APIS AND SERVICES
-
Enter "cloud translate" in the search. Select and Enable the Cloud Translation API
-
Repeating steps above: Search for "Google Sheets" , Select and Enable the Google Sheets API
-
Confirm from the APIs and services Dashboard, the following APIs have been enabled in your GCP Project
- Dialogflow API
- Cloud Translation API
- Google Sheets API
-
In the GCP Cloud Console, navigate to Vertex AI and to the Vertex Workbench
As this is the first time accessing Vertex Workbench in this GCP project, we will need to Enable the Notebooks API.
-
Create a new User-Managed Notebook
Click on + NEW NOTEBOOK and Select Python 3 from the drop-down list
Name your Notebook and select the region (us-central1)*.
Most importantly, click on the Pencil icon to edit the Notebook properties. We are going to change the Machine Type as the default (4 vCPUs & 15GB RAM) is a bit of an overkill for our purposes.
us-central1* - we select this region as it is the same region as our Dialogflow CX bot. You may use any other region. For our purpose of design-time API calls to Dialogflow, there is little to no network performance penalty.
Scroll down to Machine configuration and select a more sensible Machine type of n1-standard-1. The pricing estimate is the upperbound per month with the assumption that you are never shutting down your Notebook VM.
Scroll to the bottom of the page and click Create
-
Wait for your new Notebook to be provisioned and the pre-installed JupyterLab is ready.
Click on OPEN JUPYTERLAB to open a new tab with the Notebook environment ready to go.
-
Clone this repo into the Notebook
Enter the HTTPS field (above) by copying it from GitHub Code button (as shown below)
-
Open the
CX-Bot-Translate__MAIN.ipynbNotebook
Scroll down the CX-Bot-Translate__MAIN.ipynb notebook to find the ipynb env: section
The lines %pip install <packages> are meant to be run just once to setup your JupyterLab environment
To run this cell, select it (blue border on the left) and hit Shift-Enter on your keyboard
Screenshot above shows the code running and installing the additional Python packages
Once this has been completed, please edit the cell and comment out the %pip install <packages> lines with a Hash (#) as shown below.
This will skip these %pip install <packages> lines when we run multiple cells sequentially later.
Open a new tab on your browser and surf to "sheets.new", this will automatically create a new and empty Google Sheets.
Name your Google Sheets "CX Bot - Healthcare - Multilingual" or as desired. The more important part is to note the actual URL for this Google Sheets (in red box). We will access this Google Sheets programmatically in our JupyterLab notebook.
Navigate back to the CX-Bot-Translate__MAIN.ipynb notebook and locate the cell PROVIDE Required Information in the cell below:. This should be the cell just above ipynb env: section.
Add your new Google Sheets URL into the Google_Sheets_URL variable
From the Dialogflow CX Console, navigate to the main project area by selecting the Agent drop-down on the top of the screen and select View all agents
At the project level view in Dialogflow CX Console, click the 3-dots next to your Healthcare agent and select Agent Link to copy the link of the Healthcare agent.
Navigate back to the CX-Bot-Translate__MAIN.ipynb notebook and locate the cell PROVIDE Required Information in the cell below:.
Add your Dialogflow CX Healthcare Agent Link into the CX_Agent_URL variable
7.4. Create Service Account Key, Update CX-Bot-Translate__MAIN.ipynb notebook and Share it with Google Sheets
TL;DR, we need to provide a service account key (JSON) so that our JupyterLab Notebook can have edit access to Google Sheets. The GSheets_JSON variable in the CX-Bot-Translate__MAIN.ipynb notebook should point to that JSON key file.
In the GCP Cloud Vertex Workbench Console, if we hover our mouse pointer over "Service account" under the Permission heading for the "cx-bot-language-translation" Vertex Workbench VM, we will see the generated service account in the format of an email (eg XYZABC-compute@developer.gserviceaccount.com). Take note of your service account name.
Navigate to the IAM & Admin - Service Accounts page of the GCP Cloud Console.
Locate your service account and click the 3-dots menu on the right and select Manage keys
Click on ADD KEY and select Create new key
Select JSON for Key type and click Create. The service account key in a JSON file will be downloaded onto your system. Be mindful of where you store this file. You will need to upload this file into the Vertex Workbench in the next step.
Navigate back to your Vertex Workbench JupyterLab notebook and click the Upload File icon
Select and Upload the service account key JSON file that you downloaded previously. Confirm the file has been uploaded to the same directory as your CX-Bot-Translate__MAIN.ipynb notebook
In the CX-Bot-Translate__MAIN.ipynb notebook, update the GSheets_JSON variable with the name of the JSON key file.
Finally, in your Google Sheets, grant Edit access with your service account (eg XYZABC-compute@developer.gserviceaccount.com) via the Share button (top-right).
In the CX-Bot-Translate__MAIN.ipynb notebook, update the Cloud_Translate_Project_ID variable with the name of the GCP Project ID. This is straight-forward because we have enabled Cloud Translation API in the same GCP Project as Dialogflow CX.
In the CX-Bot-Translate__MAIN.ipynb notebook, scroll down to the section named "1. Run / Execute below:" and select it.
In the JupyterLab menu above, select Run -> Run All Above Selected Cell
Verify that your Workbench setup has successfully connected with Google Sheets and Dialogflow CX Healtcare Agent.
With the previous Step 7 (One-time setup) completed, we can proceed with running the CX-Bot-Translate__MAIN.ipynb notebook that will orchestrate between Google Sheets, Dialogflow CX and Cloud Translation, to enable the existing Healthcare bot to now converse in Spanish and Simplified Chinese. We are orchestrating AI to enable AI.
Use the Shift key and Up/Down arrow keys to multi-select cells. With the two cells selected, use Shift-Enter on your keyboard to execute the two cells sequentially.
Upon successful execution as shown in the example above, switch to your Google Sheets to inspect what has been added.
The Init Sheets cells in CX-Bot-Translate__MAIN.ipynb notebook has added the following Sheets:
- CX_Lang_REF - imported the Dialogflow CX Language Reference table
- Training_Phrases
- Parameters
- Entities
- Flows
- Pages
- Route_Groups
We also detected the two supported languages Spanish and Simplified Chinese in the Healthcare bot and added them in the respective Sheets with both the Dialogflow CX and Cloud Translation language_code respectively.
Next, we will inspect the configuration of the Dialogflow CX Healthcare bot, selectively extract parts which require translation from the default English language and populate into the respective Google Sheets.
Scroll down to Run the [Agent] Notebook and Update Sheets section.
Use the Shift key and Up/Down arrow keys to multi-select cells. With the two cells selected, use Shift-Enter on your keyboard to execute the two cells sequentially.
The following in Dialogflow CX Bot are inspected and written to the respective sheets in Google Sheets:
| Dialogflow CX Component | Google Sheets Name |
|---|---|
| Intent Training Phrases | Training_Phrases |
| Intent Parameters | Parameters |
| Entity Types | Entities |
| Flows | Flows |
| Pages | Pages |
| Route Groups | Route_Groups |
Upon successful execution as shown in the snippet above, switch to your Google Sheets to inspect the updated content. Below is just a snippet of the Route_Groups sheets.
Next, we will use Cloud Translation API to translate the English language parts of the Dialogflow CX Healthcare bot to the target languages of Spanish and Simplified Chinese. There are features of Dialogflow CX that require some nuances in translation logic and this has been accounted for in our code. A simple example is to ignore session parameters in Dialogflow CX denoted with \$ and followed by the session parameter name (eg. $session.params.email).
Scroll down to Run the [Translation] Notebook and Translate Agent in Sheets section.
Use the Shift key and Up/Down arrow keys to multi-select cells. With the two cells selected, use Shift-Enter on your keyboard to execute the two cells sequentially.
Upon successful execution, switch to your Google Sheets to inspect the translated content from English to Spanish and Simplified Chinese. Below is a snippet of the Flow sheets.
We now have a grid view of the different Dialogflow CX components where language has been translated. We can use this for quality assurance and review work by linguists, analysts and conversation architects.
The final step is to take these translations and update Dialogflow CX Healthcare bot.
Scroll down to Update Translations in Sheets to Agent section.
Select the only cell and use Shift-Enter on your keyboard to execute the cell. This will probably take a while to complete and you can see the progress via the verbose informational messages below the cell.
Screenshots below to illustrate a sample of components in Dialogflow CX Healthcare Bot that were updated:
| Intent: "healthcare.schedule_appointment" |
|---|
 |
| Default Language: English |
|---|
 |
| Additional Language: Simplified Chinese |
|---|
 |
| Entity Type: "@coverage-category" |
|---|
 |
| Default Language: English |
|---|
 |
| Additional Language: Simplified Chinese |
|---|
 |
| Page: "Condition for Coverage Category" at Flow: "Find a Doctor" |
|---|
| Default Language: English |
| ------------------------- |
 |
| --------------------------------------------------------------------------------------------- |
| Additional Language: Simplified Chinese |
| --------------------------------------- |
 |
Testing the Healtcare Bot in the new language:
The following was before we applied our translation and we tried to converse with the Healthcare bot in Simplified Chinese
Now, after applying Cloud Translation to Dialogflow CX components:
We also tested it in Spanish:
This is not a supported product and the intention of this tool is to accelerate the translation of existing Dialogflow CX agents/bots from one language to another. Machine Translation will never be perfect; we leverage it to undertake the initial load of translation and have linguist and experts curate and review these translations before we apply them back into Dialogflow CX. Even then, there might be further refinement by conversational architects as human communication across language and cultures are never a 1:1 mapping.
Now that you have setup your Vertex Workbench with Dialogflow-CX-Bot-Language-Translation capability, feel free to try it out on other Dialogflow CX bots and be productive in advancing Conversational AI. Cheers!
PS: Don't forget to Stop the Vertex Workbench Notebook when you are done. Otherwise, the underlying VM will continue to run and you'll be charged for utilization.