Click image below to watch demo video:
Currently, many classes, meetings, talks, and academic conferences (CVPR and PLDI this summer are two I'm aware of) around the world are being conducted online via Zoom due to the COVID-19 pandemic. Previously, people around the world would have had to travel for many of these events, but they now have access to these resources due to their virtual nature. A major barrier, however, is language. For instance, a person from the U.S. may have always wanted to attend an annual French conference. Yet, even though it's now conducted on Zoom, that person still cannot attend because he/she does not understand French. This is where Tranzlate comes in. The user interface is minimalistic, with a Zoom meeting screen on top and the translated text on the bottom. It is also easy to use: simply enter your meeting ID, select the speaker's language and the language you'd like to translate to, and you are all set.
This app uses the IBM Watson Speech to Text API and the Google Translate API, so you would need these appropriate API keys. Optionally, you may also install a virtual audio device software.
IBM Cloud
- Sign up for an IBM Cloud account
- Download the IBM Cloud CLI
- Create an instance of the Speech to Text service and get your credentials:
- Go to the Speech to Text page in the IBM Cloud Catalog
- Log in to your IBM Cloud account
- Click Create
- Click Show to view the service credentials
- Copy the
apikey
andurl
values
Google Cloud
- Sign up for a Google Cloud account
- Create a project via the Google Cloud console
- Enable the Translation API in your project and create an API key
Virtual Audio Device (optional, see "Using the application" below)
- Windows and Mac OS: VB-Cable
- Linux: PulseAudio Volume Control and Audacity
-
In the project directory, copy the .env.example file and create a file called .env
cp .env.example .env
-
Open the .env file and add the service credentials that you obtained in the previous step
Example .env file:
SPEECH_TO_TEXT_IAM_APIKEY=X4rbi8vwZmKpXfowaS3GAsA7vdy17Qh7km5D6EzKLHL2 SPEECH_TO_TEXT_URL=https://gateway-wdc.watsonplatform.net/speech-to-text/api GOOGLE_TRANSLATE_APIKEY=X4rbi8vwZmKpXfowaS3GAsA7vdy17Qh7km5D6EzKLHL2
-
Install the dependencies
npm install
-
Run the application
npm start
-
View the application in a browser at
localhost:3000
Enter your Zoom meeting ID upon first loading the website. You will then see your Zoom meeting window with a text box below, where the text translation will be displayed. Below the text box, select the speaker's language on the left and select the language you want to translate to on the right. When you want to start translating, click the "Start Transcribing / Translating" button and voilà !
Important: As Zoom's SDK does not provide the capability to obtain the meeting's live audio stream, this app transcribes and translates the meeting based on your microphone recording; thus, you need some way of playing the meeting audio through your microphone. The easiest way to accomplish this without any external software would be to connect your audio output to an external speaker, which your microphone can then record. Otherwise, you would need to follow some simple additional steps. If you are on Windows or Mac OS, use VB-Cable; if you are on Linux, follow these instructions.
This project uses IBM's Speech to Text demo as starter code. All modifications are clearly denoted Modified or Created at the top of relevant files, in compliance with the Apache-2.0 License.
Thank you to the staff of HackThis for organizing this great event; I had a lot of fun working on this project!