Whether we're attending general staff meetings or getting together with classmates to work on a group project, no one ever wants to be the note taker. You can't pay as much attention to the meeting since you have to concentrate on taking notes, and when meetings get long, it's often difficult to maintain the attention required to take diligent notes.
We wanted a way to stay more focused on the meeting rather than the memo. What if we could utilize the growing smart speaker market in conjunction with NLP algorithms to create comprehensive notes with code?
Scribblr takes notes for you! Not only that, but it automatically sends an e-blast to meeting attendees at the end and adds discussed deadlines and upcoming meetings to your calendar.
Just tell Alexa to start the meeting. Carry out the meeting as normal and, when you're done, tell Alexa the meeting is over. She will immediately begin to create a summary of your meeting, making note of the most important discussion points:
- Approaching deadlines and due dates
- Newly scheduled meetings
- The most important decisions that were made
- A short paragraph summary of the overall meeting
Additionally, anything discussed during the meeting associated with a date will be summarized into a calendar event:
- Date and time of the event
- Title of the event/task
- Important topics associated with the event/task
When the meeting is over, those who participated in the meeting will receive an email with the automated Alexa notes and all calendar events will be added to the official company calendar. Why take meeting notes when Alexa can do it for you?
The hack begins with an Alexa skill. We created a custom Alexa skill that allows the user to start and stop the meeting without skipping a beat. No more asking who is willing to take notes or hoping that the note-taker can keep up with the fast-pace -- just tell Alexa to start the meeting and carry on as normal.
The meeting is then assigned a unique access code that is transmitted to our server via an AWS Lambda Function which initiates the audio recording. Upon completion of a meeting, Alexa makes a request to the server to transcribe the text using the IBM Watson Speech to Text API.
But at the core of Scribblr are its Natural Language Processing (NLP) algorithms:
- The final transcript is first preprocessed, involving tokenization, stemming, and automated punctuation. Automated punctuation is accomplished using supervised machine learning, entailing a recurrent neural network model trained on over 40 million words.
- The Transcript Analyzer then integrates with the IBM Watson Natural Language Understanding API to detect keywords, topics, and concepts in order to determine the overarching theme of a meeting. We analyze the connections between these three categories to determine the most important topics discussed during the meeting which is later added to the email summary.
- We also isolate dates and times to be added to the calendar. When a date or time is isolated, the NLP algorithms search surrounding text to determine an appropriate title as well as key points. Even keywords such as "today", "tomorrow", and "noon" will be identified and appropriately extracted.
- Action items are isolated by searching for keywords in the transcript and these action items are processed by performing POS tagging, facilitated by a trained machine-learning module, ultimately being appended to the final meeting summary of the most important points discussed.