Digital-ease
Capstone for Epicodus C#/.NET.
This is an idea for a hands free web-developement app. Using speech-to-text technology, the user will be able to interact with the app and build/style a simple website.
Specs/Wishlist
| Sections of developement | Implemented? |
|---|---|
| API call using curl or postman | Postman - X |
| api call using RestSharp | NO WAY |
| use the Watson Nuget Packages to successfully transcribe speech to text | X |
| find a way for a user to record then save audio directly from the page | X |
| output text on page | X |
| build corpus for code-specific words/sentences | O |
| build keyword file | O |
| build keyword references | O |
| build code snippets that link to a keyword | O |
| parse returned json for transcript | X |
| split json parse into array | X |
| parse array for keywords | X |
| change view based on code-snippets | X |
| loading gif/img while user waits | X |
| TO DO | Example (if needed) |
|---|---|
| Account for different ways of structuring sentences | "add color background yellow" vs "add background yellow color" etc |
| Add ability to create a form | "create form with parameters two text fields, three number fields, one date field" |
| Restructure method calls to make the data path easier to follow | split into if statements regarding change/add |
| Add buttons to allow user to revisit a successful change | Change background color to green (as a button) |
| Persist correct statements for the user to retrieve and be able to change multiple aspects of the site at a time | change bg color AND font color |
Project Build
Download from the repo
Open Viual Studio 2015 and choose the Open Project option.
Navigate to the directory in which you downloaded the repo and open the project file.
Packages
Nuget Packages to ensure are installed:
- IBM.WatsonDeveloperCloud.SpeechToText.v1 - documentation
- Newtonsoft.Json
Watch for Reference Restoration errors, you may need to add System.Runtime.4.1.0 to the project.
Running the Server
After installation , press f5 or the play button in the toolbar to run the server and have the site appear in your default browser.
Known Bugs
Speech to text is buggy. Its almost impossible for a human to fully understand what another human says every time, so lets not assume the computer knows either. Need to account for much more variation, which will help.
Technologies Utilized
- C#
- Visual Studio
- ASP.NET MVC
- ASP.NET
- Javascript
- JQuery
- CSS
- IBM's Watson Speech-to-Text API
- Postman
- CameraMouse
- Knockout
Credits
- David Wilson - main project and idea
- see comments in Javascript files