Sign recognition umbrella issue
Opened this issue · 0 comments
See da4a0f0 for latest code changes on methodology. Current approach imitates this blog, repo link, which stores signs as a list of frames, where each frame is a list of all angles between every landmark on the hand. We then compute similarity between input video and a database of reference signs. Dynamic Time Warping allows for frames at different times in a video to be compared so that speed of signing is irrelevant.
Our next steps are to:
- rewrite scraper to download ASL (as opposed to french sign language) videos as reference set (#17)
- figure out how to classify signs continuously
- figure out how to classify signs more efficiently
If the last two are impossible with the DWT method, we can always switch to training a RNN or CNN for the task.