MSU-AI/SignLanguageTranslator

Sign recognition umbrella issue

Opened this issue · 0 comments

See da4a0f0 for latest code changes on methodology. Current approach imitates this blog, repo link, which stores signs as a list of frames, where each frame is a list of all angles between every landmark on the hand. We then compute similarity between input video and a database of reference signs. Dynamic Time Warping allows for frames at different times in a video to be compared so that speed of signing is irrelevant.

Our next steps are to:

  • rewrite scraper to download ASL (as opposed to french sign language) videos as reference set (#17)
  • figure out how to classify signs continuously
  • figure out how to classify signs more efficiently

If the last two are impossible with the DWT method, we can always switch to training a RNN or CNN for the task.

‼️Create PRs to merge functional code into the main repo. Only functioning code should be merged, but you should always push your latest developments to your own repo so that everyone else can view them and contribute.