/AI-for-Sign-language

Apache License 2.0Apache-2.0

AI for Sign language

AI for Sign language is a curated list of Artificial Intelligence for Sign language papers, articles, tutorials, slides and projects. Star this repository, and then you can keep abreast of the latest developments of this booming research field. Thanks to all the people who made contributions to this project. Join us and you are welcome to be a contributor.

What is AI for Sign language?

AI for Sign language provides papers, methods and processes to make machine translate of Sign language available for Deaf people and researcher, to improve efficiency of intelligence research and to accelerate research on Sign language and gesture.

Paper

Collection

  • 2020 | 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives | LREC 2020 | PDF
  • 2020 | 14. International Conference on Sign Language and Acquisition | ICSLA 2020 | link
  • 2020 | 1st International Workshop on Automatic Translation for Sign and Spoken Languages | I:\Cloud\Yandex\YandexDisk\Library\0_Авторы\КлезовичАнна\2021.mtsummit-at4ssl.pdf

Article

  • 2019 | Arxiv | Neural Sign Language Translation based on Human Keypoint Estimation | Sang-Ki Ko, Chang Jo Kim, Hyedong Jung, Choongsang Cho arxiv.org
  • 2018 | ScienceDirect | Selfie video based continuous Indian sign language recognition system | G. AnanthRaoa P.V.V.Kishore | link
  • 2018 | frontier | Visual Iconicity Across Sign Languages: Large-Scale Automated Video Analysis of Iconic Articulators and Locations | Robert Östling, Carl Börstell, Servane Courtaux | link
  • 2016 | Journal of Intelligent Systems | Movement Epenthesis Detection for Continuous Sign Language Recognition | Ananya Choudhury , Anjan Kumar Talukdar, Manas Kamal Bhuyan, Kandarpa Kumar Sarma link PDF
  • 2018 | | Sign Language Production using Neural Machine Translation and Generative Adversarial Networks | Stephanie Stoll, Necati Cihan Camgoz, Simon Hadfield, Richard Bowden | pdf GitHub
  • 2017 | | SubUNets: End-to-end Hand Shape and Continuous Sign Language Recognition | Necati Cihan Camgoz - University of Surrey, Guildford, UK; Simon Hadfield - University of Surrey, Guildford, UK; Oscar Koller - RWTH Aachen University, Germany; Richard Bowden - University of Surrey, Guildford, UK | GitHub
  • 2020 | Arxiv | Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation | Necati Cihan Camgoz(1), Oscar Koller(2), Simon Hadfield(1), Richard Bowden(1) - (1)CVSSP, University of Surrey, Guildford, UK, (2)Microsoft, Munich, Germany| PDF GitHub
  • 2020 | Arxiv | Real-Time Sign Language Detection using Human Pose Estimation | Amit Moryossef, Ioannis Tsochantaridis, Roee Aharoni, Sarah Ebling, Srini Narayanan | PDF | YouTube | Google | GitHub | Google GitHub | Sign Language Processing GitHub
  • Predict & Cluster GitHub
  • Video-to-HamNoSys Automated Annotation System aclweb.org GitHub
  • Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition www-i6.informatik.rwth-aachen.de/ YouTube|YouTube
  • 2018|Sign Language Production using Neural Machine Translation and Generative Adversarial Networks|Stephanie Stoll, Necati Cihan Camgoz, Simon Hadfield, Richard Bowden|cihancamgoz.com|github
  • 2021|Fingerspelling Detection in American Sign Language|Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu|arxiv|github
  • 2022|Classification of Phonological Parameters in Sign Languages|arxiv.org
  • 2020 | Phonologically-meaningful Subunits for Deep Learning-based Sign Language Recognition | Mark Borg, Kenneth P. Camilleri pdf from Pedro Dal Bianco
  • 2022 | LSA-T: The first continuous Argentinian Sign Language dataset for Sign Language Translation | Pedro Dal Bianco, Gastón Ríos, Franco Ronchetti, Facundo Quiroga, Oscar Stanchi, Waldo Hasperué, Alejandro Rosete | arxiv.org
  • 2022 | Keypoint based Sign Language Translation without Glosses | Youngmin Kim, Minji Kwak, Dain Lee, Yeongeun Kim, Hyeongboo Baek | arxiv.org from from Pedro Dal Bianco

Book

Projects

  • 2019 | Spatial Temporal Graph Convolutional Networks for Sign Language (ST-GCN-SL) Recognition (Not Tested) | GitHub Arxiv Arxiv
  • 2018 | Sign Language gesture recognition from video sequences (Tested) | Github Springer
  • SigNN (Colab,OpenPose, MediaPipe) (Not tested) | GitHub
  • 2020 | BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues | link GitHub Arxiv
  • 2017 | sign2text | GitHub

Other aeras

Linguistic

Collection

  • 2020 | Formal and Experimental Advances in Sign Language Theory | FEAST 2020: Online | sites.google.com

Book

  • 1998 | A Prosodic Model of Sign Language Phonology Language, Speech, and Communication| Brentari, D. | private cloud

Articles

  • Many-to-English Machine Translation Tools, Data, and Pretrained Models Arxiv
  • 2019 | Исследования и разработки в области новых информационных технологий и их приложений | Межъязыковые особенности жестовых языков(на материале жестов в знаковой форме) | Мясоедова М.А., Мясоедова З.П. | PDF
  • 2018 | Modern Information Technologies and IT-Education | Жестовые нотации и их сравнительный анализ | Мясоедова М.А., Мясоедова З.П. | pdf
  • 2018 | Исследования и разработки в области новых информационных технологий и их приложений | Корпус жестов в письменной форме как инструмент для исследования особенностей их формирования(на примере русского жестового языка) | Мясоедова М.А., Мясоедова З.П. | pdf

Projects

  • Symbol Font for ASL. How can you read and write American Sign Language? github.io
  • Multi-modal Ensemble
    • Arxiv.org | Skeleton Aware Multi-modal Sign Language Recognition | Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, Yun Fu arxiv.org | github
    • Arxiv.org | Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble | Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, Yun Fu arxiv.org | github

Q&A

  • Как правильно лингвистически назвать движения рук, когда они только поднимаются от корпуса тела, чтобы показать жест и опускаются уже после показа жеста? Между несколькими жестами есть “переходные жесты” - это понятно. А вот при исполнении одного жеста.
    • Начальное - это подготовительное движение. Конечное, когда рука возвращается в исходное положение, -- нет специального термина. Но вообще можно использовать термины из лингвистики звуковых языков, обозначающие фазы артикуляции звука. Здесь, по сути, ведь то же самое: 1. Экскурсия( подготовка органов к артикуляции). 2. Выдержка(фаза артикуляции). 3. Рекурсия(возврат органов артикуляции в исходное положение).

AI

Articles

Research

  • | Arxiv | Unsupervised Learning of Video Representations using LSTMs | Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov | pdf GitHub dataset
  • | Arxiv | A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective | Yuji Roh, Geon Heo, Steven Euijong Whang, Senior Member, IEEE | pdf
  • | Arxiv | A Survey of Deep Learning for Scientific Discovery | Maithra Raghu(1,2), Eric Schmidt(1,3) - (1)Google, (2)Cornell University, (3)Schmidt Futures | pdf
  • | CBMM | Object-Oriented Deep Learning | Liao, Q, Poggio, T | MIT pdf
  • | Arxiv | Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates | Leslie N. Smith, Nicholay Topin | pdf. On the recommendation of Jeremy Howard
  • | Arxiv | Context Based Emotion Recognition using EMOTIC Dataset | Ronak Kosti, Jose M. Alvarez, Adria Recasens, Agata Lapedriza | pdf
  • | Arxiv | Structured Query-Based Image Retrieval Using Scene Graphs | Brigit Schroeder, Subarna Tripathi | pdf YouTube
  • | CVPR15 | Image Retrieval using Scene Graphs | Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David A. Shamma, Michael S. Bernstein, Li Fei-Fei | Stanford pdf GitHub GitHub from Google GitHub
  • | Arxiv | A Framework For Contrastive Self-Supervised Learning And Designing A New Approach | William Falcon, Kyunghyun Cho | Arxiv | Author is creator of PyTorch Lightning
  • | Arxiv | RAFT: Recurrent All-Pairs Field Transforms for Optical Flow | Zachary Teed, Jia Deng | Arxiv GitHub
  • | Arxiv | Synthetic Data for Deep Learning| Sergey I. Nikolenko | GitHub

Paper

  • Machine Learning: Detecting Dropped Pacifiers medium

Project

  • IPN Hand: A Video Dataset for Continuous Hand Gesture Recognition GiyHub.io
  • PrototypeML: Powerful & Intuitive Visual Neural Network Design Platform for PyTorch offical
  • DeepStack AI Server Open Source: Build and deploy AI powered applications with in-built and custom AI APIs, all offline and Self-Hosted offical|deepquestai.com | medium
  • MotionGPT: Human Motion as Foreign Language offical | Arxiv | GitHub | dataset |

Awesome

  • Awesome-AutoML-Papers | GitHub
  • Awesome-AutoML | GitHub
  • Awesome-data-labeling | GitHub
  • Hand Pose Estimation | GitHub
  • Awesome production machine learning | GitHub
  • Awesome jupyter GitHub
  • Awesome Mobile Machine Learning GitHub
  • Awesome production machine learning GitHub
  • Awesome AI Guidelines GitHub
  • Awesome gesture recognition GitHub
  • Awesome Spark GitHub
  • Awesome Dataset Tools github
  • Awesome Sign Language Rcognition and Sign Language Translation github
  • Awesome open data-centric AI github

Demo

  • Try out deep learning models online on Colab with a single click | GitHub

Course

  • Full Stack Deep Learning | link

Book

  • Data Visualization: A Practical Introduction | Kieran Healy

Roadmap

Labeling data and active learing machine learning

  • modAL Active learning | GitHub | Arxiv | мой пример Colab from Interactive labeling with Jupyter GitHub
  • Jupyanno is simple labeling data tool in Jupyter GitHub
  • Label Studio(There is about Active Learning) GitHub offical
  • pigeonXT - Quickly annotate data in Jupyter Lab GitHub TowardsDataScience
  • Human in the Loop: как сократить ресурсы на разметку данных NeuroHive

Framework

  • PyTorch Lightning official | GitHub
    • Seamlessly train hundreds of Machine Learning models on the cloud from your laptop official
    • Talks # 13: William Falcon; Stop engineering, start winning - How to Kaggle with PyTorch Lightning YouTube
  • Lightly: Lightly is a computer vision framework for self-supervised learning lightly.ai GitHub

TODO