Awesome AI Sign Language Papers & Popularization of Professional Knowledge [UPDATING]

This repository is all you need to open the door about AI sign languages!

Keywords: Sign Language, Sign Language Translation (SLT), Sign Language Recognition (SLR), Sign Language Production (SLP), Sign Language Retrieval, Sign Language Linguistics

What's New

`Update 2023-12-23:` More concise layout & updated contents & Add the introduction of sign language knowledge.

Table of contents

Introduction
Popularization of Professional Sign Language Knowledge
Survey/Review Papers
AI Sign Language in Timeline [Papers]
- 2024
- 2023
- 2022
- 2021
- 2020
- 2019
- 2018 -> True SLT began in 2018
- 2017
- Earlier
AI Sign Language in Well-Known Institutions [Papers]
- XMU (厦门大学) SL -> active
- USTC (**科学技术大学) SL
- ZJU (浙江大学) SL -> active
- THU (清华大学) SL
- Germany-UK (英德) SL
Sign Language Datasets
AI Sign Language Related Fields

Introduction

This repository is for those interested in the field of AI sign language (SL). The collected papers have been categorized according to different criteria (btime, type, institution, etc.) for ease of searching.

If useful, please Star this repo, we are keeping it updated.

NOTE: There is overlap between the different categories. Please feel free to submit your Pull Requests for any updates.

Collaborators & Contributors (welcome to join us!):
- Jiangbin Zheng, Westlake University & Zhejiang University, China
- Yidong Chen & Pei Yu & Rui Zhao, Xiamen University, China
- Haodong Zhang, Zhejiang University, China
- Junfu Pu, University Of Science and Technology of China

Popularization of Professional Sign Language Knowledge

- Why AI Sign Language Research?

Unfolding Details

Figure 1. Communication gap between the hearing and the deaf.

Sign languages are the primary language of the deaf community. However, most hearing people find it difficult to understand sign languages. With the development of AI, researchers are trying to help people understand sign languages using AI techniques that are designed to convert sign languages into spoken languages in textual form.

- Do you know the differences between SLT and SLR?

Unfolding Details

Figure 2. SLR (Sign Language Recognition) vs. SLT (Sign Language Translation).

At the very beggining, I wanna explain the difference between SLT and SLR, as shown in Fig. 2. I'm sure this is very important for most of you!

In early efforts, researchers explored this problem (sign languages -> text-form glosses) as a recognition problem (i.e., SLR), which converts sign languages to glosses word by word according to the sign languages sequentially. Although glosses are in textual form, they do not provide meaningful interpretations of what a signer is saying because sign languages and glosses have their own specific linguistic rules, which are quite different from spoken languages.

As a result, researchers find it terrible to ignore the linguistic properties of sign language. Contrary to SLR, sign language translaton (SLT) systems aim to translate sign language videos into spoken sentences directly.

As far as AI technology is concerned, SLR belongs to the field of Computer Vision + Text Recognition, while SLT belongs to the field of Computer Vision + Text Translation.

- More details you must know about SLR tasks?

Unfolding Details

By default, we refer to the current Continuous SLR as SLR because this type of SLR is the mainstream of the deep learning era.

In fact, it is important to note that SLR tasks are generally divided into three categories: finger-spelling recognition, isolated word recognition and continuous sign language recognition. In the early days (before about 2018), SLR works mainly focused on lexical-level tasks, such as finger-spelling recognition and isolated word recognition. Nowadays, the more practical continuous SLR has become mainstream sign language research

For more categorization and details, we recommend you read this survey/review paper for a detailed look at AI Sign history.

Dataset	Language	Classes	Samples	Data Type	Language Level
ASLG-PC12 paper	American	-	87,709	GLOSS&Sentences	isolated
CSL Dataset I	Chinese	500	125,000	Videos&Depth from Kinect	isolated
CSL Dataset II	Chinese	100	25,000	Videos&Depth from Kinect	continuous
RWTH-PHOENIX-Weather 2014	German	1,081	6,841	Videos	continuous
RWTH-PHOENIX-Weather 2014 T	German	1,066	8,257	Videos	continuous
ASLLVD	American	3,300	9,800	Videos(multiple angles)	isolated
ASLLVD-Skeleton	American	3,300	9,800	Skeleton	isolated
SIGNUM	German	450	33,210	Videos	continuous
DGS Kinect 40	German	40	3,000	Videos(multiple angles)	isolated
DEVISIGN-G	Chinese	36	432	Videos	isolated
DEVISIGN-D	Chinese	500	6,000	Videos	isolated
DEVISIGN-L	Chinese	2000	24,000	Videos	isolated
LSA64	Argentinian	64	3,200	Videos	isolated
GSL isol.	Greek	310	40,785	Videos&Depth from RealSense	isolated
GSL SD	Greek	310	10,290	Videos&Depth from RealSense	continuous
GSL SI	Greek	310	10,290	Videos&Depth from RealSense	continuous
IIITA -ROBITA	Indian	23	605	Videos	isolated
PSL Kinect	Polish	30	300	Videos&Depth from Kinect	isolated
PSL ToF	Polish	84	1,680	Videos&Depth from ToF camera	isolated
BUHMAP-DB	Turkish	8	440	Videos	isolated
LSE-Sign	Spanish	2,400	2,400	Videos	isolated
Purdue RVL-SLLL	American	39	546	Videos	isolated
RWTH-BOSTON-50	American	50	483	Videos(multiple angles)	isolated
RWTH-BOSTON-104	American	104	201	Videos(multiple angles)	continuous
RWTH-BOSTON-400	American	400	843	Videos	continuous
WLASL	American	2,000	21,083	Videos	isolated

ethio-artifical/SL_Papers

Awesome AI Sign Language Papers & Popularization of Professional Knowledge [UPDATING]

Introduction

Popularization of Professional Sign Language Knowledge

- Why AI Sign Language Research?

- Do you know the differences between SLT and SLR?

- More details you must know about SLR tasks?

SURVEY/REVIEW PAPERS

AI Sign Language in Timeline [Papers]

2024

2023

2022

2021

2020

2019

2018

2017

Earlier

AI Sign Language in Well-Known Institutions [Papers]

XMU for AI Sign Language

USTC for AI Sign Language

ZJU for AI Sign Language

THU for AI Sign Language

Germany-UK for AI Sign Language

Datasets

AI Sign Language Related Fields