binbinjiang/SL_Papers

Latest AI Sign Language Papers & Survey & Review

Awesome AI Sign Language Papers & Popularization of Professional Knowledge [UPDATING]

This repository is all you need to open the door about AI sign languages!

Keywords: Sign Language, Sign Language Translation (SLT), Sign Language Recognition (SLR), Sign Language Production (SLP), Sign Language Retrieval, Sign Language Linguistics

What's New

`Update 2023-12-23:` More concise layout & updated contents & Add the introduction of sign language knowledge.

Table of contents

• Introduction

• Popularization of Professional Sign Language Knowledge

  • Why AI Sign Language Research?
  • Do you know the differences between SLT and SLR?
  • More details you must know about SLR tasks?

• Survey/Review Papers

• AI Sign Language in Timeline [Papers]

  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018 -> True SLT began in 2018
  • 2017
  • Earlier

• AI Sign Language in Well-Known Institutions [Papers]

  • XMU (厦门大学) SL -> active
  • USTC (**科学技术大学) SL
  • ZJU (浙江大学) SL -> active
  • THU (清华大学) SL
  • Germany-UK (英德) SL

• Sign Language Datasets

• AI Sign Language Related Fields

Introduction

This repository is for those interested in the field of AI sign language (SL). The collected papers have been categorized according to different criteria (btime, type, institution, etc.) for ease of searching.

If useful, please Star this repo, we are keeping it updated.

NOTE: There is overlap between the different categories. Please feel free to submit your Pull Requests for any updates.

• Collaborators & Contributors (welcome to join us!):
  • Jiangbin Zheng, Westlake University & Zhejiang University, China
  • Yidong Chen & Pei Yu & Rui Zhao, Xiamen University, China
  • Haodong Zhang, Zhejiang University, China
  • Junfu Pu, University Of Science and Technology of China

Popularization of Professional Sign Language Knowledge

- Why AI Sign Language Research?

Unfolding Details

Figure 1. Communication gap between the hearing and the deaf.

Sign languages are the primary language of the deaf community. However, most hearing people find it difficult to understand sign languages. With the development of AI, researchers are trying to help people understand sign languages using AI techniques that are designed to convert sign languages into spoken languages in textual form.

- Do you know the differences between SLT and SLR?

Unfolding Details

Figure 2. SLR (Sign Language Recognition) vs. SLT (Sign Language Translation).

At the very beggining, I wanna explain the difference between SLT and SLR, as shown in Fig. 2. I'm sure this is very important for most of you!

In early efforts, researchers explored this problem (sign languages -> text-form glosses) as a recognition problem (i.e., SLR), which converts sign languages to glosses word by word according to the sign languages sequentially. Although glosses are in textual form, they do not provide meaningful interpretations of what a signer is saying because sign languages and glosses have their own specific linguistic rules, which are quite different from spoken languages.

As a result, researchers find it terrible to ignore the linguistic properties of sign language. Contrary to SLR, sign language translaton (SLT) systems aim to translate sign language videos into spoken sentences directly.

As far as AI technology is concerned, SLR belongs to the field of Computer Vision + Text Recognition, while SLT belongs to the field of Computer Vision + Text Translation.

- More details you must know about SLR tasks?

Unfolding Details

By default, we refer to the current Continuous SLR as SLR because this type of SLR is the mainstream of the deep learning era.

In fact, it is important to note that SLR tasks are generally divided into three categories: finger-spelling recognition, isolated word recognition and continuous sign language recognition. In the early days (before about 2018), SLR works mainly focused on lexical-level tasks, such as finger-spelling recognition and isolated word recognition. Nowadays, the more practical continuous SLR has become mainstream sign language research

For more categorization and details, we recommend you read this survey/review paper for a detailed look at AI Sign history.

SURVEY/REVIEW PAPERS

[Back to TOP]

• 【IEEE Access 2019 (Recommended)】Technical approaches to Chinese sign language processing: A review. Suhail Muhammad Kamal; Yidong Chen; Shaozi Li; Xiaodong Shi; Jiangbin Zheng. [paper] Tags: Survey

AI Sign Language in Timeline [Papers]

2024

[Back to TOP]

• 【AAAI 2024】Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment. Rui Zhao, Liang Zhang, Biao Fu, Cong Hu, Jinsong Su, Yidong Chen. [paper] [code] Tags: SLT
• 【ICASSP 2024】An Explicit Multi-Modal Fusion Method For Sign Language Translation. Cong Hu, Biao Fu, Pei Yu, Liang Zhang, Xiaodong Shi, Yidong Chen. [paper] code Tags: SLT

2023

[Back to TOP]

• 【CVPR 2023 (Highlight Paper)】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code] Tags: SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University
• 【NeurIPS 2023】Auslan-Daily: Australian Sign Language Translation for Daily Communication and News. Xin Shen, Shaozu Yuan, Hongwei Sheng, Heming Du, Xin Yu. [paper] [code] Tags: Dataset
• 【David Uthus, Garrett Tanzer, Manfred Georg】YouTube-ASL: A Large-Scale. * Open-Domain American Sign Language-English Parallel Corpus.* [[paper]](NeurIPS 2023) [code] Tags: https://arxiv.org/pdf/2306.15162.pdf; https://github.com/google-research/google-research/tree/master/youtube_asl
• 【NeurIPS 2023】PopSign ASL v1.0: An Isolated American Sign Language Dataset Collected via Smartphones. Thad Starner, Sean Forbes, Matthew So, David Martin, Rohit Sridhar, Gururaj Deshpande, Sam Sepah, Sahir Shahryar, Khushi Bhardwaj, Tyler Kwok, Daksh Sehgal, Saad Hassan, Bill Neubauer, Sofia Anandi Vempala, Alec Tan, Jocelyn Heath, Unnathi Utpal Kumar, Priyanka Vijayaraghavan Mosur, Tavenner M. Hall, Rajandeep Singh et al.. [paper] code Tags: Dataset
• 【CVPR 2023】Continuous Sign Language Recognition with Correlation Network. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. [paper] [code] Tags: SLR
• 【CVPR 2023】Gloss Attention for Gloss-free Sign Language Translation. Aoxiong Yin, Tianyun Zhong, Li Tang, Weike Jin, Tao Jin, Zhou Zhao. [paper] [code] Tags: SLT
• 【CVPR 2023】Distilling Cross-Temporal Contexts for Continuous Sign Language Recognition. Leming Guo, Wanli Xue, Qing Guo, Bo Liu, Kaihua Zhang, Tiantian Yuan, Shengyong Chen. [paper] code Tags: SLR
• 【CVPR 2023】Natural Language-Assisted Sign Language Recognition. Ronglai Zuo, Fangyun Wei, Brian Mak. [paper] [code] Tags: SLR
• 【CVPR 2023】CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning. Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, Wenqiang Zhang. [paper] [code] Tags: Sign Language Retrieval
• 【CVPR 2023】Ham2Pose: Animating Sign Language Notation into Pose Sequences. Rotem Shalev Arkushin, Amit Moryossef, Ohad Fried. [paper] [code] Tags: Others
• 【CVPR 2023】Sign Language Video Retrieval with Free-Form Textual Queries. Amanda Duarte,Samuel Albanie,Xavier Giro-i-Nieto ,Gul Varol. [paper] [code] Tags: Sign Language Retrieval
• 【ICCV 2023】Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining. Benjia Zhou, Zhigang Chen, Albert Clapes, Jun Wan, Yanyan Liang,Sergio Escalera, Zhen Lei, Du Zhang. [paper] [code] Tags: SLT
• 【NAACL 2023】Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation. Yong Cao, Wei Li, Xianzhi Li, Min Chen, Guangyong Chen, Long Hu, Zhengdao Li, Hwang Kai. [paper] [code] Tags: SLT
• 【ICLR 2023】SLTUNET: A Simple Unified Model for Sign Language Translation. Biao Zhang, Mathias M��ller, Rico Sennrich. [paper] [code] Tags: SLT
• 【NeurIPS 2023】ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition. Aashaka Desai,Lauren Berger,Fyodor O. Minakov,Vanessa Milan,Chinmay Singh,Kriston Pumphrey,Richard E. Ladner,Hal Daum�� III,Alex X. Lu, Naomi Caselli, Danielle Bragg. [paper] code Tags: nan
• 【IJCAI 2023】Efficient Sign Language Translation with a Curriculum-based NAR Decoder. Pei Yu1, Liang Zhang, Biao Fu1, Yidong Chen. [paper] code Tags: SLT
• 【ICASSP 2023】A Token-level Contrastive Framework for Sign Language Translation. Biao Fu, Peigen Ye, Liang Zhang, Pei Yu, Cong Hu, Yidong Chen, Xiaodong Shi. [paper] code Tags: SLT
• 【AAAI 2023】Self-Emphasizing Network for Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang liu, Wei Feng. [paper] code Tags: SLR
• 【EACL 2023】Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation. Jinhui Ye, Wenxiang Jiao, Xing Wang, Zhaopeng Tu. [paper] code Tags: Sign Gloss Translation

2022

[Back to TOP]

• 【Arxiv 2022】Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation. [paper] code Tags: SLT
• 【CVPR 2022】A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation. Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, Stephen Lin. [paper] code Tags: SLT
• 【CVPR 2022】MLSLT: Towards Multilingual Sign Language Translation. Aoxiong Yin, Zhou Zhao, Weike Jin, Meng Zhang, Xingshan Zeng, Xiaofei He. [paper] code Tags: SLT
• 【CVPR 2022】C2SLR: Consistency-Enhanced Continuous Sign Language Recognition. Ronglai Zuo, Brian Mak. [paper] code Tags: SLR
• 【NeurIPS 2022】Two-Stream Network for Sign Language Recognition and Translation. Yutong Chen, Ronglai Zuo, Fangyun Wei, Yu Wu, Shujie Liu, Brian Mak. [paper] code Tags: SLT & SLR
• 【EMNLP 2022】Open-Domain Sign Language Translation Learned from Online Video. Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu. [paper] code Tags: SLT
• 【NAACL 2022 Findings】Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation. Yong Cao, Wei Li, Xianzhi Li, Min Chen, Guangyong Chen, Long Hu, Zhengdao Li, Hwang Kai. [paper] code Tags: SLT
• 【ACL 2022】WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language. Federico Tavella, Viktor Schlegel, Marta Romeo, Aphrodite Galata, Angelo Cangelosi. [paper] code Tags: Dataset
• 【ACL 2022】Searching for fingerspelled content in American Sign Language. Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu. [paper] code Tags: Fingerspelled SL
• 【ECCV 2022】Automatic dense annotation of large-vocabulary sign language videos. Liliane Momeni, Hannah Bull, K R Prajwal, Samuel Albanie, G��l Varol, Andrew Zisserman. [paper] code Tags: Annotation
• 【ECCV 2022】Temporal Lift Pooling for Continuous Sign Language Recognition. Lianyu Hu, Liqing Gao, Zekang Liu, Wei Feng. [paper] code Tags: SLR
• 【ACMMM 2022】MC-SLT: Towards Low-Resource Signer-Adaptive Sign Language Translation. Tao Jin, Zhou Zhao, Meng Zhang, Xingshan Zeng. [paper] code Tags: SLT
• 【WACV 2022】Sign Language Translation With Hierarchical Spatio-Temporal Graph Neural Network. Jichao Kan, Kun Hu, Markus Hagenbuchner, Ah Chung Tsoi, Mohammed Bennamoun, Zhiyong Wang. [paper] code Tags: SLT

2021

[Back to TOP]

• 【Neurocomputing 2021】Enhancing neural sign language translation by highlighting the facial expression information. [paper] code Tags: SLT
• 【TMM 2021】Spatial-Temporal Multi-Cue Network for Sign Language Recognition and Translation. Hao Zhou, Wengang Zhou, Yun Zhou, Houqiang Li. [paper] code Tags: SLT & SLR
• 【ICCV 2021】YouRefIt: Embodied Reference Understanding With Language and Gesture. Yixin Chen; Qing Li; Deqian Kong; Yik Lun Kei; Song-Chun Zhu; Tao Gao; Yixin Zhu; Siyuan Huang. [paper] code Tags: SLR
• 【ICCV 2021】Speech Drives Templates: Co-Speech Gesture Synthesis With Learned Templates. Shenhan Qian; Zhi Tu; Yihao Zhi; Wen Liu; Shenghua Gao. [paper] code Tags: SLR
• 【ICCV 2021】Audio2Gestures: Generating Diverse Gestures From Speech Audio With Conditional Variational Autoencoders. Jing Li; Di Kang; Wenjie Pei; Xuefei Zhe; Ying Zhang; Zhenyu He; Linchao Bao. [paper] code Tags: SLR
• 【ICCV 2021】Aligning Subtitles in Sign Language Videos. Hannah Bull; Triantafyllos Afouras; G��l Varol; Samuel Albanie; Liliane Momeni; Andrew Zisserman. [paper] code Tags: SLR
• 【ICCV 2021】Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives. Ben Saunders; Necati Cihan Camgoz; Richard Bowden. [paper] code Tags: SLR
• 【ICCV 2021】SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition. Hezhen Hu; Weichao Zhao; Wengang Zhou; Yuechen Wang; Houqiang Li. [paper] code Tags: SLR
• 【ICCV 2021】Visual Alignment Constraint for Continuous Sign Language Recognition. Yuecong Min, Aiming Hao, Xiujuan Chai, and Xilin Chen. [paper] [code] Tags: SLR
• 【ICCV 2021】Self-Mutual Distillation Learning for Continuous Sign Language Recognition. Aiming Hao, Yuecong Min, and Xilin Chen. [paper] code Tags: SLR
• 【CVPR 2021】Improving Sign Language Translation With Monolingual Data by Sign Back-Translation. Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, Houqiang Li. [paper] code Tags: SLR
• 【CVPR 2021】How2Sign: A Large-Scale Multimodal Dataset for Continuous American Sign Language. Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, Xavier Giro-i-Nieto. [paper] code Tags: SLR
• 【CVPR 2021】Fingerspelling Detection in American Sign Language. Bowen Shi, Diane Brentari, Greg Shakhnarovich, Karen Livescu. [paper] code Tags: SLR
• 【CVPR 2021】Read and Attend: Temporal Localisation in Sign Language Videos. G��l Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, Andrew Zisserman. [paper] code Tags: SLR
• 【CVPR 2021】iMiGUE: An Identity-Free Video Dataset for Micro-Gesture Understanding and Emotion Analysis. Xin Liu, Henglin Shi, Haoyu Chen, Zitong Yu, Xiaobai Li, Guoying Zhao. [paper] code Tags: SLR
• 【CVPR 2021】Body2Hands: Learning To Infer 3D Hands From Conversational Gesture Body Dynamics. Evonne Ng, Shiry Ginosar, Trevor Darrell, Hanbyul Joo. [paper] code Tags: SLR
• 【CVPR 2021】Model-Aware Gesture-to-Gesture Translation. Hezhen Hu, Weilun Wang, Wengang Zhou, Weichao Zhao, Houqiang Li. [paper] code Tags: SLR
• 【AAAI 2021】Hand-Model-Aware Sign Language Recognition. Hezhen Hu, Wengang Zhou, Houqiang Li. [paper] code Tags: SLR
• 【AAAI 2021】Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition. Benjia Zhou, Yunan Li, Jun Wan. [paper] code Tags: SLR
• 【WACV 2021】Hand Pose Guided 3D Pooling for Word-level Sign Language Recognition. Al Amin Hosain; Panneer Selvam Santhalingam; Parth Pathak; Huzefa Rangwala; Jana Kosecka. [paper] code Tags: SLR
• 【WACV 2021】Whose hand is this? Person Identification from Egocentric Hand Gestures. Satoshi Tsutsui; Yanwei Fu; David Crandall. [paper] code Tags: SLR

2020

[Back to TOP]

• 【ACMMM 2020】INCLUDE: A Large Scale Dataset for Indian Sign Language Recognition. Sridhar, Advaith, Rohith Gandhi Ganesan, Pratyush Kumar, and Mitesh Khapra. [paper] code Tags: SLR
• 【ACMMM 2020】Boosting Continuous Sign Language Recognition via Cross Modality Augmentation. Pu, Junfu, Wengang Zhou, Hezhen Hu, and Houqiang Li. [paper] code Tags: SLR
• 【ACMMM 2020】Recognizing Camera Wearer from Hand Gestures in Egocentric Videos. Thapar, Daksh, Aditya Nigam, and Chetan Arora. [paper] code Tags: SLR
• 【NeurIPS 2020】TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation. Li, Dongxu, Chenchen Xu, Xin Yu, Kaihao Zhang, Benjamin Swift, Hanna Suominen, and Hongdong Li. [paper] code Tags: SLR
• 【FG 2020】Feature Selection for Zero-Shot Gesture Recognition. Naveen Madapana, Juan Wachs. [paper] code Tags: SLR
• 【FG 2020】Image-Based Pose Representation for Action Recognition and Hand Gesture Recognition. Zeyi Lin, Wei Zhang, Xiaoming Deng, Cuixia Ma, Hongan Wang. [paper] code Tags: SLR
• 【FG 2020】Neural Sign Language Translation by Learning Tokenization. Orbay, Alptekin, and Lale Akarun. [paper] code Tags: SLT
• 【FG 2020】Sign Language Recognition in Virtual Reality. Jacob Schioppo, Zachary Meyer, Diego Fabiano, Shaun Canavan. [paper] code Tags: SLR
• 【FG 2020】SILFA: Sign Language Facial Action Database for the Development of Assistive Technologies for the Deaf. Emely Puj��lli da Silva, Paula Dornhofer Paro Costa, Kate Mamhy Oliveira Kumada, Jos�� Mario De Martino. [paper] code Tags: Dataset
• 【FG 2020】FineHand: Learning Hand Shapes for American Sign Language Recognition. Al Amin Hosain, Panneer Selvam Santhalingam, Parth Pathak, Huzefa Rangwala, Jana Ko?eck��. [paper] code Tags: SLR
• 【FG 2020】Introduction and Analysis of an Event-Based Sign Language Dataset. Ajay Vasudevan, Pablo Negri, Bernabe Linares-Barranco, Teresa Serrano-Gotarredona. [paper] code Tags: Dataset
• 【FG 2020】Towards a Visual Sign Language Dataset for Home Care Services. D. Kosmopoulos, I. Oikonomidis, C. Constantinopoulos, N. Arvanitis, K. Antzakas, A. Bifis, G. Lydakis, A. Roussos, A. Argyros. [paper] code Tags: Dataset
• 【ECCV 2020】BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues. Samuel Albanie, G��l Varol, Liliane Momeni, Triantafyllos Afouras, Joon Son Chung, Neil Fox, Andrew Zisserman. [paper] code Tags: SLR
• 【ECCV 2020】Progressive Transformers for End-to-End Sign Language Production. Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. [paper] [code] Tags: SLP
• 【ECCV 2020】Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition. Niu Zhe, Brian Mak. [paper] code Tags: SLR
• 【ECCV 2020】Fully Convolutional Networks for Continuous Sign Language Recognition. Ka Leong Cheng, Zhaoyang Yang, Qifeng Chen, and Yu-Wing Tai. [paper] code Tags: SLR
• 【ECCV 2020】Collaborative Learning of Gesture Recognition and 3D Hand Pose Estimation with Multi-Order Feature Analysis. Yang, Siyuan, Jun Liu, Shijian Lu, Meng Hwa Er, and Alex C. Kot. [paper] code Tags: SLR
• 【ECCV 2020】Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach. Chaitanya Ahuja, Dong Won Lee, Yukiko I. Nakano, and Louis-Philippe Morency. [paper] code Tags: SLR
• 【ECCV 2020】Towards Efficient Coarse-to-Fine Networks for Action and Gesture Recognition. Quader, Niamul, Juwei Lu, Peng Dai, and Wei Li. [paper] code Tags: SLR
• 【CVPR 2020】Decoupled Representation Learning for Skeleton-Based Gesture Recognition. Jianbo Liu, Yongcheng Liu, Ying Wang, V��ronique Prinet, Shiming Xiang, Chunhong Pan. [paper] code Tags: SLR
• 【CVPR 2020】An Efficient PointLSTM for Point Clouds Based Gesture Recognition. Yuecong Min, Yanxiao Zhang, Xiujuan Chai, Xilin Chen. [paper] code Tags: SLR
• 【CVPR 2020】Transferring Cross-Domain Knowledge for Video Sign Language Recognition. Dongxu Li, Xin Yu, Chenchen Xu, Lars Petersson, Hongdong Li. [paper] code Tags: SLR
• 【WACV 2020】Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. Dongxu Li, Cristian Rodriguez, Xin Yu, Hongdong Li. [paper] code Tags: Dataset
• 【WACV 2020】Neural Sign Language Synthesis: Words Are Our Glosses. Jan Zelinka, Jakub Kanis. [paper] code Tags: SLP
• 【CIN 2020】Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. [paper] code Tags: SLT
• 【arXiv 2020】Multi-channel Transformers for Multi-articulatory Sign Language Translation. [paper] code Tags: SLT
• 【CVPR 2020】Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation. Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. [paper] [code] Tags: SLT
• 【arXiv 2020】Better Sign Language Translation with STMC-Transformer. Kayo Yin, Jesse Read. [paper] [code] Tags: SLT
• 【arXiv 2020】Neural Sign Language Translation by Learning Tokenization. Alptekin Orbay and Lale Akarun. [paper] code Tags: SLT
• 【ECCV 2020】Stochastic Fine-grained Labeling of Multi-state Sign Glosses for Continuous Sign Language Recognition. [paper] [code] Tags: SLR
• 【ECCV 2020】Fully Convolutional Networks for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【AAAI 2020】Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【ECCV 2020】BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues. [paper] code Tags: SLR
• 【WACV 2020】Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison. [paper] [code] Tags: SLR
• 【IJCV 2020】Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks. Stephanie Stoll, Necati Cihan Camgoz, Simon Hadfield, Richard Bowden. [paper] code Tags: Text2Sign
• 【ECCV 2020】Progressive Transformers for End-to-End Sign Language Production. [paper] code Tags: Text2Sign
• 【TCSVT 2020】Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition. [paper] code Tags: SLR

2019

[Back to TOP]

• 【IEEE Access 2019 (Recommended)】Technical approaches to Chinese sign language processing: A review. Suhail Muhammad Kamal; Yidong Chen; Shaozi Li; Xiaodong Shi; Jiangbin Zheng. [paper] code Tags: Survey
• 【BigMM 2019】Deep Grammatical Multi-classifier for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【ICIP 2019】Continuous Sign Language Recognition via Reinforcement Learning. [paper] code Tags: SLR
• 【ICME 2019】Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【IJCAI 2019】Dense Temporal Convolution Network for Sign Language Translation. [paper] code Tags: SLR
• 【IEEE TRANSACTIONS ON MULTIMEDIA 2019】A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training. [paper] code Tags: SLR
• 【CVPR 2019】Iterative Alignment Network for Continuous Sign Language. [paper] code Tags: SLR
• 【TPAMI 2019】Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos. [paper] [code] Tags: SLR
• 【arXiv 2019】SF-Net: Structured Feature Network for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【IEEE ACCESS 2019】Dynamic Sign Language Recognition Based on Video Sequence With BLSTM-3D Residual Networks. [paper] code Tags: SLR
• 【ICCV 2019】Temporal Accumulative Features for Sign Language Recognition. [paper] code Tags: SLR
• 【ICCCM 2019】Thai Sign Language Recognition Using 3D Convolutional Neural Networks. [paper] code Tags: SLR
• 【DSAA 2019】Sign Language Recognition Analysis using Multimodal Data. [paper] code Tags: SLR
• 【ICANN 2019】Spatial-Temporal Graph Convolutional Networks for Sign Language Recognition. [paper] [code] Tags: SLR
• 【ICCV 2019】Fingerspelling recognition in the wild with iterative visual attention. [paper] code Tags: SLR
• 【SITIS 2019】Translation of sign language glosses to text using sequence-to-sequence attention models. [paper] code Tags: Gloss2Text

2018

[Back to TOP]

• 【TCSVT 2018】Attention based 3D-CNNs for Large-Vocabulary Sign Language Recognition. [paper] code Tags: SLR
• 【AAAI 2018】Video-based sign language recognition without temporal segmentation. [paper] code Tags: SLR
• 【AAAI 2018】Hierarchical LSTM for Sign Language Translation. [paper] code Tags: SLT
• 【ACMMM 2018】Connectionist Temporal Fusion for Sign Language Translation. [paper] code Tags: SLT
• 【arXiv 2018】Neural Sign Language Translation based on Human Keypoint Estimation. Sang-Ki Ko, Chang Jo Kim, Hyedong Jung, Choongsang Cho. [paper] code Tags: SLT
• 【CVPR 2018】Neural Sign Language Translation. Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, Richard Bowden. [paper] [code] Tags: SLT
• 【IJCAI 2018】Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition. [paper] [code] Tags: SLR
• 【ETRI 2018】Human-like sign-language learning method using deep learning. [paper] code Tags: SLR
• 【IJCV 2018】Deep Sign: Enabling Robust Statistical Continuous Sign Language Recognition via Hybrid CNN-HMMs. [paper] code Tags: SLR
• 【NIPS 2018】Attention in Convolutional LSTM for Gesture Recognition. [paper] [code] Tags: SLR

2017

[Back to TOP]

• 【CVPR 2017】Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs. [paper] [code] Tags: SLR
• 【CVPR 2017】Recurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization. [paper] code Tags: SLR
• 【TOMM 2017】Online Early-Late Fusion Based on Adaptive HMM for Sign Language Recognition*. [paper] code Tags: SLR
• 【ICCV 2017】SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition. [paper] [code] Tags: SLR
• 【ICCV 2017】Learning Spatiotemporal Features Using 3DCNN and Convolutional LSTM for Gesture Recognition. [paper] [code] Tags: SLR

Earlier

[Back to TOP]

• 【ICME 2016】Chinese sign language recognition with adaptive HMM. [paper] code Tags: SLR
• 【PCM2016】Sign Language Recognition with Multi-modal Features. [paper] code Tags: SLR
• 【ICME2015】Sign language recognition using 3D convolutional neural networks. [paper] code Tags: SLR
• 【ICIP 2016】Sign language recognition based on adaptive HMMS with data augmentation. [paper] code Tags: SLR
• 【Pattern Recognit.Lett.2016】Continuous sign language recognition using level building based on fast hidden Markov model. [paper] code Tags: SLR
• 【TACCESS 2016】Sign Transition Modeling and a Scalable Solution to Continuous Sign Language Recognition for Real-World Applications. [paper] code Tags: SLR
• 【ICPR 2016】Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition. [paper] code Tags: SLR
• 【CVPR 2016】Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks. [paper] code Tags: SLR
• 【CVPR 2016】Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled. [paper] code Tags: SLR
• 【BMVC 2016】Deep Sign: Hybrid CNN-HMM for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【ICIP 2016】Sign Language Recognition With Long Short-Term Memory. [paper] code Tags: SLR
• 【ECCV 2016】Iterative Reference Driven Metric Learning for Signer Independent Isolated Sign Language Recognition. [paper] code Tags: SLR
• 【LREC 2016】Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【CVPRW 2015】Hand Gesture Recognition with 3D Convolutional Neural Networks. [paper] code Tags: SLR
• 【ICCVW 2015】Deep Learning of Mouth Shapes for Sign Language. [paper] code Tags: SLR
• 【ICME 2015】Sign Language Recognition using 3D convolutional neural networks. [paper] code Tags: SLR
• 【ACCV 2014】Curve Matching from the View of Manifold for Sign Language Recognition. [paper] code Tags: SLR
• 【ICIMCS 2014】A Threshold-based HMM-DTW Approach for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【ICTA 2013】English-ASL Gloss Parallel Corpus 2012: ASLG-PC12, The Second Release. [paper] code Tags: Dataset
• 【SLPAT 2013】Improving Continuous Sign Language Recognition: Speech Recognition Techniques and System Design. [paper] code Tags: SLR
• 【IWSLT 2013】Using Viseme Recognition to Improve a Sign Language Translation System. [paper] code Tags: SLR
• 【AFGR 2013】Sign Language Recognition and Translation with Kinect. [paper] code Tags: SLR
• 【CVPR 2012】Sign Language Recognition using Sequential Pattern Trees. [paper] code Tags: SLR
• 【JMLR 2012】Sign language recognition using sub-units. [paper] code Tags: SLR
• 【CVPRW 2011】Advances in phonetics-based sub-unit modeling for transcription alignment and sign language recognition. [paper] code Tags: SLR
• 【Eng.Appl.Artif.Intell.2011】American Sign Language word recognition with a sensory glove using artificial neural networks. [paper] code Tags: SLR
• 【INTERSPEECH 2007】Speech Recognition Techniques for a Sign Language Recognition System. [paper] code Tags: SLR
• 【TSMC 2007】Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models. [paper] code Tags: SLR
• 【TPAMI 1998】Real-time American sign language recognition using desk and wearable computer based video. [paper] code Tags: SLR

AI Sign Language in Well-Known Institutions [Papers]

XMU for AI Sign Language

[Back to TOP]

• 【AAAI 2024】Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment. Rui Zhao, Liang Zhang, Biao Fu, Cong Hu, Jinsong Su, Yidong Chen. [paper] [code] Tags: SLT
• 【ICASSP 2024】An Explicit Multi-Modal Fusion Method For Sign Language Translation. Cong Hu, Biao Fu, Pei Yu, Liang Zhang, Xiaodong Shi, Yidong Chen. [paper] code Tags: SLT
• 【CVPR 2023 (Highlight Paper)】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code] Tags: SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University
• 【IJCAI 2023】Efficient Sign Language Translation with a Curriculum-based NAR Decoder. Pei Yu1, Liang Zhang, Biao Fu1, Yidong Chen. [paper] code Tags: SLT
• 【ICASSP 2023】A Token-level Contrastive Framework for Sign Language Translation. Biao Fu, Peigen Ye, Liang Zhang, Pei Yu, Cong Hu, Yidong Chen, Xiaodong Shi. [paper] code Tags: SLT
• 【Arxiv 2022】Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translation. [paper] code Tags: SLT
• 【Neurocomputing 2021】Enhancing neural sign language translation by highlighting the facial expression information. [paper] code Tags: SLT
• 【CIN 2020】Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences. [paper] code Tags: SLT
• 【IEEE Access 2019 (Recommended)】Technical approaches to Chinese sign language processing: A review. Suhail Muhammad Kamal; Yidong Chen; Shaozi Li; Xiaodong Shi; Jiangbin Zheng. [paper] code Tags: Survey

USTC for AI Sign Language

[Back to TOP]

• 【ACMMM 2020】Boosting Continuous Sign Language Recognition via Cross Modality Augmentation. Pu, Junfu, Wengang Zhou, Hezhen Hu, and Houqiang Li. [paper] code Tags: SLR
• 【AAAI 2020】Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【TCSVT 2020】Semantic Boundary Detection with Reinforcement Learning for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【BigMM 2019】Deep Grammatical Multi-classifier for Continuous Sign Language Recognition. [paper] code Tags: SLR
• 【IJCAI 2019】Dense Temporal Convolution Network for Sign Language Translation. [paper] code Tags: SLR
• 【IEEE TRANSACTIONS ON MULTIMEDIA 2019】A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training. [paper] code Tags: SLR
• 【CVPR 2019】Iterative Alignment Network for Continuous Sign Language. [paper] code Tags: SLR
• 【AAAI 2018】Video-based sign language recognition without temporal segmentation. [paper] code Tags: SLR
• 【AAAI 2018】Hierarchical LSTM for Sign Language Translation. [paper] code Tags: SLT
• 【ACMMM 2018】Connectionist Temporal Fusion for Sign Language Translation. [paper] code Tags: SLT
• 【IJCAI 2018】Dilated Convolutional Network with Iterative Optimization for Continuous Sign Language Recognition. [paper] [code] Tags: SLR
• 【ICME 2016】Chinese sign language recognition with adaptive HMM. [paper] code Tags: SLR
• 【PCM2016】Sign Language Recognition with Multi-modal Features. [paper] code Tags: SLR
• 【ICME2015】Sign language recognition using 3D convolutional neural networks. [paper] code Tags: SLR
• 【ICIP 2016】Sign language recognition based on adaptive HMMS with data augmentation. [paper] code Tags: SLR
• 【ICIP 2016】Sign Language Recognition With Long Short-Term Memory. [paper] code Tags: SLR

ZJU for AI Sign Language

[Back to TOP]

• 【CVPR 2023 (Highlight Paper)】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code] Tags: SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University

THU for AI Sign Language

[Back to TOP]

• 【CVPR 2023 (Highlight Paper)】CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignment. Jiangbin Zheng, Yile Wang, Cheng Tan, Siyuan Li, Ge Wang, Jun Xia, Yidong Chen, Stan Z. Li. [paper] [code] Tags: SLR; Westlake University & Zhejiang University & Xiamen University & Tsinghua University
• 【CVPR 2022】A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation. Yutong Chen, Fangyun Wei, Xiao Sun, Zhirong Wu, Stephen Lin. [paper] code Tags: SLT

Germany-UK for AI Sign Language

[Back to TOP]

• 【arXiv 2020】Multi-channel Transformers for Multi-articulatory Sign Language Translation. [paper] code Tags: SLT
• 【CVPR 2020】Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation. Necati Cihan Camgoz, Oscar Koller, Simon Hadfield, and Richard Bowden. [paper] [code] Tags: SLT
• 【TPAMI 2019】Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos. [paper] [code] Tags: SLR
• 【CVPR 2018】Neural Sign Language Translation. Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney, Richard Bowden. [paper] [code] Tags: SLT
• 【CVPR 2017】Re-Sign: Re-Aligned End-to-End Sequence Modelling with Deep Recurrent CNN-HMMs. [paper] [code] Tags: SLR
• 【ICCV 2017】SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition. [paper] [code] Tags: SLR

Datasets

[Back to TOP]

Dataset	Language	Classes	Samples	Data Type	Language Level
ASLG-PC12paper	American	-	87,709	GLOSS&Sentences	isolated
CSL Dataset I	Chinese	500	125,000	Videos&Depth from Kinect	isolated
CSL Dataset II	Chinese	100	25,000	Videos&Depth from Kinect	continuous
RWTH-PHOENIX-Weather 2014	German	1,081	6,841	Videos	continuous
RWTH-PHOENIX-Weather 2014 T	German	1,066	8,257	Videos	continuous
ASLLVD	American	3,300	9,800	Videos(multiple angles)	isolated
ASLLVD-Skeleton	American	3,300	9,800	Skeleton	isolated
SIGNUM	German	450	33,210	Videos	continuous
DGS Kinect 40	German	40	3,000	Videos(multiple angles)	isolated
DEVISIGN-G	Chinese	36	432	Videos	isolated
DEVISIGN-D	Chinese	500	6,000	Videos	isolated
DEVISIGN-L	Chinese	2000	24,000	Videos	isolated
LSA64	Argentinian	64	3,200	Videos	isolated
GSL isol.	Greek	310	40,785	Videos&Depth from RealSense	isolated
GSL SD	Greek	310	10,290	Videos&Depth from RealSense	continuous
GSL SI	Greek	310	10,290	Videos&Depth from RealSense	continuous
IIITA -ROBITA	Indian	23	605	Videos	isolated
PSL Kinect	Polish	30	300	Videos&Depth from Kinect	isolated
PSL ToF	Polish	84	1,680	Videos&Depth from ToF camera	isolated
BUHMAP-DB	Turkish	8	440	Videos	isolated
LSE-Sign	Spanish	2,400	2,400	Videos	isolated
Purdue RVL-SLLL	American	39	546	Videos	isolated
RWTH-BOSTON-50	American	50	483	Videos(multiple angles)	isolated
RWTH-BOSTON-104	American	104	201	Videos(multiple angles)	continuous
RWTH-BOSTON-400	American	400	843	Videos	continuous
WLASL	American	2,000	21,083	Videos	isolated

AI Sign Language Related Fields

[Back to TOP]

The related fields, such as Large Pretrained Language Model,Corss-modal/Multi-modality, Action Recognition, Machine Translation, Speech Recognition, Video Captioning, can help you gain more inspiration and knowledge