Generative Adversarial Graph Convolutional Networks for Peruvian Sign Language Synthesis.
-
Python 3.8.11
-
Create a new environment using
conda create -y -n psl-gan python=3.8.11
-
N_CUDA
environment variable is defined to choose the GPU in case of having more than onepip3 install -r requirements.txt
This repository contains the implementantion of Kinetic GAN applied to Peruvian Sign Language. It is used as a data augmentation method.
The Peruvian Sign Language dataset was created by https://github.com/gissemari/PeruvianSignLanguage. This folder contains the following data:
segmented_signs.zip
: zip file with the segmented signs in MP4 formatraw_data_cocopose.json
: located in zip file. Contains the landmarks got by COCO-WholeBody pose estimation modelraw_data_mediapipe.json
: Contains the landmarks got by MediaPipe pose estimation model (body and hands)
To generate the pickle file with the shape of N x C x T x V, where N is the number of samples, C the number of coordinates, T the number of frames, and V the number of joints, run the following code
- 27 number of joints
python3 create_dataset.py --minframes <MIN_FRAMES> --mininstances <MIN_FRAMES> --leftHandLandmarks --rightHandLandmarks --rawCocoDataset <PATH_JSON_COCOPOSE> --inputPath <PATH_SEGMENTED_SIGNS> --nLandmarks 27 --useCoco --rawDataset <PATH_JSON_MEDIAPIPE>
- 24 number of joints
python3 create_dataset.py --minframes <MIN_FRAMES> --mininstances <MIN_FRAMES> --leftHandLandmarks --rightHandLandmarks --rawCocoDataset <PATH_JSON_COCOPOSE> --inputPath <PATH_SEGMENTED_SIGNS> --nLandmarks 24 --useCoco --rawDataset <PATH_JSON_MEDIAPIPE> --addExtraJoint
- Run the following code for training without wandb tracking.
python3 train.py