Repository for the Similarity Modeling 1 and 2 project at University of Technology Vienna.
For a more detailed documentation please read the documentation.
Make sure you have at least Python 3.6.3 and TensorFlow up and running on your system. The primary development system was a Windows 10 machine using a Nvidia 1080 and Nvidia 770, Ryzen 7 1700 with 32GB RAM. Training might take quite long using only CPU.
pip3 install -r requirements.txt
The expected folder structure is the following:
./root/dataset/<training videos>
./root/audio_dataset/<training audio> # Generated by extract_audio.py
./root/Repo/<git repository>
If your folder structure is differently, some paths inside the python scripts must be changed.
Download the video data set from the similarity modeling course website. To extract audio ffmpeg must be properly installed. Execute extract_audio.py.
cd ./path_of_repository/
python3 extract_audio.py
To start training the jump detection model, execute SM1/jump_detection.py or SM2/winch_detection.py
Make sure the audio files have been extracted before properly. This might take very long (including feature extraction it took like 8 hours on my machine).
cd ./path_of_repository/
python3 SM1/jump_detection.py
cd ./path_of_repository/
python3 SM2/winch_detection.py
To use hand detection the skin neural network has to be trained before. To train this neural network execute SM2/skin_detection.py before.
After that you can start the hand detection by executing SM2/hand_detection.py. Which is using the previously trained models to display the winch usage and jump detection.
cd ./path_of_repository/
python3 SM2/skin_detection.py
python3 SM2/hand_detection.py