/Tennis-Tracking

Open-source monocular HawkEye 🔥

Primary LanguagePythonThe UnlicenseUnlicense

Tennis Tracking 🎾

Objectives

  • Track the ball
  • Detect court lines
  • Detect the players

To track the ball we used TrackNet - deep learning network for tracking high-speed objects. For players detection yolov3 was used.

Example using sample videos

Input Output
input_img1 output_img1
input_img2 output_img2
input_img3 output_img3

How to run

This project requires compatible GPU to install tensorflow, you can run it on your local machine in case you have one or use Google Colaboratory with Runtime Type changed to GPU.

  1. Clone this repository 
    2. git clone https://github.com/ArtLabss/tennis-tracking
  2. Download yolov3 weights (237 MB) from here and add it to your Yolov3 folder.
  3. Install the requirements using pip 
    4. pip install -r requirements.txt
  4. Run the following command in the command line 
    5. python predict_video.py --input_video_path=VideoInput/video_input3.mp4 --output_video_path=VideoOutput/video_output.mp4 --minimap=0
  5. If you are using Google Colab upload all the files to Google Drive
  6. Create a Google Colaboratory Notebook in the same directory as predict_video.py and connect it to Google drive 
    7. from google.colab import drive
drive.mount('/content/drive')
  7. Change the working directory to the one where the Colab Notebook and predict_video.py are. In my case, 
    8. import os 
os.chdir('MyDrive/Colab Notebooks/tennis-tracking')
  8. Install the requirements 
    9. !pip install -r requirements.txt
  9. Inside the notebook run predict_video.py 
    10.  !python3 predict_video.py --input_video_path=VideoInput/video_input3.mp4 --output_video_path=VideoOutput/video_output.mp4 --minimap=0

    After the compilation is completed, a new video will be created in VideoOutput folder if --minimap was set 0, if --minimap=1 three videos will be created: video of the game, video of minimap and a combined video of both

    P.S. If you stumble upon an error or have any questions feel free to open a new Issue

What's new?

  • Court line detection improved
  • Player detection improved
  • The algorithm now works practically with any court colors
  • Faster algorithm
  • Dynamic Mini-Map with players and ball added, to activate use argument --minimap
--minimap=0 --minimap=1
input_img1 output_img1

Further Developments

  • Improve line detection of the court and remove overlapping lines
  • Algorithm fails to detect players when the court colors aren't similar to the sample video
  • Don't detect the ballboys/ballgirls
  • Don't contour the banners
  • Detect players on videos with different angles
  • Find the coordinates of the ball touching the court and display them
  • Code Optimization
  • Dynamic court mini-map with players and the ball

Current Drawbacks

  • Slow algorithms (to process 15 seconds video (6.1 Mb) it takes 28 minutes 16 minutes)
  • Algorithm works only on official match videos

References

- Yu-Chuan Huang, "TrackNet: Tennis Ball Tracking from Broadcast Video by Deep Learning Networks," Master Thesis, advised by Tsì-Uí İk and Guan-Hua Huang, National Chiao Tung University, Taiwan, April 2018. - Yu-Chuan Huang, I-No Liao, Ching-Hsuan Chen, Tsì-Uí İk, and Wen-Chih Peng, "TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications," in the IEEE International Workshop of Content-Aware Video Analysis (CAVA 2019) in conjunction with the 16th IEEE International Conference on Advanced Video and Signal-based Surveillance (AVSS 2019), 18-21 September 2019, Taipei, Taiwan. - Joseph Redmon, Ali Farhadi, "YOLOv3: An Incremental Improvement", University of Washington, https://arxiv.org/pdf/1804.02767.pdf