/LIDIC

Live Interactive Drone Imaging Control

Primary LanguagePythonMIT LicenseMIT

LIDIC - Live Interactive Drone Imaging Control

This repository is an implementation of machine vision and deep neural networks to be able to predict hand gestures and control a Tello EDU drone.

The default repository recognizes five different gestures: stop, up, down, left & right. The default model has an input shape of (42,) and a output shape of (5,) the 5 corresponding to the gestures predicted.

This repository also allow to control the drone using your keyboard, an xbox controller as well as by gestures, the latter is the focus of this repository.

Installation & Use:

  1. Install the requirements to be able to run our program, I am using a conda environment for easy setup.

    pip install -r requirements.txt

  2. Connect your computer to the drone's access point (WIFI).

  3. Run the command below to start the program, choose one of the options below in the square brackets.

    python tello.py -c [keyboard, gesture, xbox_controller]

  4. Controller Types:

    1. Xbox Controller:
    2. Keyboard:
      A - Left
      D - Right
      W - Forward
      S - Backwards
      X - Takeoff / Land R - Reset RC (ZERO WHEN YOU WANT TO STAY STILL)
    3. Gesture
      Stop = Takeoff / Land
      Hike Right = Go Right
      Point Left = Go Left
      Point Up = Go Up
      Thumbs Down = Go Down

      up gesture left gesture right gesture down gesture stop gesture
      Multi Commands:
      Stop + Hike Right = Rotate Right
      Stop + Point Left = Rotate Left
      Stop + Point Up = Forward
      Stop + Thumbs Down = Backwards
      up gesture left gesture right gesture down gesture

  5. To land the drone and finish the program, press Q.

Demo Video:

Demo Video

Research Paper

Project Structure:

  1. tello.py

    File for actuall drone control, use this file when you want to connect and control the drone.

  2. train.py

    File for actually training and creating the gesture recognizer

  3. utillities.py

    File for classes and functions frequently used in our program

  4. tellogui.py

    Graphics which is used across files and function to draw information about the tello drone and displaying it on a cv2 image feed.

  5. ./controllers

    Different ways to control the drone, e.g xbox_controller, gestures and keyboard

  6. ./deprecated

    Files which was used earlier in development to test different types of algorithms to e.g, detect shapes as well as isolating the hand.

  7. ./data

    Different training sets to train our models on.

  8. ./models

    Folder to contain different neural network models.