/drone-speechcommand

Flying a drone using voice commands by training a model right in the browser.

Primary LanguageJavaScriptDo What The F*ck You Want To Public LicenseWTFPL

AI and JavaScript, flying a drone using voice commands

Hardware Used

  • DJI Tello Drone
  • Extra Batteries - it comes with one and I bought two extra. For continual development I'd say you only need two — one in the charger and one in the drone. For flying away from your house you definitely need at least 3 as they only last about 10-15 mins.
  • Fast Battery Charger with 4 slots
  • A cooling desk fan to keep the drone cool while you're testing with it

Using This Code

retraining the SpeechCommand model

  1. cd spcmd
  2. yarn
  3. yarn watch

The first button is to send the init command to drone and make it ready.

There are four buttons below that, pressing the first three, you could say a word (I've used take off, land, and flip), record at least 150 samples for each of those. Make sure you record some background noise as well (I had to play a drone video off YouTube for that 😁).

Once done, click on the brain button to retrain the model. When it's done, you will see the accuracy of your model.

At last, press the listen button and say your recorded word, that should send the command to drone.

Backend

  1. cd backend
  2. npm install
  3. connect to drone via wifi
  4. npm start

Troubleshooting

Docs for Tello are available here

If you let the drone's WIFI connection lapse, you have to restart the server by typing rs into the terminal. This will re-run the command command that puts the drone in SDK mode. If you don't do this, it will ignore any commands you send it.

Credits

Credits to Wes Bos for the backend part of this code base. Thanks to him I could just focus on the AI part.