ESP32-CAM: remoted object detection camera

Demonstration

Descrition

The ESP32-CAM would stream the images to PC via WebSocket packages, then the remote software in PC would receive the images and do the object detection with Tensorflow Lite(SSD Mobilenetv2) or Keras (YOLOv3).

Hardware setup

Components are used: ESP32-CAM module, RC Servo x2, 5V Battery.

Software setup

The software contains 2 parts

ESP32-CAM firmware

Dependency: Download https://github.com/Links2004/arduinoWebSockets as zip file and add this library to your Arduino Libraries by Sketch/Include Library/Add Zip...
Use Arduino IDE to flash firmware to ESP32-CAM module.

Remote control software (Can be run on Ubuntu or Windows PC)

ubuntu: Install virtualenv

sudo apt-get install python3-pip
sudo pip3 install virtualenv
cd ESP32CamObjectDetection/remote
virtualenv -p python3 <env_name>
source <env_name>/bin/activate
(<env_name>)$ pip install -r requirementVirtualenv.txt

windows 10: Not yet checked but it should be similar

Run remote control:

(<env_name>)$ cd remote
(<env_name>)$ python3 main.py

Note: If your computer does not have GPU, update setting.ini by changing GPU = 0, or you can change like this to run the code in CPU.

For use GPU, download the pretrained model here and place in the remote folder.

CNN models reference sources

SSD Mobilenetv2: https://github.com/tensorflow/models/tree/master/research/object_detection