This is my Masters final year individual project, where I am building a low-cost, all-in-one robitic hand, capable of being controlled via gesture controll over the internet in real time. Scroll down for demo.
- Microcontroller embedded in the robotic hand. This will control the servo motors used to actuate the 5 fingers on the hand. It will also be able to handle any other peripherals that could be added in the future.
- A single board computer (SBC) such as a Raspberry Pi. It will receive a stream of movement and positional data over the network from the client that will be relayed to the robotic hand and robotic arm. Data will be sent from the SBC to the microcontroller in the robotic hand over either USB or I2C.
- The scope of this project does not extend to building a full robotic arm. However, the project will be designed so that an arm could be connected to the SBC and sent the stream of positional data received by the SBC. Attaching the robotic hand to an arm would allow for more degrees of movement and flexibility.
- A camera module will be connected to the SBC. This will stream live video footage back to the client application, allowing the user to see the actions of the robot remotely and in real time.
- Initially, IP addresses will be used so that the client is able connect to the SBC. A stretch goal of this project is to allow access to the robot from anywhere with an internet connection and to facilitate this, a server is needed to manage the connection. This server could be hosted using a cloud provider such as Azure and AWS or hosted locally.
- Client desktop application. This will run on the user’s PC and will handle the computer vision aspect of project. The user will require a PC or laptop with a webcam to capture the gestures. This movement data will be sent to the SBC over the network. At the same time, the video stream from the SBC will be displayed in the desktop application to allow the user to see the real-time movements of the robotic hand.
demo.mp4
The communication now works over UDP.
- The CAD and .obj files for the robotic hand can be found in the "3D Models" folder.
- The desktop application can be found under QT/Application
- The code running on the microcontroller can be found under ESP32.