Arduino & C# Automated desktop webcam robot.
This is a project to make a (somewhat) expressive, desk toy webcam that looks for and attempts to follow movement ( and maybe more, eventually. ) It's based on Hackerbox kit #24 with a few added parts where needed for additional functionality.
| Progress | Description |
|---|---|
| Done! | Initial build. |
| Done! | Write Initial Arduino Serial interface API |
| Done! | Write Initial C# desktop app to prove out tracking and Serial interfacing. |
| Done! | Put together github for everything. |
| Done! | Put together tutorial video on initial build, initial code, key learnings and demonstration. |
| 50% | 2nd pass at Arduino code, clean things up, break them into seperate files, etc. |
| 90% | 2nd pass at C# desktop app, clean things up, break them into seperate classes, etc. |
| 0% | 3d print nicer cover for electronics and paint / finish stand. |
There are 2 sides to the software on this project.
- The Arduino code that creates a Serial Interface to control the robot and other hardware.
- The C# code to create a desktop app to read the webcam, look for interesting activity and control the robot ( via Arduino Serial Interface mentioned above. )
The hardware consists of: ( Amazon links added for beginners looking for easier options to purchase. )
- An Arduino Nano V3 to control everything Amazon Link
- A single Neopixel RGB LED Amazon Link
- 2 MG996R Servos Amazon Link
- A Pan / Tilt Assembly Amazon Link
- A laptop webcam assembly + USB adapter Amazon Link
- A repurposed power adapter from an old cellphone Amazon Link
- 2 usb cables
- Scrap floor samples as a weighted base + cardboard for 'feet'
- A tiny breadboard + prototyping connectors to keep things open to experimentation. Amazon Link
Awful hand drawn schematic:
The arduino code is pretty straightforward. There is a set of commands to control every piece of hardware. I generally write the Serial interpretter to take commands in the form of <COMMAND:PARAMETER>, that way if we end up with multiple commands in the buffer at a time they can easily be broken apart and dealt with individually. I also like to write this code so that you can define where you want the state of things to be, and how fast you want to move towards those states. That way you can just send one command that says "Hey, I want the light to fade to green slowly" with a few commands and not have to actually send all of the rgb values to animate that yourself.
The C# code is currently using a wrapper around the amazing Open CV library for movement detection from the webcam feed. You can find this library Here ( although you should just install the package via the Nuget Package Manager in Visual Studio. ) At the moment I am just hacking up the movement detection example to suite my needs, but eventually I will start my own, more organized, app.
I would love for the CamBot to have some personality, it would look for faces and follow them. When it see's people it 'likes' it would light up with a happy color and beep in delight. Something that just sits on the desk and looks adorable. Maybe it would unlock your computer for you, or do other more useful things. Some kind of google home type integration?!
I would also love to rebuild it to be much smaller, use a raspberry pi for the controller, and maybe have a higher resolution webcam?
You will have to be able to run .net 4 code with Visual Studio 2017 ( Community Edition is fine, ) and you will also have to install the EMGU libs via the Nuget Package Manager.
If you use this on a computer that already has a webcam, you will have to disable that webcam so this doesn't pick that up instead. You could add code to find your bot webcam, specifically, but I decided not to add that as it's too specific to one user.
Movement detection is VERY sensitive to alot of things. Take care to minimize the amount of noise in your video feed. For example; dark rooms will be a problem if your camera doesn't use infrared. Cheaper cameras will have alot more noise in general. Your robot will need to be seated on a totally still surface. ANY unplanned movement will 'unsettle' your motion detection ( it will think EVERYTHING is moving until it has time to settle again. )
If you open the Desktop app solution file and open VideoSurveilance.cs you will see some settings towards the top of the file that you can play with to fine tune things alittle.