OpenCV and Unity3D integration.

A simple way to connect Opencv with Unity3D using socket.

I was looking for a solution to integrated OpenCV with Unity3D easily. Most of the solution were for Windows with very complex setup procedure / expensive plugins / with feature restriction. As a Unix user those were'nt very helpful. I wanted to utilise full feature of OpenCV and Unity3D regardless OS and then came up with this very simple solution.

28th Sept 2018 Update

Unity 2018.2 compatible

30th January 2017 update

Make it compatible for Unity 5.5 and above
Change port from commonly used 5005 to a safer 5065
Add comments on how to get position Y value in gameDemo.py file

I've used Open CV Camshift algorithm to detect and track object in Python. Then I send the position of the object into Unity 3D using UDP Socket, read the stream from Unity 3D and change the position of a game object.... Pretty simple.

Here is the demo video - https://www.youtube.com/watch?v=aE4_VtUDORw

In order to make this demo work

You need to install Unity3D (http://unity3d.com)
Install and configure Python and OpenCV (https://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_setup/py_setup_in_windows/py_setup_in_windows.html#install-opencv-python-in-windows)
Change the 3rd line of OpenCV/gameDemo.py to appropriate installation directory of OpenCv
Run the OpenCV/gameDemo.py
Drag your mouse over an object. You will find the program start tracking it while you change position. The position will be printed in the command line.
Open the Unity3d Scene Unity3D/Assets/Scenes/socketTest.unity in Unity3D
Run the game and you will find the Unity's game object is following the real life tracked object

I'm just a tweet away (https://twitter.com/hasanavi) if you find any problem. For other contact details. http://hasanavi.me/

License

This is strictly WTFPL (http://en.wikipedia.org/wiki/WTFPL) license ;)