Kinect playground

This project is my playground for experimenting with the Kinect depth camera. Right now I'm interested in how to stream depth data with minimal overhead so it can be consumed by higher-level languages.

If you're looking for an easy way to get started with the Kinect and OpenKinect/libfreenect in C, or if you're curious about depth image processing in JavaScript, the code here might be a good starting point for you.

What you need to get started

A Microsoft Kinect. You can buy one at any major electronics retailer. It's sold as an Xbox accessory, but it comes with a USB adapter that will plug right into your computer.
A C compiler. OS X users can install Xcode for this.
libfreenect and libpng. OS X users can install both libraries using Homebrew.
A recent version of Node.js (>= 0.8) if you'd like to run the browser example.

To compile the example programs, check out this repository and run make. You may need to edit the Makefile to adjust the path to libfreenect's header files.

Example: Watching depth data from the command line

The kinect-depth-ascii program was largely accidental. I needed a way to make sure I was reading the Kinect's depth data properly, and the simplest way to debug it was just to write it out to the terminal.

Source code: kinect-depth-ascii.c

Example: Streaming depth PNGs to another program

PNG is a good format for encoding Kinect depth data because it's lossless, universally readable, and compresses reasonably well. The kinect-depth-png program encodes the Kinect's depth images in 320x240@8bpp PNG images and streams them over stdout using the netstrings format.

Source code: kinect-depth-png.c

Example: Watching depth PNGs in the browser

You can pipe kinect-depth-png's output to the browser/server program, a Node.js web server that in turn streams the PNGs as Base64-encoded data: URIs using HTML5 Server-Sent Events.

The example client page simply sets an <img> tag's src attribute to the URI after each frame is received without further processing.

🎥 Watch video: Depth images streamed simultaneously to multiple browsers

Usage: Run ./kinect-depth-png | browser/server in a terminal, then visit http://localhost:5600/ in a browser.

Source code: browser/server, browser/client.html

Example: Depth edge detection in the browser

Building on the previous example, the edge.html example page demonstrates how to perform live edge detection of Kinect depth images in the browser.

The streamed depth images are drawn to a <canvas> element using the drawImage function. The depth image is then extracted into a typed array using getImageData and passed to a Web worker, where it is processed by the Sobel operator to detect edges. Then the computed edge image is returned back to the page and composited atop the depth image.

🎥 Watch video: Live edge detection of Kinect depth images in the browser

Usage: As above, but visit http://localhost:5600/edge.html.

Source code: browser/edge.html