iDrum is a game in which the user can play along to his or her favorite songs simply by using a pair of drumsticks and a webcam. The format is similar to that of the popular Guitar Hero and offers a unique twist: the user plays the drums using a laptop webcam equipped with computer vision and can thus interact with the games simply by moving drumsticks in the air. Furthermore, the game has built-in audio processing intelligence; the user can input his or her own M4A music files, and iDrum will automatically convert them into playable levels. There were four distinct components to the project: audio processing, computer vision, graphics, and the back-end scoring algorithms.
The audio processing was done in MATLAB and could automatically generate Guitar Hero-like levels from an input music file. This allows users to customize their experience and play along to any song of their choosing; no more getting tired of playing the same levels over and over again! The audio processing algorithm outputs a .gh file (a nod to Guitar Hero), which is sent to the graphical component of the project.
The graphical component of the game was done in C++ with OpenGL, and is reminiscient of a DDR set-up. The "notes" to hit fall down onto the drum-like pads on the bottom of the game window. There are two cymbals and three drums, and the user must hit them when their corresponding notes pass over them.
This "hitting" motion comes from the computer vision aspect of the project, done in C++ and OpenCV. Using the webcam, the tips of the user's drumsticks are automatically detected and tracked.