stephen-huan/anime-music-quiz

Computer plays AMQ

anime-music-quiz

[Video] [Lecture]

A bot which automatically plays animemusicquiz. Based on minimizing the l2 norm between the given clip and each song in its database, though the Fast Fourier Transform (FFT).

Future work: make it based off fingerprints, like Shazam.

It is recommended to use the dejavu branch which uses the Dejavu audio fingerprinting library to do recognition. This drastically improves accuracy and speed, and scales much better with the number of songs in the database. It also includes features the master branch doesn't have, for example, automatically logging mistakes.

Installing and Running

To install dependencies, run:

pipenv install

Before the first run, the mp3 database needs to be set up.

A browser driver for Selenium is required, see the Selenium documentation for more information. Common drivers include chromedriver for Google Chrome and geckodriver for Firefox.

The program needs to be able to take in system sound. You can create a loopback device; the exact steps depend on your operating system. For example, in macOS, one program is soundflower.

Once you have the loopback program setup get its device ID with the following command:

python -m sounddevice

Create a JSON file called params.json as follows, where the device ID was found from the previous command:

{
  "IN": DEVICE_ID
}

Finally, run amq.py as follows:

python amq.py

It will prompt you for a username and password. To avoid putting in the login credentials every time, you can store the data in a JSON file login.json as follows (GPG encryption TODO):

{
 "username": "USERNAME",
 "password": "PASSWORD"
}

If everything is working properly, it will then prompt you for a room ID and password. Once that information is given, it will join the room and once the game starts, will begin play.

It will then warn you if the audio it receives is suspiciously quiet --- check that the system output is going to the loopback device properly.

Database

Songs (mp3 files) are stored in the songs folder. Currently, this folder has to be manually created:

mkdir songs

Also manually create the sampled_down cache directory in songs:

mkdir songs/sampled_down

Add *.mp3 files to the songs folder and use the command

python db.py update

to update the database after new songs have been added.

Run python db.py size to see the total number of songs in the database.

Testing

In order to test the system is working properly, clips can be generated by db.py. To generate a sample, run

python db.py clip --length 10

The optional --length flag provides the length of the clip in seconds, defaulting to 10 seconds. The --play flag can be added to the same command to hear the clip, or python db.py play can be run afterwards.

To get the system's best guess as to where the clip came from, run

python main.py