/gmu-mmimdb

Source code for training Gated Multimodal Units on MM-IMDb dataset

Primary LanguagePythonMIT LicenseMIT

Source code for Gated Multimodal Units for Information Fusion.

GMU model

Dependencies

Make dataset

You can download the ready-to-use Fuel format version: multimodal_imdb.hdf5 (archive.org mirror) and metadata.npy (archive.org mirror).

Alternatively, you can build it manually:

Getting more movies

You can extend the dataset by adding more IMDb IDs to the links.csv file and run get_data.py script to crawl other movies.

Train and eval the model

Generate random configurations:

python3 generators/gmu.py gmu.json

Train the model and then report performance in test set (e.g. best conf for GMU model #23):

python3 run.py json/gmu_23.json