Memory Box
Demo (click on GIF for the full video) :
Inspiration
Wished for a dropbox which works with videos! Well wait no more!! Search moments that you relish, you want to share!
What it does
The app looks through a library of videos, samples frames every few seconds, and generates natural language captions using a CNN+LSTM for each video frame, written in PyTorch. Next, all the captions are indexed using lucene and users can search this index using a web interface.
How we built it
We will explain in the presentation
Challenges we ran into
Setting up the infrastructure for video feature extraction. Setting up the indexing and searching aspects.
Accomplishments that we're proud of
What we learned
Bunch of technologies like Lucene, PyTorch, LSTM and CNN.
What's next for MemoryBox
Scaling to 1000s of videos database