Learning to hash-tag videos with Tag2Vec

The project contains the resources for Tag2Vec project. The repository contains the link to the resources for ~3 million hashtagged vines, 3000 train-test vines with their associated hashtags. Apart from it the steps also describe how to use the Improved Dense Trajectories code to generate video descriptors.

Currently working on integrating zslearning and other required libraries into the docker

Citing

@inproceedings{Singh:2016:LHV:3009977.3010035,
 author = {Singh, Aditya and Saini, Saurabh and Shah, Rajvi and Narayanan, P J},
 title = {Learning to Hash-tag Videos with Tag2Vec},
 booktitle = {Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing},
 series = {ICVGIP '16},
 year = {2016},
 keywords = {Tag2Vec, hash-tag recommendation, video tagging},
}

Resources

How To Run

Improved Dense Trajectories(iDT)

It's a fairly old project and might not run with the latest version of OpenCV or Ubuntu. I have created a Docker image using Ubuntu14.04 as a base and built openCV 2.4.13 and iDT.

To install Docker you can follow the instructions provided here or any other resource on the net.
Execute the run command. docker run -it aditya27singh/improved_dense_trajectory:success
Navigate to /home/aditya/iiit. This will be reffered to as the $ROOT_DIR
Following the command described on the author's README one can generate features for the videos. $ROOT_DIR/iDT/release/DenseTrackStab video_name.vob -H video_name.bb > video_name.ft

Embedding Learning

The work utilized the Zero-Shot learning to obtain the mapping from Video to Word descriptors

Tag2Vec

Note Requires precomputed iDTs Preprocessing

Remove duplicate videos. Based on md5 hash one can remove duplicate video data
Process Hashtags and remove data which doesn't contain #CLASSNAME
Remove Hashtags based on TF-IDF and lowest in-class frequency

paganpasta/Tag2Vec