KangSooHan/S2VT_tensorflow

S2VT_tensorflow

Environment

• Python3.6
• Tensorflow 1.8.0

Directory Tree

S2VT_tensorflow

How To Run

1. Download MSVD dataset in data/youtube_videos

   Dataset

2. Extract Video Features

   $ python extract_RGB_feature.py

3. Train model

   • run ipython

     $ CUDA_VISIBLE_DEVICES=0 ipython

   >>> import model_RGB
   
   >>> model_RGB.train()

4. Test model

   • run ipython

     $ CUDA_VISIBLE_DEVICES=0 ipython

   >>> import model_RGB
   
   >>> model_RGB.test()

   You can change save model

5. Evaluate with COCO

   • move S2VT_Description.txt, S2VT_results.txt to ./caption-eval

     mv S2VT_Description.txt, S2VT_results.txt to ./caption-eval
     
     $python create_json_references.py -i S2VT_Description.txt -o S2VT_Description.json
     
     $python run_evaluations.py -i S2VT_results.txt -r S2VT_Description.json

References

S2VT model by chenxinpeng

Vgg model by AlonDaks/tsa-kaggle