kirbiyik/caption-it

Image Captioning Web Application with PyTorch and Flask - Implementation of "Show and Tell: A Neural Image Caption Generator"

caption-it

NOTE: Deployment of ML models has been evolved a lot. At the time this projects is created, there were no(or very little) guidelines for deploying a Pytorch model properly. The methods might be outdated. Use this project only conceptually.

Image Captioning Web Application

This project is the implementation of the paper Show and Tell: A Neural Image Caption Generator

Please visit below to see deployed web app.

http://omerkirbiyik.com/caption-it

Training

• Clone repository
• Download dataset with src/download_dataset.sh
• Preprocess with src/build_vocab.py and src/resize.py
• Train model giving appropriate arguments with src/train.py

Prediction

• Make a nginx configuration to run Flask app. This part can be tricky. See this link for further.

References

• https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-16-04
• https://github.com/yunjey/pytorch-tutorial/tree/master/tutorials/03-advanced/image_captioning
• http://flask.pocoo.org/docs/1.0/deploying/
• https://arxiv.org/pdf/1411.4555.pdf