/Captioner

A bilingual en2end RNN-based approach for captioning images in English and Persian

Primary LanguagePython

Captioner

A bilingual end2end RNN-based approach for captioning images in English and Persian

  • Overview

Bilingual_image_captioning (1)

  • Examples

Image English Persian
A woman in black shirt is sitting on the floor and playing the guitar زنی جوان روی زمین نشسته و در حال نواختن گیتار است
A person is riding a brown horse زنی در حال سوارکاری است
Two children are brawling with each other دو کودک در حال دعوا کردن هستند
Someone is putting colorful flowers into a vase فردی در حال گذاشتن گل های رنگارنگ در داخل یک گلدان شیشه ای است

  • Train

After making a torch env do the following:

  1. Download Flicker8k Dataset
  2. By cloning the repo you have en_farsi_captions.txt which includes the following: image_id, English caption, Persian caption
  3. Set the related path variables in train.py module

  • Inference

  1. Set the related path variables in infer.py module

  • ToDos

  • Refactor based on OOP
  • Provide pretrained ckpts

  • Refs

  • A big 👍 for Aladdin Persson tutorial
  • I have used Abadis and selenium for automatically translating English ground-truth captions