General idea behind the Pic2Story
Pic2Story is the idea of an NLP tool which, given a set of images, provides the user with generated text - a type of story inspired by the inputted pictures.
The emphasis is put on the latter part of the project; therefore, the image object recognition aspect is not to be considered the core of the application (YOLO has been utilized).
The main points of interest are:
- creating an English story-based corpus,
- developing a tool for automatic text generation,
- utilizing the text-generation tool based on the corpus, alongside the annotated images with the detected object.
To be considered in further stages:
- custom object detection,
- deployment of the app as a web application to allow accessability for the online users.
Install Pic2Story
Download the source code from github:
git clone https://github.com/amillert/pic2story.gitRunning
In order to properly run all the commands one must ensure that the current directory is pic2story/:
cd pic2story/Install YOLO
The script installs YOLO and downloads required weights for the model
./data/yolo_objects/install-YOLO.shDownload Textual Books
The dataset comes from the Project Gutenberg's website. In order to download the data and unzip recursively located *.txt files, one should run the follwoing command:
./data/gutten/getgutenbergdata.shThe following script will first download the data, then run unzipping python script.
Create and Activate Conda Virtual Environment
conda env create -f pic2story.yml; conda activate pic2storyScript's Help
In order to find out about the arguements required to successfully run the scrip, one should run:
./scr/main.py -hRun Pic2Story
In order to run the whole application with sample parameters, one should run:
./run.sh