/IE-System

Image Editing Dialogue System

Primary LanguageJupyter Notebook

Conversational Image Editing

Getting Started

To start the system website, you will need

  1. Conversational Image Editing(this repo)
  2. EditmeTagger(NLU)
  3. MongoDB(for session and logging purposes)

Installation

Create a conda environment and setup with the necessary packages

# Creates a conda environment named "cie" in python3.5
conda create -n cie python=3.5
conda activate cie
pip install --uprade pip
pip install -r requirements.txt
conda install -c menpo opencv3 
sudo apt-get update
sudo apt-get install libgtk2.0-dev
python
import nltk
nltk.download('stopwords')
exit()

Start service

After starting EditmeTagger and MongoDB on localhost (sudo service mongod start), run the following command to start the service. Current config uses Rule-based Policy. Note: If you plan to use EC2 instance, create a security group that allows all incoming traffic to TCP port 2000.

./realuser_server.sh

TODO

  • Evaluation

    • Set maximum number of turns, or execution
    • Customize goal index instead of random sampling
    • Record selected policy

  • User Interface

    • Debug gesture_click on object location image
    • object_mask_str user input should remove system queries
    • Display turn count

  • Photoshop

    • Optimize image processing, this is the current bottleneck