/llid_proto

Language Learning Image Description Feedback

Primary LanguagePython

llid_proto ContributorsAugSepOctNovDecJanFebMarAprMayJunJulAugSunTueThuSat

LLID Feedback Prototype

This repository contains the automatic assessment system for Image Description by Language Learners.

Overview

We focus on image description and a corresponding assessment system for language learners. To achieve automatic assessment of image description, we construct a novel dataset, the Language Learner Image Description (LLID) dataset, which consists of images, their descriptions, and assessment annotations. Then, we propose a novel task of automatic error correction for image description, and we develop a baseline model that encodes multimodal information from a learner sentence with an image and accurately decodes a corrected sentence. Our experimental results show that the developed model can revise errors that cannot be revised without an image.

Usage

The easiest way to use LLID Feedback and its dependencies is using docker.

docker-compose build
docker-compose up

Pretrained Model and Image Features