/teaspn-server

A sample implementation of the TEASPN server

Primary LanguagePython

TEASPN Server

Overview

This is a sample implementation of the TEASPN server supporting neural auto-completion and paraphrasing.

Prerequisites

Install

  1. Build an image from the Dockerfile and pull images from the registry. Note that this may take a while (10-30 mins) depending on your environment.

    docker-compose build
docker-compose pull

  2. Download and build the Elasticsearch index. This also takes a while.

    mkdir elasticsearch_indexes
curl -sLJ --output elasticsearch_indexes/tatoeba_ja.index 'https://teaspn.s3.amazonaws.com/server/0.0.1/assets/tatoeba_ja.index'

docker-compose up -d elasticsearch
docker run --net=host --rm -ti -v $PWD/elasticsearch_indexes:/tmp taskrabbit/elasticsearch-dump --input=/tmp/tatoeba_ja.index --output=http://localhost:9200/tatoeba_ja

docker-compose stop

  3. Set the PATH environment variable.

    echo "export PATH='$PWD:$PATH'" >> ~/.bash_profile
    • If you are using zsh, modify your ~/.zshenv file instead of ~/.bash_profile.

Features

Syntax highlighting

Powered by the spaCy dependency parser.

Head tokens with specific dependency (ROOT, nsubj, nsubjpass, and dobj in the CLEAR style tag set) relation are highlighted in different colors.

Grammatical Error Detection & Grammatical Error Correction

Powered by LanguageTool and its python wrapper

GED GEC Demo

Completion

This implementation provides two types of completion:

  1. Suggesting the likely next phrases given the context using DistilGPT2 developed by HuggingFace.
  2. Suggesting a set of words consistent with the characters being typed.

Completion Demo

Text Rewriting

This provides paraphrase suggestions for the selected text.

We built a paraphrase model trained on PARANMT-50M using fairseq.

Paraphrasing Demo

Example Search

Provide full-text search feature using Tatoeba and Elasticsearch. Currently, this only supports Japanese-to-English search.

Search Demo

Reference Jump

This lets you jupm from a selected expression to its antecedent. Powered by NeuralCoref.

Jump Demo

Mouse Hover

Show the definition of a hovered word using WordNet.

Hover Demo