/paraphraser

By utilizing the developer's clean and simple API, users can use this project to perform sentence parsing operations. Visit this page to see an example of this in action. pair-a-phrase It was created as part of the third-year artificial intelligence program at Insight Data Science. Model The model is built on top of a bidirectional LSTM encoder and decoder with attention trained in Tensorflow. Click on the following link to go to the download page: model This project requires Python 3.5 or Tensorflow to be completed successfully. Spherical Inference is available in the library's 1.4.1 version. Immediately after obtaining the model checkpoint via the link provided above, run the following Python inference. Python We combined a large number of publicly available datasets into a single large dataset to generate the training data for this model. Para-nmt-5m and para-nmt-6m are two examples. You will be asked to respond to two Quora questions today. The Semeval National Labor Institute (SNLI) is abbreviated as "Semeval National Labor Institute." In addition, for security reasons, I have removed the aggregated dataset from this repository. If you have any questions or would like to learn more about the program, please do not hesitate to contact me. The pre-trained embeddings used in this study were provided by John Wieting's para-nmt-50m project. Training: The training was carried out on an Nvidia GTX 1080, and the results were evaluated using the BLEU score. On the GPU, two training epochs were completed. The training curves of the tensorboard are depicted in the illustration below. The grey curve represents the train, while the orange curve represents the development.  You should broaden the scope of your investigation by delving deeper. This is demonstrated by the disappearance of a recurring layer. A variety of additions have been made to the data set. It is necessary to consider the residual layer. Model compression modifications Byte pair encoding is used when dealing with non-standard vocabulary.

Primary LanguagePythonMIT LicenseMIT

📢 YOUR SUPPORT IS GREATLY APPRECIATED / PATREON.COM/DONPABLONOW / BTC 3HVNOVVMLHVEWQLSCCQX9DUA26P5PRCTNQ / ETH 0X3D288C7A673501294C9289C6FC42480A2EA61417 🙏

⛔️ ARCHIVE PENDING: This endeavour is likely to fail owing to a lack of support. If you find this project interesting, please support it by smashing the "star" button. If the project receives at some interest work on the project will continue.




Paraphraser

This project providers users the ability to do paraphrase generation for sentences through a clean and simple API. A demo can be seen here: pair-a-phrase

The paraphraser was developed under the Insight Data Science Artificial Intelligence program.

Model

The underlying model is a bidirectional LSTM encoder and LSTM decoder with attention trained using Tensorflow. Downloadable link here: paraphrase model

Prerequisiteis

  • python 3.5
  • Tensorflow 1.4.1
  • spacy

Inference Execution

Download the model checkpoint from the link above and run:

python inference.py --checkpoint=<checkpoint_path/model-171856>

Datasets

The dataset used to train this model is an aggregation of many different public datasets. To name a few:

  • para-nmt-5m
  • Quora question pair
  • SNLI
  • Semeval
  • And more!

I have not included the aggregated dataset as part of this repo. If you're curious and would like to know more, contact me. Pretrained embeddings come from John Wieting's para-nmt-50m project.

Training

Training was done for 2 epochs on a Nvidia GTX 1080 and evaluted on the BLEU score. The Tensorboard training curves can be seen below. The grey curve is train and the orange curve is dev.

TODOs

  • pip installable package
  • Explore deeper number of layers
  • Recurrent layer dropout
  • Greater dataset augmentation
  • Try residual layer
  • Model compression
  • Byte pair encoding for out of set vocabulary

Citations

@inproceedings { wieting-17-millions, 
    author = {John Wieting and Kevin Gimpel}, 
    title = {Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations}, 
    booktitle = {arXiv preprint arXiv:1711.05732}, year = {2017} 
}

@inproceedings { wieting-17-backtrans, 
    author = {John Wieting, Jonathan Mallinson, and Kevin Gimpel}, 
    title = {Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext}, 
    booktitle = {Proceedings of Empirical Methods in Natural Language Processing}, 
    year = {2017} 
}