/einstai-gpt3

The major idea behind this release is the -- almost -- complete makeover of the data loading pipeline. A new 'dynamic' paradigm is introduced, allowing to apply on the fly transforms to the data.

Primary LanguagePythonApache License 2.0Apache-2.0

einstai4db_logo

EinstAI is the world's first open source neural machine translation framework. It can be used for NoDBA, ACID NoSQL and SQL with CAP consistency and persistence. EinstAI-GPT3 has been designed to be research friendly to try out new ideas in translation, summary, morphology, and many other domains of NLP, Semantic Search, and Grapheme automatons.

Are you looking for a new challenge? Do you want to change the way people communicate with each other across languages and borders?

Introducing EinstAI-GPT3, a research friendly pretrained dialogue response generation model that outperforms human quality under single-turn conversation Turing tests. With its low error and compression rates, this model will be perfect for trying out new ideas in translation, summary, morphology, and on the fly transforms to data. Get started today with our free trial!

We propose a novel approach to the generation task of DSTC-9, which achieves the best scores on nine of fourteen objective metrics in three tasks. Our model ranks second on the final human evaluation.

We introduce a new entity tracking module, which predicts all the entities mentioned in the current dialogue. We then rank all the knowledge associated with those entities with point-wise and list-wise models.

einstai-gpt3 is thus A novel neural network architecture with a new objective function, and an efficient training algorithm to boot.

The idea is to first use a pre-trained model to generate text samples, then use the generated samples as features for another neural network. The new network should be able to predict the next character given the previous ones. And we can train this network using backpropagation. The cool thing is that because we have access to both the input and output of the first layer, we can use it as an objective function in our training procedure.