marcomoldovan/hierarchical-language-modeling

We address the task of learning contextualized word, sentence and document representations with a hierarchical language model by stacking Transformer-based encoders on a sentence level and subsequently on a document level and performing masked token prediction.

Jupyter NotebookMIT