marcomoldovan/hierarchical-language-modeling
We address the task of learning contextualized word, sentence and document representations with a hierarchical language model by stacking Transformer-based encoders on a sentence level and subsequently on a document level and performing masked token prediction.
Jupyter NotebookMIT