koaning/whatlies

Add context to vocab based language models

koaning opened this issue · 1 comments

koaning commented

We could add one layer of self-attention to all the vocabulary-based models. That way there'd be context possible and we could have a unified API for [bank] of the river.

koaning commented

This idea totally does not work.