I'm planning on fine tuning BERT on rap lyrics and calling it Lil BERT. The idea is to make an interactive, mad-lib style poetry generator where the user "asks" for words with [MASK] tokens.
Using 1362 Hip Hop artists metioned on Wikipedia (accessed on 2019-05-22), the lyrics were scraped from Genius. Checkout the Dataset Datasheet for more detailed information.
- Inspiration: Kevin Knight's Poetry Generator
- BERT: Original Paper, Explained, PyTorch Implementation
- Genius API: Client
CPU Env:
conda env create .
. activate GG
ipython kernel install --user --name=GG
python -m spacy download "en"