Train in 5m your own Neural Network on someone's tweets and tweet them back your awesome predictions!
This project fine-tunes a pre-trained transformer on a user's tweets using HuggingFace.
Training and results are logged on W&B (which is integrated in HuggingFace).
If you just want to test the demo, click on below link and share your predictions on Twitter with #huggingtweets!
To understand how the model works, check huggingtweets.ipynb or use the following link.
My favorite sample is definitely on Andrej Karpathy, start of sentence "I don't like":
I don't like this :) 9:20am: Forget this little low code and preprocessor optimization. Even if it's neat, for top-level projects. 9:27am: Other useful code examples? It's not kind of best code, :) 9:37am: Python drawing bug like crazy, restarts regular web browsing ;) 9:46am: Okay, I don't mind. Maybe I should try that out! I'll investigate it :) 10:00am: I think I should try Shigemitsu's imgur page. Or the minimalist website if you're after 10/10 results :) Also maybe Google ImageNet on "Yelp" instead :) 10:05am: Looking forward to watching it talk!
I had a lot of fun running predictions on other people too!
Lot more research to do:
- test training top layers vs bottom layers to see how it affects learning of lexical field (subject of content) vs word predictions, memorization vs creativity ;
- losses are not the same based on people (Karpathy is the hardest to predict) ;
- data pre-processing can be optimized (padding, end tokens…) ;
- augment text data with adversarial approaches ;
- what about hashtags? #ConvNets #iloveGANs ;
- need to test more models and do some fine-tuning ;
- pre-train on large Twitter dataset of many people and fine-tune on single user!
- try few shots learning as we have very limited data per user (there is only a limited number of writing styles)
- implement a pipeline continuously train the network on new tweets.
Built by Boris Dayma
My main goals with this project are:
- to experiment with how to train, deploy and maintain neural networks in production ;
- to make AI accessible to everyone.
To see how the model works, visit the project repository.
Disclaimer: this project is not to be used to publish any false generated information but to perform research on Natural Language Generation.
- A Step by Step Guide to Tracking Hugging Face Model Performance
- W&B Forum: If you have any questions, reach out to the slack community
I was able to make the first version of this program in just a few days.
It would not have been possible without these people and these open-source tools:
- W&B for the great tracking & visualization tools for ML experiments ;
- Huggingface for providing a great framework for Natural Language Understanding ;
- Tweepy for providing a great API to interact with Twitter (used in the dev notebook) ;
- Chris Van Pelt for hacking with me on the demo ;
- Lavanya Shukla for her great feedback on the demo ;
- Colab for letting people access free GPU!