This is a sample program to test speed and memory usage for different approaches for n-gram creation in Python
make venv
make run
This command will do the following:
- Create a file of random length lorem ipsum sentences in one sentence per line (SPL) format
- Tests character and word 3-grams and 5-grams and print the output
See results in results.txt
This program could not be created without the joint efforts of Iván Mittelholcz, Bálint Sass, Bence Nyéki and Balázs Indig
This code is licensed under the MIT license