How to execute scrapper ?
pip install scrapy bs4
scrapy runspider ./scrapper/physics_scrapper_uncategorized.py -o uncategorized_corpus.csv -t csv
scrapy runspider ./scrapper/physics_scrapper_categorized.py -o categorized_corpus.csv -t csv
To process scrapped content run the following script
python ./text_processing/processor.py