News-Shift-Apache-Spark
A real time news aggregation system that scrapes news articles from the internet through RSS feeds, classifies them into news categories on an Apache Spark cluster using MLlib and automatically generates hyperlinks to Wikipedia pages of named entities extracted from the news articles. Built using Python, Beautiful Soup, PHP, Apache Spark, PostgreSQL and hosted on AWS.