/Spark-frequency

Double word frequency,Word-count

Primary LanguagePython

Spark-frequency

Double word frequency,Word-count

Wordcount.py : Count one-word frequency as in the wordcount example.

Wordcount2.py : Count double-word frequency. For example, ”I am given an opportunity to use EC2 so I am very happy and very happy and very happy.” The output would be something as follows (in any reasonable order): I am 2, am given 1, given an 1, an opportunity 1, opportunity to 1, to use 1, use EC2 1, EC2 so 1, so I 1, am very 1, very happy 3, happy and 2, andy very 2

Wordcount3.py : Find frequency of one-words in another given list. It is to use the given list to find one-word frequency in above data set. For example, small list has ”hello world gator”. Then you need to find frequency of ”hello”, ”world”, and ”gator” in given dataset.