/fcup-bdcc

Projects developed for 'Big Data & Cloud Computing', a first year subject of the Master's Degree in Network and Information Systems Engineering @ FCUP.

Primary LanguagePython

fcup-bdcc

Projects developed for 'Big Data & Cloud Computing', a first year subject of the Master's Degree in Network and Information Systems Engineering @FCUP.

Projects

Projects Theme Report Folder Grade
#1 Write Python functions involving these datasets and the use of the TF-IDF and Jaccard Index. R1 F1 19
#2 Write Pyhon classes capable of manipulate datasets and output human knowledge, concerning entries on an hospital, as well as a method of predicting next patients waiting time. R2 F2 19

Assignments

Assignments Assignment questions Grade
#1 Aq1 20
#2 Aq2 18

Summary Content

While studying for the exam, a Summary was developed containing the lectured material. This includes an introduction to:

  • Cloud Computing – Architecture and Services
  • MapReduce
  • PySpark - RDDs
  • Spark data frames
  • Data partitioning and persistence in Spark
  • HDFS
  • YARN.

The Summary can be checked here.

Author