/spark-and-python-for-big-data-with-pyspark

Course on Udemy by Jose Portilla

Primary LanguageJupyter Notebook

Python-and-Spark-for-Big-Data

Course Notebooks for Python and Spark for Big Data

Course Outline:

  • Course Introduction

    • Promo/Intro Video
    • Course Curriculum Overview
    • Introduction to Spark, RDDs, and Spark 2.0

  • Course Set-up

    • Set-up Overview
    • EC2 Installation Guide
    • Local Installation Guide with VirtualBox
    • Databricks Notebooks
    • Unix Command Line Basics and Jupyter Notebook Overview

  • Spark DataFrames

    • Spark DataFrames Section Introduction
    • Spark DataFrame Basics
    • Spark DataFrame Operations
    • Groupby and Aggregate Functions
    • Missing Data
    • Dates and Timestamps

  • Spark DataFrame Project

    • DataFrame Project Exercise
    • DataFrame Project Exercise Solutions

  • Machine Learning

    • Introduction to Machine Learning and ISLR
    • Machine Learning with Spark and Python and MLlib
    • Consulting Project Approach Overview

  • Linear Regression

    • Introduction to Linear Regression
    • Discussion on Data Transformations
    • Linear Regression with PySpark Example (Car Data)
    • Linear Regression Consulting Project (Housing Data)
    • Linear Regression Consulting Project Solution

  • Logistic Regression

    • Introduction to Logisitic Regression
    • Logistic Regression Example
    • Logistic Regression Consulting Project (Customer Churn)
    • Logistic Regression Consluting Project Solution

  • Tree Methods

    • Introduction to Tree Methods
    • Decision Tree and Random Forest Example
    • Random Forest Classification Consulting Project - Dog Food Data
    • RF Classification Consulting Project Solutions
    • RF Regression Project - (Facebook Data)

  • Clustering

    • Introduction to K-means Clustering
    • Clustering Example - Iris Dataset
    • Clustering Consulting Project - Customer Segmentation (Fake Data)
    • Clustering Consulting Project Solutions

  • Recommender System

    • Introduction to Recommender Systems and Collaborative Filtering
    • Code Along Project - MovieLens Dataset
    • Possible Consulting Project ? Company Service Reviews

  • Natural Language Processing

    • Introduction to Project/NLP/Naive Bayes Model
    • What are pipelines?
    • Code Along

  • Spark Streaming

    • Introduction to Spark Streaming
    • Spark Streaming Code-along!