/dataframe-examples

Apache Spark with Scala and AWS - DataFrame Examples

Primary LanguageScala

Dataframe Examples

  1. Challenges with RDD
  2. RDD to DataFrame conversion
  3. Data ingestion from
    1. RDBMS (MySQL)
    2. NoSQL (MongoDB)
    3. SFTP
    4. Amazon S3 bucket
    5. Amazon Redshift Database
  4. Applying transformation using
    1. DSL (Domain Spaecific Language)
    2. Spark SQL
  5. Window/analytics function (lead(), lag(), rank(), dense_rank(), etc.)