This repository is for demonstrating the capability to do SQL-based UPDATES, DELETES, and INSERTS directly in the Data Lake using Amazon S3, AWS Glue and Delta Lake.
Generates the schema for the Delta Table
This code takes a Delta Lake table and does an UPSERT operation using Spark SQL.
This code takes a Delta Lake table and does a DELETE operation using Spark SQL.
This code takes a Delta Lake table and does an INSERT operation using Spark SQL.
Full documentation of functions can be found here and a more detailed one with examples can be found here