SCE-Development/Skylab-pipeline

Discord ETL (lambda)

jerrylee17 opened this issue · 1 comments

Create an AWS Lambda function that takes raw data from discord and outputs it as the format in the Data Warehouse

INPUT: Massive JSON file
OUTPUT: Fields we want from the JSON file

The main difference between this and the Core-v4 JSON is that the Discord ETL will require some string parsing rather than just selecting fields.

For raw data info, see here