This Chapter will help us to understand :
- How to decide the best option for Data Pipeline orchestation on AWS between MWAA(Airflow) & AWS Step Functions ?
- DEMO - 1
- Study case about - How to extract data from Youtube Channel and orchestrate it with AWS Step functions
- Data Stats Channel
- Data Video Features
For our Step Function state machine, we are going to start by using a Lambda function that checks the extension of an incoming file to determine the type of file. Once determined, we'll pass that information on to the next state, which is a CHOICE state. If it is a file type we support, we'll call a Lambda function to process the file, but if it's not, we'll send out a notification, indicating that we cannot process the file.
If the Lambda function fails, we'll send a notification to report on the failure; otherwise, we will end the state machine with a SUCCESS status.
- Go to Data 👉 Stats Channel
- Go to Data 👉 Video Featuresl