Lab time: 30 minutes
In this exercise, you will explore data using Spark engine.
Understanding data through data exploration is one of the core challenges faced today by data engineers and data scientists as well. Depending on the data's underlying structure and the specific requirements of the exploration process, different data processing engines will offer varying degrees of performance, complexity, and flexibility.
In this lab, you will play the role of various persona: a data engineer, a business analyst, and a data scientist. The workspace is already set up to focus on some of the core development capabilities of Azure Synapse Analytics.
By the end of this lab, you will have performed a non-exhaustive list of operations that combine the strength of Big Data and Spark Analytics into a single platform.
Once data is properly understood and interpreted, moving it to the various destinations where processing steps occur is the next big task. Any modern data platform must provide a seamless experience for all the typical data wrangling actions like extractions, parsing, joining, standardizing, augmenting, cleansing, consolidating, and filtering.
Azure Synapse Analytics provides two significant categories of features - data flows and data orchestrations (orchestrations covered in this lab). They cover the whole range of needs, from design and development to triggering, execution, and monitoring.
In the world of massively distributed big data processing platforms, Spark is almost an implicit choice. Azure Synapse Analytics provides Apache Spark to support data science workloads. They enable data scientists to perform all their specific tasks while benefiting from the integration with the other Synapse services.