In this track, you’ll discover how to build an effective data architecture, streamline data processing, and maintain large-scale data systems. In addition to working with Python, you’ll also grow your language skills as you work with Shell, SQL, and Scala, to create data engineering pipelines, automate common file system tasks, and build a high-performance database.
Through hands-on exercises, you’ll add cloud and big data tools such as AWS Boto, PySpark, Spark SQL, and MongoDB, to your data engineering toolkit to help you create and query databases, wrangle data, and configure schedules to run your pipelines. By the end of this track, you’ll have mastered the critical database, scripting, and process skills you need to progress your career.
Please note this track assumes a fundamental knowledge of Python and SQL.
Discover how data engineers lay the groundwork that makes data science possible. No coding involved!
Skill Assessment Python Programming Advanced Score: 154 | Percentile: 96%
3 Introduction to Data Engineering Learn about the world of data engineering with an overview of all its relevant topics and tools!
4 Streamlined Data Ingestion with pandas Learn to acquire data from common file formats and systems such as CSV files, spreadsheets, JSON, SQL databases, and ...
Learn to write efficient code that executes quickly and allocates resources skillfully to avoid unnecessary overhead.
Learn to use best practices to write maintainable, reusable, complex functions with good documentation.
The Unix command line helps users combine existing programs in new ways, automate repetitive tasks, and run programs ...
Learn powerful command-line skills to download, process, and transform data, including machine learning pipeline.
9 Introduction to Bash Scripting Bash scripting allows you to build analytics pipelines in the cloud and work with data stored across multiple files.
10 Unit Testing for Data Science in Python Learn how to write unit tests for your Data Science projects in Python using pytest.
Dive in and learn how to create classes and leverage inheritance and polymorphism to reuse and optimize code.
12 Introduction to Airflow in Python Learn how to to implement and schedule data engineering workflows.
13 Introduction to PySpark Learn to implement distributed data management and machine learning in Spark using the PySpark package.
14 Building Data Engineering Pipelines in Python Learn how to build data engineering pipelines in Python.
15 Introduction to AWS Boto in Python Learn about AWS Boto and harnessing cloud technology to optimize your data workflow.
16 Test your newly acquired skills... Skill Assessment Data Analysis in SQL (PostgreSQL)
17 Introduction to Relational Databases in SQL Learn how to create one of the most efficient ways of storing data - relational databases!
Learn to design databases in SQL.
19 Introduction to Scala Begin your journey with Scala, a popular language for scalable applications and data engineering infrastructure.
20 Big Data Fundamentals with PySpark Learn the fundamentals of working with big data with PySpark.
21 Cleaning Data with PySpark Learn how to clean data with Apache Spark in Python.
22 Introduction to Spark SQL in Python Learn how to manipulate data and create machine learning feature sets in Spark using SQL in Python.
23 Cleaning Data in SQL Server Databases Develop the skills you need to clean raw data and transform it into accurate insights.
24 Transactions and Error Handling in SQL Server Learn to write scripts that will catch and handle errors and control for multiple operations happening at once.
25 Building and Optimizing Triggers in SQL Server Learn how to design and implement triggers in SQL Server using real-world examples.
26 Improving Query Performance in SQL Server In this course, students will learn to write queries that are both efficient and easy to read and understand.
27 Introduction to MongoDB in Python Learn to manipulate and analyze flexibly structured data with MongoDB.
Statement of Accomplishment