This guide is meant to help you run a data engineering team. By data engineering team, I mean any team that has at least one data engineer on board. I know that’s vague, but that’s the nature of the game, whether you call it analytics, data science, or data platform.
Data engineering teams cover lots of different kinds of functions to very different degrees:
- Data engineering at its core - building and running complete data pipelines
- Analytics engineering - transforming data (often using SQL)
- Platform engineering - running data orchestrators BI tools and providing data products to other teams
- Analytics - providing insights, dashboards, and reports to decision makers.
This guide tries to cover most of these functions and, in addition, all metafunctions.
CONTRIBUTE: FEEL FREE TO CONTRIBUTE GREAT RESOURCES. You can contribute links to podcasts, videos, books, tools, and blog posts. Just make sure to find the right category. There are still a ton of gaps, so please help me update this guide!
- Approaches to build the right data engineering team [free, video]
- How do you build great data teams? What is the best advice or lessons you have learnt? [free, Reddit discussion]
- How to build a modern data team: structure, skill sets, and common mistakes [free, article]
- Data Demystified, Part 4: Building an efficient data team [free, article]
- Building Data Engineering Teams | Datadog [free, video]
- How to Build an On-Call Culture in a Data Engineering Team [free, article]
- Data Science for Business - Provost & Fawcett [book, paid]
- The Data Engineering Handbook [free, GitHub]
- Fundamentals of Data Engineering: Plan and Build Robust Data Systems [paid, book]
- # The dbt Reading Guide [free, guide]
- # The Data Engineers Reading Guide [free, guide]
- Organizing data teams - where to make the cut [free, article]
- Hello Product Data Team, Goodbye Ad Hoc Work [free, article]
- Product Data Teams 101 [free, article]
- How should our company structure our data team? [free, medium article]
- Building The Modern Data Team [free, article]
- Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric [paid, book]
- Locally Optimistic [free, blog]
- Locally Optimistic Slack Community [free, slack]
- The Data PM Gazette [free, newsletter]
- Data Gibberish [free, newsletter]
- The Joe Reis Show [free, podcast]
- Product Discovery for Analytics Teams - Talking To Internal Customers [free, email course]
- Team Topologies: Organizing Business and Technology Teams for Fast Flow [paid, book]
- How Mercateo is Rolling Out a Modern Data Platform [free, article]
- Building a Data Platform in 2024 [free, article]
- Thinnest Viable Platform (TVP) Example using a Data Platform [free, GitHub]
- Managing Analytics Teams With Nate Sooter [free, video]
- What makes a data analyst excellent? [free, medium article]
- 5 Ways To Ensure High Functioning Data Engineering Teams [free, article]
- High-Performance Data Teams Don’t Care About Data Quality [free, medium article]
- How The Modern Data Stack Is Reshaping Data Engineering [free, article]
- Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems [paid, book]
- Data Mesh: Delivering Data-Driven Value at Scale [paid, book]
- Data Mesh in Action [paid, book]
- What Companies REALLY Want in an Analytics Engineer [free, medium article]
- Why Data Product Management is Different — 3 Important Lessons Applied [free, medium article]
- Scaling data teams
- leadership resources (focused on actually leading, e.g. people management)
- more video content
- more on data product management (as long as relevant!)
- ...