/Azure-Data-Integration-DevOps-Training

Training workshop content on building CI/CD within Azure DevOps for Azure Data Factory & Azure Synapse Analytics

BSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

Azure-Data-Integration-DevOps-Training

Kamil Nowinski

Prerequisites

You have at least some basic knowledge and experience with these products:

  • Azure Data Factory
  • Synapse Analytics
  • Azure DevOps

You don't need to have experience with PowerShell - basic knowledge of any other scripting language would be suffice.

Workshop Agenda

  • Module 1: Introduction & tooling
    • Environments & isolation
    • Azure DevOps - overview (repo)
    • Visual Studio Code
    • PowerShell

  • Module 2: Azure Data Factory - Deployment Patterns
    • Two deployment approaches
    • Branching strategy
    • Lab: PowerShell - install modules
    • Lab: #adftools

  • Module 3: ADF - Azure DevOps: Build Pipeline
    • Theory
    • Lab: npm library from Microsoft (ARM)
    • Lab: Build with adftools
    • Lab: Validate with adftools

  • Module 4: ADF - Azure DevOps: Release Pipeline
    • Theory
    • Lab: Deploy using ARM Template
    • Lab: Deploy using adftools
    • Lab: Parametrisation

  • Module 5: ADF - Infrastructure as a Code
    • Infra scope for ADF
    • Terraform vs ARM vs BICEP
    • Lab: Set up Build & Release pipeline

  • Module 6: ADF - Post-deployment test
    • Testing connections
    • NUnit Test
    • Testing with Pester

  • Module 7: ADF - Self-Hosted Integration Runtime
    • Manual installation
    • Automated deployment

  • Module 8: Synapse Analytics - deployment Patterns
    • Synapse Services Review
    • Lab: Set up environment
    • Infra scope for Synapse
    • Lab: Infra deployment

  • Module 9: Synapse - Azure DevOps: Build Pipeline
    • Lab: Build CI with YAML Pipeline
    • Lab: Parametrisation

  • Module 10: Synapse - Azure DevOps: Release Pipeline
    • Lab: CD with YAML Pipeline

  • Module 11: Synapse - databases deployment
    • SSDT for Dedicated SQL Pool
    • Deployment for SQL Serverless Pool

  • Module 12: Synapse - Testing
    • Testing connections
    • Testing notebooks
    • Conclusions

About Me

Blogger, speaker, #sqlfamily member. Microsoft Data Platform MVP. Data passionate, Data Engineer and Architect.
Over 20 years of programming and experience with SQL Server databases (since 2000 version) he confirmed by certificates MCITP, MCP, MCTS, MCSA, MCSE Data Platform & Data management & analytics. He worked both as developer and administrator big databases designing systems from the scratch. Recently focused on Data Platform in Azure as a certified (Azure Dev-Ops Engineer Expert, Azure Developer Associate) Data Engineer and Azure Architect.
Passionate about optimization of database systems, an advocate of code transparency, open-source projects and automation, DevOps and PowerShell fan.

Since 2015 he has been living and working in the UK. Currently professionally associated with Avanade, an international consulting company.

For many years tied with Data Community Poland (former PLSSUG), between 2012-2018 acted as a Member of the Audit Committee. He worked a couple of years as a volunteer and now as a co-organizer and speaker of the biggest SQL Server conference in Poland (SQLDay).

An originator of the "Ask SQL Family" podcast and founder of AzurePlayer blog.
Privately happy husband and father of two wonderful girls.

Contact: https://azureplayer.net/kamil-nowinski/