DP-203: Data Engineering on Microsoft Azure

Course Description

Build scalable, enterprise-grade data pipelines and analytics platforms with Data Engineering on Microsoft Azure (DP203). In this comprehensive, instructor-led four-day course, you’ll architect and implement modern data storage, processing, and integration solutions using Azure services like Azure Data Lake Storage Gen2, Azure Synapse Analytics, Azure Databricks, Azure Data Factory, Stream Analytics, and Event Hubs. You’ll design data warehouses, build batch and streaming pipelines, optimize query performance, and apply robust security, monitoring, and governance—preparing you for real-world enterprise projects and the Azure Data Engineer Associate certification.

Target Audience

Ideal for:

• Data Engineers, Data Architects, and BI Professionals designing and managing analytics solutions on Azure

• Data Analysts and Data Scientists looking to enhance their expertise in data engineering and pipeline optimization

Prerequisites:

• Proficiency in SQL and programming languages like Python, Scala, or Spark SQL

• Familiarity with data formats (e.g., CSV, JSON, Parquet) and cloud concepts; experience with Azure is a plus 

Course Outline

Module1: Explore Compute and Storage for Data Engineering on Azure

• Understand compute/storage selection—Synapse pools, Databricks, and Data Lake Gen2

• Design data lake zones, refine file formats, and optimize data layouts

Module2: Build Lakehouse and Serverless Analytics with Synapse

• Query data files with Synapse serverless SQL pools using T-SQL

• Implement data transformations via CREATE EXTERNAL TABLE AS SELECT (CETAS) and pipelines

Module3: Perform Data Engineering with Spark in Synapse

• Explore and transform data using Apache Spark DataFrame APIs

• Leverage Delta Lake capabilities for reliable, ACID-compliant processing

Module4: Load and Transform Data in Synapse Pipelines

• Use Synapse pipeline orchestration and Spark notebook activities

• Build modular, parameterized ETL/ELT workflows

Module5: Implement Data Warehousing with Dedicated SQL Pools

• Design star schemas, load data using PolyBase, COPY, and CTAS

• Monitor and optimize performance across relational pools

Module6: Design HTAP Architectures with Synapse Link

• Implement Hybrid Transactional Analytical Processing (HTAP) using Synapse Link for Cosmos DB and SQL

• Enable real-time insights with operational data integration

Module7: Stream Real-Time Data with Stream Analytics & Event Hubs

• Ingest streaming data from devices and services

• Perform real-time aggregation, windowing, and load insights into Synapse or Power BI

Module8: Build Data Lakehouse Pipelines with Databricks

• Use Azure Databricks for large-scale ETL and streaming with Spark

• Implement Delta Live Tables, notebooks integration, and workflow orchestration

Module9: Secure, Monitor, and Govern Data Solutions

• Implement enterprise data governance using Azure Purview metadata and cataloging

• Secure data at rest/in transit, use Synapse security features—network, row- and column-level

• Monitor pipeline performance, SLAs, and lineage

HandsOn Experience

40–50% of class time is practical. You’ll design and build data lakehouses, orchestrate batch/stream pipelines, optimize warehousing, secure data environments, and monitor production systems—all driven by real-world scenarios.

Skills You’ll Gain

By the end of DP203, you will be able to:

• Architect scalable data solutions using Azure Synapse, Databricks, and ADLS Gen2

• Design efficient ETL/ELT pipelines with Synapse pipelines, Spark, Databricks, and Stream Analytics

• Build data warehouses, implement HTAP, and support real-time intelligence

• Apply enterprise-grade security, monitoring, and data governance