Azure Data Factory Essentials For Data Engineers 2024

A Complete Guide to Building and Managing Data Pipelines in Azure – Your Step-by-Step Journey in Azure Data Factory

What you’ll learn
Creating and configuring an Azure account.
Understanding and using Azure Cost Management to monitor and optimize cloud spending
Setting up and connecting to an Azure SQL Database using the Azure portal, and resolving common connection issues.
Using SQL Server Management Studio (SSMS) to interact with Azure SQL databases.
Creating and managing an Azure Storage Account and connecting via Azure Storage Explorer to upload and manage data.
Setting up an Azure Data Factory instance and exploring the ADF user interface.
Creating Linked Services (LS) and Datasets (DS) to connect data sources and destinations.
Building and configuring pipelines to automate the movement of data between Azure services using Copy Activities.
Monitoring and managing pipeline executions to ensure smooth data workflows.

Requirements
Basic understanding of data engineering concepts is helpful but not required.
Familiarity with cloud computing or Azure fundamentals is a plus.
A PC or laptop with internet access to create and manage an Azure account.

Description
This course is a comprehensive introduction to Azure Data Factory (ADF) and essential Azure services, designed to give you practical, hands-on experience in building data pipelines. Whether you’re new to Azure or looking to automate data workflows, this course will equip you with the knowledge and skills needed to move and manage data efficiently.We start by helping you create your own Azure account, providing an overview of Azure Cost Management to control cloud expenses. From there, we dive into setting up an Azure SQL Database, resolving common connection issues, and connecting to it using SQL Server Management Studio (SSMS). You’ll also learn how to create and manage an Azure Storage Account, and upload data through Azure Storage Explorer.The core of the course focuses on Azure Data Factory. You will learn how to create an ADF instance, navigate the user interface, and create Linked Services and Datasets for both your SQL database and Storage Account. Through step-by-step videos, we’ll guide you in building a pipeline to copy data between these services. You’ll also learn to monitor and manage pipeline executions, ensuring smooth data transfers between systems.By the end of the course, you’ll be confident in building and automating data pipelines using Azure Data Factory and integrating other Azure services effectively.

Overview
Section 1: Introduction

Lecture 1 Introduction

Section 2: Basics Of Azure Account

Lecture 2 Create Azure Account

Lecture 3 Azure Cost Management and Budget

Section 3: Basics Of Azure Data Factory

Lecture 4 Sample Project 1

Lecture 5 Create Source Systems

Lecture 6 Access Azure SQL From Azure Portal

Lecture 7 Access Azure SQL Using SSMS

Lecture 8 Create Target Systems

Lecture 9 Access Azure Storage Account Using Storage Explorer

Lecture 10 Create Azure Data Factory Instance

Lecture 11 Azure Data Factory First Time

Lecture 12 Create Linked Services

Lecture 13 Create Data Sets

Lecture 14 Create Pipelines

Lecture 15 Cost Optimization Strategies

Beginners in cloud computing and data engineering who want to learn the basics of Azure Data Factory.,Developers and data engineers looking to automate data workflows using Azure services.,IT professionals seeking hands-on experience with Azure SQL Database, Azure Storage Accounts, and Azure Data Factory.,Students and professionals preparing for certifications in Azure data services.