A.B.C.D Apache Airflow 2.3.x – Helm | EKS | Minikube

What you’ll learn
Create your own Helm Chart
Deploy Airflow on AWS EKS
Create a Scalable Cluster
Create Pipeline for Deployment
Configure airflow using templates

Requirements
Little Knowledge around Shell/ Python Scripting
Basic Knowledge about Kubernetes
Basic Knowledge about Airflow Components

Description
In this course, you will learn how to

Create a Script to seamlessly deploy Airflow for Local Development

Create your own Helm Chart

Template Scripts and YAML files

Configure Airflow using Helm

Create a Scalable EKS Cluster using eksctl

Deploy an ALB Ingress Controller for Load Balancing and accessing the Airflow UI

Mounting EFS for Persisting Kubernetes Executor Logs

Creating a Pipeline to Deploy Airflow using AWS Code Pipeline

Prerequisites

Basic Knowledge about Airflow Components

Basic Knowledge about Kubernetes

Basic Knowledge about AWS

Familiar with working on an IDE

Basic knowledge of Python

Dependencies: Linux OS, Windows, AWS Account (For Non-Local Deployment )

This course is lined up with the Production Guide of Apache Airflow to deploy a Highly Scalable Airflow on EKS.

Who this course is for

If you are a DevOps Engineer and want to know the technical dependencies for deploying Airflow such as using EFS for using a Kubernetes Executor.

If you are a Data Engineer and want to use Airflow for Development but don’t want to spend a huge amount of time learning how to configure it.

If you are a Full Stack Engineer and want to learn about various frameworks revolving around Airflow such as Helm, AWS EKS.

If you want to focus on development and get rid of all the frustration coming from trying to set up Airflow with all the core components.

Who this course isn’t for: If you want to know what Airflow is or learn how to create DAGs or pipelines.

Note: This course includes using AWS resources such as EKS which is not free tier eligible.

Who this course is for
Airflow beginners who wants to deploy it in a few commands
Devops, Platform Engineers
Data Engineers looking to leverage scalability of Airflow