What You Will Learn!
- Understand the overall Airflow system
- Learn about DAGs, operators, sensors, connections, hooks, etc.
- Get an introduction into advanced topics, like XCOMs, conditional execution, etc.
- Do a high level review of the Apache Airflow code base
- Get the ability to unblock yourself if you run into any issues running Airflow workflows
Description
Airflow is an Apache project, used to programmatically author, schedule and monitor workflows. It was originally developed by Airbnb and then later open-sourced. Today, many technology companies use Airflow to orchestrate their workflows.
This course is designed to get you up and running with Airflow v2 in an hour. The specific focus of the course is on teaching you the fundamentals of using Airflow rather than setting up a production environment. We will set up a sandbox environment with Airflow and PostgreSQL Servers running for you to code along and practice what you have learned. The skills you learn in this course will be transferable to your work, where a production environment is already available and maintained. Alternatively, if you are interested in using Airflow for hobby projects, the sandbox environment that you set up on your machine will likely be sufficient.
When you are taking this course, you will:
Learn the core Airflow concepts
Write an end to end workflow that takes data, cleans it up and writes into a database
Learn a number of different operators and get the hand of using operators so that you can easily incorporate new ones into your workflow
Understand how you can unblock yourself if you run into issues running your code
Who Should Attend!
- Data scientists and data engineers who would like to start writing workflows in Airflow v2 quickly
