Apache Airflow

The industry-standard platform for authoring and scheduling DAG workflows.

Apache Airflow is a battle-tested open-source platform to programmatically author, schedule, and monitor workflows defined as directed acyclic graphs in Python. It is the de facto standard for data engineering orchestration, with a vast provider ecosystem, a rich web UI for monitoring runs, and pluggable executors for scaling across Celery, Kubernetes, and more. It is governed by the Apache Software Foundation.