Airflow CI/CD: Github to Cloud Composer (safely)

Опубликовано: 24 Июль 2020
на канале: Apache Airflow
12,552
135

Jacob Ferriero


Deploying bad DAGs to your Airflow environment can wreak havoc. This talk provides an opinionated take on a mono repo structure for GCP data pipelines leveraging BigQuery, Dataflow and a series of CI tests for validating your Airflow DAGs before deploying them to Cloud Composer.