ELT batch pipeline with Cloud Storage, BigQuery and Composer/Airflow
This video shows a real wold use case for an ELT batch pipeline, with Cloud Storage, BigQuery, Apache Airflow and Cloud Composer.
The Extract is done from Cloud Storage.
The Load is done from Cloud Storage to BigQuery.
The Transform is done with a BigQuery SQL query.
Everything is orchestrated by Cloud Composer and Apache Airflow.
#googlecloud #ELT #CloudStorage #BigQuery #Airflow #CloudComposer #SQL #Python #Orchestration #Batch #Pipeline
▸ Medium : / elt-batch-pipeline-with-cloud-storage-bigq...
▸ Github : https://github.com/tosun-si/teams-lea...
▸ LinkedIn : https://www.linkedin.com/posts/mazlum...
▸ Twitter : https://twitter.com/MazlumTosun3/stat...
Feel free to subscribe to the channel and click on the bell 🔔 to receive notifications for the next videos.
📲 Follow me on social networks :
▸ Articles : / mazlum.tosun
▸ X : / mazlumtosun3
▸ LinkedIn : / mazlum-tosun-900b1812
▸ WhatsApp : https://whatsapp.com/channel/0029VaCj...
