ELT with Dataform on Google Cloud
🚀 Building an ELT Pipeline with Dataform on Google Cloud | Real-World Football Use Case ⚽
In this video, I walk you through a complete ELT pipeline using Dataform and BigQuery on Google Cloud, showcasing a real-world football-related use case. We'll go through staging and mart steps, building tables dynamically, and leveraging SQLX & JavaScript to optimize our workflows.
What You’ll Learn:
✅ Setting up a Dataform repository manually and linking it with a GitHub repo 📂
✅ Automating the process with Terraform for a seamless GitOps workflow ⚙️
✅ Dataform models in SQLX with a reusable JS function
✅ Dataform model in Javascript for dynamic tables and views generation 🔄
✅ Documenting tables directly in SQLX or in a separate JS file 📖
This project follows GitOps best practices, with GitHub as the source of truth for managing our Dataform models.
🔔 Subscribe for more Data & Cloud content! Don't forget to like and drop a comment if you found this useful. 🚀
