Mage AI Pipeline - API to BigQuery

Опубликовано: 31 Май 2023
на канале: Data Slinger

6,924

128

This video is a walkthrough for making a simple Mage AI Pipeline that pulls from an API and loads the data into BigQuery. It is very cost efficient using Instance Schedules.

Github repo for Docker and Docker Compose Install:
https://github.com/MichaelShoemaker/D...

Partitioning and Clustering Google Documentation:
https://cloud.google.com/bigquery/doc...

Install Docker and Docker Compose on GCP VM Manually:
   • Installing Docker and Docker Compose ...

Chicago Data Portal Sign Up Walkthrough:
   • How to Retrieve Data from the Chicago...

Videos Explaining Partitioning and Clustering MUCH better than I can:
   • DE Zoomcamp 3.1.2 - Partioning and Cl...

Big thanks to Xiaoyou Wang in the Mage Slack for helping me through the timestamp conversion. :-)