Creating an Optimized Data Pipeline for Data-Heavy Applications // Subsurface Summer 2020
Building an Efficient Data Pipeline for Data Intensive Workloads is a presentation that Ryan Murray, OSS Developer at Dremio, will be delivering at Subsurface Summer 2020. It will cover how to improve the data pipeline for machine learning and IoT workloads with a modern transport mechanism that includes Apache Arrow Flight. Attendees will learn about the architecture and features of the Arrow Flight protocol, see a demo of a machine learning pipeline running in Spark with data microservices powered by Arrow Flight, as well as how much faster and simpler the Flight interface makes this example pipeline.
Data lakes are rapidly becoming essential components of modern data architectures. As organizations amass more data, they need to ensure their pipelines are efficient and predictable to get the most out of their investments. With Apache Arrow Flight and a modern transport mechanism, they can do just that.
At Subsurface Summer 2020, Ryan Murray will discuss how attendees can improve their data pipelines with Apache Arrow Flight and build microservices for Spark with it. This is an amazing opportunity to gain insight into how to use this open source technology to make your pipelines faster and simpler while handling more data intensive workloads like IoT and machine learning in production.
No matter if you’re already using data lakes or just getting started on your journey towards building one, Subsurface Summer 2020 has something for everyone. Make sure you don’t miss out on the chance to learn from experts like Ryan Murray about how Apache Arrow Flight can help you optimize your data lakehouse or warehouse lake engine pipelines for better performance!
Connect with us!
Twitter: https://bit.ly/30pcpE1
LinkedIn: https://bit.ly/2PoqsDq
Facebook: https://bit.ly/2BV881V
Community Forum: https://bit.ly/2ELXT0W
Github: https://bit.ly/3go4dcM
Blog: https://bit.ly/2DgyR9B
Questions?: https://bit.ly/30oi8tX
Website: https://bit.ly/2XmtEnN
