Understanding Databricks & Apache Spark Performance Tuning: Lesson 02 - Spark Hardware
Following up on Databricks Performance Tuning with the best place to start: allocating Spark clusters. If you don't allocate sufficient resources, nothing else will fix the problem. How many nodes? How large should the driver and workers be? Do you need GPUs or CPUs? Should you use Photon? These and many more questions will be covered in detail.
Support me on Patreon
https://www.patreon.com/bePatron?u=63...
Slides
https://github.com/bcafferky/shared/b...
Referenced Video by Daniel Tomes, Databricks
• Apache Spark Core—Deep Dive—Proper Optimiz...
See the Full Playlist Here:
• Databricks and Apache Spark Performance Tu...
