Data Scientists are More Valuable than Hardware: The Design of Modin - Devin Petersohn, Intel
Data Scientists are More Valuable than Hardware: The Design of Modin - Devin Petersohn, Intel
As tools trend toward scaling to larger and larger data, their requirements are growing as well, which presents a non-trivial human cost. To bring the focus of data science tools back to the data scientist, we built Modin, a platform that scales data scientist capabilities without requiring them to learn about distributed computing concepts. In this talk, we discuss the design of Modin and how Ray enables our modular design to scale. We also discuss our experience with enabling the data scientist to connect to multiple Ray clusters from the same notebook.
