MLOps Community
timezone
+00:00 GMT
Sign in or Join the community to continue

Orchestrating Spark Jobs with Kubeflow

Posted Jul 14, 2021 | Views 507
# Open Source
# Coding Workshop
# Presentation
Share
SPEAKERS
Sadik Bakiu
Sadik Bakiu
Sadik Bakiu
Freelance ML Engineer @ Freelance

Sadik is a Freelance ML Engineer focused on creating production-grade ML workflows. Since the early beginning of his career, more than a decade ago, he was fascinated by Data and Information management systems and has been working in this field ever since. Sadik also writes occasionally about technology topics.

+ Read More

Sadik is a Freelance ML Engineer focused on creating production-grade ML workflows. Since the early beginning of his career, more than a decade ago, he was fascinated by Data and Information management systems and has been working in this field ever since. Sadik also writes occasionally about technology topics.

+ Read More
Demetrios Brinkmann
Demetrios Brinkmann
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More
SUMMARY

Apache Spark and Kubernetes have been established as de facto standards for data processing and container orchestration respectively. This talk will cover how these technologies can be integrated under the orchestration of Kubeflow. (Kubeflow has been emerging as a platform to make ML workflows easy to work with and deploy.) All codes are available here: https://github.com/sbakiu/kubeflow-spark/blob/main/kubeflow_pipeline.py

+ Read More

Watch More

1:04:18
Posted Mar 04, 2022 | Views 486
# Coding Workshop
# Presentation
# ML Orchestration
# Prefect
# Prefect.io
1:26:03
Posted Jul 21, 2021 | Views 647
# Open Source
# Coding Workshop
# Presentation
# Kubeflow
# Feature Store
# publicissapient.com
57:42
Posted Apr 23, 2023 | Views 5.7K
# Spark
# Open Source
# Databricks