MLOps Community
timezone
+00:00 GMT
SIGN IN
Sign in or Join the community to continue

Airflow Sucks for MLOps

Posted Jan 17, 2023 | Views 416
Share
SPEAKER
Stephen Bailey
Stephen Bailey
Stephen Bailey
Data Engineer @ Whatnot

Stephen has worked as a data scientist, analyst, manager, and engineer, and loves all the domains equally. He currently works at Whatnot, a collectibles marketplace that focuses on live shopping, and has previously worked in privacy tech at Immuta. He has his Ph.D. from Vanderbilt University in educational cognitive neuroscience, but it has yet to help him understand why his three children are so crazy.

+ Read More

Stephen has worked as a data scientist, analyst, manager, and engineer, and loves all the domains equally. He currently works at Whatnot, a collectibles marketplace that focuses on live shopping, and has previously worked in privacy tech at Immuta. He has his Ph.D. from Vanderbilt University in educational cognitive neuroscience, but it has yet to help him understand why his three children are so crazy.

+ Read More
SUMMARY

Stephen discusses his experience working with data platforms, particularly the challenges of training and sharing knowledge among different stakeholders. This talk highlights the importance of having clear priorities and a sense of practicality and mentions the use of modular job design and data classification to make it easier for end users to understand which data to use.

Stephen also mentions the importance of being able to move quickly and not getting bogged down in the quest for perfection. We recommend Stephen's blog post "Airflow's Problem" for further reading.

+ Read More

Watch More

1:11:12
Posted Jul 21, 2022 | Views 359
# Airflow
# Orchestration
# ML Engineering
45:20
Posted Aug 30, 2022 | Views 959
# DoorDash
# Leverage
# Core Infrastructures