MLOps EngineeringLabs: What We Learned Building End-to-end ML Applications on Flyte

Flyte is the backbone for large-scale Machine Learning and Data Processing (ETL) pipelines at Lyft, Spotify, BlackShark, and many others. It is used across business-critical applications ranging from ETA, Pricing, Mapping, Autonomous, and many more. At its core is a Kubernetes native workflow engine that executes 10M+ containers per month as part of thousands of workflows.

The talk will focus on:

What does a Simple ML User Journey look like? The sacrifices ML Practitioners have to do to achieve Robustness? How does Flyte help bridge the gap?

This talk will provide some background on the March 2022 MLOps EngineeringLab in collaboration with the team behind Flyte, then go into lightning talks given by the three winning teams.

During this talk, Amale will present the business context and the implementation of their solution that they have submitted to the MLOps EngineeringLabs. They built 2 end-to-end pipelines on Flyte that train and apply a Named Entity Recognition model on Twitter posts to extract brands from them.

Implementation of a real-life business challenge using Flyte orchestrator Their feedback on Flyte

Ali and Omar are presenting for team BraveHyenas2.

The team's ability to develop a complex and resilient machine learning project in such a short period.

In this talk, Matheus will present the Vamos-Dalhe's project for the MLOps EngineeringLabs hackathon: a destination similarity algorithm capable of recommending other places to go based on a given location. They used a custom scraper to retrieve data from Wikimedia pages and build a dataset with Brazilian cities' information. Team Vamos-Dalhe then applied the BERTimbau language transformer to compute embedding vectors for the cities. Finally, the similarity was defined as the Euclidian distance between the vectors. They used Flyte to help develop and deploy the entire system.