MLOps Community

Using LLM in production. That's right. Hype or here to stay? The conversation answers some of the questions that have been asked by our community members like; performance & cost of production, the difference in architectures, Reliability issues, and a bunch of random tangents. We have some heavy hitters for this event!

Saahil shares insights into the You.com search engine approach, which includes a focus on a user-friendly interface, third-party apps, and the combination of natural language processing and traditional information retrieval techniques. Saahil highlights the importance of product thinking and the trade-offs between relevance, throughput, and latency when working with large language models. Saahil also discusses the intersection of traditional information retrieval and generative models and the trade-offs in the type of outputs they produce. He suggests occupying users' attention during long wait times and the importance of considering how users engage with websites beyond just performance.

Forecasting supply and demand, serving restaurant recommendations, and predicting delivery times. These are just a few examples of how Machine Learning is being applied at Wolt. Now with over 20 million users, scaling the ML infrastructure has been a significant challenge. This talk highlights those challenges and how they were addressed by building an end-to-end MLOps platform on Kubernetes. You'll learn about the open-source frameworks that Wolt integrated, specifically Flyte, MLFlow, and Seldon Core.