MLOps IRL

It's very hard to find existing solutions to the current problems so you might be working on a problem that you're certain that somebody has faced the same problem before but you can't really find the solution that will take you off the ground quickly. QuantumBlack had four key issues in the firmware doing analytics projects that lead to the creation of Brix.

Kenny Chen discusses robust production QA Practices for multimodel systems in shifting ground truth and UX environments. This talk covers, QA practices, object metadata tagging, regenerative testing framework, and production model rollbacking.

In this talk, Kyle Kranen discusses some of the challenges in scaling large graph neural networks.

Simon talks about how TMNL use one of the main tools they use at Sagemaker pipelines and how they are using them to fight money laundering. Simon also relates how the matryoshka doll has to do with Sagemaker pipelines!

Using Machine Learning to solve a problem is expensive, both in terms of talent necessary, time, and infrastructure. Creating a project that fails during or at delivery becomes costly, so what can we do to understand whether we should even think about using ML?

In this talk, the speakers discuss collaboration with the Outerbounds, NVIDIA Merlin, and Comet teams to release as open source code a realistic data and ML pipeline for cutting-edge recommender systems “that just works”. Anyone can cook great ML, not just Big Tech if you know how to pick and choose your tools.

The R&D workflow of an AI-based product is inherently characterized by the experimental nature of the deep-tech research process. Adding to the challenges of edge technology - working on various ARM-based SOMs with multiple GPUs and DSP types, the inevitable conclusion is that a bespoke R&D methodology is required. This talk discusses SightX AI's design and successful application of an end-to-end MLOPS methodology. The proposed design enabled us to tackle the management of deep learning research aimed to be deployed on various platforms and to become faster and better with every version release. SightX AI recently added a feedback loop to this methodology which gets us a step closer to the holy grail of automated and continuously learning R&D workflow for edge AI.

Deploying machine learning models is hard. Deploying multiple machine-learning models in parallel is harder. In this talk, we discuss Ironscales' strategy for decomposing a machine learning system and providing components that allow highly parallelized development.

With 2 open-source ingredients and a tsp of Python Sidecar, you can automate the deployment of your models into accelerated infra and also, control the end-to-end lifecycle of your ML models from a single tool.

Drive your ML like Mario Andretti. Get it operationalized. Speed demon Mario Andretti is a racing legend. His success can be attributed to well-established routines that have been fully operationalized. Operationalized you say? What does that mean? It means every component of his racing career has been tuned for success. It includes a high-performance vehicle (tools), the support of a seasoned crew (people), and loads and loads of track time on tough-to-navigate courses (process). MLOps defines the people, the process, and the tools to achieve Mario Andretti-like performance for your machine learning models. In this talk Kevin Hartman, Practice Director of Data Science and Machine Learning Engineering at Unify Consulting, will dive into the “pit crew” of MLOps and take a look at who does what (and when) to make your models fully operational.

Accelerating ML Research and Prototyping with Ray Spotify started evaluating Ray on Kubernetes as a distributed compute platform that enables ML research and experimentation for ML workflows in early 2022. Ray and its ecosystem provide researchers and data scientists at Spotify with a better model development experience, instant access to distributed computing, and an expressive programming interface. These features greatly complement our existing production ML workflow. In this talk, we share the story of how Ray started at Spotify, what our long-term goals are with Ray, and how Ray enabled tangible business impact for Spotify by accelerating an ML research use case for improving podcast recommendations. Supporting Model Serving at Scale: From Backend to On Device Recently our ML serving team expanded its scope from backend infra to on-device models. The talk covers some (early) observations on this project and the new challenges that came with it.

With the volume of data increasing, a lot of data practitioners are needing to migrate existing Python or pandas code to distributed computing frameworks such as Spark and Dask. In this tutorial, we discuss the possible solutions and their specific behaviors. Pandas-like frameworks such as Modin (for Dask) and Koalas (for Spark) offer the promise of a drop-in replacement for Pandas. Fugue, on the other hand, chooses to deviate away from the Pandas interface. Fugue users instead write minimal additional code to port existing Python and pandas code. To learn the tradeoffs of these approaches, we will learn underlying distributed computing concepts. Attendees will deepen their understanding of distributed computing and understand the pros and cons when evaluating these options.

The Lisboa MLOps Community fires up another Meetup in Lisbon on the 27th of October. After taking inspiration from prior roundtables from Deep Learning Sessions Portugal, they discussed multiple subjects within MLOps in a roundtable fashion, along with their primary focus: ML Platform thinking.

Adam gives an overview of MLOps and: - why it's important - where it's going - where it's come from - what the major challenges are - how maturity models can be dangerous

Do you want to start tackling Machine Learning Operations (MLOps) and know more about it? Or do you want to find more data engineering and MLOps solutions for your business needs? In this talk we will be dealing with MLOps principles and practices, lessons learned and the architectural solutions implemented in Trustpilot. In the last 3 years Machine Learning (ML) engineers, data scientists, and data engineers have joint efforts to bring Trustpilot to have a more robust and consistent use of artificial intelligence (AI) under MLOps principles. From identifying and defining MLOps gaps within the data science journey and listening to business needs, this joint collaboration effort resulted in the creation of the MLOps infrastructure within the Google Cloud Platform (GCP). As a result, the entire ML journey has been drastically improved, easing models prototyping, data retrieval processes and lowering models development and deployment time from months to weeks.

How do you design and build an ML feature from scratch with little ML knowledge and limited resource? How do you make sure your ML stays grounded in human needs? In this talk, we'll learn how Sewei, an indie maker, incorporates ML into her mobile app via a human-centered design process and everything else she's learned along the way.

A/B Testing for MLOps A/B testing is used heavily in marketing campaigns to improve their effectiveness. In addition, it can be applied very well in Machine Learning applications to help understand which model or which version works better. This talk describes how to practically implement A/B testing for the MLOps context in a microservice architecture. Tool Agnostic MLOps with ZenML The MLOps landscape is exploding with new tools for specific needs. As an ML practitioner, it can be disorienting which tools to use, and even harder, how to do it in a production setting. In this talk, Hamza shows how you can use ZenML to build tool-agnostic MLOps solutions tailored to your needs. He walks you through how you build an ML pipeline and experiment on your local machine, and scale the pipeline into a full-fledged production-ready solution with minimal code changes.

Traditional software monitoring best practices are not enough to detect problems with machine learning stacks. How can you detect issues and be alerted in real-time? This talk will give you a practical guide on how to do machine learning monitoring: which metrics should you implement and in which order? Can you use your team’s existing monitoring and dashboard tools, or do you need an MLOps Platform?

Setting Healthy Boundaries: Generating Geofences at Scale with Machine Learning Stephanie describes how project44 developed a new technique for generating geofences for customer facilities using historical data combined with machine learning. The audience learns how Stephanie and her team did it, including the pitfalls of deploying a product like this to production. She makes a special point to discuss the varied open source tools the team used including DBSCAN, Dask, geopandas, folium, Airflow, and more. Cars-Forge: Spot EC2s Made Easy Nikhil showcases cars-forge, a tool developed at Cars.com that was recently open sourced. Cars-forge is a command line tool that starts EC2 instances and runs scripts on them. Additionally, Nikhil also previews Skelebot, a command-line tool for developing ML projects and executing them in docker.

Pachyderm provides Kubernetes-native pipeline orchestration with data versioning and lineage, which can greatly improve the reproducibility and auditability of your ML experiments. Pachyderm helps with experimentation by allowing data scientists to track and compare results from different experiments.

The First MLOps Community in Africa - MLOps Lagos Community Meetup features a discussion with experts about their experiences with MLOps at their companies. The MLOps Community was introduced with its goals and presents on scaling machine learning innovation with MLOps, XAI, and A/B testing. The meetup concludes with a panel featuring practitioners and students sharing their experiences with MLOps.

This talk focuses on the application of machine learning techniques in the detection of phishing attacks. The domain discussed involves deploying and calibrating machine learning models, as well as an automated workflow, to accurately detect phishing attempts. The objective is to provide a comprehensive overview of the current state-of-the-art techniques used in the field and how they can be effectively integrated into an automated system to minimize the risk of phishing attacks.

This learning session is led by Elhay Efrat, a Cloud Solution Architect for Startups. Elhay guides attendees through the practical application of Open AI on Azure, discussing how to leverage the platform to improve Machine Learning performance and efficiency in real-world scenarios. Topics are given with an overview of the Machine Learning lifecycle in the real world, comparing cognitive services vs Open AI. The session also covers ML operations on Azure, and how to use Open AI to enhance performance and efficiency in real-world scenarios.

Ultraleap do a surprising amount of interesting ML – mostly hand tracking from cameras - they had headsets to try and a cool screen ball pit demo thing. They’ve got this interesting homebrew ML platform because they were shipping ML in a product early. It’s got some interesting features and a ton of stuff in it, so they’re having a fun time working out how to replace it, nothing modern quite fits. Maybe partly as they don’t have the normal model serving problem. They’re hiring across a surprising range of things.

Deploying ML models to low-powered hardware brings a whole bunch of new challenges. In this talk, Matt walks through how they trained a vision model to play a card game on an Nvidia Jetson Nano and discusses the MLOPs tooling and approaches we used to make it happen.

Forecasting supply and demand, serving restaurant recommendations, and predicting delivery times. These are just a few examples of how Machine Learning is being applied at Wolt. Now with over 20 million users, scaling the ML infrastructure has been a significant challenge. This talk highlights those challenges and how they were addressed by building an end-to-end MLOps platform on Kubernetes. You'll learn about the open-source frameworks that Wolt integrated, specifically Flyte, MLFlow, and Seldon Core.

Tomi outlines their goals for the year, which involve building platforms and tools for accelerating AI development and promoting data-driven recognition. They are part of a machine learning and advanced analytics team in a company called Wärtsilä, which has over 17,000 employees and is known for its heavy machinery for marine and energy markets. Wärtsilä supports and consults all the other businesses in the company, and their work involves applying MLOps principles to the development and deployment of machine learning models. Tomi discusses the evolution of their team's work with amulets and future prospects.

With the increasing number of deep learning models in production, Etsy faces new challenges in terms of scaling and managing the models. The talk highlights the custom feature transformations, ideal infrastructure settings, and early feedback loop on the latency of models as some of the challenges faced. Etsy has developed new processes and tooling to address these challenges and improve the performance of the platform.

Alaa shares their experiences in working on ML projects. In one example, their team attempted to introduce a new deep neural net architecture for predicting ad clicks but encountered challenges in revamping the entire ecosystem around the model. After overcoming these challenges, the team faced a new issue where the model suddenly stopped working. They discovered that the issue was caused by applying a log transform to a feature with a parallel distribution, resulting in NAND predictions when the feature became zero.

Neal Lathia, a staff ML engineer at Monzo, provides an overview of Monzo's journey from a prepaid card to a fully-fledged bank with over 7 million customers. Lathia shares his experience of building Monzo's ML platform and team and highlights some of the challenges they faced. He explains how Monzo's ML stack is built on top of their existing engineering and analytics foundations, and how they developed an ML Ops path to enable the deployment of ML models in a flexible and reliable way. Lathia also discusses the ML frameworks used by Monzo and the importance of speed and determinism in ML when dealing with transaction data.