MLOps Community
timezone
+00:00 GMT
Sign in or Join the community to continue

Real-Time Exactly-Once Event Processing with Apache Flink, Kafka, and Pinot

Posted Apr 29, 2022 | Views 1.2K
# Uber machine learning platform
# Uber Machine Learning
# Real-time Machine Learning
Share
SPEAKERS
Jacob Tsafatinos
Jacob Tsafatinos
Jacob Tsafatinos
Staff Software Engineer @ Elemy

Jacob Tsafatinos is a Staff Software Engineer at Elemy. He led the efforts of the Ad Events Processing system at Uber and has previously worked on a range of problems including data ingestion for search and machine learning recommendation pipelines. In his spare time he can be found playing lead guitar in his band Good Kid.

+ Read More

Jacob Tsafatinos is a Staff Software Engineer at Elemy. He led the efforts of the Ad Events Processing system at Uber and has previously worked on a range of problems including data ingestion for search and machine learning recommendation pipelines. In his spare time he can be found playing lead guitar in his band Good Kid.

+ Read More
Demetrios Brinkmann
Demetrios Brinkmann
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More
Mihail Eric
Mihail Eric
Mihail Eric
Co-founder @ Storia AI

Mihail is a co-CEO of Storia AI, an early-stage startup building an AI-powered creative assistant for video production. He has over a decade of experience researching and engineering AI systems at scale. Previously he built the first deep-learning dialogue systems at the Stanford NLP group. He was also a founding member of Amazon Alexa’s first special projects team where he built the organization’s earliest large language models. Mihail is a serial entrepreneur who previously founded Confetti AI, a machine-learning education company that he led until its acquisition in 2022.

+ Read More

Mihail is a co-CEO of Storia AI, an early-stage startup building an AI-powered creative assistant for video production. He has over a decade of experience researching and engineering AI systems at scale. Previously he built the first deep-learning dialogue systems at the Stanford NLP group. He was also a founding member of Amazon Alexa’s first special projects team where he built the organization’s earliest large language models. Mihail is a serial entrepreneur who previously founded Confetti AI, a machine-learning education company that he led until its acquisition in 2022.

+ Read More
SUMMARY

A few years ago Uber set out to create an ads platform for the Uber Eats app that relied heavily on three pillars; Speed, Reliability, and Accuracy. Some of the technical challenges they were faced with included exactly-once semantics in real-time.

To accomplish this goal, they created the architecture diagram above with lots of love from Flink, Kafka, Hive, and Pinot. You can dig into the whole paper (https://go.mlops.community/k8gzZd) to see all the reasoning for their design decisions.

+ Read More

Watch More

58:24
Posted Nov 22, 2022 | Views 1.1K
# Real-time Machine Learning
# Accountability
# MLOps Practice
# Claypot AI
# Claypot.ai
51:55
Posted Nov 29, 2022 | Views 722
# Real-time Machine Learning
# ML Inference
# ML Features
# LinkedIn
48:16
Posted May 30, 2022 | Views 674
# Juggling
# Dr. Seuss
# Feature Stores
# Real-time AI/ML
# Redis.io