Real-Time Exactly-Once Event Processing with Apache Flink, Kafka, and Pinot
Speakers

Jacob Tsafatinos is a Staff Software Engineer at Elemy. He led the efforts of the Ad Events Processing system at Uber and has previously worked on a range of problems including data ingestion for search and machine learning recommendation pipelines. In his spare time he can be found playing lead guitar in his band Good Kid.

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

Mihail is a co-CEO of Storia AI, an early-stage startup building an AI-powered creative assistant for video production. He has over a decade of experience researching and engineering AI systems at scale. Previously he built the first deep-learning dialogue systems at the Stanford NLP group. He was also a founding member of Amazon Alexa’s first special projects team where he built the organization’s earliest large language models. Mihail is a serial entrepreneur who previously founded Confetti AI, a machine-learning education company that he led until its acquisition in 2022.
SUMMARY
A few years ago Uber set out to create an ads platform for the Uber Eats app that relied heavily on three pillars; Speed, Reliability, and Accuracy. Some of the technical challenges they were faced with included exactly-once semantics in real-time.
To accomplish this goal, they created the architecture diagram above with lots of love from Flink, Kafka, Hive, and Pinot. You can dig into the whole paper (https://go.mlops.community/k8gzZd) to see all the reasoning for their design decisions.

