MLOps Community
+00:00 GMT
Agents in Production 2024
# streamline workflows
# Agentic
# memoryrank

Planning is the New Search // Fabian Jakobi // Agents in Production

In an era where knowledge workers are inundated with manual tasks and corporate memory systems, agents hold massive potential to streamline and document their workflows. Agents also hold the potential to unlock a vast amount of skills and knowledge from the front-line employees of global businesses. This talk explores how dynamic agents can capture work patterns, from inboxes to task sequences, using clustering and process mining techniques. These methods ground agentic planning and provide more robust, company-specific workflow orchestration. We’ll also discuss how knowledge graphs power effective memory management, and how Interloom’s innovative MemoryRank™ system leverages reflective agents to create and serve agentic memories, optimizing task management for both human workers and specialized agents. Attendees will gain insights into how agentic memory systems can drive efficiency and productivity for end-to-end processes and workflows.
Fabian Jakobi
Fabian Jakobi · Nov 26th, 2024
Popular topics
# Machine Learning
# LLMs
# Machine learning
# Interview
# Deployment
# LLM
# GetYourGuide
# RAG
# Machine Learning Engineer
# Model Serving
# Cultural Side
# Scaling
# ML Platform
# Responsible AI
# Observability
# A/B Testing
# Video Game
# Arize
# MLOps Cycle
# Rule-bases Systems
All
Shaun Wei
Andrew Tanabe
Shaun Wei & Andrew Tanabe · Nov 26th, 2024
This talk explores how AI can automate the invisible processes in healthcare, like insurance calls, to reduce administrative burdens. I'll share our journey with HeyRevia and how we create AI solutions to help providers focus on patient care.
# Healthcare
# HeyRevia
# AI Agents
24:56
Pietro Gagliano
Pietro Gagliano · Nov 26th, 2024
The Creative Singularity is here! Welcome to the age of transformable media, where audiences can generate and remix their entertainment at will – and AI systems will regenerate the content in return. This talk will take you behind the scenes on several of Transitional Forms' dynamic media projects, designed to use machine intelligence as a real-time creative collaborator to generate the content itself.
# Singularity
# Transitional
# Forms
26:24
In a world dominated by Python-centric AI frameworks, JavaScript developers have often found themselves at a disadvantage when it comes to building intelligent, multi-agent systems. KaibanJS aims to change that. In this talk, we’ll explore how this first-of-its-kind, JavaScript-native framework enables developers to create and integrate AI agents effortlessly into their projects. We’ll break down real-world use cases, dig into the technical features, and demonstrate how KaibanJS is helping to bridge the gap for JavaScript developers in the AI space.
# javascript
# multi-agent
# AI Systems
23:31
As conversational AI systems become increasingly autonomous and mission-critical, ensuring their reliability presents novel challenges that parallel those faced in self-driving vehicle development. This talk explores how simulation-based testing and evaluation strategies from autonomous vehicles can be adapted to build more robust AI agents. We'll examine how traditional software engineering practices are evolving in response to AI systems, where deterministic unit tests give way to probabilistic performance metrics and behavioral analysis. Drawing from real-world examples, we'll demonstrate how comprehensive telemetry — both in pre-production simulation and production environments — provides crucial insights beyond simple pass/fail metrics. The presentation will delve into the critical balance between computational cost, latency requirements, and signal quality in AI system evaluation. We'll introduce a framework for developing evaluation strategies based on reliability requirements across different use cases, from bug detection tools where any true positive provides value, to medical assistance systems that demand near-perfect accuracy. Attendees will learn practical approaches to implementing simulation-based testing pipelines, techniques for meaningful telemetry collection and analysis, and strategies for defining appropriate reliability thresholds based on application context. This session will benefit ML engineers, software architects, and engineering leaders working on production AI systems.
# Conversational Agents
# Coval
# Agents in Production
33:58
Neal Lathia
Andrew Tanabe
Neal Lathia & Andrew Tanabe · Nov 26th, 2024
AI agents change what we can achieve with software; but also change how we have to think about building software systems. In this talk, I'll share some of the lessons we've learned while building a powerful AI agent for complex support settings.
# LLMs
# AI Agents
# Gradient Labs AI
26:19
Few companies have shipped an agent at the scale of Replit, but with scale comes challenges. This talk explores how Replit built and scaled its agent, and how it adapted from a small engineering team to involving almost half of its engineering organization almost overnight.
# Replit Agent
# Repls
# Replit
32:31
Nishikant Dhanuka
Beatriz Ferreira
Demetrios Brinkmann
Nishikant Dhanuka, Beatriz Ferreira & Demetrios Brinkmann · Nov 22nd, 2024
OLX has a catalog of millions of unique items across a very diverse range of categories. While existing technologies for navigating this assortment are reasonably effective, LLMs offer the potential for a much improved user experience. In this session, we will talk about how we utilized the Agentic flow to transform a customer’s standard search on OLX into something highly personal, fully reflecting the customer’s demands and feelings. Aside from Generative AI, we will also share how we redesigned the search pipeline to work well with agentic architecture. And lastly, we will talk about some lessons learned along the way!
# olx
# Prosus
# AI Agents
# Agentic
# GenAi
29:58
Software Engineering can serve as a diverse, rich testbed for evaluating the next generation of language models and driving their development. This talk introduces a line of three works that has established the potential of this research direction and guided industry advancements towards autonomous software engineers. First, SWE-bench is a benchmark that evaluates an Al system's capability to resolve real world GitHub issues, featuring 2294 task instances collected from 12 distinct Python repositories. Second, SWE-agent is an autonomous system that uses a language model to interact with a computer to solve software engineering tasks, setting a state-of-the-art 12.5% resolved rate on the SWE-bench test set. Lastly, SWE-bench Multimodal, a new dataset of 617 task instances from 17 JavaScript repositories, shows how many existing coding agents are overfit to Python, raising the property of “generalizability as an overlooked but desirable trait of AI systems.
# shot code
# autonomous
# AI Agents
26:05
Integrating Large Language Models (LLMs) into production-level data workflows presents both significant challenges and opportunities. In this talk, we'll introduce Numbers Station, a platform that automates data analytics workflows using LLMs, Retrieval Augmented Generation (RAG) over a Knowledge Layer, and a customizable multi-agent architecture. We'll start by discussing practical use cases for analytics, such as dashboard search, query generation, or automatically adding analysis summaries to slide deck presentations. We'll then delve into the methodologies for deploying LLMs within data analytics workflows, focusing on a detailed case study to build a SQL agent from the ground up. We will cover the architectural considerations necessary to support agent-based analytics, including the role of dynamic control flows and the importance of incorporating business context through a unified Knowledge Layer. This session aims to provide a deep technical insight into transforming theoretical AI frameworks into practical, scalable solutions that advance organizational data capabilities.
# AI Agents
# analytics
# Gen AI
24:26
Graham  Neubig
Demetrios Brinkmann
Graham Neubig & Demetrios Brinkmann · Nov 22nd, 2024
In this talk, I will discuss practical issues and technical fundamentals of coding agents. This will include both the promise of what coding agents may be able to do if they are implemented properly, what tasks they can do well now, some challenges in using them appropriately, and some of the challenges that we need to solve to make them more effective -- file localization, file editing, planning, error recovery, and "good taste". I will also give a few demos of our open-source OpenHands toolkit (https://github.com/All-Hands-AI/OpenHands) to illustrate each of the points.
# Coding
# Agents
# Autonomous
28:00
Popular
Price per token is going down. Price per answer is going up.
Demetrios Brinkmann