Agents in Production - Prosus x MLOps

Prosus has shipped over 7949 agents. 15% have worked. The rest have been learning experiences. Let's talk about what we have learned, and where we see things going.

In this session, Ricky Doar, VP of Solutions at Cursor, shares actionable insights from leading large-scale AI developer tool implementations at the world’s top enterprises. Drawing on field experience with organizations at the forefront of transformation, Ricky highlights key best practices, observed power-user patterns, and deployment strategies that maximize value and ensure smooth rollout. Learn what distinguishes high-performing teams, how tailored onboarding accelerates adoption, and which support resources matter most for driving enterprise-wide success.

This talk covers an overview of AI coding tools and different levels of coding automation. It also discusses workflow patterns that have emerged and how they will change over time.

The rapid spread of digital misinformation requires solutions that address the entire lifecycle, moving beyond single-LLM limitations. This talk, based on the author’s ICWSM research paper, offers a practitioner's guide to a novel, five-agent system—Classifier, Indexer, Extractor, Corrector, and Verification—designed for maximum scalability, modularity, and explainability. This paper aims at automating the working of fact-checkers, which is traditionally done through a team of experts, saving millions and increasing efficiency with a human-in-the-loop system. We will get the details for each specialized agent, detailing crucial elements like model sizing and fine-tuning—for example, matching small, fine-tuned encoder models for the Classifier's high-confidence multi-class labeling against the need for a strong reasoning LLM in the Corrector Agent. Topics include building an efficient Indexer Agent and reranking with retrieval through hybrid keyword and vector embeddings, enabling the Corrector Agent to use external search APIs for cross-validation, and the function of the Verification Agent as the final quality check for high precision.. The talk concludes by covering agent coordination protocols, cost, holistic evaluation, offline evaluation and online A/B testing and post-deployment metrics.

This presentation discusses the evolution of AI agent interaction, focusing on transitioning from low-engagement text-based chat to more intuitive, GUI-driven experiences. It outlines critical challenges in creating an intuitive and impactful experience for busy dealers, proposing solutions that include quick actions, efficient data streaming, and agent interactivity to create a great user experience.

In this session, we share the development journey of Ailo, iFood's conversational AI agent. We cover the practical challenges and victories of navigating from concept to production, highlighting how robust MLOps practices and the integration of our proprietary Large Commerce Model (LCM) enable us to interpret complex intents and create the best personalization experience for our users.

At Elyos AI, we’re not building AI to pretend to be human - we’re building AI that can perform like one: completing real tasks, with real quality and real transparency. Our agents will tell you they’re AI. We’re not replacing humans - we’re building trust that AI can help humans work better. In this talk, I’ll share what it actually takes to make that vision real in production. From designing low-latency pipelines, to managing dialogue context and emotional tone across long calls, we’ve learned that delivering natural, useful conversations is as much an infrastructure problem as it is a language one. We’ll cover: How Elyos built a resilient real-time stack combining STT, LLM orchestration, and neural TTS. Engineering patterns for error recovery, context engineering, context retention and conversational coherence. The metrics (and human feedback) that actually predict trust and task completion. If you’re building or deploying AI agents that interact with people, this is a behind-the-scenes look at what breaks, what works, and how we’re bringing transparent, high-performance voice AI to the real world.

MCP has revolutionized how AI agents interact with the world. However, with over 13,000 MCP servers launched in 2025 alone, it has also opened a Pandora's box of security vulnerabilities that most organizations aren't prepared to handle: 10% are known to be malicious, the rest of the 90% are exploitable. This presentation guides you through the MCP threat landscape, showcasing real-world exploits already in the wild. We'll examine the most dangerous attack vectors including tool poisoning (hidden instructions lurking in tool descriptions), rug-pulls (bait-and-switch tactics that change behavior post-approval), conversation history theft, and cross-server tool shadowing. We won't leave you defenseless. For each vulnerability demonstrated, you'll learn practical defensive strategies and implementation patterns to safeguard your MCP deployments. Whether you're a security engineer protecting AI agents, a developer building MCP servers, or a a business user integrating your CRM to Claude, you'll walk away with:A comprehensive understanding of the MCP attack surface Practical knowledge of how these exploits work A security checklist for MCP implementations Strategies for detecting and responding to MCP-based attacksAs enterprises adopt MCP faster than security teams can assess the risks, this session provides the essential knowledge needed to stay ahead of attackers in the age of autonomous AI agents.

The discussion centers on highly technical yet practical themes, such as the use of advanced post-training techniques like Direct Preference Optimization (DPO) and Parameter-Efficient Fine-Tuning (PEFT) to ensure LLMs maintain stability while specializing for e-commerce domains. We compare the implementation challenges of Computer-Using Agents in automating legacy enterprise systems versus the stability issues faced by conversational agents when inputs become unpredictable in production. We will analyze the role of cloud infrastructure in supporting the continuous, iterative training loops required by Reinforcement Learning-based agents for e-commerce!

MCP and similar protocols solved tool discovery, but not tool execution. Raw exposure of APIs pollutes model context, bloats prompts, and degrades performance. This talk draws on lessons from enterprise-scale deployments at Fortune 100 tech companies and shows how deploying a thin layer within your tooling service improves accuracy, cost, and performance for AI in a real enterprise setting.

We are in a golden age of AI agent development, with frameworks and models proliferating at an incredible pace. Yet, this Cambrian explosion has created a new ""Tower of Babel."" Our agents, each brilliant in their own silo, cannot effectively collaborate. They speak different languages, follow different customs, and lack a shared understanding of how to work together. This fragmentation is the single greatest barrier to building truly powerful, multi-agent systems that can tackle complex, real-world problems. This talk will explore the critical need for a universal standard for agent-to-agent communication. We will survey the current landscape of emerging protocols, including Anthropic's Model Context Protocol (MCP), the Linux Foundation's Agent Communication Protocol (ACP), and Google's Agent-to-Agent (A2A) protocol. By comparing their philosophies, technical approaches, and trade-offs, we will build a clear picture of the challenges involved. Join this session to understand the fundamental principles of agent interoperability and to explore a conceptual framework for a future standard that could finally allow our agents to speak a common language.

Context Engineering 2.0 treats retrieval, tools, and memory as one surface that agents can navigate. The aim is to make documents, databases, events, and live APIs addressable and navigable through a single MCP native interface. Think GraphQL for agents. RAG works well for one shot queries from textual corpora like help centers and docs. With Redis's vector database, users can index, embed, and retrieve relevant chunks. Sources like relational databases and APIs are out of reach through RAG. Teams paste large ad hoc JSON objects into prompts, rely on Text2SQL, or struggle with OpenAPI to MCP wrappers. It is not reliable and it does not scale across the organization. With Redis Context Engine we are engineering a better way to expose data to agents. A unified, schema driven, MCP native layer connects all your data and powers real time, reliable agent workflows. Define a semantic schema and structured data enters the same path as unstructured text. Agents blend semantic search with structured filters in one call, traverse relationships, call APIs, and keep state via memory. All powered by Redis.

Most agent demos are toys fun prototypes with no pressure. Immigration is the opposite: forms that break lives if you get them wrong, laws that change every week, and stressed users who need both speed and trust. At NaviSmart AI, we built agents that don’t just chat but execute critical workflows checking eligibility, parsing forms, prepping applicants for interviews, and tracking real-time progress across jurisdictions. This talk breaks down how we moved from concept to production with a regulated, high-stakes use case. You’ll learn the engineering realities of handling hallucinations, adding human-in-the-loop safeguards, making agents interoperable with MCP, and scaling reliably for thousands of real users.

The next generation of AI agents won’t just respond to what we say—they will sense how we feel. As large language model–powered agents move from research prototypes into production, a critical frontier is the integration of multi-modal affective computing: combining voice, text, facial expressions, and interaction patterns to detect the learner’s or user’s emotional state in real time. This talk explores the challenges and opportunities of deploying emotion-aware AI tutors in production environments. Drawing from ongoing research at MIT Media Lab and Harvard, and from startup experience building GlowingStar, I will share how multi-modal signals—speech tone, facial micro-expressions, response latency, and even silence—can be fused into affective state estimates that meaningfully improve user experience. We will unpack the technical lessons learned from moving affective sensing beyond the lab: designing architectures that combine ensemble LLMs with sensor inputs, diagnosing when modalities conflict or sabotage each other, and establishing guardrails for privacy and consent in sensitive domains like education. In parallel, I will highlight multi-agent orchestration patterns—including critic–rewriter loops and role-based ensembles—that make it possible to personalize instruction, generate equitable feedback, and sustain engagement across diverse learners. By the end of this session, attendees will have a clear picture of what it takes to move multi-modal, affect-sensing agents from demos to durable production systems: the architectures, the pitfalls, and the metrics that matter. More importantly, we will consider how these lessons extend beyond education to any industry where AI agents must not only think, but also feel with and for the human in the loop.

The rapid evolution of AI agents is fueled by the collaborative power of the open-source community. This panel explores how the democratization of foundational components—including open model weights, permissible datasets, and open RL/planning algorithms—is dramatically reducing the barrier to entry for production-ready agents. When these assets are combined with open communication protocols, development velocity skyrockets. We'll discuss the practical benefits of this approach: developers can fine-tune specialized agent models without massive proprietary costs, researchers can transparently audit the full stack for safety and reliability, and organizations gain the agility to adapt algorithms and protocols to unique business needs. This panel talks about the strategic necessity of an entirely open-source approach to ensure the future of production agents is trustworthy, accessible, and fast-moving.

MCP lets your agents connect to Slack, GitHub, your database, and whatever else you throw at it. Great for productivity. Terrible for security. When an agent can call any tool through any protocol, you've got a problem: who's actually making the request? What can they access? And when something breaks—or gets exploited—how do you even trace it back? This talk covers what breaks when agents go multi-protocol: authentication that doesn't account for agent delegation, permission models designed for humans not bots, and audit trails that disappear when Agent A spawns Agent B to call Tool C. I'll walk through real attack scenarios—prompt injection leading to unauthorized API calls, credential leakage across protocol boundaries, and privilege escalation through tool chaining. Then we'll dig into what actually works: identity verification at protocol boundaries, granular permissions that follow context, not just credentials, and audit systems built for non-human actors. You'll leave knowing how to implement MCP without turning your agent system into an attack surface, and what to build (or demand from vendors) to keep agent-to-tool communication secure.

Modern AI agents depend on vast amounts of context, data, features, and intermediate states, to make correct decisions. In practice, this context is often tied to specific datasets or infrastructure, leading to brittle pipelines and unpredictable behaviour when agents move from prototypes to production. This talk introduces mloda, an open‑source Python framework that makes data, feature, and context engineering shareable. By separating what you compute from how you compute it, mloda provides the missing abstraction layer for AI pipelines, allowing teams to build deterministic context layers that agents can rely on. Attendees will learn how mloda's plugin‑based architecture (minimal dependencies, BYOB design) enables clean separation of transformation logic from execution environments. We'll explore how built‑in input/output validation and test‑driven development will help you build strong contexts. The session will demonstrate how mloda can generate production‑ready data flows. Real‑world examples will show how mloda enables deterministic context layers from laptop prototypes to cloud deployments.

Recently, we’ve heard smart people we respect say things like: “There's a lot of buzz around MCP. I'm not convinced it needs to exist.” In this talk, we will argue that MCP differs from other modern protocols like gRPC and HTTP primarily due to its inherent statefulness. And, that this statefulness is required to bring about the rich, intelligent, long-lived, human-like communication that we want with AI. It is a feature, not a bug. Consider a human-to-human phone conversation: you're on the phone, the call drops, and you dial back. You don’t expect to begin your conversation all over again. Instead, you'd anticipate resuming from where you left off, or at least very close to it. To achieve something similar from human-to-computer, the protocol needs ways to pick up where the chat left off. More so than schemas and a clean separation of prompts, resources, and tools, this is what excites us the most about MCP. This talk will explore the aspects of MCP that enable fault-tolerance as well as features that have remained relatively obscure like elicitation (which enables the MCP server to ask questions or elicit feedback from humans), and sampling (which enables the MCP server to invoke the LLM itself). We will also discuss the current MCP client landscape as well as some important SEPs (Specification Enhancement Proposals) coming down the pike, like SEP-1391, in which we have been involved.

Agents and Model Context Protocols (MCPs) are being rapidly adopted across enterprises: over 80% of professional developers now using AI tools daily, and agentic coding platforms like Claude Code seeing significant growth. This session will explore emerging patterns and security risks, along with strategies to mitigate them. We’ll share insights from observing agent–MCP interactions, discuss methods to detect and prevent potentially harmful behaviors, and outline practical approaches for establishing robust guardrails to ensure safe and controlled MCP deployment.

When your head of data goes on paternity leave, you learn whether your AI agent actually works. For a logistics SaaS company with a 2.5-person data team, our AI analyst ""Wobby"" became the unexpected backup, handling 60% of incoming data questions from the business. This talk shares the hard-won lessons from taking an AI agent from concept to daily use. You'll learn why we abandoned our web UI for Slack, why BIRD benchmark scores meant nothing for our actual success, and how we built an eval system that caught real failure modes instead of synthetic ones. We'll cover the technical decisions that mattered: context engineering, metadata design, and latency optimization. We'll also cover the non-technical ones that mattered more: channel design, user onboarding, and building trust with skeptical business users. This is a practitioner's guide to agent deployment. What worked, what failed spectacularly, and what we'd do differently next time.

Traditionally, an enterprise needs to go under transformation for business processes or IT system due to the arrival of any internal (such as mergers and new service) or external (e.g. new technology like GenAI, pandemic etc.) disruptor. This led to a long catch-up time for an enterprise. We can design our applications to be resilient to these disruptions using adaptable-IT systems, by leveraging future advancements in computing, and scalable to envision zero-downtime. Today’s AI based applications may involve traditional computing, SQL processing , deep learning and Gen AI model inference, so are heterogenous in their demands for computing and memory bandwidth. Some of the applications, like molecular simulation and portfolio optimizations are intractable and not solvable even using traditional compute. Also, the workloads on these applications keep varying throughout their life cycle. On the deployment side,, the range of computing accelerators today extends beyond traditional general purpose computing to low power AI specialized hardware such as Inferentia, Graphcore, Sambanova and Cerebras, many of which are accessible on public and private clouds, and special hardware like Quantum computer for intractable problems. Further, due to death of Moore’s law and need of power hungry AI models, there is shift from silicon based computing to physics based computing for AI applications, such as photonics, neuromorphic, analog and DNA computing paradigms. Achieving the optimal balance of latency, throughput, cost, and energy requires large design space exploration across hardware architectures. This motivates us to build intelligent middleware for pareto optimal mapping of application components to heterogenous hardware ecosystem. This asset can recommend high performance low cost, low energy deployment options for enterprise applications.

I went viral for spending $9k on Cursor in 1 month, and I wrote half a million lines of code building Zo Computer over the last 4 months. In this talk, I'll share everything I would've wanted to know when I was just getting started. Coding with AI done right is *leveraged thinking*, and I'll share the workflows we use on our team to ship high-quality code at breakneck speed.

Search is still the front door of most digital products—and it’s brittle. Keyword heuristics and static ranking pipelines struggle with messy, ambiguous queries. Traditionally, fixing this meant years of hand-engineering and expensive labeling. Large language models change that equation: they let us deploy agents that act like search engineers—rewriting queries, disambiguating intent, and even judging relevance on the fly. In this talk, I’ll show how to put these agents to work in real production systems. We’ll look at simple but powerful patterns—query rewriting, hybrid retrieval, agent-based reranking—and what actually happens when you deploy them at scale. You’ll hear about the wins, the pitfalls, and the open questions. The goal: to leave you with a practical playbook for how agents can make search smarter, faster, and more adaptive—without turning your system into a black box.

We reflect on how the complexity of an agent analytics project at an international pharma taught us to move from prompt engineering to context engineering, empowering agent with interactive tooling to build their context dynamically.

There's never been a better time to be a hacker. With the explosion of vibe-coded solutions full of vulnerabilities and the power and ease that LLMs and Agents lend to hackers, we are seeing an increase in attacks. This talk dives into several vulnerabilities that agent systems have introduced and how they are already being exploited.

Personalization at scale needs deep understanding of each customer. You must collect data from many sources, read it, reason and infer, plan, decide, act, and write to each person. One agent doing everything gave us poor and inconsistent quality. Multi-agent systems changed that. They deliver mass personalization. They also break in edge cases, contradict each other, and are hard to debug. I will share how we addressed this with Cortex UCM, a unified customer memory, and Generative Tables. We map noisy data into a clean, structured layer that agents read and write. We began with email for both outbound and inbound communication. Then we personalized websites and product pages for e-commerce at scale. I share customer stories. For example, one customer had over 60,000 product pages that required customization for thousands of communities and product offerings. I will present our decentralized shared-memory orchestration briefly and how it stays transparent and debuggable. It opens safe paths for external agents. What failed. What worked. What we are building next.

In this session, we’ll explore how developing and deploying AI-driven agents demands a fundamentally new testing paradigm—and how scalable simulations deliver the reliability, safety and human-feel that production-grade agents require. You’ll learn how simulations allow you to: - Mirror messy real-world user behavior (multiple languages, emotional states, background noise) rather than scripting narrow “happy-path” dialogues. - Model full conversation stacks including voice: turn-taking, background noise, accents, and latency – not just text messages. - Embed automated simulation suites into your CI/CD pipeline so that every change to your agent is validated before going live. - Assess multiple dimensions of agent performance—goal completion, brand-compliance, empathy, edge-case handling—and continuously guard against regressions. - Scale from “works in demo” to “works for every customer scenario” and maintain quality as your agent grows in tasks, languages or domains. Whether you’re building chat, voice, or multi-modal agents, you’ll walk away with actionable strategies for incorporating simulations into your workflow—improving reliability, reducing surprises in production, and enabling your agent to behave as thoughtfully and consistently as a human teammate.

Developing AI agents for shopping is just the first step; the real challenge is reliably running them in production across complex, mission-critical e-commerce systems—a significant MLOps hurdle. In this talk, we'll talk about Alfred, our agentic orchestration layer. Built with tools like Langgraph, LangFuse, LiteLLM, and Google Cloud components, Alfred is the critical piece that coordinates LLMs with our entire e-commerce backend—from search and recommendations to cart management. It handles the complete execution graph, secured tool calling, and prompt workflow. We’ll share our journey in designing a reusable agent architecture that scales across all our digital properties. We’ll discuss the specifics of our tech stack and productionization methodology, including how we leveraged the MCP framework and our existing platform APIs to accelerate development of Alfred.

Single-agent LLM systems fail silently in production - they're confidently wrong at scale with no mechanism for self-correction. We've deployed a multi-agent orchestration pattern called ""structured dissent"" where believer, skeptic, and neutral agents debate decisions before consensus. This isn't theoretical - we'll show production deployment patterns, cost/performance tradeoffs, and measurable reliability improvements. You'll learn when multi-agent architectures justify the overhead, how to orchestrate adversarial agents effectively, and operational patterns for monitoring agent reasoning quality in production. Our first deployment of the debate swarm revolves around MCP servers - we use a security swarm specially built for MCP servers to analyze findings from open source security tools. This provides more nuanced reasoning and gives a confidence score to evaluate the security of unknown MCP tools.

Stop thinking of `POST /predict` when someone says ""serving AI"". At Delivery Hero, we've rethought Gen AI infrastructure from the ground up, with async message queues, actor-model microservices, and zero-to-infinity autoscaling - no orchestrators, no waste, no surprising GPU bills. Here's the paradigm shift: treat every AI step as an independent async actor (we call them ""asyas""). Data ingestion? One asya. Prompt construction? Another. Smart model routing? Another. Pre-processing, analysis, backend logic, even agents — dozens of specialized actors coexist on the same GPU cluster and talk to each other, each scaling from zero to whatever capacity you need. The result? Dramatically lower GPU costs, true composability, and a maintainable system that actually matches how AI workloads behave. We'll show the evolution of our project - DAGs to distributed stateless async actors - and demonstrate how naturally this architecture serves real-world production needs. The framework is open-source as `Asya`. If time permits, we'll also discuss bridging these async pipelines with synchronous MCP servers when real-time responses are required. Come see why async isn't an optimization — it's a paradigm shift for AI infrastructure.