AI Agent Builder Summit SF

The future of customer engagement is voice-first. In this joint talk, Deepgram and Sierra explore how autonomous AI agents are evolving from text-based systems into natural, multi-language voice experiences. Speakers will share the MLOps considerations that make production-grade voice agents possible, including real-time speech recognition, model adaptability, data privacy, and deployment across regions. Attendees will gain practical insights into architecting voice agents that scale, adapt, and deliver human-like responsiveness for enterprise-grade applications.

Wait, what?

The last thing you want to do is learn how to use yet another agent framework. So when we launched ours, we made sure you wouldn't have to - by building an agent that builds agents.

In this talk, we'll share:

• What we learned while building our agent builder
• How we used evals and monitoring to maximize iteration speed
• And the surprising challenges of building an agent framework for both the most and least technical users

For ML engineers and infrastructure teams building AI agent applications, the shift from traditional ML models to AI agents introduces challenges that traditional MLOps paradigms do not address. Unlike static models, agents engage in multi-turn conversations, execute complex reasoning chains, and make dynamic decisions that can be difficult to trace and evaluate. This talk first explores the core challenges of AgentOps: - How do we track and debug an agent's chain of reasoning across multiple conversation turns? - What does it mean to evaluate an agent's performance when success metrics aren't clear-cut? - How do we capture and incorporate human feedback on agent behavior at scale? - How can we deploy agents with confidence? We'll then examine practical solutions to these challenges through MLflow new agent-focused capabilities, demonstrating patterns that work in production environments. You'll learn about implementing reliable agent tracing, setting up meaningful evaluation frameworks, and monitoring production agent performance.