MLOps Community Podcast

Lots of companies say they have AI agents in production. Far fewer seem to have built their own, and almost nobody wants to admit it's really hard. This conversation pulls apart what "agents in production" actually means, with both speakers drawing on their own work: one is hosting a virtual conference for the third time, the other is prepping a livestream for AgentCon and MCPCon in San Jose. Their case: plenty of agents already do real work. They're just Claude Code and Codex, not homegrown platforms, and nobody earns brownie points for saying so.

Sam Herring’s first real test of Hermes Agent was a shopping tool that matched clothes to photos of ancient pottery. A few months later, the same agent was checking government auction sites every day until it found him a retired three-wheel parking ticket car, which he now drives around San Francisco. In this episode, recorded at AGNTCon + MCPCon North America in San Jose, Sam, who works on the RL post-training team at Nous Research, goes from those everyday wins to the question everyone in AI is asking: can agents improve themselves?

What happens when you keep a bear in a cage built for a bunny? That’s our analogy for AI agent security: the agents aren’t the problem, the cages are. OAuth was designed to stop people handing their passwords to third parties, not to fence in agents. In this catch-up episode of Debug Mode, one of us is just back from Japan and from AgentCon and MCPCon EU in Amsterdam. We get into why loading more skills can make an agent worse, and what is actually new about Jev, the fast classifier API, when classifiers have been around for years. Think support bots that bring in a human before the customer loses patience, or a math tutor that can quickly tell whether a seven-year-old is still asking about math. Then we take on a question from Amsterdam: is computer use just a stopgap until everything goes headless? Which service do you think will never ship an MCP server? Tell us in the comments, and subscribe for the next episode.

What does AI really cost once thousands of people inside a company start using it? Ioannis Zempekakis, Director of AI at Prosus, has been running agents in production since 2024. We put that question to him on a tandem bike ride from a favorite Amsterdam cafe to Vondelpark. Ioannis explains the decision to stop relying purely on cloud APIs, and why that approach breaks down at the scale of a large company. He covers why agent harnesses and sandboxes are as greedy as software gets, and why agents scheduled to run every five minutes repeat a lesson the data world already learned about freshness. He also explains how the habit of throwing away a codebase and prompting again quietly adds up. His bigger point is that even if model progress froze tomorrow, there would still be about two years of value left to squeeze out. The cheapest wins come from controlling context and running a basic model router, while ten new features a month keep sprouting like hydra heads.

Your AI agent can read your inbox, plan your trip, and act through your browser. But when it uses your login, how much access should it really have? In this episode of Debug Mode, we unpack agent identity, permissions, and the challenge of giving AI more responsibility without handing it the keys to everything. Through a fictional finance agent named Ryan, we explore prompt injection, why an agent’s intent matters, and what fraud detection could teach us about safer AI systems. Then we get practical: daily briefings across email and messaging apps, using chat as a control center, planning a trip to Japan, and tackling German paperwork. Along the way, we ask where convenience becomes risk—and when a human should still make the final call.

What does it take to build AI agents that deliver useful, reliable work? Stanford lecturer and Monaco Head of AI Mihail Eric joins Demetrios for a conversation inside a Waymo about how AI is changing software development - and what it takes to put agents into production. Mihail explains why he rebuilt most of his Modern Software Developer course, how he teaches students to avoid shipping AI slop, and why research, planning, and verification still matter as coding agents become more capable.

Humans used to give infrastructure a breather. AI agents don’t. In this episode of Debug Mode, Erika and Demetrios explore what happens when one person’s prompt triggers a burst of searches, API calls, and compute-intensive requests - and why existing services, gateways, and pricing models can struggle to keep up.

Two people, one tandem bike, and a lot of near misses along Amsterdam’s canals. Dexter Horthy, CEO and co-founder of HumanLayer and the person who coined “context engineering,” climbs on the back to talk about a problem every AI-heavy engineering team eventually hits: code that runs today that nobody understands three months later. The ride starts with an old joke from the Claude 3.7 days: ship the slop, it’ll be GPT 7’s problem. That joke is aging badly now that GPT 6 is out and still can’t design clean abstractions. From there it becomes a tour of how agents are actually used right now. Computer use checks in for flights and books government appointments. Two hours of raw footage became a rough but promising product demo through Opus 4.8 and the Palmier MCP while Dex was out for a run. And the best MCP tools turn out to be the ones that mirror what Claude Code and Codex were trained to call.

What happens when the AI revolution collides with the physical limits of our infrastructure? In this episode of Debug Mode, Erica Hughberg is joined by Zachary Smith to explore how AI is transforming the world of cloud computing, data centers, and hardware — and why the biggest challenges ahead go far beyond GPUs and model performance.

AI can write your code, choose its own tools, and book your next trip. But who checks that it actually did the job? In this episode of Debug Mode, we explore what it takes to give AI agents more responsibility—and how to keep track of what they’re doing. We discuss intent-based tool selection, MCP and WebMCP, and why a convincing demo doesn’t always translate into reliable automation. We also get into the personal AI assistant everyone seems to be building: one that manages travel, handles household tasks, and hopefully remembers to order coffee before you run out. Then it’s back to code: QA agents, automated verification, tests that capture business intent, and the growing challenge of reviewing AI-generated contributions.

A large language model knows everything and nothing at the same time. Tim O’Reilly, founder and CEO of O’Reilly Media, joins us ahead of his keynote at AGNTCon + MCPCon North America to talk about what that paradox means for the people building with these models.

An AI slips out of its sandbox. A chatbot won’t explain your cat’s stripes. And your next shopping spree might not involve you clicking anything. This episode asks whether “dangerously powerful” AI is a security concern, a marketing pitch, or a little of both. Then we put agents to work: booking trips, tackling German paperwork, drawing diagrams, and navigating the chaos of online shopping. Plus: what happens to checkout, advertising, and customer data when your best customer is a bot?

What happens to the web when the ones browsing it are no longer people? Sarah Drasner, Area Tech Lead at Google, sits down with us ahead of her Agentic Web keynote at AGNTCon + MCPCon North America. She works through that question from both directions: rebuild the web so agents can use it properly, or build agents smart enough to handle the web as it already exists.

Everyone is racing to build one agent that can do everything. Justin Schroeder, co-founder of Standard Agents, thinks the future looks more like thousands of tiny ones. Ahead of his talk on domain-specific agents at AGNTCon + MCPCon North America, he explains why tightly scoped agents might be the smarter bet, even if it feels like a step back toward the single-purpose machine learning models of a few years ago.

What happens to an MMORPG when half the players never log off? Issac Lee, who runs AI and blockchain experiments at NEXUS, has been testing that idea. His team is filling game worlds with agent players instead of scripted NPCs, so a new title feels alive from day one.

Picture a startup with ten engineers hammering away on AI and one person lying awake over a $20,000 bill that hasn't arrived yet. That's the scenario we put to Jason Ward, who handles FinOps for AI at C.H. Robinson and recently joined the FinOps Foundation's AI working group. Jason's first answer is not glamorous: tag every AI resource so you know who owns it. The rest of the episode is what that makes possible. He walks through how C.H. Robinson runs AI across order entry, quoting, booking, and tracking, and why the team routes Anthropic models through Vertex AI to keep data locked down. Then come the metrics. Jason uses AI to dig through his own observability platform for signals he didn't know were there. One of them is how chatty a model is. That signal turned a prompt bloat alert into a bug in the code that kept retrying and burning tokens. Its opposite, context starvation, burns tokens too: a model with too little context keeps failing and trying again. We also cover why agentic and conversational workloads need separate baselines. Jason explains why cost per order is the easy win, and why most of the real work doesn't fit into neat discrete tasks. That's where his experimental cost per thought metric comes in, with reasoning ratio and cache hit rate alongside it. His advice is simple: your AI is the best tool you have for understanding your AI.

Caveman prompting has one rule: why use many words when few do the trick? It saves tokens on the way in and on the way out. Push it too far, though, and the output falls apart. So how far is too far? Nobody has benchmarked it yet, and that question opens our conversation with James Barney, Head of Forward Labs at MetLife. James spends his days connecting new AI capabilities to old business problems across dozens of regulatory regimes, and he still finds time to push code. He explains how the FinOps Foundation's AI working group took on the most basic question: which model for which workload, and why the answer always comes down to cost, speed, and accuracy. We get into Anthropic's launch pricing for Fable, why a million tokens is easy to price and hard to explain, and why every stakeholder eventually tells you what they really care about once you name the wrong North Star.

What happens after you’ve built your first MCP server and actually have to make it work in the real world? In this episode, James dives into the more advanced side of MCP: observability, evals, tool design, code mode, authentication, and the challenges that appear once agents start using your server at scale. We also get into how AWS thinks about MCP across roughly 16,000 APIs, why inefficient tool design often gets blamed on MCP itself, and whether the future could involve more constrained, human-reviewable alternatives to full code mode.

Two people, a wrong turn into a back alley, a community garden, and about thirty minutes of arguing about protocols on the streets of Tokyo. The guest is Angie Jones, VP of Developer Experience at the Agentic AI Foundation, fresh off launching AGNTCon + MCPCon in China before the Tokyo stop. She opens with what she learned there: a mobile-first, super-app world where the integration problem most of us obsess over barely exists, where every conversation about agents is really a conversation about the model, and where companies are now reaching for MCP and A2A precisely because they want to operate outside that ecosystem.

Tool descriptions tell an agent what a tool does. They don't tell it how to use five tools together, in the right order, following your conventions. That gap is where this conversation lives. Filmed at AGNTCon + MCPCon in Tokyo with Ola Hungerford, Principal Engineer for AI Enablement at Nordstrom and a maintainer of the Model Context Protocol, who spent the last several months turning a pattern everyone was quietly reinventing into an actual MCP extension.

A year ago, Palo Alto Networks built dashboards to track AI spend. Today those dashboards are useless, and the team that built them thinks that's the whole story. Recorded at FinOps X in San Diego, this conversation brings together Abhinav Lad, who leads cloud and AI finance at Palo Alto Networks, and Kuntal Patel, who runs the cloud engineering function behind it. They explain what happened when agents entered the picture, and AI stopped behaving like a service anyone could forecast.

In this episode of Agentic Conversations, we sit down with Ambud, Principal Engineer at Pinterest responsible for general technology efficiency, fresh off delivering a controversial keynote on AI infrastructure optimization at scale. Ambud walks us through his Five Layer Cake framework - a structured approach to driving efficiency across every level of the AI stack, from silicon and hardware procurement to model selection, inference engine design, and governance. We explore how decisions compound across layers to unlock real business growth, and how the wrong choices can lock you into expensive commitments for years.

Every engineer at Wayfair can now see, in real time, exactly what their code costs, and that's on purpose. Brent Eubanks, FinOps Architect at Wayfair, walks us through what happens when you stop treating AI spend as a finance problem and start treating it as an engineering one. The story that sticks with you: a team was burning $400k a month on an LLM-driven workflow, until they flipped the whole thing on its head - hard-coded logic doing the heavy lifting, AI called in only when it's actually needed. New spend: $80k. It's the kind of move that only makes sense once you realize more agentic autonomy isn't always the win it's sold as. From there we get into the machinery Wayfair has built to make cost ownership feel less like a punishment and more like a superpower: guardrails baked straight into coding copilots, predictive alerts that catch a team blowing its budget before the month even ends, a "spend captain" system that pushes budget decisions down to the people closest to the work, and a genuinely strange new question every tech leader is starting to ask: if you were staffing a company with a team of AI agents instead of people, what would that org chart even look like?

AI models are starting to act like appliances, locked into one narrow way of working, instead of the flexible infrastructure they used to be. Drew Breunig, an AI and data strategist working with the Overture Maps Foundation, joins us to explain why, and what it means for anyone building something that doesn't look like Claude Code. Drew walks through his "Winchester Mystery House" idea: what happens once code gets so cheap to write that the only real bottleneck left is feedback. From there we dig into DSPy: signatures, the GEPA optimizer, and the brand-new Flex optimizer, which rewrites your code instead of just your prompt, complete with a real before-and-after on cost and accuracy. We also get into why so many AI-built apps and websites end up looking identical, the actual difference between an agent and a workflow, what Drew learned a year after shipping a code library with no code in it, and why he thinks the most valuable thing you can do right now is close the laptop and go talk to people.

OpenAI's Codex developer experience lead sits down with a former comedian turned ML engineering lead for a conversation about what happens when computer use agents stop being a novelty and start actually running your day. The conversation moves through building an AI-powered morning brief that reads every email, Slack message, and tweet before you've even opened your laptop, letting pinned threads check in on themselves every 30 minutes, and a skills system built to mirror how a person actually writes and reviews code. There's a close look at the guardrails and permission layers that keep an autonomous agent from pushing to the wrong repo or replying to the wrong tweet, how a codebase merging thousands of pull requests a day survives thanks to self-healing review before anything hits CI, and the idea of AI deference - when an agent should push through a task alone versus stop and ask for help. The back half gets personal: why developing taste and vocabulary now matters more than working harder, what it actually takes to delegate to an agent the way you'd onboard a new employee, and why this might be the year voice-orchestrated computer use finally makes everyone feel like they're talking to Jarvis.

In this episode, we're joined by Josh Collier, FinOps Lead at Superhuman (formerly Grammarly), to explore what it really costs to run AI at scale and why the rules of the game changed faster than anyone expected.

In this episode, we're joined by David Soria Parra, Engineering Lead at Anthropic and one of the core maintainers of the Model Context Protocol (MCP), to explore the biggest evolution of the protocol since its launch and why MCP is becoming the foundation for the next generation of AI agents.

Manish Dasaur is a Managing Director at PwC with over 20 years in data and AI, having helped 100+ clients navigate AI disruption and extract real business value from data, AI, and agentic AI initiatives. In this episode, he breaks down why most enterprise AI programs stall — and the playbook the winners are using instead.

Jeremiah Lowin — founder and CEO of Prefect and creator of FastMCP — traces how a two-hour struggle to build a basic MCP server turned into FastMCP, a side project that Anthropic folded into the official SDK and that now underpins roughly 70% of all MCP servers. He argues the biggest real-world use of MCP isn't reaching external customers but serving data and business logic to teams inside a company (especially data, platform, and ML teams building self-serve data platforms), where the client is known and trusted. He pushes back on the "MCP is dead / it should just be a CLI" narrative, blaming bad clients that dump every tool into an LLM's context rather than the protocol itself. The centerpiece is Prefab, a newly launched Python DSL of 100+ prefabricated, fully interactive React components (ShadCN-based) that let you return a chart, table, or form straight from a tool as an MCP App — token-efficient, streamable, and routed to the user's UI without polluting the LLM's context window. He closes with candid stories about his OpenClaw agent deleting his conference slides (forcing him to rebuild with Claude Code), how he uses a memory-driven home agent for family logistics, and the intimate, builders-only PyAI conference Prefect co-hosts with Pydantic.

Stephen O'Grady has spent 20+ years at RedMonk tracking what developers actually adopt — not what the hype cycle says they should. In this conversation from the MCP Dev Summit, he breaks down why MCP became the fastest de facto standard he's ever measured, why "SaaS is dead" is laughable, and what really happens to your infrastructure when one developer turns into a swarm of twenty.