Accuracy & Reliability of AI Agents: Virtual Summit

MEETING

3:00 PM - 5:00 PM GMT

October 15, 2026

Accuracy & Reliability of AI Agents: Virtual Summit

As AI agents move from demos into real-world workflows, accuracy and reliability become critical. An agent can produce impressive results in a controlled demo, but how do we know it will make the right decision consistently, handle unexpected situations, and continue to perform reliably in production?

Join the AAIF Accuracy & Reliability Working Group for a virtual summit exploring the practical challenges of building, evaluating, and deploying dependable AI agent systems.

The event will bring together practitioners and researchers to share how agents are evaluated today, what we can actually measure, where current approaches fall short, and what teams are learning from real-world deployments.

The session will feature talks and open discussion covering agent evaluation, benchmarks, reliability, verification, failure analysis, and practical lessons from deploying agent systems.

We’ll also dig into the harder questions: What does “reliable” actually mean for an AI agent? How should we measure it? What happens when existing evaluations don’t reflect real-world behavior? And what should the community be working on next?

Expect practical examples, lessons learned, and an open conversation about the challenges and opportunities in making AI agent systems that people can genuinely depend on.

Speakers

Rahul Parundekar

Founder @ A.I. Hero, inc.

Sanjeev Suresh

Staff ML Engineer @ Uber

Stephan Rabanser

Postdoctoral Researcher @ Princeton University

Giulio Grassi

Engineering Technical Leader @ Cisco Systems

Nikhil Simha

Chief Technology Officer @ Zipline AI

Casper Nielsen

AI Engineering Team Lead @ Diagrid

Agenda

3:00 PM

3:10 PM

GMT

Opening / Closing

Welcome & Introduction

Event introduction, overview of the session, and AAIF/AgentCon updates

+ content:video.read_more_button

3:10 PM

3:35 PM

GMT

Presentation

Towards a Science of AI Agent Reliability

In this talk I show how we can study agent reliability as a distinct axis from agent capability and how recent model releases have mostly improved capability but not reliability.

+ content:video.read_more_button

3:35 PM

4:00 PM

GMT

Presentation

Missing Abstractions for High-stakes agentic systems

Lessons from building large-scale customer-facing agentic systems at massive scale

+ content:video.read_more_button

4:00 PM

4:25 PM

GMT

Presentation

Silent Failures, Loud Consequences: Cognitive Challenges in Multi-Agent Systems

How hidden cognitive failures compound in multi-agent systems and what it takes to detect and mitigate them.

+ content:video.read_more_button

4:25 PM

4:50 PM

GMT

Presentation

Crash-Proof Agents: Durable Execution as the Reliability Layer for AI Agents

How durable execution, persisting every step of an agent's loop as a workflow, lets agents resume exactly where they left off, retry safely, and avoid repeating side effects

+ content:video.read_more_button

4:50 PM

5:00 PM

GMT

Opening / Closing

Closing & Open Discussion

A final discussion bringing together the key themes from the summit, with speakers and attendees reflecting on the biggest reliability challenges, practical approaches, and open questions around building dependable AI agents.

+ content:video.read_more_button