How iFood Designed Rosie Around Latency

# AI Agents

# Agentic AI

# Multi-Agent System

# IFood Rosie

How iFood built an async-first, multi-agent support system that embraces latency instead of fighting it.

July 28, 2026

Demetrios Brinkmann

When an iFood order goes wrong, the customer doesn’t care which API is slow, which tool call is pending, or which agent is waiting on a callback. They care that their dinner hasn’t arrived.

iFood, Brazil's leading food delivery platform, connects millions of customers with over 400,000 restaurants and merchants across the country.

Rosie, iFood’s support agent which is built for that very moment, sits inside the same chat interface people already use to place orders. But underneath it is a multi-agent system designed around one awkward production question: what should the user see while the system is waiting?

I spoke to César Gomes , a Data Scientist at iFood, about how his team designed Rosie for the moments when the system has to call external services. A lot of the conversation came back to latency. They don’t try to engineer it away. They design around it.

Designing for the wait

When an agent has to check an order status, request a cancellation, look up restaurant details, or call any external service, latency is part of the system. You can reduce it, but you can’t remove it.

So the question changes. Instead of asking “how do we make this fast?”, the team asks “what should the user see while we’re waiting?”

That framing changes the architecture.

The first decision is that everything is async by default. César described every agent in their stack as something that “wastes time to answer.” You can’t have one agent’s latency holding up the whole pipeline, and you can’t force a global timeout on the system because one service occasionally takes longer than the rest.

So the agents communicate through callbacks and event listeners. The user sends a message, the agent acknowledges it, fires off whatever it needs to fire off, and continues handling new messages from the user in the meantime. When the async event resolves, the response comes back.

Multi-agent and the right model for the job

The multi-agent setup matters here because not every step needs the same model.

iFood runs separate agents for separate jobs. Partly that’s about parallelising work - one agent consolidating context, another generating responses - and partly it’s about matching the model to the task.

The output reviser agent can run on a small, fast model. A risk assessment step can use a reasoning model. Splitting the work means you’re not paying for reasoning capabilities on every turn just because one part of the workflow needs them.

Although, it’s worth being honest about what this costs. Multi-agent systems introduce coordination problems that single-agent systems don’t have to think about, and César’s team has had to build patterns around those problems.

One of those patterns is rollback.

Double texting and rollback

Watch a real user interact with a chatbot and you see this constantly. They type “hi”, then “I have a problem,” then “with my order from last night,” each in a separate message, hitting enter as they go.

A responsive agent will reply to each one individually, and the conversation starts to look like four overlapping monologues.

The standard solutions are either to buffer incoming messages for some threshold of seconds, or to use a sentence detector. César’s team tried both and threw both out.

Buffers are threshold-dependent. Change your latency profile and the buffer feels wrong. Sentence detectors miss multi-intent messages, which are common when a user is upset or in a hurry. People don’t write one neat sentence per turn when their food is late.

What the team landed on is a rollback strategy, borrowed from a LangGraph piece. Before the agent sends a response, it checks the queue. If a new message has arrived during processing, the agent rolls back its state, joins the new message with the previous one, and sends the combined input back through.

César framed it as the way humans handle this. You don’t talk over someone. You wait, consolidate, then respond.

That pattern depends on the async-first architecture. If the agent is blocking on every message, there’s nowhere for the new input to go.

Memory at the end of the conversation

Memory is a topic where the pattern matters more than the storage.

César’s team uses the standard taxonomy of procedural, episodic, and factual memory. The interesting decision is when memory gets consolidated.

During a conversation, the agent uses short-term memory. No external service calls, no latency penalty. At the end of the conversation, a separate agent consolidates what happened into long-term memory.

If you consolidated after every turn, you’d be calling out to external services constantly and eating the latency cost on every message. Doing it at the end means the user never waits for memory operations.

The same thinking shapes how iFood handles personalisation. The agent doesn’t only pull a customer’s past orders into the prompt. It pulls context from the in-progress chat with the restaurant, from the delivery driver’s conversation, and from prior support sessions, so the agent doesn’t ask questions it already has the answers to.

That context also changes the response style. The agent can be more concise for users who write tersely, and more detailed for users who tend to ask follow-ups.

High-value hallucinations and the reviser agent

A lot of discussion treats hallucinations as a binary: the model either lies or it doesn’t, and the goal is zero lies.

César’s framing is more useful in production. The goal is zero high-value hallucinations.

A high-value hallucination is one that costs the business money or the user trust. The agent confidently confirms a refund that wasn’t authorised. The agent invents a policy the company doesn’t have.

A low-value hallucination is something the user can correct mid-conversation. The agent misreads what was said, the user clarifies, and the conversation gets back on track.

You can’t engineer hallucinations to zero. You can engineer your tooling and architecture to catch the expensive ones before they reach the user.

iFood does this with three things:

Structured outputs through Pydantic, so the agent’s responses parse into validated objects

LLM-as-judge with full tracing on the conversation

A reviser agent on the output path, which checks every outgoing message against iFood’s communication guidelines and guardrails before it goes to the user

The reviser adds latency. César was clear that it does.

He was also clear that the cost is worth it, partly because the reviser can run on a smaller model, and partly because the alternative - sending a hallucinated message to a hungry customer at 11pm - is worse than waiting another half-second.

For actions that can’t be undone, like cancellations and refunds, the agent confirms with the user before firing the tool call.

This is the place where tool-call evaluation matters most. If your agent has a non-zero chance of misfiring a destructive tool call, and you’ve designed away the confirmation step, you’ve shipped a bug into production.

Trace-level evals miss the point

César’s view is that a lot of eval tooling is too trace-centred.

A trace is one message and one response. Most of what makes an agent succeed or fail happens across multiple turns. You can’t evaluate task completion by looking at individual exchanges.

There’s also a cost problem with default-on eval tools. Run evals on 150,000 conversations, where the vast majority were fine, and you’ve spent a lot of tokens telling yourself things you already knew.

The signal is in the conversations that went wrong, and finding those is more work than the eval pass itself.

César pointed at end-state evaluation, referencing Anthropic’s tau-bench in this context, as a more useful pattern. Did the agent complete the task the user came in with? That’s the metric.

Toxicity scores and per-turn quality checks have a place, but they’re not the metric that tells you whether the agent is doing its job.

Closing

César’s simplest description of Rosie came near the end of the conversation: the agent is a conversational front end whose mission is to shorten the path to resolution.

That’s worth remembering once the architecture starts getting complicated.

Rosie is in production, handling real complaints from real users with cold pizza. The architecture exists because César’s team has spent a long time watching it break under load.

The patterns here - rollback, the reviser agent, end-of-conversation memory - weren’t in the first version they shipped. They got there by paying attention to what kept failing.

‍

For the ordering side of iFood’s agent work, listen to Daniel Wolbert and Rafael Borger explain Ailo, their conversational food discovery agent, and the “Goldilocks zone” between too fast, too slow, and just believable enough here: https://home.mlops.community/home/videos/the-latency-goldilocks-zone-explained