The Control-vs-Magic Spectrum Building Agents

Thiago Cardoso: [00:00:00] I like to put agents and AI in a spectrum. Like in, in one end you have a lot of control, and on the other end you have a lot of magic

Demetrios: We're working on iFood Pago, which is the fintech arm of iFood

Thiago Cardoso: Exactly. So a lot of people know iFood, especially in Brazil. Most of Brazilians know iFood. It's a food delivery app and marketplace that, uh, the market leader in Brazil But a few people know that we have all this other business. W- one of those is our fintech business.

Thiago Cardoso: So our fintech business is how can we help and enable our restaurants to grow and have more support to grow inside our platform. So it's, it's not a, a bank that lives alone in his space, but it's a bank that serves [00:01:00] iFood and the iFood ecosystem. So success for our bank, our fintech, is seeing our customers, our restaurants grow inside the platform.

Demetrios: And I'm assuming you know a lot about the restaurants, so you can help them in different ways by giving them loans, but also understanding their credit and how risky the loans are.

Thiago Cardoso: We know how the restaurants operate, so we can, we can give loans to restaurants that probably wouldn't get a loan outside our fintech- Mm

Thiago Cardoso: but we can give this loan because we know that he's a good restaurant, that he's serving the clients, that he's actually delivering the food, and so on.

Demetrios: But why wouldn't they be able to get it outside?

Thiago Cardoso: Because around 80% of our restaurants are actually small restaurants-

Demetrios: Mm-hmm ...

Thiago Cardoso: in Brazil. So sometimes you see a single person that takes care of the restaurant, the packaging, preparing the meal, and then, um, taking care of the finance of the business as well, so it's a very small [00:02:00] business, and sometimes they don't have access to credit in, in regular banks.

Thiago Cardoso: So what we do is, since we know this is a good restaurant that is really operating inside iFood, we can, um, give a loan to that person.

Demetrios: And you can know what their order volume is, and you can understand- Exactly ... like, yeah, this is sustainable.

Thiago Cardoso: Exactly, because having working capital is very important for a restaurant.

Thiago Cardoso: Yeah. So they cannot grow if they don't have access to some capital, um, to start growing the business. Right. So this is very important for them. And since we know how it operates and, um, th- how it's actually working in the platform, we can choose the right amount, the, the right size of loan for this restaurant.

Demetrios: Yeah, and how are you bringing the agentic experience into that whole life cycle?

Thiago Cardoso: Yeah, we have a lot of different agents inside Pago. Um, in that [00:03:00] life cycle ex- specifically, we have Chat Bank, which, which is like the bank manager for the, for those restaurants, and it's a, a different kind of bank manager because it's not only, um, saying things about, like, your, um, your balance- Or gen- general information about your account, but it actually helps the entrepreneur to run the restaurant.

Thiago Cardoso: Mm. So it says, "Okay, you need to pay your employees." The, the, uh, the entrepreneur can say, "I want to pay my employees. I have these five employees, and those are the accounts." And then the, the bank manager can do the, the bank transfers. Nice. Um, it also can help-

Demetrios: And the bank, it... So me as a restaurant owner, I keep all my money in the bank, and I'm using iFood as my bank as opposed to whatever, like a ING or N26 type of thing.

Thiago Cardoso: [00:04:00] Different people choose different ways of working, but a lot of our restaurants use our account because of all these different features that we have for them. Yeah.

Demetrios: It makes it a lot easier, I can imagine.

Thiago Cardoso: Yes, exactly. So this- Mm ... chat bank assistant, the, the bank manager, it helps the, the, the restaurant owner with the day-to-day life of the restaurant.

Thiago Cardoso: So making payments, knowing that he has an outstanding invoice that he needs to pay, and this also is an opportunity for helping him with the working capital as well. Mm. So for example, we can say, "We see that you have an outstanding payment, but you don't have the money in your account right now. But I, I see that you have access to credit.

Thiago Cardoso: Like, you have these three different sources of credit you can use, but I'm seeing that the cheapest one is this one. So you can access the cheapest possible credit just to solve your problem right now, um- Mm ... in the, in the quickest [00:05:00] and cheapest way possible."

Demetrios: And this is all done through a chat interface?

Thiago Cardoso: Yes, exactly. WhatsApp is very popular in Brazil. And the restaurant owners are usually operating in WhatsApp as well. Uh-huh. So he's talking to suppliers, he's talking to customers in WhatsApp. Uh-huh. So helping him in WhatsApp works really well because it's, it's very comfortable for him.

Demetrios: Oh, so they, I imagine a world where they can just forward invoices to ChatBank WhatsApp, and then it's automatically uploaded?

Thiago Cardoso: Yeah. Is that right? It actually does that.

Demetrios: Yeah, yeah.

Thiago Cardoso: Yeah. He can even take a, a picture of something and say, "I want to make a Pix for this number." Pix is the Brazilian trans- money transfer protocol- Okay ... one of the, the protocols. So he can say, "I just want to transfer money for this account," and take a picture, and then the ChatBank takes care of the rest.

Demetrios: Yeah. It's like, "Pay this shit." Yeah. "I don't wanna deal with it."

Thiago Cardoso: Exactly.

Demetrios: Wow. And it will transfer the funds. It's just [00:06:00] I send a WhatsApp message-

Thiago Cardoso: Yes ...

Demetrios: and then you-

Thiago Cardoso: Um ...

Demetrios: make the transfer.

Thiago Cardoso: Um, it's, o- of course it's a secure procedure- Yeah ... so the, the restaurant owner has to tap his password, but- Mm ... everything is very well integrated.

Thiago Cardoso: Like, the user experience is very, um, seamless for him.

Demetrios: Mm-hmm.

Thiago Cardoso: So he just types the password, and then the transfer is made.

Demetrios: Wow.

Thiago Cardoso: Yeah.

Demetrios: Behind the scenes, what's it look like?

Thiago Cardoso: Behind the scenes, we have an agent there that for every message the user sends-

Demetrios: Mm ...

Thiago Cardoso: it reacts to it. So it, it, it receives the message, creates an execution plan.

Thiago Cardoso: So what should I do for the me- this message? So for example, I need to, um, start a money transfer. operation. So he calls this function, and then this function, um, becomes a UI element inside WhatsApp for the user to confirm the password, and then the, after the password's [00:07:00] confirmed, then the transfer is created in the backend.

Thiago Cardoso: Oh. So it's, the agent works like a, a front end, like a web front end, and you have all these tools in the backend that interface with the different APIs we have inside the, the fintech. Oh. So we have APIs for, um, the money transfer. We have APIs for getting the account balance. For authentication, we need to know if a WhatsApp number is attached to a fintech account, what account is that account?

Thiago Cardoso: Yeah. So th- th- these are all the different backend functions that we need. And, and we have other agents and other business as well.

Demetrios: Mm.

Thiago Cardoso: So inside our fintech, we have Beneficios, which is a corporate meal voucher that is very common in Brazil. It's like a credit card, but only for meals. And we have an agent that helps small companies [00:08:00] hire this, this service, and also operate this service in a day-to-day basis.

Demetrios: A- and hire the service, you mean, like, go and get you a credit card?

Thiago Cardoso: Um, not for the person, but the company's HR. Yeah. So I'm from a company HR, and I go there, and I want to hire these beneficios for all my employees- Oh ... and have this credit card for all my employees, and the thing with the agents here is that we can scale a very custom experience for this person.

Thiago Cardoso: So we can say, "Okay, what's the size of your company? What are your, um, needs for beneficios, for the, the meal voucher? Um, what are you worried about?" And sometimes people ask, "What's the difference between this product here and this other product?" And then we can- Yeah ... help the user guide his decision and, and make the best decision.

Thiago Cardoso: So this is one of the ways we use these agents.

Demetrios: Yeah. [00:09:00] Talk to me more about this idea of scaling personalization.

Thiago Cardoso: Yeah. '

Demetrios: Cause I think that's something fascinating that you all are doing, and It is mainly happening right now through chat, right?

Thiago Cardoso: Yeah.

Demetrios: But what are your thoughts around it?

Thiago Cardoso: Yeah. So, um, for me, when I started using AI, when I started working with AI, the, the interesting thing for me is that you can bring a very, um, customized experience to a whole new level.

Thiago Cardoso: You can, like, scale that, um, to a huge population. So for example, wh- when you look at iFood, not only the fintech part, but, but the whole iFood, uh, company, you have a lot of opportunities that you want to be more, um, tailored to the customers. So when you're ordering food, I want to offer you what's good for you.

Thiago Cardoso: Yeah. Um, when I'm trying to [00:10:00] help you find and hire beneficios, I want to clarify for you and make easier for you to understand how beneficios can help you in your company, not someone else's company.

Demetrios: Yeah.

Thiago Cardoso: And, and with agents, we can bring that to everyone. So you can talk to the person in their own context.

Thiago Cardoso: Yeah. So it's way easier to do that. And today, this is, this mostly happens by chat because it's a very open-ended interface. It's, it's kind of easy to put everywhere. But what I'm seeing right now is more and more we're starting to creating, like, um, custom UIs that are more tailored to the specific cases.

Thiago Cardoso: So for example, we, we have a case in iLu, which is our food recommendation agent, that having the, the right UI for the user not only saves time because he can just tap the dish that he wants, but it [00:11:00] also saves a lot of processing the background- Mm ... because you reduce, like, LLM calls.

Demetrios: Yeah.

Thiago Cardoso: So it's not only about having the right experience, which is easier and ni- and, and nicer to use, but it's also about efficiency as well.

Demetrios: Dude, that is so true. The tightrope that you have to walk to make sure that you're not just sending everything to the LLM. It's that fine line, and you get to be almost an artist about where you throw in the LLM and where you don't.

Thiago Cardoso: Yes. A- as the models improves, it's getting easier because it's not, um, that sensitive to your co- your, the, the way you write your prompts and everything.

Thiago Cardoso: Yeah. But there is still some magic to it. Like, maybe it's one word that you put that changes how the agent works.

Demetrios: Yeah.

Thiago Cardoso: Um, so how you evaluate that and, and how you [00:12:00] choose what you put inside the context is, it's very important.

Demetrios: Yeah, and just thinking about where the human plays the role, where do you actually have the human be part of the process versus not-

Thiago Cardoso: Mm-hmm

Demetrios: is also fascinating in my mind, because I imagine you're thinking a lot about how to get the merchant the least friction way- Exactly ... possible to do something. So for you, anywhere that the human has to do something is, it's like it better be a must that they have to actually do it, because otherwise you're not saving them as much time.

Thiago Cardoso: Yeah. But, but at the same time, you have to build trust as well.

Demetrios: Mm-hmm.

Thiago Cardoso: So when you use an agent and it starts doing everything for you, maybe you can get uncomfortable, like, "I don't want you to do that," or, "Just wait, uh, you need my approval to [00:13:00] do that." Yeah. So it's, it's a, a fine line you have to walk between giving the automation and making the person comfortable, um, with the decisions that are being taken.

Demetrios: Well, especially with money.

Thiago Cardoso: Yeah, exactly. So it's, it's a very sensitive topic, yeah.

Demetrios: Yeah. You're deal- I, I mean, if the wrong pizza shows up at my door, that's one thing. Yeah. But if you send my employees an extra two grand-

Thiago Cardoso: Yeah ...

Demetrios: I'm gonna-- That's a totally different problem to have, and if it's the agent's mistake on both of those parts, like one is a little excusable.

Thiago Cardoso: Yeah.

Demetrios: The other is like, "Hey, I want that two grand back, but I'm not going to not give it back to my..." Like it w- it's just a messy situation that you're in.

Thiago Cardoso: Yeah, exactly. Um, the, the hardness we build for different types of products and applications is completely different.

Demetrios: Yeah.

Thiago Cardoso: Like I, I, I like to put agents and AI in a spectrum.

Thiago Cardoso: Like in, in one [00:14:00] end you have a lot of control, and in, on the other end you have a lot of magic. Uh. So you have to choose where you're going to put your product in this spectrum, because sometimes i- it will feel a bit more stiff, but you want that because you need more control, and sometimes the, the risks are low, so you want to see more magic.

Thiago Cardoso: Yeah. You want things to happen and, and people to feel that wow moment when using your product.

Demetrios: Are there moments where you've just let it go completely to the magic side of that spectrum?

Thiago Cardoso: Yeah. I don't think that in fintech business we have that, but in iFood, in Ailo, for example, for, um, recommending food, yes, it, we want it to be a magical experience.

Thiago Cardoso: So instead of saying, "I just want something that I like," or, "I j- I just want a pizza here," we want people to ask, um, "What should I get for a romantic dinner with my wife?" Mm. So this is, uh, a different [00:15:00] level, um, of openness that we want from our users.

Demetrios: You talked a little bit about how the chat interface is changing and how we're starting to create these new components.

Demetrios: Mm. I'm also fascinated by that. I've seen-- There's a, there's a project that I saw on Figma that was, I can't rem- I think it's called Language GUI.

Thiago Cardoso: Mm-hmm.

Demetrios: And it's a bunch of Figma components that you can use inside the chat to help render different Things that you wouldn't necessarily get in a chat.

Demetrios: Maybe that's an embedded tweet- Mm-hmm ... or it's a voice note, or it's a snippet of a podcast, so that instead of always getting back text, I get different, more enriched data points, and now we're seeing that a ton with the mCPUI. Yeah. Which is that taken to the extreme, and I'm excited to see what people build with it.

Demetrios: I know that [00:16:00] some people in the MLOps community Slack were talking about how hard it was to work with mCPUI right now, just because it's so nascent, and you kind of expect that. But are you thinking through different components that you're now rendering? You talked about how you tap for authorization, so that's one.

Demetrios: What else is there that you're, like, trying to bring into the chat that you don't have to have the user leave for any reason?

Thiago Cardoso: Those are usually very specific and tailor-made for each case. Of course, you have like a generic agentic UI- Yeah ... toolkit that can be used, but I think the, the real experience, it, it becomes really good when you tailor the component exactly for the use case that we have and- Mm

Thiago Cardoso: what the user needs. So it's, it's not about the, only about the agentic part, the AI, but it's u- [00:17:00] it's the how the, the customer is using your product, um, in a day-to-day basis and the experience that he has. So those special components are very important, I think. We have this for, um, passwords, for confirming information with the user to make sure that he's aware of what the agent is doing and what's happening.

Thiago Cardoso: So this is something we are, we are experimenting about. But when I think of, um, agentic UI components and so on, I also see that spectrum from control to magic. Mm-hmm. Like, on one side you have all these agentic UI toolkits popping up right now with different CSS, HTML, or React components already packed that the agent can choose.

Thiago Cardoso: And on the other side of the spectrum, you have, like, free HTML with JavaScript and CSS, which, which also is working really well nowadays. So for, for some [00:18:00] internal tools, we are actually doing that a lot. Oh. So generate on the fly the HTML with this visualization, um, for this specific case.

Demetrios: So- Yeah, that's kind of like the playgrounds that I was showing you, yeah?

Thiago Cardoso: Exactly, yeah. Uh, like the playgrounds you were showing. For example, we had one, um, one problem in a, in a, in a dataset that we had, and we wanted to investigate in debug what was happening, and we created a simple tool, like, called Code, created this tool. I want to investigate this, this, and that. I want to see the lineage of the dataset- Mm

Thiago Cardoso: and why this happened, and it just generates the HTML with the visualization- Yeah ... the graph with the datasets being built, and it was very easy to see why the problem happened.

Demetrios: Yeah. I, I've... That's one of my primary use cases for this playground skill, is debugging-

Thiago Cardoso: Yeah ...

Demetrios: and getting visual representations, and then [00:19:00] being able to hone in on what the problem is and where it's happening.

Demetrios: And then you can just copy over the prompt and be like, "All right, fix that for me, please."

Thiago Cardoso: Yeah, exactly. It's a, a really cool application.

Demetrios: When you're talking about these different components that you're serving up to customers, do you feel like that is also personalization at scale? So now I'm going to get on the app and it knows the top four things that I normally do when I talk to it on WhatsApp or whatever, so it instantly says, "Hey, welcome back.

Demetrios: Here's... Do you wanna do any of these?" Like, quick buttons, you know, so I don't even have to think.

Thiago Cardoso: I'm talking more about context, um, in general- Mm ... that affects everything. So for example, I know that you have two employees, I know that you're not opening your restaurant tomorrow. Those are the things in context that start summing up to give a, a nice experience for the user.

Thiago Cardoso: Got it. So today I'm not going to say for, um, I'm not going to send you a [00:20:00] message, um, for you to pay your employees because I know you're not opening today or something like that. So- Yeah ... this context enriches every experience be- and, and one being the, the buttons I'm going to show, in what order, and what's- Ah

Thiago Cardoso: the user going to use first, like what's the most frequently used app-

Demetrios: Yeah ...

Thiago Cardoso: right?

Demetrios: And now behind the scenes to create that context, I know that isn't the easiest problem. How are you going about making sure that you have all of the right context at the right time?

Thiago Cardoso: Yeah, I think it's something that is more of a emergent behavior, because you have different systems.

Thiago Cardoso: Some are pretty simple but bring this feeling, and some are pretty complex. In, in one end of the spectrum, the complex part, we have LCM, which is our own custom, um, LLM tailored for e-commerce data that has the profiles, [00:21:00] um, that can build profiles for the user and say, "Thiago really likes, um, Japanese food, and during the weekends he likes to eat pizza as well," and so on.

Thiago Cardoso: So this is LCM that brings a whole new level of, of personalization for our, um, customers in iFood. But on the other end of the spectrum, maybe just counting how frequently you, you use e- every feature and use that to sort- Ah ... the, the menu is enough. So I think this is more of a emergent behavior that when we start assembling all the pieces and putting everything together, it seems like, um, it's pretty complex, but some components are complex, some are very simple, and they, wh- when they are all aligned, they bring this, this feeling.

Demetrios: Is there stuff with, like, your context engineering that is the low-hanging fruit?

Thiago Cardoso: Usually it's the, the basic [00:22:00] profile that you have on the restaurant or the person and so on. So for example, I will give an example with the, the customer-facing part because I think it's easier to visualize. But if I have my last 10 orders inside my context, it's pretty easy to see what I like eating.

Thiago Cardoso: Yeah. So for the restaurants, this can be the, the, the way it operates, so where it is located, um, what's the opening hours, and what's the cuisine. Mm-hmm. All these things can help creating this experience for the restaurant. So for example, we have a product which is iFood Shop that sells supplies for restaurants, and knowing the cuisine is, and the dishes that the restaurant serves is the information we use to recommend what products Um, he can buy.

Thiago Cardoso: So for example, if it's a pizza restaurant, I'm going to sell, for example, cheese [00:23:00] because I know it's something that he uses a lot.

Demetrios: Yeah, you're not gonna recommend chopsticks.

Thiago Cardoso: Yes, exactly.

Demetrios: Yeah. It feels like there is a law of diminishing returns with the context that you give it versus what actually is the most valuable in that output and what your, your end user is experiencing.

Thiago Cardoso: Mm-hmm. Depends a lot on the task you're actually trying to solve.

Demetrios: Uh.

Thiago Cardoso: So when, when I'm building agents, I see that if the technology was perfect and the models w- were perfect, we would, we would have only one single agent, agentic prompt that says everything the agent has to do with all the guides and all the information and all the tools, and it would have access to, like, 100 tools, and everything would be perfect.

Thiago Cardoso: But the reality is not like that. As you increase the prompt size, as you increase the number of different instructions the agents has to follow, as you [00:24:00] add more tools, it gets, um, harder and harder for the agent to, um, to have high accuracy, to do the right thing- Yeah ... at the right time. So what we see is that as the complexity grows, you start breaking this agent in s- sub graphs of agents, um, that, that solves a specific problem.

Thiago Cardoso: So for example, this agent starts really complex. I see that, um, when I'm talking about payments for the employees, I, I should have a special care when dealing with this part, so I will create a, a different prompt with a different context, with different tools for the agent to handle that specific case.

Thiago Cardoso: So when we look at the context, for me, this is very important, what we're trying to solve, and this context is going to change, um, given the task we're trying to solve.

Demetrios: Do you feel like there is ever a point where you're just gonna have sub-agent bloat because you've [00:25:00] created so many sub-agents- Mm-hmm ... now that the...

Demetrios: At some point, there's gotta be that Bottleneck, right? And so if, uh, or is it just layers of abstraction?

Thiago Cardoso: Um, I see more as layers of abstraction. It's like breaking a complex problem in smaller problems, so it's usually easier to handle that way. So I haven't reached a point of, uh, sub-agent bloat or something like that, and I don't even like to think a lot as sub-agents, because when we're building an agent, I see more like a graph.

Thiago Cardoso: Mm. Like, I have these nodes, and this node execute a specific operation, and it can go to other nodes, and so on. So a, a sub-agent can be a subgraph, can be a set of different nodes that solve a task. Um, and it's a very, uh, seamless thing. It's, it's not something like you, you say, "Oh, I'm going to have those three [00:26:00] sub-agents."

Thiago Cardoso: It's more something like grows into three sub-agents. Ah. So when I'm building a, a, a single product, I think more of a, an agent that starts to be assembled by different pieces and start to, um, being created by different pieces. When I think of sub-agents, I, I think in something more similar to, like, microservices.

Thiago Cardoso: Like, I have two different teams. One team builds one specific agent to solve a task. I have another team building an agent that handles another part of the business, and sometimes one will need to call the other, and then I have this exchange of information.

Demetrios: Tell me more about how it grows.

Thiago Cardoso: If you start with this single node that solves everything, this is the way I think all agents should start.

Thiago Cardoso: You always start with the simplest approach p-possible, which is a single node, um, w-with access to tools, and then it can choose what to do. And you [00:27:00] have this set of evaluations that for every case that you have, you evaluate what's the accuracy of your agent. Um, as you progress and as you get more data and you start expanding that evaluation set and you start, um, having real users with your agent, then things start not working so well, and you start to detect which points are not working really well.

Thiago Cardoso: So when the user tries to do action A, it's usually the, the, the agent usually messes up what it's trying to do. So here's something that we should break. So you go to your evals, you're going to see that in your eval, this specific case is not performing really well, and then you break that node in two nodes, and now I have one node that takes care of only this specific case.

Thiago Cardoso: Hmm. And then you go back to your evals and see how it's working. Now everything's fine. We, we keep, um, working with this agent As you progress, as you get more [00:28:00] data, people use it more, then you're going to identify another use case that's not working well, and then you do the same thing. Now I have to break this specific node in two nodes and so on.

Thiago Cardoso: So it's more of a iterative process. It's not something that you, I, well, I believe it's not something you can plan in advance. Mm-hmm. So I'm going to have this agent, I'm going to have all this, um, 10 nodes and it's going to be the best architecture possible. I think the architecture is being discovered as you gain knowledge of the problem.

Demetrios: Yeah, you don't try and boil the ocean with that first agent.

Thiago Cardoso: Exactly, yeah.

Demetrios: You see what it's doing right, what it's doing wrong, and then you try and create those edge cases and continue to have that feedback loop. I, I like that vision. Now I understand why You say it grows because it's that iterative loop and you water your little plant by giving it more evals and looking at it again, seeing where we can break it up.

Demetrios: [00:29:00] Okay, now there's a new branch. Now it's sprouted and, and this side of the tree is flowering. Cool, let's go there, let's water that a little bit more and give it more eval sets.

Thiago Cardoso: Yeah. In, in my opinion, complexity should never be planned. Mm. It, it should be a result- Yeah ... of, of, um, time.

Demetrios: Uh-huh.

Thiago Cardoso: It's not something that you prepare, you're going to start complex.

Thiago Cardoso: You have to reach that point after a lot of time of work. And, and for me, what makes a, a very effective team at building agents is actually running this feedback loop really quick. So if you can get new instance for your evals, run your evals, detect what's, what's wrong, growing your agent, and doing that loop really quick, those are the teams that are winners, and those are the agents that are winners as well.

Demetrios: I've never met an engineer that [00:30:00] told me, "I wish this system was more complex."

Thiago Cardoso: Yeah.

Demetrios: If anything, you try and go the other way, and like you said, you wanna try and keep it as simple as possible, but that feedback loop, you allow it to enrich the experience, and complexity comes from that, but that's only when you're faced with the reality of the world.

Thiago Cardoso: Exactly. It guides you when to add complexity and where.

Demetrios: So I know you had hot takes yesterday- Mm-hmm ... on Claude Bot, which is Modbot, which is OpenClaude now.

Thiago Cardoso: Uh, I really like OpenClau. It's something that, um... It's, it's a game changer. It, it doesn't bring a lot of n- new techniques or anything like that, but it's a very, um, good interface.

Thiago Cardoso: Like, you, you have it 24/7 by your side answering your questions using the tools, and the, the, the thing that is [00:31:00] different from OpenClau is that it actually develops itself. So for me, this is the, the biggest thing for OpenClau, because I can say, "I want you to create a new skill to search the internet for me and do this."

Thiago Cardoso: So for, I'll give you an example. For my OpenClau, I asked it to create a new skill that researches the internet and Twitter and see what are the trending topics for the day, and then it reflects in a art, uh image to create, and it has its consistent style. Oh, nice. So it has, like, a book of styles that it can use, and then it chooses for this reflection he's creating for this specific day, um, what is the style he's going to use, and then he generates the description for an image, and then generates an image with this reflection that represents this reflection.

Thiago Cardoso: So this is something I created there. And, and this is not trivial to do [00:32:00] because he had to create, um, a specific script to use the X API to fetch training data. It had to get data from the internet as well, then summarize this, and then he had to create a script to call the, um, image generation API and so on, and it's all nicely packaged, and I only had to talk to him to create that.

Thiago Cardoso: Mm. So this self-development process is what makes, um, OpenClaw so interesting, in my opinion. But at the same time, all this complexity and all this, um, freedom for it to create also brings a lot of security risks as well. So OpenClaw expands a lot on the attack surface that we have, um, not only by installing software and downloading software and, and doing...

Thiago Cardoso: getting skills from somewhere on the internet- Uh-huh ... and so on, which is one attack surface. But [00:33:00] also when you look at prompt injection and other problems that can arise from processing the, the textual information as well. And for the prompt injection, um, it's not enough to control the entry point. Like, for example, if you don't talk to my OpenClaude, then I'm safe.

Thiago Cardoso: This is not true because maybe you can get a calendar invite that has a prompt injection, or maybe someone- Oh ... sends you a message in WhatsApp with a prompt injection, or you receive an email with a prompt injection.

Demetrios: Yeah, because it's monitoring all of it.

Thiago Cardoso: Yes.

Demetrios: Yeah.

Thiago Cardoso: So the, the amount of, um, access you give, it's also the amount of risk you're taking.

Demetrios: You had mentioned, too, before the idea of how much your job has changed. As you were talking about-

Thiago Cardoso: Mm-hmm ...

Demetrios: what you've created with OpenClaude, I was like, "Oh, so basically what you were doing in 2009, you basically can put together in 20 minutes now with OpenClaude." Yeah. And probably more. Can you talk me [00:34:00] through now, like, how you think about working through the, all those changes that you've had over the years?

Thiago Cardoso: Yeah, I think that the, the cost of software, of writing code, it's already a fraction of what it was a few years ago. We're still in the adaption period in, in which people are actually understand how to use and how to make it work. But one example for my personal experience as well. It's been easier sometimes to write a tool to solve a problem than to find a, a suitable candidate in, in Google.

Demetrios: Oh,

Thiago Cardoso: wow. So for example, I have one website that I put all the tools, small tools that I code. Um, one example, I sometimes will create a, a presentation deck, some slides, and I want to put a video converted as a GIF. So I want to get a video, convert to GIF, and put on my presentation. And I always used to search for a [00:35:00] website, and I al- always was thinking if this website is storing my video there or not.

Thiago Cardoso: And everything- Which it is. Yeah. And, and what I did was I opened one, um, agentic programming tool, and I prompted to create, um, I want this small tool only using HTML and JavaScript, so it runs on the browser. Mm-hmm. And you're going to use FFmpeg- Yeah ... in WebAssembly to convert this video to GIF, and then it created, like, one shot the, the application.

Thiago Cardoso: Wow. So I have this repo that I put all those tools, and I have it online. Um, and for the specific case, it's easier to do that than to find a suitable candidate. So right now, this is getting very common, and as the models progress and get better, um, it's going to be more and more common to see that happen.

Demetrios: Actually, you are bringing up such a good point. Yesterday I saw a tweet from [00:36:00] Andrej Karpathy, and he said that he trained a GPT-2-

Thiago Cardoso: Mm-hmm ...

Demetrios: model. It was Capable of the same stuff for under 100 bucks Yeah, under 100.

Thiago Cardoso: Yeah

Demetrios: That is so crazy because you think about how much it cost to train back in, whatever, 2019 when they first did it, or 2017 when they were doing it, till now.

Demetrios: And so not only is software going trending towards being zero price, but the models themselves- Yeah ... are trending towards zero. It, it is seven years later, but still, man, if you can train it for under 100 bucks- Yeah ... in seven years from now, you're telling me that we're gonna be able to train Codex 5.2 for under 100 bucks?

Thiago Cardoso: You have access to a huge amount of intelligence for a, a very small price.

Demetrios: Yeah.

Thiago Cardoso: And when you look at code in this scenario, it's, it's so cheap that, um, you're going to get software for free. Yeah. And what [00:37:00] happens when we live in a world that software is free? Um, so my, my personal take is that it's going to increase a lot of demand for software because it, it's now, um, easier to build, so we, we want to build more stuff.

Thiago Cardoso: Mm-hmm. So for example, you, you said, "I'm going to create a small tool here just to debug a problem that I have." Mm-hmm. "Or just to experiment with a, a different character I want to put in my game."

Demetrios: Yep.

Thiago Cardoso: And, and this is not something that you used to do some

Demetrios: years ago. Even the game itself, I'm making a game for my daughter, and I would have never thought, I'm going to code a game so that my daughter can play.

Thiago Cardoso: Exactly.

Demetrios: I would have gone to Google- Mm ... and I would have found a game, and it m- probably would fit a little bit, but it goes back to that personalization at scale.

Thiago Cardoso: Mm-hmm.

Demetrios: And now I can have our own game that I choose the character names, I choose how they look and what they're doing, and what it's related to.

Demetrios: This one's [00:38:00] specifically around geometry, and it has horses, 'cause my daughter loves that, right? Mm-hmm. That's the personalization for me, N of one.

Thiago Cardoso: Yeah.

Demetrios: Well, and my daughter, so I guess

Thiago Cardoso: N of two. Yeah, and, and I believe that everyone will have, um, his or hers tool belt-

Demetrios: Yeah ...

Thiago Cardoso: of programs that you're going to build, and sometimes those will be disposable.

Thiago Cardoso: So I create a- Mm ... a program just to solve this specific case, and it's so cheap and fast that, okay, I can create, I solve my problem, and I'll never touch this software again.

Demetrios: Yeah. Yeah, and you'll want the novelty. Like, if I had this horsey game for my daughter, if it was faster and easier to create- Mm-hmm

Demetrios: 'cause right now I'm spending, whatever, a weekend on it or a week If it was easier to create, then I would probably make more games. I, like, today I'm playing the horsey game that's r- revolving around geometry. Tomorrow I'm going to [00:39:00] play the dog game that revolves around, whatever, vocabulary expansion or- Mm-hmm

Demetrios: something like that to try and help my daughter's learning journey in that regard, and then you augment all of the different disposable things you're, you're creating because it is so fast and easy to build.

Thiago Cardoso: Yeah, I, I, I saw one case recently that, um, one guy had his MRI scan.

Demetrios: Mm-hmm.

Thiago Cardoso: And there's only one software or a few software that can do that, and it, and it runs on Windows, and he said, "I want to, to see my files, and I want to see in my, uh, Mac or Linux or whatever."

Thiago Cardoso: So he went there to CloudCode and vibe coded his MRI- Wow ... file visualizer that runs on the browser. So this is something crazy. Wow. Yeah. It wouldn't happen, like, a few years ago.

Demetrios: Wow. You at iFood, I'm sure, in your current position, you're using a lot of software [00:40:00] still that is, like... You're, and you're probably paying for a lot of software.

Demetrios: Are you gonna just stop paying for it?

Thiago Cardoso: Um, no, I, I, I don't think so. I think, um, when you build a company around software, it's not only the code. Like, you have a lot of things happening around the code that are very important.

Demetrios: Mm.

Thiago Cardoso: Um, you have the operation, you have support, you have a lot of other things happening there.

Thiago Cardoso: And so I, I think this is not going to happen. It's not going to stop. But maybe we'll want to build more stuff that previously we wouldn't like to build. So for example, I have this back office thing. Maybe I'm using a external provider today, but now the cost to build a similar software is so, um, small and, and if we had a customize, a custom-made one would be so much better for us that now it makes sense [00:41:00] to actually put the, the team, the time, and the effort to build that because, um, the cost is decreased.

Thiago Cardoso: So I think that that could happen, yeah.

Demetrios: Yeah. With the MLOps community, we were able to save a ton of money because we vibe coded a newsletter platform.

Thiago Cardoso: Mm-hmm.

Demetrios: We were, were spending like... 'Cause once you get to a certain amount of subscribers, you're paying per subscriber-

Thiago Cardoso: Mm-hmm ...

Demetrios: every time you send a newsletter.

Oh.

Demetrios: And well, each platform has different cost structures, but the platform we were using, we were paying up the ass for it.

Thiago Cardoso: Mm-hmm.

Demetrios: And it was like, well you know what? Maybe we can try and vibe code something so that we don't have to pay 1,000 bucks a month, and it totally worked. And so now we saved 12 grand.

Thiago Cardoso: Yeah. Yeah, the, the, the world already changed. Like, in, in, in my opinion, what we're seeing is that we completely moved the [00:42:00] abstraction level that we work with code.

Demetrios: Ah.

Thiago Cardoso: So right now we're not thinking... It, it's the same thing that happened when we moved from Assembly to Fortran, for example. Yeah. This is...

Thiago Cardoso: Imagine John Backus arriving and saying, "Hey, I created this new programming language. It's really cool. It's called Fortran, and it's easier to program a computer in that." I imagine that people would say, "Oh, this is, this is not that good, because if I code in Assembly, I can choose every register. I can make every decision.

Thiago Cardoso: I can choose every instruction the computer is using." And, and he say, "Yes, but this is a higher level of abstraction," and so on. And this is something that completely took the world. Like, right now, most of programming happens in a high level programming language.

Demetrios: Yeah.

Thiago Cardoso: And with those AI tools, what I see is that we're giving another hop in this, um, abstraction level ladder, so we're working in a new higher level of [00:43:00] abstraction.

Thiago Cardoso: It's the, it's the, kind of the same shift that we had before.