Sign in or Join the community to continue
Your AI Can Click “Buy.” Should It?
Posted Nov 03, 2026 | Views 6
Share
Speakers
Erica Hughberg
Community Advocate @ Tetrate
Erica Hughberg is a technical leader and community advocate passionate about helping engineering teams build scalable, secure, and human-centric application platforms. With a background in software engineering and a deep understanding of cloud-native technologies, she specializes in driving the adoption of open-source projects like Envoy Gateway, Istio, and Kubernetes Gateway API, which enable organizations to simplify traffic management, security, and API distribution.
As a maintainer of Envoy AI Gateway, she plays a key role in shaping the future of API infrastructure. She focuses on features to ensure organizations can securely and efficiently integrate AI-powered services while simplifying traffic management, security, and API distribution. In the Envoy community, she drives collaboration, mentorship, and contributions that advance the project and its adoption.
Lastly, as a believer in the power of storytelling, Erica enjoys translating complex technical concepts into engaging, accessible narratives in the form of social media posts, conference talks, podcasts, and educational content.
+ Read More
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community
At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.
+ Read More
SUMMARY
Your AI agent can read your inbox, plan your trip, and act through your browser. But when it uses your login, how much access should it really have?
In this episode of Debug Mode, we unpack agent identity, permissions, and the challenge of giving AI more responsibility without handing it the keys to everything. Through a fictional finance agent named Ryan, we explore prompt injection, why an agent’s intent matters, and what fraud detection could teach us about safer AI systems.
Then we get practical: daily briefings across email and messaging apps, using chat as a control center, planning a trip to Japan, and tackling German paperwork. Along the way, we ask where convenience becomes risk—and when a human should still make the final call.
+ Read More
CONTENT & TRANSCRIPT
Demetrios: [00:00:00] What I've realized is I'm starting to use the chat as my control center
Erica Hughberg: for
Demetrios: your life,
Erica Hughberg: not
Demetrios: just work. These agents are using your credentials to log in to things through the browser and using the UI that the humans are using.
Erica Hughberg: And I can use your browser. If you have like a long running logged in session in your Gmail now I can start acting as you.
Demetrios: I'm assuming like, oh, the models are getting good enough now to where they're not gonna go rogue, but you can never guarantee that.
Demetrios: Talk about agent identity. We don't-- We also wanna talk about use cases that we have- Yeah ... for our agents. I've got a ton these days, and I've got some big questions that I wanna hit with you. [00:01:00] But first, tell me about- Yeah ... agent identity.
Erica Hughberg: Well, okay. So, uh, for context, I've always been really interested in authorization and authentication and stuff like that, which, to be fair, is a very big topic to be interested in.
Erica Hughberg: Mm-hmm. And so last Friday, I was hanging out with my friend Brett and my friend and CEO, JJ, here in Atlanta. And if you don't know the BeltLine, if you come to Atlanta, you've gotta just, like, walk the BeltLine. Anyway, we're walking the BeltLine. We had dinner. We went for some drinks. And of course, the most entertaining topic anyone could talk about on a Friday night is i- agent identity and authorization.
Erica Hughberg: That's what you do on the BeltLine in Atlanta. That's what happens if you-
Demetrios: Just, like, tell me you're a nerd without telling me you're a nerd.
Erica Hughberg: Yeah, exactly. And, uh, oh, there's so many fun things we're talking about. But anyway [00:02:00] So my friend Brett, he goes like, "Why is agent identity even like different from anything else we've ever experienced?"
Erica Hughberg: He's like, "What's the deal?" This is-- And I wa- And one part of me is like, is it just marketing, right? Is it just a way to create another segment of the industry to say, "Hey, we do agent identity," right? Uh-
Demetrios: Yeah. What does agent identity even mean? That's the big question that I have, because- Yeah ... there's agents acting on your behalf, or there's- Yeah
Demetrios: the agents that we have now that's very popular with like Hermes and GroqBot, where you give them their own name and their own kind of- Yeah ... personas. So when you're talking about it, what are you referring to?
Erica Hughberg: So of course, this was like the first thing where me and my, my dear friends were like, "Wait a minute," like, "What are we talking about here?"
Erica Hughberg: Like, that was like step one, because are we talking about autonomous agents [00:03:00] that like just live out there, like that you live in your system, or are we talking about like your chatbot? Like which type of agent even are we talking about? We talking about-- Or are we talking about your coding agent? The, which is, by the way, coding agents that you use like Claude Code or Codex is much more human-supervised, right, than, uh, say more autonomous agents than may, you have may be setting up in your system.
Erica Hughberg: Right. So the most interesting part to me is like, okay, so if you are an authentication authorization nerd, you'd be like, "Haven't we solved a lot of these problems already?" Okay. As many things with software, loads of great ideas have existed for many, many decades, like half a century even, right? And the problem we have been running into Okay, so it's like what type of agent identity are we talking about?
Erica Hughberg: Let's just talk about the type of [00:04:00] agent that maybe... Hmm. So the, uh, the use case that I came up with to sort of anchor the conversation I was in. So it's like, okay, it- let's just pretend that we turn one of the, uh, our people that work, work for you into a agent, okay? So you have a sort of autonomous agent to which you delegate the responsibility.
Erica Hughberg: So let's, uh, as we, in our Friday night conversation called this agent, it's called Ryan. May be based on a real human, may not be r- based on a real human.
Demetrios: That sounds suspiciously on point. Like, do you work with a Ryan? I don't know, but-
Erica Hughberg: Yeah, no. Yeah. There, there is a real Ryan, but we do not think we can automate Ryan away.
Erica Hughberg: But anyway, let's, let's say we have a Ryan. And, um, but we gave Ryan a completely different job that Ryan's never done, and that was Ryan, say Ryan is now responsible for reporting and analyzing our [00:05:00] company's financial, you know, situation. So imagine Ryan is responsible for looking at our accounts every day.
Erica Hughberg: Ryan is responsible for also notifying the right people. Like, okay, so well maybe there are some, uh, invoices that are past due, right? Ryan needs to make a call. At
Demetrios: this company? Yeah.
Erica Hughberg: What's he gonna do, right? Like, if you have a human that you want
Demetrios: to- You're sending emails. Yeah, you're chasing down accounts receivable.
Demetrios: You're looking at- Yeah ... how much money's actually in the bank account, what hit, when you're expecting- Yeah ... money to hit. And then you have the expenses from the employees, and-
Erica Hughberg: Yeah ...
Demetrios: so looking at how much are we spending there. And then you're looking, you're trying to provision the next month and say, "Well, next month we're expecting to have to pay XYZ."
Erica Hughberg: Yeah.
Demetrios: You know, we have the fixed costs that are this much, but maybe there's some variable costs on top of it. So we all know-
Erica Hughberg: And then we'd ex- ...
Demetrios: how that works, you know.
Erica Hughberg: Yeah. And then we would, like, expect Ryan to, like, also create, [00:06:00] like, monthly and quarterly reports on all of this. But we don't want to, like, necessarily sit as a human to, you know, every day be like, "Ryan, go and look at the accounts."
Erica Hughberg: Yeah. "Ryan, it's end of month. Remember you need to send a summary," right, "of the stuff." We don't wanna have to do that. As with a human, we just wanna say, "Hey, Ryan, you are responsible for this, and we trust that you are intelligent enough to assign your own tasks." Okay. So where does this become quite interesting to me was, okay, but wait a minute.
Erica Hughberg: Let us just take a look at what we did. We just-- Ryan is now a piece of software, right? So Ryan is not human, so he can't, you know- Ryan is not human, so it's not like we can be like, "Oh, Ryan," sitting here in your performance review and be like, "Ryan, you missed every monthly report the last three months.
Erica Hughberg: This is a problem." Like they were always, they always had to remind you, like-
Demetrios: We're gonna put you on a PIP.
Erica Hughberg: Yeah, we've gotta put you on a PIP, right? But to be fair- [00:07:00] Like, so I said that sort of jokingly, but then my friends were like, "Yeah, but kind of don't you need to actually have a way of giving feedback to your agents, though?
Erica Hughberg: Like, are they behaving and doing the job you want them to do?" And that's kind of true. So is this a sort of fun evaluation situation, right? But okay, so then this c- so I was like, "Okay, but we have a responsibility, and then we have jobs." Okay? And this is, was quite interest- Like tasks? Tasks, yeah. Let's... Job, task.
Erica Hughberg: So not like... This is actually what confused me in my conversation when my friends were like, talked about jobs, and I was like, "Wait a minute. Can we, like, distinguish this?" So in this scenario, job and task, same thing. Like you br- you have a responsibility, and then you break it down into tasks that need to happen.
Erica Hughberg: Mm-hmm. And this is where authorization, you could argue, haven't we already sort of solved this? Because say we have Orion, a human, who is responsible [00:08:00] for this. Orion has been given access to certain systems in a company. Orion has, is authorized to do certain things, and maybe if Orion needs to access something specific, Orion can request access to something, and it could be temporary, right?
Erica Hughberg: So we solve those things for humans. We're like, "Okay, we're gonna get you temporary access to this for X amount of time." And that seems pretty straightforward, right? But keep in mind this, this is much s- this is quite slow-moving. This is what we always run into with agents, though. Like when it was humans that requested access from other humans, yes, we're slow-moving, but humans were a lot busy, and we only had, like, so much we could do at once.
Erica Hughberg: But if you have loads of agents, now this entire process becomes quite, like tough to handle, right? Are you gonna have a human that is saying yes, no to, to like, "Oh, I wanna do this task and [00:09:00] I need some extra access to be able to do it." You may even recognize some of these challenges when you even sit. If you don't put everything on auto mode and auto-approve all the access when you s- just sit with a coding agent, you have this like, "Oh, uh, am I allowed to do this thing?"
Erica Hughberg: And it's like allow once, allow always, and stuff like that.
Demetrios: Yeah.
Erica Hughberg: So the,
Demetrios: you can- These different policies that the companies have don't quite cut it either because you don't realize, oh, the agent needs that until it needs it, and it's a lot of time probably like a long tail of what you need and the access that you need and the scope that you need in that moment.
Erica Hughberg: Yeah.
Demetrios: The- I'm sure there's a lot of stuff that does fit into the 80/20 principle, like 80% of the cases are gonna fit into it, but then you have that long tail of 20%, I guess, if I'm thinking about it. Uh, I'm probably confusing you, but I don't know if [00:10:00] the majority would be one-offs for these authorization requests or if it would be like something that could fit into the policy.
Erica Hughberg: So this is what quite interesting, right? So if you go back to like the fundamentals of authorization and authentication and permissions and so forth, you want to be as precise and as limited as possible. That's like your best practice. Yeah. And as short as possible, and revocable, right? So you want it to l- you want to effectively operate in I don't trust anything that's operating in my system.
Demetrios: Uh, is that the like zero trust policy, or is that-
Erica Hughberg: So zero, so zero trust is
Demetrios: A little different
Erica Hughberg: It's more about-- Well, zero trust is related to this philosophy. Zero trust ends up being how you do not trust any system calling another system. Mm-hmm. Right? And everything has to prove their identity [00:11:00] all the time.
Erica Hughberg: And then you're on top of identity. Identity does not necessarily mean access right. So even if I can prove I am Erica, that doesn't mean I'm allowed to do something. So then I also have to say, "I am Erica, and I am authorized to take this action." So zero trust falls into that. But zero trust was fairly straightforward, and why was it straightforward?
Erica Hughberg: When I say straightforward, people be like, "Oh, Erica, zero trust is really, like, hard to get right." But there is one benefit that traditional software have over agentic software. Gonna come back to this a lot. But traditional software, you had code, you compiled it, you deployed it. The code that is running and that workload is always doing the same thing.
Erica Hughberg: Like, it isn't changing, right? Yeah. And this is actually why we ended up with a lot of more like microservices and also more defined services, because if you [00:12:00] imagine like one massive piece of code that could do 100 different things, managing that from an identity authorization permissioning point of view is actually harder than something that is more specific and more like purpose-built.
Erica Hughberg: But now we have systems that can improvise. They're like, "Mm, I'm gonna do this now." Right? The, they have, if you will, like this ability to not just do what they were originally told to do. So it was kind of fun with that, is one of-- So it breaks some of our principles that we've been working with, and there is a paper, I believe, from, is it 1988?
Erica Hughberg: That is about the confused deputy And we, and the agents, the improvising agents, like the problem was [00:13:00] described a very long time ago, effectively, like 40 years ago, right? Or is it more than 40? About 40 years ago, right? '88 to 2026, 40 years ago. So we talked about the confused deputy. So ima- imagine again our agent Ryan.
Erica Hughberg: Ryan now has an invoice, but in this invoice, you, this says, and says like, "Uh, I also want you to send an email to so and so," or, "I also want you to deposit X amount of money into this other account."
Demetrios: Mm-hmm.
Erica Hughberg: Right? That's effectively you can call that prompt in- injection, right?
Demetrios: Yeah.
Erica Hughberg: So we are now confusing.
Erica Hughberg: We're having contradicting instructions or additional instructions that weren't there. And So, so the problem was described a long time ago, and what are some of the things that we can do to [00:14:00] help these problems, right? The good part is that we actually have solved a lot of it, and it's got not so much to do with actual identity or authorization.
Erica Hughberg: It has more to do with risk determination of an action. So you know how we have been having systems for identifying fraud in banking,
Demetrios: right? Oh, so it's taking that same idea from the fraud-
Erica Hughberg: Protection ...
Demetrios: like the risk of this as being fraud and saying the risk of this as being the wrong action for the agent?
Erica Hughberg: Yes. So now this is the interesting part. How can you-- You need to have intent, right? So in- why are you doing the thing? What's the original intent? So if we come back to delegating responsibility, to dividing up tasks, what's also becoming different in what we need to do in authorization, it's not so much about [00:15:00] agent identity, I would like to say.
Erica Hughberg: It's more about identity-- uh, agent authorization. When I say that, I mean to authorize the action that the agent wants to take, we need to understand where it originated from. What is the intent, right? So Ryan is responsible for managing our finances. Ryan's intent right now is to create a monthly report for the leadership team on the finances.
Erica Hughberg: Now we have intent. How can-- So with understanding the intent and the action that's trying to be taken, we can now also apply effectively fraud detection to the action. So then I will-
Demetrios: Are you always going to get the intent, I guess is the big question, because a lot of times maybe it feels like you're in a privileged position if you know the intent.
Demetrios: I guess if it's always in your own [00:16:00] company trying to do things with your own data and money-
Erica Hughberg: Yeah ...
Demetrios: you should have that as a prerequisite. You don't have some random agents being able to cr- control this stuff. Uh-
Erica Hughberg: Yeah.
Demetrios: But yeah, it is, it, it is fascinating to think about, all right, we've got this whole world of fraud detection.
Demetrios: Can we mirror that to agent fraud?
Erica Hughberg: Yeah. And I, I believe we should be able to, right? And what I started wondering about, like how compute-intensive and how hard is it to do? So when I started to look into it, there's two levels apparently where you can look at fraud detection. You can actually apply-- The more expensive one is that you can actually apply agen- like agents, fraud agents, fraud detec- detection agents, right?
Erica Hughberg: Like you actually now add AI or even you can add like AI into it, right? To see, ooh, [00:17:00] is this fraud based on this intent? So if an agent has to always describe its intent and then all subsequent actions in the chain of getting the task done carries this intent with it. But this-- Then we could also analyze it.
Erica Hughberg: So that's quite expensive, right? That's an expensive action to sort of determine fraud. That is not actually how most fraud detection systems work. They don't throw AI at it, right? We've been doing fraud detection way before AI became commonplace. We have-- Then you can be like, okay, we have more traditional like machine learning spaces for fraud detection.
Erica Hughberg: Again, a little bit expensive. Then you have like the most basic fraud detection that is literally code, but like, well, you sh-
Demetrios: Rules-based system. Yeah.
Erica Hughberg: Yes.
Demetrios: So like- If it does e- it's basically a big long if statement.
Erica Hughberg: Yeah, like policies, like literal policies- Yeah ... being applied, like-
Demetrios: Literal policy. Exactly.
Erica Hughberg: [00:18:00] So there are like a lot of things that are happening here. The other thing about agent identity, which is a bit different, is we are spawning up way more identities than- Oh ... we did before, even with machine identities. There are some statistics assessing a general enterprise, like don't think AI, but like in a general enterprise, machine identities outnumber humans one to 20.
Erica Hughberg: So for each, there is like 20 times more, uh, machine identities than there are- You know what that reminds
Demetrios: me of?
Erica Hughberg: No.
Demetrios: In New Zealand. It's like New Zealand and their sheep. It, it-
Erica Hughberg: Oh, yeah, yeah. I've heard about that. The sheep. Yeah.
Demetrios: The sheep outnumber the humans 20 to one. Yes. So-
Erica Hughberg: Oh, what? Is that like... That's so funny.
Erica Hughberg: But yeah, so basically, they have a lot of sheep, uh, companies, right? Enterprises, they have a lot of sheep, but there is a lot of machine identities around, and now with agents, agents spawning agents, if you will, [00:19:00] and how do you inherit like the correct type and how do you narrow down per task effectively the authorization that needs to exist, the permissions, and how do you narrow it down?
Erica Hughberg: The good part is a lot of these ideas have already been explored over time, and why is it becoming a problem? It's becoming a problem because software engineers hate authorization and authentication. They've been taking shortcuts for the last 30 years. Yeah. Of course they have, and that's what we pay is,
Demetrios: is- OAuth doesn't just fix everything?
Demetrios: I thought OAuth-
Erica Hughberg: OAuth is- ...
Demetrios: throw OAuth at it and you're good.
Erica Hughberg: O-O-OAuth and tokens is literally the shortcut we took because it was sort of the best middle ground we could f- get to agree on. Yeah. And, um, it's- For example, tokens are difficult [00:20:00] because they, uh, tokens are minted at a specific time, right? And at that point they carry like scopes we call them, like what was authorized.
Erica Hughberg: But you only know what was authorized at that point in time at the minting of the token. If those things change, it is actually very expensive activity to, to revoke that.
Demetrios: To swap out. Yeah.
Erica Hughberg: Uh, to revoke it, like this token now lives out in the wild. It has an expiration time, so people have tried to work around it with like giving them really short, uh, expiration times.
Erica Hughberg: But yeah, so this is quite tricky and s- they don't carry intent. Again, they do not carry why you needed these things. So that to me is quite interesting, right? W- what do we need to adapt in this more dynamic world? [00:21:00]
Demetrios: But don't you think, take this example of an invoice coming through and having some kind of a prompt injection- Yeah
Demetrios: so that it's trying to not only get the invoice paid, but get a random amount of money transferred to another bank account. Yeah. And that prompt injection, as long as it is sophisticated enough-
Erica Hughberg: Mm-hmm ...
Demetrios: it's going to mask the intent in a way that some other agent will not be able to tell.
Erica Hughberg: Hmm.
Demetrios: Does that make- So- So the- Yeah
Demetrios: intent is going to be like everything's hunky-dory because the prompt injection isn't just saying, "Oh yeah, by the way, transfer this random amount of money to this random bank account." It's saying something that is plausible, and it is now when you have the LLM as a judge that's looking at it, it's gonna go, "Oh, okay, cool.
Demetrios: That seems all right." [00:22:00]
Erica Hughberg: Yeah. That's why you need the policies though, because that's where we need to go back to standard fraud detection. And-
Demetrios: And I think a lot of the fraud detection systems that I've seen have that cascading system, where it's like first things first, if it can't make it past the random hard-coded stuff-
Erica Hughberg: Yeah
Demetrios: it doesn't go through. That's like the great filter. Yeah. And then you've got some machine learning models, and then you've got some AI, because y- as you were saying-
Erica Hughberg: Yeah ...
Demetrios: this is expensive. And the machine learning models are, it's not just one. It's like-
Erica Hughberg: Yeah ...
Demetrios: a whole lot of them.
Erica Hughberg: They l- look at patterns of fraud.
Erica Hughberg: They like- Mm-hmm ... modeled out these patterns, 'cause people have been trying to do fraud for a long time, so we have a lot of great patterns.
Demetrios: But- Yeah. And they're getting updated continuously because the fraudsters are always trying to poke around and see how they can... It's a, it's literally a cat, game of cat and mouse.
Erica Hughberg: Yeah. Because also like hu- humans, so we talk about social [00:23:00] engineering with humans, right? And we may call it prompt injection when we talk about agents, but humans get prompt injected too, if you will.
Demetrios: Yeah.
Erica Hughberg: Because we have social engineering. When I went to university, I had this lecturer with, uh, he was researching social engineering.
Erica Hughberg: And what he did as one of his research, um, what do you call it, projects? I know. He wrote paper in the end. So this was a bit older, right? It was not in the day where you downloaded software on the internet. It was in the day where you literally, to your customers, you would send out CD-ROMs, like CDs for installing the software.
Erica Hughberg: So he Created this very believable packaging and this very important letter to send to all these companies that used, um, a piece of server software and said, "It's an important security update." And he made the paper look [00:24:00] very believable, the letter- Mm-hmm ... and the thing say, "Okay, you need to install it."
Erica Hughberg: Obviously, he was doing research, so all this software did was reporting back that it got installed, right? Mm-hmm. So he wanted to see how many companies actually believed it and installed it on their servers, and I think he found, like, it was near 70%. 70. Oh, boy. So th- yeah. But that's, like, still high, right?
Erica Hughberg: I think it was something- That's still high ... something quite high, like near 60, 70% of companies actually installed this mal- malicious. It wasn't malicious. All it did was say, "Hey, I got installed," and reported back- Yeah ... to the researcher. But imagine it was an attacker. The point was this was literally humans.
Erica Hughberg: They got a letter, they were told to do something, and they did it. So prompt injection, is it just social, you know, social engineering, hacking, kind of same thing, and w- we see these things happen. So yes, th- th- Well, can we
Demetrios: talk, uh, [00:25:00] can we change gears? 'Cause I wanna talk about different use cases for-
Erica Hughberg: Go for it
Demetrios: the ways that I'm using my agents these days, and I think it's fun.
Erica Hughberg: Like productive ways or like entertaining ways?
Demetrios: Yeah. No, productive only. Entertaining-ish, I guess- Yeah ... for I'm planning a trip to Japan right now with my daughter, and so we're exploring what we can do in Japan, and so that could be entertaining, but it's not- And you-
Demetrios: entertaining like you've been doing.
Erica Hughberg: Yeah. But, like, your daughter is about seven-ish-
Demetrios: Yeah ...
Erica Hughberg: years old? Yeah, and it's, like, the 10-year Pokémon anniversary, so I'm hoping you're doing-
Demetrios: Oh, I didn't realize that ... something fun. LLM didn't tell me that one.
Erica Hughberg: Oh. It feels like if you're going to Japan, you have to do something Pokémon related.
Demetrios: Well, I'm going to now. Thank you.
Erica Hughberg: Uh- Unless you wanna save your child from Pokémon exposure. I don't know
Demetrios: From the great capitalism of [00:26:00] Pokémon. Oh, what a j- they... Yeah, they were just using me so they could have this data, you know, the Pokémon Go. Yeah. And that's all they wanted, this
Erica Hughberg: spatial data. Was us scanning.
Erica Hughberg: Yeah, I, I mean, I contributed. I scanned loads of stuff in my neighborhood.
Demetrios: Yeah. They had us all fooled, and good on them. They did it, so- I'm a...
Erica Hughberg: They made
Demetrios: me walk ... whatever. They found a way to gamify it.
Erica Hughberg: Well, they made me walk. Yeah. I, I mean-
Demetrios: Yeah ...
Erica Hughberg: it's a, it's a fair
Demetrios: exchange ... it was a two-way street. We had some entertainment, and we got healthier for it, so the least they could do is get some data out of it.
Demetrios: Yeah. Anyway, the ways that I am using my agents right now, first of all, I am using computer use to the max.
Erica Hughberg: Okay.
Demetrios: Like-
Erica Hughberg: Oh, yeah. We talked about that last time, and I did also give Codex a go, 'cause I've been in my little Claude bubble. Yeah. So I went to Codex. I mean, I, I'm happy with it, but it is like switching from, like, a Windows to a Mac.
Erica Hughberg: Like-
Demetrios: No way.
Erica Hughberg: L- as [00:27:00] in- No way. I don't know how to explain it. No, but like, it's more like your brain is, like, trying to, like- ... figure stuff out. But I am very impressed with where OpenAI is today. But anyway, are you using Codex for your computer use for this trip planning?
Demetrios: So I'm u- I'm just using the normal ChatGPT desktop app in the- Yeah
Demetrios: work mode. And then- Yeah ... I use the keyword computer use. And one thing that I've started to do is I've set these triggers, much like we were talking about-
Erica Hughberg: Yeah ...
Demetrios: before, where, okay, right now we can't just give the responsibility to the agent and expect it to set up its own tasks for itself to complete that responsibility.
Demetrios: And there's a bigger question there, too, just, like, the outcomes that we're going for. And- Yeah ... so stepping back, saying, "These are your responsibilities," like, whatever, you're the head marketer. But- Yeah ... the head marketer, you're going to expect, is going to create some outcomes that they're going to drive towards, and then they're gonna have [00:28:00] tasks that are related to that.
Demetrios: But that's a different- Yeah ... conversation. Right now, let's stick to my use case. Uh, I have set up recurring computer use tasks because I can have my daily briefs traverse all of my message platforms. And so instead of it just connecting to one email address, it connects to all three or five of my email addresses, like my three personal ones, my two work ones, you know?
Demetrios: Yeah. And then it also looks at my LinkedIn inbox, it looks at my WhatsApp, it looks at my Telegram, it looks at my Slack, and then it gives me the daily briefing, boom, right there. Yeah. All the things that I need to keep in mind and any balls that I'm, like, dropping. That is one that is incredible, and what I've realized is I'm starting to use the chat as my control center because I don't go-- I don't [00:29:00] get this briefing and then say, "Okay, cool.
Demetrios: Let me go and switch over to Telegram now to write a response to that whatever message that came through. Yeah. What I do is in line, I just first I press a button and I talk to my computer, right? Mm-hmm. Because I'm using one of these dictation services. Oh. And I go line by line, "Oh, hey, for this message on Telegram, respond with this.
Demetrios: For the email that just came through, draft a response before you send it, and let me edit and read it. Uh, for this Slack message, make sure to forward it to this person." Yeah. And I'm going, and it's now become my control center. Hmm. For your life, not just work. That's... For, I'm trying to throw as much as possible at it and- Yeah
Demetrios: see where it's failing, and it's very rare that it does fail. It fails... The thing that is a little bit [00:30:00] frustrating is sometimes I will have too many threads going at once- Mm ... and computer use like trips over itself.
Erica Hughberg: Oh.
Demetrios: It's like, oh, Chrome keeps getting, the tab keeps getting switched because of this or that.
Demetrios: Mm. And I can't use the in ChatGPT browser for one reason or another because- Yeah ... I'm not logged in or on some reason here or there. But- Like it's- ... for the most
Erica Hughberg: part I just had this... Okay. I have to tell you something I do. I sit at my kitchen table, or not kitchen, it's like a dining room table, it's this big.
Erica Hughberg: Mm. And I have my, I have my work laptop and my personal laptop. Mm-hmm. Then I got my iPad up, and my husband laughs at me. But why do I sometimes feel like I need to use my personal and my work laptop and my iPad? Because this like segre- I feel like sometimes I need to segregate stuff out.
Demetrios: Yeah.
Erica Hughberg: And
Demetrios: I- Segregate your agents.
Demetrios: Yep.
Erica Hughberg: Yeah. So I do tend to like, yeah, have my personal stuff on my personal, right? My personal [00:31:00] weird projects. Like because, I mean- I hear you ... sometimes I have, I, I have like weird ideas sometimes I need to explore. So anyways, uh-
Demetrios: Random stuff. You're like, "I don't wanna burn work tokens for that."
Erica Hughberg: Oh, like I'm such a good citizen.
Erica Hughberg: Like I, I barely use my work tokens. Like ev- like I will use my Claude Max, like I don't know. I just like, unless it feels like necessary for work, I do not use my work tokens, but-
Demetrios: That is nice of you ...
Erica Hughberg: you know. I'm
Demetrios: very fine. Yeah. But, well, tell me, riddle me this one. In a world where computer use is my control center or- Yeah
Demetrios: and the chat is my control center, and everything is getting done through computer
Erica Hughberg: use-
Demetrios: Mm ... what does agent identity mean in that world? Because we know- No,
Erica Hughberg: that, that hurts my [00:32:00] head, dude. That, that hurts my head.
Demetrios: It's not like you have agents that are talking to agents or agents that are talking to services.
Demetrios: These agents are using your credentials to log in to things through the browser and using the UI that the humans are using.
Erica Hughberg: You, you are literally a cyborg, aren't you? You have like this- A little ... you have like this extension to yourself, uh- Yeah ... that is tied to physical hardware machine, but it's a mental extension.
Erica Hughberg: No, but, but, but to be fair, like that is what hurts my head because I do think that the conversation about authorization and identity and so forth is quite different when people are talking about almost like enterprise agents and like- Mm ... agent meshes. And I do think often in the world of AI, I see this disconnect between like [00:33:00] actual like day-to-day human usage versus enterprise architecture, system design, security, et cetera.
Erica Hughberg: Like-
Demetrios: Mm ...
Erica Hughberg: I find that happening a lot. And even if you go into this other segment of like AI startups, I feel they operate in a different model again from the enterprises. Yes. So it's, to me Like different worlds of challenges and problems. And what can be really confusing is when you try to take a model that applies in one of these worlds and you want to apply it to the other, whichever direction, and the model
Demetrios: breaks
Erica Hughberg: Don't
Demetrios: you think you're getting the pressure of the consumer use cases?
Demetrios: Like- Yeah ... me as a consumer using computer use-
Erica Hughberg: Yeah ...
Demetrios: I am going to inadvertently want that ease and that, like, [00:34:00] no friction experience, that user experience that I have, I wanna- Yeah ... transfer that to my work scenario.
Erica Hughberg: Yeah.
Demetrios: Because I see how easy it can be, and then I look at my work scenario and I go like, "Oh, this is so cumbersome.
Demetrios: Like, can't we just make it easy?"
Erica Hughberg: Yeah. And I think, like, what does the-- I think there's a difference again between the enterprise worker experience versus the enterprise autonomous agent deployment experience. Yeah. I would expect- Exactly ... those two differ- differ, and I would l- I would like to see a way of making that enterprise, you know, worker assistant, like the actual more like The assistant to the work of the person being as s- similar in experience as possible, because otherwise it's frustrating, right?
Erica Hughberg: I mean, I had this experience at work [00:35:00] where we were applying all of these, like, policies and guardrails and things, and suddenly the most innocent thing got flagged by a guardrail, right? Yeah.
Demetrios: That's-
Erica Hughberg: I, I literally just asked the coding agent, like, "What can you do?" And the thing that got flagged was the system pro- prompt of the harness, not what I wrote, right?
Erica Hughberg: So that's what I'm saying the most innocent. And to me, that's how do you balance it correctly? How do you get the right level of control? Because it's hard. It's not super easy. So guardrails, controls is like this one part of it, and then like hard identity and authorization, but also who did it. So yeah, if you imagine you have your agent that is local on your computer taking actions, very much an extension of you, who is the-- what is the identity?
Erica Hughberg: Is [00:36:00] Demetrios the identity? But also keep in mind, Demetrios does have different authorization levels in different scenarios, even as a human, right? So that's authorization. But proof of identity, the agent should probably be its own identity. We have to separate identity and principle. So this is another key, like auth- auth- authorization, authentication space.
Erica Hughberg: So Demetrios is the principal, right? So you're the principal. You're the human behind this. Then you have, in normal software, we talk about workload identities, the actual identity of the software that's running- Mm-hmm ... on behalf of the principal.
Demetrios: Mm.
Erica Hughberg: So therefore, the agent should have its own identity being the workload that is running.
Erica Hughberg: So it should still present, "I am workload Demetrios agent, and my principal is Demetrios," and then it should be able to take [00:37:00] actions. It should be. But then you have like all of, that's like a massively, as you say, like are you over-- There's probably a lot of overprivileged agents out there, and because we should, in theory, be scoping.
Erica Hughberg: So say you're asking your agent to do all of this, like managing your life. In theory, in a very clean architecture way, you-- when it reads your email, that should be scoped and temporary, so when it's done looking through your emails, that permission should disappear.
Demetrios: Yeah.
Erica Hughberg: And then not until it needs to access your email again should it be having...
Erica Hughberg: And that's where we talk about short-lived access. But again- Yeah ... that becomes really cumbersome to manage, and now we're back to why do we take so many shortcuts in authorization and permissioning, because it's easier to just over-permission. And-
Demetrios: That is so true. I haven't thought about that. I was int- I was initially thinking, well, if I have access to it as [00:38:00] Demetrios, then shouldn't my agent get access to it?
Demetrios: But it sets you up for a whole world of hurt if the agent does decide to go rogue and do something that you, as Demetrios, wouldn't want to do. I'm assuming like, oh, the models are getting good enough now to where they're not gonna go rogue, but you can never guarantee that. So what you want is-
Erica Hughberg: Yeah ...
Demetrios: the scope and the permissions so that it can't be Johnny Drop Tables and delete your whole database, right?
Erica Hughberg: Yeah. And Johnny Drop Tables, love it. Uh, but it's also like this idea that we We do a lot of authorization and permissioning and, uh, that happens in software behind the scenes where you may not necessarily see it. So like you open your Gmail, right? You have stuff in your browser that has like established that you are, like you have this, there's loads of things they [00:39:00] c- they do, right?
Erica Hughberg: To, there's cookies, there are things are stored in there to make sure that you are getting the right authorization as you access it. Mm. And so when we talk about these computer use agents, are we effectively relying on that they are using these existing interfaces? But again, browsers are notorious for being wonderfully hacked, and, uh, there are, you know, great things that people are doing to circumvent those common attacks.
Erica Hughberg: But again, just imagine that you can hack into someone's computer. If I could get onto your computer, however we wanna say I did, and I can use your browser. If you have like a long-running logged in session in your Gmail- Yeah ... now I can start acting as you.
Demetrios: Or all my bank accounts, all of that stuff. Of course, you can put me into a whole world of hurt, buy a bunch of [00:40:00] plane tickets, 'cause I'm logged in everywhere.
Demetrios: And for me as the user who wants to use m- the chat as my control center, it's just easier if I'm always logged in so that the agent can not have to log in and get in, get stuck- Yeah ... on this long-running task. When I say, "Go buy me a ticket to wherever, and make sure it's on these dates, and it doesn't cost more than this"-
Erica Hughberg: Yeah
Demetrios: it can go do that because I'm already logged in. It already has my credentials for my credit card, or they're saved already in the-
Erica Hughberg: Yeah, in the like browser memory. Yeah.
Demetrios: So yeah, that I do see the, the, there's a whole lot of danger there.
Erica Hughberg: And how do we make it easy? How do we make it safe
Demetrios: and- That's the price of convenience.
Demetrios: Yeah. It's like, wow, there's this world of convenience that I want to live in, but I'm really living on the edge, and it could come back and bite me in the ass.
Erica Hughberg: It's [00:41:00] like, it's tricky, but it's also that if you think about when you log into your bank, at least I've had this experience many times with my bank, and I want to transfer some money between accounts or even to someone else, then if it's under a certain amount, I can do it with my standard login.
Erica Hughberg: But if I'm trying to do a larger amount, it often wants me to reauthenticate, right? So it wants to know that I am me again. So however Since I am in the workflow, this, yes, it's a little bit inconvenient, but I do it and I move on, right?
Demetrios: Mm-hmm.
Erica Hughberg: And that is me gaining additional permissions during my workflow.
Erica Hughberg: So Re- reading or seeing my accounts is different from, you know, transferring money or adding a new recipient. I think we've all done that. We wanna add a new person, a new account to send money to, and we have to prove we are ourselves [00:42:00] again. So, uh, but how do we apply some of these learnings to agents?
Erica Hughberg: How do we do that in a good way? Yes, OAuth, the token exchange flow has a lot of it built in, but not necessarily, again, it doesn't capture intent. So, okay, but you're going to Japan though. Did the agent buy your tickets?
Demetrios: Uh, it found the tickets for me, but then- Yeah ... when it was time to actually pull the trigger and buy them, I went and I did-- It filled out all the forms.
Demetrios: Okay,
Erica Hughberg: great.
Demetrios: Like my passenger info and all of that, but then I went and I double-checked everything, and I ended up, like, putting the... I used my one password and then put the-
Erica Hughberg: Okay ...
Demetrios: credit card details into the im- payment information. So yes and no, I guess, is the answer there.
Erica Hughberg: Yeah. But it got you to the finish line and you had to [00:43:00] step over it.
Demetrios: Yeah, pretty much. It, it got me very, very close and, and, uh, few other ways that I've used it which have been absolutely game-changing have been I need to update my ID from a Spanish residency card- Oh, yeah ... to a Vernon residency card.
Erica Hughberg: Yeah, yeah.
Demetrios: So I asked it to do that. It fully filled out all of the forms that I needed to do.
Demetrios: They were all in German, and it asked f- for me just to, like, give it photos of things that it didn't already have access to.
Erica Hughberg: Yeah.
Demetrios: And I mentioned before, I think, the German- Driver's license ... driver's
Erica Hughberg: license. Yeah.
Demetrios: Uh, it did that. What it did not do, though, which I had my driver's license appointment yesterday, and I was not able to get my new driver's license 'cause I showed up there and I started- Yeah
Demetrios: saying, "Hey, I got this appointment," blah, blah, blah. And they said, "Oh, no. If you don't speak German, you need a translator." I was like, "It didn't say that [00:44:00] nowhere in the website or form," so it, it wasn't the- The
Erica Hughberg: agent sh- the agent should have booked you a translator, an interpreter or whatever.
Demetrios: It just- Yeah ... it didn't say anywhere that, uh, it...
Demetrios: That's like some unwritten rule that you need to know, and you don't know it. Oh, too bad. Like, you find out when you arrive. Show up. And this w- this was ridiculous though. This is just like the German system at its best, and- Oh, yeah ... I've come to love it because I went in there and I was like, "Well, uh, I'll just use my phone to translate."
Demetrios: And they were like, "No phones, no translator. You need a person here." And I was like, "Okay, well, uh-" Why? And they said, "You have to sign a lot of documents and you need to understand what the documents say." And I was like, "Listen up here. There is no world in which I am not going to sign those documents."
Erica Hughberg: Why do I need- But they want you to know why you s- what you're signing though.
Erica Hughberg: They want to be sure. That was what
Demetrios: they were saying. That was what... Then, [00:45:00] and they couldn't get through their head that like, okay, I will just use a translator on my phone to understand this. Like- Yeah ... it's going to be much better than my friend who I'm gonna bring with me who's not an official translator, and he's just gonna do some half-ass job of being like, "Hey bro, so this is like kind of saying this, I guess."
Demetrios: Yeah. And I-
Erica Hughberg: Yeah ... but I'm sitting there and I'm like, "I'm gonna sign the documents no matter what."
Demetrios: Yeah.
Erica Hughberg: Like, why are you not just letting me sign
Demetrios: and get on with this? And they're like, "You have to understand."
Erica Hughberg: I'm
Demetrios: like, "Oh."
Erica Hughberg: You have to follow the rules. You have to follow the rules. You need to know the rules so you can follow the rules.
Erica Hughberg: You have to
Demetrios: do that. And, uh- Exactly.
Erica Hughberg: Yeah.
Demetrios: I think- That's
Erica Hughberg: how
Demetrios: good old Germany works.
Erica Hughberg: I mean, I'm kind of a fan. I mean, I'm Swedish, so I'm a bit of a fan of the rules, but that seems, uh, that seems a little bit over the top.
Demetrios: No, th- I- This is the Germans and their rules. That's like, it's just like if [00:46:00] there weren't...
Demetrios: Like, why would we make rules if we weren't going to follow them? Yeah. You can't bend rules. No, because then otherwise why even have the rule? And so I've come to, uh, understand that that's just how it works, you know? And it was such a hard culture shock when I moved from Spain to Germany, because in Spain-
Erica Hughberg: Oh yeah
Demetrios: rules are-
Erica Hughberg: Fluid ...
Demetrios: this is- Suggestions. E- exactly. You can go with it, and depending on who you get, they enforce the rule more or less, or you can talk to someone and they can see the logic behind it. But not here. No, it's like if it is written, you follow that damn thing. Yeah. Otherwise, why would we write it?
Erica Hughberg: Yeah. Like, you need to follow the rule. I think, like, that's... I mean, I've, I've had problems the other way, right? Like coming to, I guess, the, America is not as probably as fluid as Spain. However, even going from Sweden [00:47:00] to c- growing up in Swedish culture and going to America, like- I'd be like, "Wait, why are everyone driving above the speed limit?"
Erica Hughberg: Like, there's a limit and everyone's driving above it. Like, what's going on? Like, it sa- it says like it's 75 miles per hour or whatever, and then people are driving 80. What are they up to? Like what's going on?
Demetrios: Yeah. It, it's a little fluid. It's a little fluid.
Erica Hughberg: Yeah. It's like- But- But yeah, I, I mean, I've been using AI for fun stuff.
Erica Hughberg: I think what I would recommend people-- I t- I messaged you this, but like ask AI what, based on what it knows about you, what your star sign is. Like, not based on when you're born, just based on like-
Demetrios: Just how you work.
Erica Hughberg: Yeah. What
Demetrios: you say.
Erica Hughberg: And like I had a... It literally... So I did that, and it told me, "You're a Gemini."
Erica Hughberg: And I am a Gemini, so it really messed with me. So-
Demetrios: That's incredible.
Erica Hughberg: And, and I do [00:48:00] not like the idea of believing that star signs have anything to do with your actual personality, but it-
Demetrios: Except they do ...
Erica Hughberg: I mean, that scared me. Did
Demetrios: you ask what your ascending moon
Erica Hughberg: is? I did. Like I did. Yeah, I did. Yeah. Yeah.
Erica Hughberg: And then I actually... So it got those things wrong, but then it was funny 'cause then I asked it, "Okay, but here's my actual birthday, my location, and rough timing." And then it was like, "Oh, but this makes a lot of sense because now you have all..." And I'm like, "I guess." But then it's like, it's funny though 'cause you could probably map anything to anyone and like make it make sense.
Demetrios: Yeah.
Erica Hughberg: But it, it did give me like all of these other things. I am so bad at those things. I don't even under- People talk about like moons and whatever, and there's Mars, and there's Jupiter. There's all of these heavenly bodies.
Demetrios: The house. Yeah.
Erica Hughberg: Yeah. I know.
Demetrios: Brilliant. Yeah. Well, this is awesome. Until next week.
Erica Hughberg: Yeah. I wanna find out your star sign, though. Your AI star sign, not your real one, Demetrios. Your AI star sign. I'll
Demetrios: reveal it next week. I'm gonna go do it [00:49:00] right now. I'm gonna have some fun with it.
+ Read More
Watch More
We Can All Be AI Engineers and We Can Do It with Open Source Models
Posted Nov 20, 2024 | Views 762
# AI Specs
# Accessible AI
# HelixML
How LlamaIndex Can Bring the Power of LLM's to Your Data
Posted Apr 27, 2023 | Views 2.9K
# LLM
# LLM in Production
# LlamaIndex
# Rungalileo.io
# Snorkel.ai
# Wandb.ai
# Tecton.ai
# Petuum.com
# Mckinsey.com/quantumblack
# Wallaroo.ai
# Union.ai
# Redis.com
# Alphasignal.ai
# Bigbraindaily.com
# Turningpost.com
[Exclusive] Gen AI Buy vs Build, Commercial vs Open Source
Posted Feb 02, 2024 | Views 1.2K
# GenAI Buy
# Build
# Commercial
# Open Source
# QuantumBlack