Omnigent: Composition, Control, and Collaboration for AI Agents

Denny Lee: [00:00:00] That's exactly what's going on right now, which is basically that service-based architecture, SOA, you know, um, where we've switched from this idea that we had to go ahead and originally it was one central command to understand what the budget was, and now that got pushed down to developers because the developers could actually then care, understand, and help control that.

Denny Lee: Tokenomics is the exact same thing, 100%.

Demetrios: I will have you know, I'm still waiting for the coffee from Taiwan that you promised you were gonna send me after that blog post you wrote

Denny Lee: about the- And I, and I'm glad you're recording it to basically make sure that I'm held like- This is

Demetrios: accountability 101.

Denny Lee: Yeah, this is a g- No, no, you're right, I'm wrong.

Denny Lee: Uh, the problem is that I'm actually not going this summer, um, unfortunately. But if when I do, yes, there is a specific set of farmers and a specific set of places that I wanna go to. And it's [00:01:00] actually, interestingly enough, related to agents, and we will get into that when you're ready to go. But it's actually related to agents, my friend.

Demetrios: Are, are those some of your agentic workflows that you've got going on, finding different micro fincas around the world?

Denny Lee: Not just that, but also which regions in Taiwan versus which regions of Japan or Korea- Ooh ... specifically for matcha and shit like that.

Demetrios: What? How, h- how are you getting agents to do this research?

Demetrios: Is it just, like, exploring the internet?

Denny Lee: Yeah, it's exploring. So, so basically, okay, so-

Demetrios: Reddit threads?

Denny Lee: We're probably jumping ahead here, but, like, I'm using Omniagent's, uh, Polly. Uh, uh, sorry, Debbie, excuse me. That's the one where basically they debate each other, right? And so in this case, it's basically, uh, my ChatGPT, you know, 5.5 versus my Claude Opus 4.7, right?

Denny Lee: And they basically debate each other. And so basically, I start... I, I already have enough, like, agentic memory and skills MDs and all this other shit that provide, like, a lot, a huge background on [00:02:00] what I'm specifically looking for anyways, right? So if you go to my blog, either like the davegulley.com blog, the personal one, y- you'll remember that there's this one asinine-y long Taiwanese bl- uh, blog about Taiwanese coffee.

Demetrios: Uh-huh.

Denny Lee: Right. It was supposed to be- That was the one I read. Yeah, yeah, exactly, yeah. It, the, it was supposed to be small, and then it turned out to just basically be this long diatribe that went from, like, "Here are all the great coffee shops in Taiwan," to, like, "Let me go visit a farmer in Taiwan, in Nantou Province, uh, Nantou County, specifically to go ahead and understand how he's growing the beans and how, like..."

Denny Lee: Yeah, like, and how it used to be betel leaves, right? And then how the Taiwanese government was switching over to convince people not to do, grow betel leaves, because betel is a, it, it's like a c- like cigarettes. People, uh, get super addicted to it, but also- Oh, yeah ... the leaves are huge. So the water actually, uh, degrades the land underneath it, so the Taiwanese government's trying to convince people to grow coffee.

Denny Lee: Well, the, this is already too long, but the TLDR version of this is that Nantou County's actually famous for growing tea, and first, [00:03:00] before growing coffee. So I've been actually using Omniagent to find these specific regions in Nantou to grow matcha.

Demetrios: Ooh. S- wait, with the intention of you going there and then growing matcha or partnering with-

Denny Lee: Or working with the farmers.

Denny Lee: Wow. Working with the farmers in that area to grow the matcha. And so some interesting tidbits is that, um, the oxidat- when you, when you working, when you pick the leaves, right, it, it could oxidize really fast. So you actually have to steam and dry them immediately. So y- so the problem, like Japan and Korea, they both have the infrastructure set up that next to where the leaf picking is actually happening, they can immediately deal with the steaming and drying, okay?

Denny Lee: And, um- Mm-hmm ... of the leaves. Um, for, for example, it's, this is a good setup for coffee because, um, coffee, in the coffee regions they know... There's like bourbon style, honey style, whatever else, but it's all about drying the coffee beans after you pick [00:04:00] them, and how do you do it correctly, right? And so the, for oolong and other types of teas in Taiwan, they're all set.

Denny Lee: That's, Nantou's pr- famous for that, for that producing amazing, like for example, Zhushan tea. Like they're, they're great at that stuff. But for teas that oxidize fast, like, um, they- Matcha ... they don't have the infrastructure r- yet, and so the idea is to actually ensure that they have the, the steaming and the drying technology there right away, number one.

Denny Lee: But then is there an area in Taiwan that actually can actually handle the how to deal with oxidized tea, basically? Like once you've dried, how do you process it? There's only one area in Taiwan that actually knows how to do it. It's a, it's an, it's a region called Songsha, which is, uh, near Bunchao.

Denny Lee: Bunchao is like a, a major city in Taiwan.

Demetrios: Yeah.

Denny Lee: Okay? And so, uh, ironically, that's an area that I've been trying to convince my family to move to. Like in other words, like- ... uh, like, uh, to stay in Taiwan just because I love the area a lot. Winter is, you know. It's got, well, uh, it's got a lot [00:05:00] of great cycling, dude.

Denny Lee: Like the s- the cycling there is gorgeous. There's a lot of really, really painfully painful hills. It's amazing, so. So, so I'm basically getting the agents to debate with each other to get me to dive in, and so it's showing me which areas of Songsha I can work with, specifically, uh, which, um, with farmers, which small distributors that I can work with to basically figure out how to go ahead and have them process the oxidized tea as well.

Denny Lee: Yeah.

Demetrios: Mm, yeah.

Denny Lee: So it's like, if w- if I'm actually ever decide that I wanna go into that business, then the idea is like, yeah, like obviously it's a side business, but as you can tell, I'm just a tiny bit OCD

Demetrios: on st- stuff like this.

Denny Lee: Yeah, and- Just, just a little bit ... you're pretty deep Yeah. Yeah, yeah. So just, just a little bit, so.

Denny Lee: I appreciate that about you. So because of that, then the idea is, like, the, one of my fun projects when, if I'm not cycling would be like, okay, let's figure out how to actually, like, grow high quality matcha. So obviously Japan has the lead. Korea's infrastructure's amazing in typical fashion, like Korea does.

Denny Lee: And so for me, I'm like, "Huh, can I do this in Taiwan?" Because the, [00:06:00] the final tip is Taiwan Nantou elevation is typically isn't high enough to grow some of the very interesting coffee beans. Yet, it does

Demetrios: So s- for some reason it works even though it shouldn't. Exactly. And you have the theory that that also can translate to matcha.

Demetrios: Matcha.

Denny Lee: Exactly.

Demetrios: And now with these agents debating each other, that's just giving you all the context that you would need if you wanna pull the trigger and say, "Okay, I'm gonna go do it. Now I've got it, all the information I need, I can point another agent at it and say, 'Read through this context, and then give me what I need.'"

Demetrios: Mm.

Denny Lee: Yeah. Really good. So I save obviously all the, uh, all the... Like, it's the old, like, build skills or build an agentic memory to store all the stuff so I have a reference point, because there's only so much context you can compress before the thing blows up, right? But don't forget, it's like things like, okay, cool, now I have a [00:07:00] general idea of what's going on, but then how do I, for sake of argument...

Denny Lee: My Mandarin's horrible, okay? Let's be very clear. It is disgustingly bad, okay? Um, my, at the time, five-year-old, when we were in Taiwan, makes fun of my Chinese accent, and she's... It's fair. It's fair game. It's not even like, "No, that's true," right? Because, uh, my, my Mandarin sounds like I'm speaking, like, Cantonese.

Denny Lee: Like I have a Cantonese accent Mandarin. Mm-hmm. And mainly because I've learned most of my Mandarin by watching Jackie Chan movies that were dubbed into Mandarin, and so they had a Cantonese speaker speaking Mandarin, and that's how I actually learned a lot of my Mandarin. It, it... I wish I was j- joking.

Denny Lee: This is completely freaking true, okay? So, so in other words, no, I'm, I'm bad, okay? And so now I need to go ahead and mess around with agents to go ahead and read Mandarin properly. Yeah. And a lot of it's handwritten, not digitized, because we're talking small farmers, small distributors, right? Mm. So this is when I start switching over to the, uh, [00:08:00] the DeepSeek models, except, a-ha, now here, here we get into this.

Denny Lee: The DeepSeek models and all the Chinese open source models are great, but they are for simplified Chinese, not for traditional Chinese. And traditional Chinese, because there's actually, there's actually more context, so you actually need more tokens in order to process the same amount of Chinese characters for- Oh, yeah

Denny Lee: traditional Chinese versus simplified Chinese, okay? I, I forget the name of it, but there was a really good, cool researcher. She had done a l- um, all this research, uh, about creating Taiwanese specific models. All right, uh, LLMs, and how that it was actually required more context, required more tokens to process the s- in essence, the same amount of words.

Denny Lee: And so there was some... I forgot who did it, but there was some person who was actually trying to do, like, "No, no, we'll translate the, the traditional Chinese to simplified Chinese-

Demetrios: Mm-hmm ...

Denny Lee: run against the LLMs that way, and then translate it back. And bec- because then there'll be less tokens required, and the translation [00:09:00] actually is relatively smor- straightforward.

Denny Lee: It's just a direct mapping in a lot of cases. Mm-hmm. Like, you don't, like, need thinking in that case. It's like, no, here's a simplified version, here's the traditional version. So, uh, I'm actually still testing that out. I, I, I screwed up my installations those models. Why? Because my Mandarin sucks. I think that's the recurring theme here.

Denny Lee: But the idea is that I'm, I'm trying out different things like that because then that's the whole point. Like, how do I then work with The traditional farmer, the traditional distributor, with the ones that actually have real longevity and real knowledge of the topic, right? And so, so l- part of the reason I'm bringing this up, by the way, is just because a lot of people talk about how AI agents are removing jobs, right?

Denny Lee: And, and there is obviously validity to that statement by every stretch of imagination. But I keep trying to remind people, it's like, yeah, but it's like every other technology we have. It also introduces new jobs. Right. It also introduces a way for us to [00:10:00] rediscover things that we forgot, using this small example of, like, just my obsession with matcha, or my obsession with tea or coffee in general.

Denny Lee: Right. No, just any obsession I have, right? Just-

Demetrios: And now you can task an agent to go do these things that you probably would've never gotten around to doing- Right ...

Denny Lee: before. No, because think about how much research, 'cause think how t- horrible my Mandarin is, right? Yep. And so instead, I'm having fun being a geek again, but then I'm also connecting with old farmers.

Demetrios: Mm-hmm.

Denny Lee: I'm connecting with old distributors. Like, this is fucking sweet. Like, ju- so in other words, I get to go ahead and work with people that normally I wouldn't even have a clue to n- where to even start now.

Demetrios: Yeah.

Denny Lee: Right? Which is r- which is, that's like, that's, that's cool.

Demetrios: Yeah. That is very cool. That, and so you've got these, uh, you mentioned there's the setup where the agents will debate each other.

Demetrios: Yeah, yeah. And they're debating each other just because you're saying, "Hey, look, give me a reason not to do this." Like, try and

Denny Lee: find a reason why- Oh, no, [00:11:00] it's not even that. No, no. No, no ... okay ... it, it's, it's actually better than that. It's not... Like, because most of the time, at least w- with the prompts that I use or the context I'm using, it's not about doing or not doing.

Denny Lee: No. Uh, uh, you, you know me n- not well enough to matrix. I'm always gonna go do it, right? Right? My, my problem isn't am I gonna do it or not. My problem is, like, am I gonna basically slam into a wall or how thick that wall is when I slam into it, right? That, that's usually my problem. So, so, uh, y- you can tell, like, basically how I look at the tech is actually s- uh, you can still look at it from an optimistic perspective, right?

Denny Lee: And so for me, it's not about, like, should I do it or not should I do it. It's more like, what's the right way to do it? Like, and so in other words, a- as you can probably hear or tell from the way I'm describing everything, my obsession is high quality, going back to the roots of actual, like, tea making, right?

Denny Lee: I'm not an expert in that at all, but I respect the quality. I [00:12:00] respect the art. Yeah. I respect the cultural significance of, of it. That, that I can say. Yeah, I can vibe with that. Right. And so because of that, then the idea is that what I'm asking the agents is basically, "Look, I'm giving the agents that context so then way they're debating with each other-" Like, oh, well then what region of Taiwan am I looking at?

Denny Lee: What's the elevation? What's the pH? What's the soil quality? Am I dealing with lots of, for example, like, uh, the theory is if we were to grow matcha in Taiwan, it would be more floral-based, especially in Nantou County. Why? Because, um, that's what Nantou is famous for. Like, their teas have a lot of floral, a lot of lychee.

Denny Lee: Like, so for example, that coffee, that coffee blog that I, I sent you, right? That one, his coffee beans were notor- were famous notorious is the wrong word. Famous partially because, uh, there was, um, hi- very strong, [00:13:00] very, very strong hints of lychee in the coffee beans themselves, right? And the reason why is because literally the re- lychees were growing wild in his coffee farm.

Denny Lee: So literally, like no joke, we, when we got there to try out the coffee, the, um, the grandma was literally just giving us lychees for free and she's like- That's so awesome ... "Don't you want us to pay for them?" "No, no, they ju- they grow everywhere. They're literally a weed at this point." Like, I'm like, "I've never heard lychees referred to as weeds, but okay."

Denny Lee: Like, right. But that's literally the context. It's like, oh, fi- being able to find cool things like that. And so that's the, that's the attempt to discover. That's the attempt at like try to... Like, somebody like myself who has the obsession of it or the OCD-ness, but not necessarily the knowledge, the letting the two systems debate with each other provide me the additional information to say, to a- know what other questions to ask.

Demetrios: Yeah, because when you read through that debate, then you go- Right ... "Oh, I didn't even think about that."

Denny Lee: Right.

Demetrios: That [00:14:00] is a, such a great trick that I recognize, and you know what I... One of the actually toys that I've been making, and I don't know why this isn't a common UX for dealing with agents, is I now have created a little widget that you can highlight any piece of text, whether you're inside of the chat window with the agents or you're on a website, and it will pop up a little prompt box, and it will read the context of whatever the, either the paragraph or the whole page that you're on.

Demetrios: And you can prompt whatever, ask a question to it just about... Because a lot of times what I recognize with myself is that I don't want to continue this conversation in the actual chat that I'm having. I wanna shoot off and do this on a different thread because it's just, like, something that isn't [00:15:00] necessarily needed to pollute this conversation with the agent that I'm having.

Demetrios: But I still wanna know, like, hey, n- you s- you referenced this thing here. What does that mean? Go deeper. Give me more context on this specific thing, but not in the main chat. So it's like little sidebar conversations. And I've got this, uh, I, I use this tool called Clearance to read, like, markdown files. And so now I created a little right-hand sidebar where you can have a side chat with an agent.

Denny Lee: Okay. I, I know this is definitely wasn't done on purpose for everybody that's listening to us right now, but you, you wonderful segue to Omnigen, by the way. Because in- Oh, nice. No, no, because, okay, so for example, I did this already. So I f- when I first started asking the question about matchas and things like that, like what regions, I learned a lot about, more about the regions of Japan, for starters, and then the regions of Korea.

Denny Lee: All right? [00:16:00] But then, remember my end goal was to figure out if I could do it in Nantou County in Taiwan, all right? But of course I, as I'm... To your point, I'm learning more about these regions of Japan and these regions in Korea. I'm like, "Oh, this is really cool," so I wanna know more. But now, I... So I really wish there was a way to fork my conversation.

Demetrios: Yes, that's it.

Denny Lee: So Omnijit has the ability for me to fork, so that's what I did. I basically had the conversation, I forked it off, then I allowed myself to continue one conversation specifically about the Japan and Korea areas, while the other one, k- the, the main line, I just redirected to Nantou County.

Denny Lee: And so they both had the original context by which we, uh, we had the conversation with, but now I can have two very separate conversations so I can learn more. Why? Because, uh, I'm fortunate enough where, because I live in Seattle, um, no, I'm not being paid for this, But there's direct on Alaska [00:17:00] Airlines from Seattle to Tokyo, Seattle to Seoul now.

Denny Lee: So I'm going... So yeah, so I- hopefully, maybe they'll give me something for that. But in the realm of that- Yeah.

Demetrios: Yeah, this-

Denny Lee: The reason I called that out is like- ... this episode is sponsored by Alaska Airlines ... this

Demetrios: episode's sponsored

Denny Lee: by... Right. But no, it's not. I wish. I'm not that- Yeah. We're, we're both not big e- we're both not big enough for that, right?

Denny Lee: But-

Demetrios: Exactly ... but- We need to become the true travel bloggers

Denny Lee: that we aspire to be. Exactly, yeah. We're, we're just, we're not, we're not travel influencers, so unfortunately- That's right ... that's not gonna work, but- Not

Demetrios: yet ...

Denny Lee: the, y- there we go. True. Point taken. Yeah. We, we have goals, right? But nevertheless, the i- idea being now I'll go directly to Korea, I'll go directly to Japan, so I can explore the regions myself.

Demetrios: Yeah.

Denny Lee: So that's why I started going down... So I forked the conversation specifically so I could go do that. Yeah.

Demetrios: But I don't understand why that isn't a common UX pattern with these agents, and specifically for me, it's like I have this one [00:18:00] sentence or these five words that appear together in my context, and I wanna fork them out and have a separate thread on that specific piece.

Demetrios: Like, explain more. But I haven't seen any of the labs create this as a integral part of how we interact with agents. And so it, it makes me think, like, maybe I'm the only one who cares about that

Denny Lee: enough. Like- No, no, no, no, no, no, no, no, no, no. You're, you're... Look- When we started all this, this entire AI community, right?

Denny Lee: W- think about how chat windows are still the... Even after all this time, chat windows are still the way we work with things. Even our sticking, when we decide to use CLI and terminals, it's still basically a CLI chat window. That's basically all it is- Yeah ... for all intents and purposes. Totally. Right? Just that we might have some JSON stuff shoved in there now.

Denny Lee: That's the, that's the extent of the difference, right? Um, the reason [00:19:00] lies because... A- a- and you actually see harnesses, like, being created right now. Like, a bunch of startups, a bunch... Even the major labs, they all create harnesses themselves, right? And the reason why is because, like, look, up to this point, a- and this is, this is like our, definitely our learnings within Databricks itself, right?

Denny Lee: We all had, like, five terminal windows open, six browser windows open. Yeah. We're all going back and forth, back and forth. You deep dive on a particular topic. I'm coding something for Delta Rust, right? Like, so I can get off of T for a second. Right? Yeah. So I, I'm co- I'm coding something for Delta Rust. I'm trying to understand the latest version of Arrow or, and data fusion, because they keep changing.

Denny Lee: Uh, hats off to the community. This is not a complaint. This is just... But they do c- they do change things, so we're trying to keep Delta Rust up to date, right? And so when we're trying to make those changes, then I need to run some terminal windows [00:20:00] to, ah, I need to figure out what's the current version.

Denny Lee: Another terminal set is, uh, creating the tests harnesses that I wanna go create. Another browser window is giving me, oh, what's the latest security context? What's the latest, like, you know, uh, uh, uh, security alerts associated to e- each and every one, the 20 different dependencies we have that, that we have to go through, right?

Denny Lee: And, and what are you doing? You're opening multiple windows, and you're losing context. No pun intended, right? Um, and- You, you're, you're losing all of the connections because you've got so many windows, so many terminal windows open, right? So you know how Amazon made the, the idea of, like, the f- each team should only have, like, two pizza boxes,

Demetrios: right?

Demetrios: Yep.

Denny Lee: All right. The whole reason that was created was because it was the idea that each team should only have five to seven people [00:21:00] reporting to it. Okay? So each manager should only have five to seven, because anything greater than that, they basically start losing context. And again, pun intended. Okay?

Denny Lee: Well, this was determined not... Amazon might have made it famous, but it was actually a military thing. Like, military, military command in terms of units, teams, understood that each commander could only l- logistically handle five to seven people reporting to them in order to be effective, right? And so I'm not military, but I, I am, uh...

Denny Lee: I used to be King County, um, um, um, um, ops. Um, um, the, um, search and rescue ops, excuse me. Oh, nice. I'm trying to remember, I'm trying to remember SNR, like- ... this is how tired I am, okay, dude? Uh, search and rescue operations. So we learned that from... So S- SNR ops learned that from, obviously, military, and then the idea is that this is a, a time-valued tradition, like, un- understanding it.

Denny Lee: A- and so, so in the tech world, everybody thought about two pizza boxes, but basically this is a [00:22:00] well-understood, um, logistical structure within the military. Okay? And so cool. The reason I bring that up is because, well, then, in a lot of ways, if you consider an agent almost like a junior developer that you're managing-

Demetrios: Mm-hmm

Denny Lee: Y- once you go beyond five- Five and seven.

Denny Lee: Yeah, yeah ... once you go with seven, you're losing track of everything, right? And so more than anything else, the things like, to your point exactly about I wanna go off and do that, it's actually gonna make you lose context even more so, right? Part of the reason why we, we had, and, and Omjen had basically built this idea of forking was actually not originally that idea you talked about, by the way.

Denny Lee: Like, to be very clear. Like, it was more about literally we're coding. So you fork so that way you can literally go ahead and try out different ideas- Yeah ... but have the initial same context. [00:23:00] But to your point exactly, coding and actually having a real conversation isn't actually that much different.

Denny Lee: Coding happens to be just around code, but it's still an abstraction layer, right? And so conversations are just another abstraction layer, right? Everyone's like, "Well, coding, no, that's real code." I'm like, "No, are you writing this thing in binary or assembly? No? Okay, well then stop talking to me. It's an abstraction layer," okay?

Denny Lee: And so the idea is that if you've got all these different things floating around, right, you will lose context. You will not remember why you asked in the first place.

outro: Yeah.

Denny Lee: So there's, so there's a lot of research being done by the labs, and to an extent even a lot of the harnesses, which is like how do you basically have now an agent fleet and the fleet manage itself?

Denny Lee: Mm-hmm. Right? And there's various... Like, by the time this podcast comes out, which is only a week or two, I'm sure everything will change, so I'm not gonna bother with the pretense of knowing what's gonna happen with fleets. But irrelevant of whether you, you call it a fleet or you talk about an agent or [00:24:00] there's a, some other layer, the, the whole reason we built what we built was because we understood that this was gonna be a problem either way.

Denny Lee: And just as importantly, or if not ca- in many cases more importantly, the models keep changing. Mm-hmm. The harnesses keep changing. I wanna be able to go ahead and move all that context, all the agentic memory, all that configuration. I don't wanna rebuild it so now I can use the latest OpenAI model or the latest DeepSeek model or the latest Anthropic model or whatever it is I'm using.

Denny Lee: I want this one layer that says whatever harness you're using, whatever model you're using, it'll flow through irrelevant

Demetrios: Where I get a little bit confused there is the harnesses are optimized for certain models, right? I guess if you're building a harness, you're building it for a model and there's certain things that the labs are gonna be doing on their own that's like the inner harness and then you have the outer [00:25:00] harness.

Demetrios: And the inner harness, they're RL-ing that model so that it works really well with their harness.

Denny Lee: Oh, 100%.

Demetrios: How do you, how do you, like, envision these hot swaps in a world where you know that?

Denny Lee: Oh, because that, that's actually why we value Omnigen as an open source project to begin with, right? Because the idea is that, like for example, even just building it, we notice, for example, what deems to be simple but actually changes people's workflows is, like, uh, approvals.

Denny Lee: Like- Mm-hmm ... the approval process, I, I'm not gonna get this quite right so I'm gonna stay vague so that way I can, I can not make it sound like one's better than the other because that's not true at all, okay? Just to... But the process is slightly different between harness A and harness B, and the process is slightly different between model A and model B, just to be very vague so that way, again, I'm not calling anybody out, right?

Denny Lee: And again, it's, it's not even calling because like I said, it's not like one's better or one's worse, [00:26:00] it's just purely they're different. That's all this is, okay? So if, if you look in the Omnigen code base, that's, we actually reflect that Right? And so then that way we're actually leveraging what Model A or Harness A is really good at.

Denny Lee: Because w- w- like, the creation of Omnijit isn't about going in and saying, "We're a better harness than everybody else." We're not. No, we're... That's why we call ourselves a meta harness, because we're saying, "No, no, we're on top of this so that way you, the individual, the person that's using these things, all that configuration, all that context, all that agent of memory, that is portable.

Denny Lee: That you have control over. That you can flow." Right? You can c- compose these different ideas where basically I wanna go ahead and run... Like, just to be a little bit more specific now, for sake of argument, I was using Opus [00:27:00] 4.8, actually 4.7 in the m- m- the ca- a, a real example, to generate

Demetrios: the code. Yeah, I like Owen better, to be truthful also.

Demetrios: Yeah. I, I'm glad, I- Yeah ... glad I have a, uh, kindred spirit here. I've been using way more. 4.8 is not... I'm not stoked on it,

Denny Lee: but- Yeah, yeah, yeah, yeah. Uh, it... Eh. It's a preference thing. Yeah, that's a whole other thing. But- Yeah ... let- let's not debate that, okay? I don't wanna- Yeah ... throw off the kick of my ass, okay?

Denny Lee: Right, so, right. But I'm using Opus 4.7, and then, but then I was using GBD 5.4 at the time to do the test cases. And fuck yeah. Like, it was wonderful because then I got the two using Polly to debate each other in terms of high-quality coding, and then I, and then I got one to actually go ahead and, and generate the test cases for me as well, which is amazing, right?

Denny Lee: And then maybe a week later I would've switched over to say, "Eh, you know what? ChatGPT 5.5, I like their code base a little better, so I'm using that now. I'll stick to Opus 4.7 for the, um, of to debate it. I'll use maybe Sonnet [00:28:00] to, to go ahead and do the, the test cases because those are relatively easier. To, once you put the original template, I don't actually need something as powerful to generate the test cases, I just need to sort of update them, so this is perfectly fine."

Denny Lee: Um, and then when f- you know, when Fable and all that stuff came out that rewrote everything, and then Fable got removed, and that rewrote everything. Right. The whole point is that I'm just using Omnijit, so I didn't care. I just redirected it and, like, hell, I'm using Pi for a bunch of my test cases now because it's cheap.

Denny Lee: Like, really dirt cheap, so I'm like, "Why not? It's really good, so just use that in some cases," right? So that's more or less the point. Like, it, it just gives you that flexibility to say, like, okay, you start off with everybody telling you you're supposed to use the most advanced one. And I, I get it that you're tempted to do that, and if you, if you're allowed token maxing, you will do that, right?

Denny Lee: But the reality is this stuff's starting to get expensive, right? Yeah. Prices are going up or discounts are going down, or both, right? And so my take on the [00:29:00] whole thing's like, okay, well then- Try to find the right model or the right harness for the job. And so maybe even if you are initially using the more advanced models to build it, a lot of the maintenance doesn't require the most advanced model at all.

Denny Lee: In fact, the most advanced models more times than not actually make things literally more complicated. Like you're going, "I only needed simple test cases. I didn't need this gargantuan thing that is impossible for me to read," right? That's

Demetrios: a feature, not a bug. That's why Anthropic's worth its whatever trillion dollars.

Demetrios: They are very

Denny Lee: verbose. No, but see, that's what it's like. In the end, you get my drift, right? Like you- Yeah ... it's, it's about going ahead and then... Because after all, like I, I'm definitely not meaning to get in the debate just right now, but like during, for example, Data and AI Summit, uh, during the O- Open Lakehouse contributors', um, uh, meeting, right?

Denny Lee: Uh, one of the sessions that I actually end up, uh, helping to lead was the, the dealing with coding AI slop, right? And one of the call-outs that we had, um, uh, [00:30:00] because the way we run Open Lakehouse was very much in the vein of like, yeah, I might have led the discussion, but it was meant for everybody to chat.

Denny Lee: I'm not interested in like me telling people what to do. Th- that's not interesting at all. I'm... Look, we- we're getting all these people in the, the Open Lakehouse and the AI community specifically to go debate. Like, like w- we're working on competing projects, whatever, I don't care, but like we all are suffering from same version of, of insanity in one form or another, so can we help each other out on that part at least, right?

Denny Lee: And so for example, there was a whole debate like on the, on various distribution lists including like Apache Iceberg, on the Delta List, whatever. I was like created by versus assisted by. So created by is the person who did it, assisted by is like it's Opus 47 or Sonnet or whatever it is that you were using at the time, right?

Denny Lee: And the call-out more than anything else unequivocally was that- You needed to... In the end, everybody agrees with the idea that it is still the individual, the user, that is the [00:31:00] one that submitted the code. That there still is, from a TLDR perspective, who's the neck you're gonna choke? It's not- Yeah ... I'm not choking the neck of an agent, I'm not choking the neck of Anthropic, I'm choking the neck of the person that's contributing the code, right?

Denny Lee: Sure. And so ultimately, like, whatever we're phrasing it as in terms of creative bias, bias or anything else, the point is that me, the developer, whether it's developing code or developing a business project or whatever it is you're building, it's still me. It's still you. It's still the person that's making that decision that ha- ultimately succeeds or fails.

Denny Lee: So whether it's the, the code for Delta Rust or whether it's, you know, figuring out which region of Nantel County that I'm, I wanna grow matcha leaf, right? Ultimately, it's like, it's me, right? The, the onus has to be on me alone, right? And so knowing how to follow a thread, knowing what these harnesses can do, knowing that you can move [00:32:00] your context, your thought processes across the different models so that way you yourself can succeed, ultimately that's it.

Denny Lee: And, and I think you, uh, Dimitrios, you and I have actually had this conversation on the side. You've also did, did this as part of your own set of podcasts where ultimately the main reason people fail when they're using agents is because they, they zero shot it and they have no context applied at all.

Denny Lee: They, they have no free thinking. So people are like going, "Well, of course, it's garbage that came out of it." I'm like, "Yeah, so that means these agents suck." I'm like, "Well, it, it's the old garbage in, garbage out." Like, if you if you're not putting any effort into this, it's, it's gonna be garbage coming out.

Denny Lee: Doesn't matter how good the agent is. It doesn't ma- matter how good the model is.

Demetrios: Mm-hmm. Have you been playing with the different... 'Cause basically what I understand is, hey, you bring your own model and you can bring your own harness, and then you have this meta harness above it- Right ... that is [00:33:00] unopinionated and it's like, cool, you wanna come with your OpenClaw over here and then your Hermes over here- Yep, you wanna

Denny Lee: come with your Hermes, you wanna toss, toss in integration with Ollama, have fun at it.

Denny Lee: By all means, yeah.

Demetrios: How have you... And how have you been seeing that working? And, like, is there a world where you're recognizing that now what is important is understanding what each one of these is best at?

Denny Lee: It- we built Omnigen in about two weeks, by the way, to give you some context.

Demetrios: All right?

Denny Lee: Again, pun intended.

Denny Lee: Um- We literally used Omnigent to help build Omnigent. Okay? Like, so the... Everything you see, we, we basically used Omnigent itself to basically build it, improve it, the whole kit and caboodle. The reason I bring that up is because, look, we have a ton of benchmarks, right? But before the two weeks, we, Databricks engineering product, we, we were s- we were s- all suffering from the same [00:34:00] thing, right?

Denny Lee: We were suffering from, like, which model should we use this week? Should I use Kimmy now? Should I use... Right. Uh, uh, should I be looking more into the open source ones, right? Should... Like, so if you, like, m- if you look at Databricks itself, right, you'll notice that there's a whole bunch of different models, like, that all natively run within Databricks.

Denny Lee: Uh, they're all governed. That's great. I love that fact, but then y- y- exactly to your point, I, I don't know which one's better. And so yeah, i- if you even go to the Databricks website, we actually have this thing where we update monthly. I forget the exact URL, but basically we say, like, "Here's the best ones based on what we see this month," right?

Denny Lee: And then we'll... Obviously, like, for documentation, for debate, for financial, like, whatever. We just, we actually break down all the steps because we're like, "This is what we think in real world scenarios." And you'll notice that the real world scenarios often do not match benchmarks.

Demetrios: Yeah.

Denny Lee: Right. And so that's what got us to build Omni in the first place, because we were basically...

Denny Lee: We actually had a [00:35:00] version of Omni internally. We obviously can't share that one, but, but basically used that as our starting point, mainly because we were recognizing that every single engineer needed the ability to have that flexibility to switch Right? Because they realized that maybe that particular code base that they were working on was better with X.

Denny Lee: While different team, same org, hell, same team, like same division and everything, realizing, "Oh, no, but I'm doing Y and it's better with this." And so the idea of us trying to dictate you're supposed to only use one, like we never were gonna do that in the first place just because of the nature of Databricks, but like it just...

Denny Lee: The whole thing fell on its face almost immediately. And so what we did- Or,

Demetrios: or the model provider could be down, which we see

Denny Lee: all

Demetrios: the time Right.

Denny Lee: Oh my gosh, yeah, yeah, yeah. Uh, Claude was just down yesterday, so I was... When I was just trying [00:36:00] to run my stuff, so fortunately not a big deal. Flip, uh, it automatically flipped over to Codex and all right, I'm back up and running, yippee-ki-yay.

Denny Lee: Barely, barely a blip in the whole thing, right? But- Yeah ...

Demetrios: the...

Denny Lee: S- so ultimately it was more like, okay, do I have the flexibility to change? Do I have the flexibility to, to, al- but also not to change, but control my costs? Mm-hmm. So, like within Omnigent we actually have policies put right in there, which you say, okay, policy's as simple as like, okay, can I run this command?

Denny Lee: Okay, that's one of them. But like a, like a shell command, right? For example, you and I were joking about how just starting off you had run a skill that basically shut off your... Redirected your browser, right? Well, guess what? Like Omnigent has it built in where you have a policy that says, "No, you actually have to prove such things, so it can't do that," right?

Denny Lee: But the... Also even budgets, which is like, okay, if this thing's gonna cost you at more than X dollars or X cents or whatever it is, guess what? You have to prove before you're allowed running it, right? So ju- just things like that which allow you to have control over the situation. And then the [00:37:00] final little tidbit is that inside of it, all the conversations you and I have just had, right?

Denny Lee: The one thing is that it's a v- I, I'm still very like the, a, or a developer centric, right? But in order to be successful, more times than not what we have to do is we have to work with your team. Yeah. You wanna share your learnings from your team. And so that's the, the coup de gras for Omnigent, which is like, okay, it's like a Google Doc type interface where basically now, hey, I ran this agent or had this poly do this particular debate.

Denny Lee: Why don't you take a look, you know, team leader X? And then not only can they see what's being generated, they can then themselves run- on that session themselves because you shared it with them, and they can run the session and say, "Wait, can you run this particular test? Did you take into account of X?

Denny Lee: Did you take into account of, like, the new updated, uh, data fusion?" Or whatever else that you need to do, right? And then basically now that process continues on. So you're actually [00:38:00] sharing the workload together so the teams can actually work together really well,

outro: right?

Demetrios: Yeah. That is so cool. And actually, a funny thing that you m- were mentioning, another one of my little side projects, toy projects that I'm creating for myself, is a widget that goes up into the corner of your screen and it just calculates how much money you've spent-

Denny Lee: There you go

Demetrios: by day or by week, and every time you hit another $100 increment, it goes cha-ching and co- confetti goes everywhere, and then there's a, uh- Dude,

Denny Lee: you should so do a PR in Omnijit to do that, seriously. I, I- Because we already have the basis for calculation. Do a PR, baby. Like, hell- I

Demetrios: will ...

Denny Lee: u- use, use Omnijit to create the PR for you, because that's all I've done.

Denny Lee: That's perfect.

Demetrios: Yeah. You know what else I did, though? I put a little GIF, so all of a sudden a GIF pops up of, like, uh, someone making it rain.

outro: Make

Demetrios: it rain like gifts. And so it's, like, five different random GIFs that you can get, and it's, like, making it [00:39:00] rain, and so you recognize... And I created this w- because I was at FinOps X, which is this whole conference around budgets, and all they care about are tokenomics.

Demetrios: That's the whole- Yeah ... thing now. Nobody cares about dashboards for your cloud spend. It's like, uh- Oh, God, yeah, yeah. 100%,

Denny Lee: dude. Yeah. Yeah ... so,

Demetrios: uh, we got that figured out, right? All they care about are tokenomics and how people are spending their tokens. And what I realized is that a lot of folks are creating dashboards so that their CFOs or whoever in the finance department can see how much the engineers themselves are spending.

Demetrios: But what I didn't see a lot of is pushing that visibility onto the end users. And I'm like, if I knew that I was spending 1,000 bucks a day, I might be a little bit more conscientious about the costs.

Denny Lee: Right. So call out, wonderful call out. Like, so I still remember because I'm a, I'm a longtime Microsoft [00:40:00] guy, okay?

Denny Lee: Before all this. And then when we switched to the cloud, the whole premise is that, like, this idea of CapEx to OpEx, right? Like, we're switching capital expenditures to OpEx. In that process of OpEx, it was the, into DevOps, meaning that the developers actually had to understand how much money they were spending on the infrastructure that they're building.

outro: Hmm.

Denny Lee: Right? So if they weren't, like, needlessly running a small query and spinning up a, a 50-node Hadoop cluster, right? Mm-hmm. When their laptop could've solved the problem, right? So- That's exactly what's going on right now, which is basically that service-based architecture, SOA, you know, um, where we've switched from this idea that we had to go ahead and originally it was one central command to understand what the budget was, and now that got pushed down to developers because the developers could actually then care, understand, and help control that.

Denny Lee: Tokenomics is the exact same thing, [00:41:00] 100%.

Demetrios: Yeah. That's so true. It's the same motion and you're just, we're reliving it. Yeah. But it's a little bit different now because what is funny, and I think one of the hardest questions that I saw people struggling with at the event was, how do you know what token spend is valuable versus wasteful?

Denny Lee: Yeah.

Demetrios: And so there's all kinds of tricks that you can do on these, like these details, uh, one of them being, hey, choose a different model. If it's not that big of or complex of a problem, just use a better model. Use an open source model, use whatever. But still, you have certain engineers that are burning a ton of tokens, and for some subset of those engineers, that burn is worth it.

Demetrios: And then for- It's- ... another subset of those engineers, that burn is not worth it, and it's like, [00:42:00] what are they getting done? What's actually happening?

Denny Lee: No, no, 100%. So this goes back to the idea where it's actually just like with the switch from CapEx to OpEx, right? You actually need both. You need to give the developers the tools to understand what they're doing and what the spend is, but you also need to have some central governance story so you can understand what's going on, right?

Denny Lee: Like, so like you've... I've come to you, we've actually had this conversation, it's like why do we have, you know, for example, UniCatalog, and why did we- Yeah ... Databricks introduce UniA- A- AI Gateway? That's the whole point. It, it's two sides of the same coin. Like Omnigen is there for the developer so that they themselves understand, they themselves can apply policies.

Denny Lee: But then we have UniAI Gateway to, to, for a centralized form which says, "Hey, look. Yeah, you can use Opus 4.7, 4.8," whichever one you're doing, 4.7, right?

Demetrios: Yep.

Denny Lee: But wait, like for example, if you were to look at my questions, especially in, in terms of when I was still beginning with Omnigen, when I was testing it out, there would [00:43:00] be an absurd number of which bagel's better, Montreal versus New York, right?

Denny Lee: And so if I was the central person, they'd be like, "No, this Denny guy, we, we got- He's wasting tokens ... we gotta remove this. Th- th- this is a ridiculous amount of tokens being spent on this particular question," okay? Especially because everybody knows Montreal bagels are better. So yes, I said it. Yeah. All right, but, but, but you get my drift.

Denny Lee: Like the... Obviously there was a reason for why I did it, but if I still do it today, oh my goodness, yeah, they would just log into UniAI Gateway, "No." Be like, "No, you're not allowed doing this," right? Shifting budget over to your colleague. Yeah, yeah. They're, they're shifting budgets. They're like, "Okay," like, "No, Denny, you're allowed using Pi."

Denny Lee: Yep. "You're, you're allowed using..." Like, "You're allowed using-" That's as good as you get ... maybe if we're n- if you, if, if there, if, if you're lucky, we'll let you use Kimi just for the sake of doing it. But not even a later version of Kimi, like an older version," right? Yeah. But like, "Opus? Oh, hell no. We're never gonna let you use that," right?

Denny Lee: So...

Demetrios: You know what? On this idea of choosing the models and choosing the harnesses and all of that, [00:44:00] where I see the biggest friction is when I have to manually make the decision because I am a little bit lazy and I just kind of go with whatever I've got on, you know? And so I'm not going to, especially if it's m- my company's money in a way, I'm gonna be like, "Ah, whatever, let's see if it works with this."

Demetrios: Right, yeah, yeah, I could

Denny Lee: just blow the money and ask the bagel question again.

Demetrios: That's not... Yeah, yeah. I'm not gonna be like, "Oh, let's use the smallest model." It doesn't work, and then I'm gonna go back and try and go to the next smallest and cheapest, and then it still doesn't work, and then I've wasted time, which is the whole reason I thought I was using these coding agents was to save myself time.

Demetrios: And so it's like I see that I will overcompensate with bigger models just for the fear of a smaller model. Nah, it's probably not gonna be able to get this, even though it totally will get it. Like, it's like, oh, you just changed some names in the code base and it went through and, you know, like [00:45:00] Control + F'd.

Demetrios: You did not need a 4.7 for that.

Denny Lee: Right. So w- I don't think that's completely solved yet because this is sort of like, again, same idea, like the journey from CapEx to OpEx. Like, we don't know when, when one should be applied, where like the w- like in other words, preventing you from using it in the first place is very limiting.

Denny Lee: We don't wanna do that. But we also don't wanna get to a point where you've blown a million tokens for- For nothing ... which color of mauve is the best. Yeah. Okay? Yeah. Right. Um, and so I- we're at a point where I, especially with meta harnesses and especially with things like Uni AI Gateway, that now we at least have visibility and some form of control.

Denny Lee: Um, one of the things that we've been experimenting on, and you're gonna see some of it show up actually in Omnigen itself, uh, because we've been experimenting internally first because we wanna see how many engineers are gonna kill us, right? Um, is the idea of where do we [00:46:00] auto choose for you based on the questions that's being asked, right?

Denny Lee: But that itself is hard, and that itself may require us to use an LLM to make the choice, so do we just shift the token usage that way, right? But this is where, like a, a good old-fashioned machine learning may kick in for us

Demetrios: again. Yeah. You get a little

Denny Lee: class fire. Yeah, like you know how RL's back in vogue again?

Denny Lee: Yeah. Which is, it never should have not been in vogue, by the way, but you get my drift, right? RL's back in vogue, it's like, oh, that's cheaper, that's faster, that's better. Y- yeah, we've been saying this pretty much the entire time, guys. But yeah, so goes back to the same idea. Machine learning's back in vogue.

Denny Lee: I'm like, no, a lot of these types of questions though, like in terms of the complexity of the question, you can literally use good old-fashioned machine learning to make the choices. So that's why I'm saying we can, we're definitely seeing trends like that to help with that process. And ultimately, however it's done, the key thing is, to your point, is like I, I, at least that, for example, with Omnigen, there isn't changing the code base to choose.

Denny Lee: It's [00:47:00] literally click and it automatically chooses for... And, and it, you have the, the dropdown list of which one it to go with in the first place, for starters. And so what we're hoping for is by the time we're done with everything, is that once we get into some good patterns around that- We actually tell you up front, "We're gonna switch to Model X.

Denny Lee: Here's the reason why," and let it run, right?

Demetrios: So- Well, as long as it, if it fails, it will retry with- Yes ... a better model, then I'm stoked because I don't have to spend my time being like, "Ah, this doesn't really look that good."

Denny Lee: Right, right, right, right. But then it goes back to, like, this is why we like Polly so much and, uh, and Debbie because there's different versions of what it means to say it doesn't work if you get my drift.

Denny Lee: Mm-hmm.

Demetrios: Yeah,

Denny Lee: 100%. Right, so that's why we like the idea of the, the agents debating each other because then you can walk away, let the two agents fight each other out, and basically, yeah, because they have enough context, they [00:48:00] have enough memory, they can actually figure out, okay, there, there's a reasonable assertion here that actually because they're debating each other, you will get a result that actually works, or at least even if it doesn't work, at least now you have the basis of why it doesn't work.

Denny Lee: So it let... Because in the end, especially with how powerful a lot of these open source models are these days, for me, a lot of time it's ne- it's less about which model I'm using nowadays, and it's a lot more about did I provide the data, the context, the background- Yeah ... the configuration, the memory. Did you set up...

Demetrios: Now the big thing is did I set up the correct loops? Did I give it what it needed to just be able to run- Right ... for an hour so that- Yeah ... I don't have to come back? And that is one of them. I think that's a great point, is being able to debate and recognize what good looks like so that you can trust, okay, it's a smaller, cheaper model, but I know that The debate is going to happen, [00:49:00] and if it doesn't pass the sniff test, then it will go back and it will loop again.

Denny Lee: Right. And then this goes, and why do we see yet another resurgence of evals all over again?

Demetrios: Yeah.

Denny Lee: For precisely that. I

Demetrios: can't believe we're still talking about evals.

Denny Lee: We have to, because in the end, there has to be some numerical... There has to be a number that allows us to state if what was created actually, per evaluation, actually is, it's the old definition, what is good?

Denny Lee: It goes back to that definition over and over again, right? And so in the end, like I said, it's, it's becoming more and more obvious that it's less about the models. This is not a knock on any of the models, just they're all really good now at the point. That's what I'm just trying to get at. And it's much more about, okay, now that I know the model's good, what else is missing?

Denny Lee: And then- Mm-hmm ... the evals will actually help us figure that out.

Demetrios: Yeah.

Denny Lee: Yeah.

Demetrios: Yeah. That, how can I make this harness top-notch?

Denny Lee: Exactly. 100%. And

Demetrios: I think the auto model choosing is the way to go, because [00:50:00] if it is left onto the developer, they're never gonna choose a shitty model, or quote unquote- No, no, they're always

Denny Lee: gonna choose the best model

Denny Lee: shitty- Yeah. Like, yeah. The best model

Demetrios: because- Yeah ... it's a psychological thing. Yeah. You're like, "Ah, I need to do this because what if it encounters something in this run that it needs the power? Let's just, let's do it just in case. It's a few extra cents." Or dollars, depending

Denny Lee: on what you spend on. That, yeah.

Denny Lee: I mean, if it's a few extra cents, we're probably okay with it. It's the few extra- Yeah ... dollars or hundred dollars. And unequivocally, no, you're not wrong. I, you're absolutely right. Uh, and I think, like, this is part of that story for us. Like, it's, that's part of the story why we made it open source because we're going, look, we're not pretending we have all the answers.

Denny Lee: We're just pretending, we're just stating the fact that this is the problem we faced internally. Mm-hmm. We wanna share this with the world so that way y'all can, like, we can all work together to get to that solution that you're talking about, which is like- Yeah ... how do we do a proper auto model, [00:51:00] right? How do we, how do we do a proper thing?

Denny Lee: So that is absolutely 100% vendor agnostic

Demetrios: How... You know what I had a question with? Because certain harnesses have certain ways of expressing their harness, right? And I'm thinking specifically like in Hermes and OpenClaw, you have the soul. But then in Claude Code, you don't have that. How does this meta harness of Omn agent act when you're running these different shapes

Denny Lee: of harnesses?

Denny Lee: What about... So the, the TLDR is that the root... You, you remember the DSPy project, right?

Demetrios: Yeah.

Denny Lee: Right. The root of a lot of our learnings is basic from DSPy in the first place, which is the idea of how do you, how do I go ahead and allow myself to functionally run these queries against all these different models that are actually instinct- that actually have different ways of running in the first place?

Denny Lee: Mm-hmm. Right? Different parameters, different weights, different everything, right? Um, and so with that, the TLDR is [00:52:00] We started looking at the systems that we're integrating with, and we tried our best to basically take an account of each one, and so that way any integration we have basically takes an account of, like, what each one is strong at, at the code level, right?

Denny Lee: Wow. At the config level. But the, uh, I, it would, I would be remiss with if I didn't also basically mention the fact it's like that's actually why we're reaching out to a lot of these, both the model makers and also the harness m- makers because we're going, "Look, we're not pretending we got it right. We're pretending we got a good start.

Denny Lee: That's all we're saying, right? We'd love to work with you to make sure we have configured it the best to represent, you know, your particular model, your particular harness the best." Because yeah, we're, we, we're not pretending we understand it all. We're just simply stating that, as you can probably guess, we've used a lot of them.

Demetrios: Yeah. Right.

Denny Lee: So we, we've- Absolutely ... incorporated as much of that knowledge as we possibly can, but that still doesn't replace the fact that there are people in these other... who've [00:53:00] created the harnesses themselves or created the models themselves, they would know better than us. Fair. And again, why is it an open source project?

Denny Lee: So you get to look at the code, tell us what we screwed up on. We wanna go ahead and work with you to figure out how to make it better. Simple as that.

Demetrios: Hmm. You, you know, it gi- gives me this, like, visual in my head of almost like you have your different harnesses, and so, uh, the outer harness we could consider, like, the MCP servers that I set up.

Demetrios: Maybe I do memory in a certain way, et cetera, et cetera. And then you've got the meta harness, which is the, it's outside of all these outer harnesses, and that is where you have this way to almost, like, be an overlord of all your agents. So, like, if you're looking at a, a circular thing, like the inner harness and the outer harness- Right.

Demetrios: It's basically the outside layer of the

Denny Lee: onion ... this next circle. Yeah, yeah, [00:54:00] yeah.

Demetrios: Yeah, exactly. It's the- Yeah ... it's the onion or it's the earth with the different, like, rocks that you have- Core, mantle, shelf. Yeah, yeah ... as you go to the core. Yeah, yeah. Yeah, the magma. Yeah. All of that. So that's kind of how I, I visualize it.

Demetrios: Is that- Mm-hmm ... the way that you would... I- is that a good way of thinking of it, yes?

Denny Lee: Oh, absolutely. I mean, l- look, everything we do- Business-wise, development-wise, whatever else, we, we're always trying to figure out ways to abstract it.

outro: Mm-hmm.

Denny Lee: Right. The reason for the abstraction is because it gives us flexibility, it gives us open standards, it gives us the ability to go ahead and, like, simplify said business logic, right?

Denny Lee: In order... So that way you're able to focus on the real problem, not on the little problem. I mean, if, if you and I were having this discussion five years ago when ChatGPT just came out, right? You and I would've definitely been [00:55:00] debating on things like what were the weights What were the hyperparameters used?

Demetrios: What's the

Denny Lee: prompt? Uh, b- right. Yeah. Right. We- are we still d- having those debates? No. Some damn good AI researchers? Of course they are, because they're trying to improve the models, right? But you and I as users of these systems, we're not having that debate anymore. We were barely qualified to have that debate then.

Denny Lee: We're certainly not qualified to have that debate now, right? And so, but what, what can we do? We can talk about, like, how to effectively use agents, right? Well, remember what I just said, effectively use agents, not effectively use LLMs, right? Because we've already accepted the fact that, like, whether it was called compound AI systems before or whatever else afterwards, the reality is, like, the agent is basically not just the LLM.

Denny Lee: It's actually a lot of the things that are around it as well, right? And then us saying harnesses, yes, exactly to your point, like whether it's an MCP server, if the CLI, it's reading a, a JSONs from somewhere, an XML from somewhere, [00:56:00] or hitting a database, whatever, whatever it's doing. Doesn't matter, right?

Denny Lee: The point is that we're trying to figure out how to make these things portable and stateful. Well, guess what? That is an old-fashioned business database problem that we've been doing since the '60s, right? Like, uh, why are databases back? Like, why, why do we talk about, like, serverless Postgres? Why Databricks talk about Lakebase and Neon?

Denny Lee: Why? Because literally, we have to have cheap, stateful memory for these agents to run, right? It's simple as that, right? Well then, if I have cheap stateful, then I'm like, "Okay, w- what's stateful?" The best thing is a database. Cool. Why is Postgres SQL? Why is there a resurgence? I, I would argue there never was a drop in the first place.

Denny Lee: That's what I would argue, actually. But irrelevant of, like, w- you wanna call resurgence or not, the premise is that I need something that's trustworthy that can keep state. That's it. That, that, and so that's never left us. And I mean, like [00:57:00] remember, I was like former SQL Server, so like I'm going, "Yeah, yeah," like the database is still there.

Denny Lee: It's the... No matter what we've done, no matter how much we've talked about lake houses, which obviously I'm a big fan of, we're talking about agents, which is the, the tool du jour these days, we still need something called state. Well, that state is databases, and then guess what? That's... If you can make it cheaper, you're more successful.

Denny Lee: Well, that's how these agents have run. That's how all these systems work, where we're just abstracting it away and then storing what needs to be stored, i- i.e. state, so that way you can be effective. And so everything, when it comes to, like, how we started with databases before, BI, ETL pipelines, it's the same action plan, right, with agents right now, which is like, how do I modularize this?

Denny Lee: How do I make it portable? And then for a bunch of us, how do I make it open so that [00:58:00] way it's easy for you to move from one to the other?