Context Engineering 2.0

Simba Khadder [00:00:00]: Biggest engineering part of agents for builders becomes the engine, like the context engineering. And this is my view of what context engineering looks like.

Demetrios Brinkmann [00:00:24]: All right, so feature form got acquired by Redis. Does that signal the death of the feature store?

Simba Khadder [00:00:33]: It doesn't. It's funny because you made this LinkedIn post about this.

Demetrios Brinkmann [00:00:37]: I was trolling you.

Simba Khadder [00:00:37]: You were trolling me? I knew you were baiting me, so I didn't take the bait. One, let's talk about where going back to MLOps in general, where did MLOps work and where did it not work? I think the promise of MLOps from the early days, which you remember, was that we were going to have models everywhere. We're going to have these hyper intelligent apps where every time you're using an app you would interact with hundreds and hundreds of models. That was the idea. So you needed MLOps to be able to operationalize all those models. The reality is that did actually happen, but not because of MLOps. It happened because of LM.

Demetrios Brinkmann [00:01:15]: Transformers came.

Simba Khadder [00:01:17]: Yeah. And Transformers are old, but specifically like GPT style, Transformers worked. And the nice thing about them is the bar to get them in production is way lower because there's no, you don't need to train them, they just kind of generically work. So the bar of how much ROI you need to justify doing it is solo.

Demetrios Brinkmann [00:01:40]: Yeah. And you can just hit an API.

Simba Khadder [00:01:42]: Exactly. So it's just like. So now where do you actually need or where does it makes sense for a business to actually invest to maybe have an entire mlops team, which is an expensive team and it's an operational team. So it's like a team that's whole job is to help other teams do things. So if you start thinking of like just the economics of where that would actually make sense. One, it has to happen at scale. Two, the use cases that we see very often, there's many. But the ones that come up a lot, one is recommender systems, which makes complete sense.

Simba Khadder [00:02:18]: Like if your recommendations are better, you make more money. There's a very short line between recommendations being better and money. And the other place you see is fraud. Same thing. If you, yeah. Screw up, you know, if your fraud is better, that literally equals money. And so and at scale, like for every single user of your application or every single transaction that goes through. So that's where we've seen mlops work.

Simba Khadder [00:02:41]: And we're still seeing to this day a lot of companies are deploying new feature stores. They're redoing their existing feature stores. We're actually seeing some of the most demand we've ever seen for feature stores now. And I think that's partly because it's forgotten a bit. And so in the early days when people went and built their own, it's like the maintenance cost of those things. It's kind of a why did we build this thing when there are all these solutions out there.

Demetrios Brinkmann [00:03:12]: Yeah. And the maturity curve.

Simba Khadder [00:03:14]: Maturity curve.

Demetrios Brinkmann [00:03:14]: You start to see that people understand the concept and they realize that it's not just like a postgres database.

Simba Khadder [00:03:20]: Yeah. People are starting to really feel the problems. And it's less like we need this thing because LinkedIn said so. And like hype driven development. And we're getting into like this actually is a huge pain. And if we solve this, there would be true roi.

Demetrios Brinkmann [00:03:36]: Wait, wait, wait. There's. There's things I want to know about the acquisition. One being just like, what's it like to sell a company? How much of a pain is it? I've heard due diligence takes years off of your life.

Simba Khadder [00:03:49]: So in general, I mean, raising around. We've raised multiple rounds. Acquisition, I would say is more intensive than raising around. It just. It depends. I think it depends on a lot of things. You can't really say it in any way. I think the acquisition here, it made sense for a lot of reasons.

Simba Khadder [00:04:10]: One being just as a team. There's like a DNA match, I think of Redis. I think Redis is known for usability, developer experience. It really has that dev love that we really think about at feature form, something that we obsess on. And I mean the cool thing of the acquisition for us is the whole team is there. So the featureform team came as a whole. Each form of the product continues. It will continue to exist.

Simba Khadder [00:04:38]: It's getting doubled down on. We're actually getting more resources to keep doing what we're doing and build on top of it. Our product Enrich MCP is kind of transforming into something even bigger with Context Engine and what we're working on there. And I just think Redis as a company is just in such a cool spot because it's kind of moving and up leveling from becoming just Redis, which again is powerful. It's like a Swiss army knife. But it's like providing that platform layer, the solutions on top and becoming this kind of AI and ML data platform or a context platform. So I think it just was the right fit at the right time and it's just a really good match. And I mean it's only been a month or so since it's happened and like, I'm wearing a red sweatshirt.

Simba Khadder [00:05:29]: I'm like, I really. It just has been even better than I expected.

Demetrios Brinkmann [00:05:33]: Yeah, I'm stoked for you and hopefully we'll get to talk to you in two years and hear all about how you've evolved through the company and all that fun stuff.

Simba Khadder [00:05:42]: Yeah, it's really awesome. And I think that's probably the most important part. I mean, my biggest thing of feedback is same with fundraising. Like your partner, who you work with is all that matters. Because I also know a lot of companies that have been acquired and the.

Demetrios Brinkmann [00:05:56]: Partner in the company that acquires.

Simba Khadder [00:05:58]: Yeah. So I was thinking more like VC partner, but both, like whoever. Like who, who are you working with? Who's like your champion there? Who is. Who cares about this? Are they investing in this? Are they not investing in it? How are they thinking about it? Because you can kind of get in your own head, like what you want them to think and what is reality. And I think it's really important to go figure out what is reality, what is actually happening. So you make the right decision for yourself and your team and your invest and just everyone make sure that it's a. It's a all win. And unfortunately that rarely happens.

Simba Khadder [00:06:30]: And we, I think, are super fortunate that it did for us.

Demetrios Brinkmann [00:06:34]: Wait, so Redis already had a feature store, right? Or Redis was doing stuff. I know. I remember them talking a lot about feature stores back in the day.

Simba Khadder [00:06:43]: Yeah.

Demetrios Brinkmann [00:06:44]: Feature form now is just folding into Redis. What does that path look like now that you're part of Redis?

Simba Khadder [00:06:50]: Yeah. So on the feature store part, how it looks is this. So Redis has always powered the online store. So a lot of companies come to us, we're like, we're building a feature store, we're going to use one. Maybe we're going to use feature form, maybe we're going to use Feast, whatever. Or if we're going to build something in house, we need an online store. And Redis is the best for a variety of reasons, but the general vision is like, well, why don't we meet our customers and prospects where we're at? If they're trying to solve this thing, why don't we meet them there and have a solution for them that's custom built over at this. So rather than them having to go figure it out and just use this as a component, we can actually give them the full solution.

Demetrios Brinkmann [00:07:32]: And features. Feature form traditionally was something that played nicely with everybody. Right. It was one of those things where you said the engine, the compute, the storage, it doesn't really matter. We can just be on top of that.

Simba Khadder [00:07:46]: Yes, and that's still true. In fact, we still have customers on, like, other online stores today. And that will continue to be true. Because Redis strategy, which I really appreciate and this is something that I again, really resonated with me and drew me to the company, was we want to win on our merits. We don't want to win on, like, we own this thing. So therefore, you know, we're going to, like, shut off Cassandra support. It's actually, no, let's get in and make it. I mean, the other thing with Feature form on the flip side is because it's all unified, it makes it really easy to have like Cassandra next to Redis and have both, so you don't have to kind of choose one online store or the other.

Demetrios Brinkmann [00:08:25]: Oh, nice.

Simba Khadder [00:08:26]: And so then it's like, great. Well, if we get people on Feature form, if they just use feature form and pay for feature form, fine. I mean, it's still revenue for Redis, but our view and it's showing to be true is actually now that it's easier to try Redis, you are more likely to 1 just try it because you have it. You have a contract with us to get Feature form and then we'll show you. We'll put like our kind of our money where our mouth is. We'll do a comparison on cost, on speed, on all these things, and you'll see that we are actually the best online store for this. And if we're not, you know, for certain use cases, then that's fine too.

Demetrios Brinkmann [00:09:01]: I think they call that the Trojan horse strategy.

Simba Khadder [00:09:05]: It is, but I wouldn't say it's. I mean, it's. It's sh. Sure. Like, there is some aspect to that, but I think it's actually what I like is it's in really good faith. It's like, yeah, we want to compete, we want to win. I mean, of course we do. Like every company does, but it's not.

Simba Khadder [00:09:22]: It's like, let's just make it easier for them to compare, not lose due to just like, you know, walls that actually limit competition. Like, oh, well, like, it's so hard for us to move over to this thing. So we can never. We can't, you know, we have to not only be bettering whatever they're using, we also have to justify the move cost. Now we don't have to. We can just be like, cool, what's actually better for you? And you can choose that and so it's not just a Trojan horse, it's actually a simplicity to allow the best online store to win. And we believe that's us. And we've been building towards that for a long time.

Demetrios Brinkmann [00:09:55]: So talk to me about Redis and elasticcash because I'm not sure I fully understand what the difference is between the two.

Simba Khadder [00:10:03]: Elasticache and this isn't true for all the clouds. So I'll talk about elasticache. Like for example on Azure we have AMR which is managed Redis and that actually is Redis, that's us of elasticache. There's a fork of Redis on the older version of Redis and that's actually what's powering elasticache. And there's a lot of things like availability, speed, et cetera, even just like general capabilities. Redis Core redis We released Redis 8 recently. Core Redis has really pulled away in a lot of places. The other thing though isn't just Redis the kind of open source product, but there's also Redis the company.

Simba Khadder [00:10:53]: And I think one thing, another thing that really is interesting to me about Redis coming in and another thing that really the vision that I really liked is think of feature form and feature stores. Redis has always just been that database, the core database which people developers have always used, we've always loved. And that continues to be true. But we're kind of having this almost like a platform moment. We're like cool. What are people using Red list for? What more can we do to provide more value? Maybe higher levels of abstractions and solutions for both agents for ML via feature stores and a variety. Like we have Flex which is like cold storage and SSD which is really powerful, especially actually for feature store use cases where you might have massive data sets.

Demetrios Brinkmann [00:11:41]: Vector database. I remember back in the day we did a whole before LLMs came out we had a vector search or I think it was similarity search hackathon where we were using Archive and it was with Redis.

Simba Khadder [00:11:57]: You know what's funny is the use case with Rexis and actually a lot of Vector dbs, you know, people think of them as for rag, but my sense, I actually don't know what's true at Redis, but I have seen actually just other companies and my sense is actually that a lot of revenue still comes from recommender Systems for vector DBs and not actually for Rag as people would think, which is kind of interesting for the features. For what you'll do is you will generate like user embeddings or Item embeddings. And then you will say, you know, very simple thing might be I have an item embedding for like a. A shirt.

Demetrios Brinkmann [00:12:30]: Yeah.

Simba Khadder [00:12:30]: If I do a nearest neighbor search on that, I can get similar shirts.

Demetrios Brinkmann [00:12:35]: Yeah.

Simba Khadder [00:12:36]: And so they'll do candid generation and then re ranking. And this is like how two tower recommenders work.

Demetrios Brinkmann [00:12:41]: Yeah. I remember the two tower stuff that you mentioned back in the day and I totally had forgotten about that. Yeah. Because I think funny enough, you were the first person to talk to me about embeddings and you had what an embedding store is what you were calling.

Simba Khadder [00:12:58]: It or Embedding hub.

Demetrios Brinkmann [00:13:00]: Embedding Hub. Dude, that was what, like 20, 21?

Simba Khadder [00:13:03]: Yeah, we were like, I think feature form. We were like one of the first vector databases, which is really funny. And it's because at that point I built like three of them. So I was just like, this needs to exist. What I couldn't have guessed is that, oh, in like six months or a year, this is actually going to be like super huge market because of RAG and LLMs, which hadn't even crossed my mind. I built it again for Rexis use cases because we were seeing people using feature stores for recommender systems. So that's why we built it. And it's really funny how that worked out.

Simba Khadder [00:13:35]: And then we kind of bowed out because it became like a database thing in feature form, it itself. We just weren't like a true database company. We didn't really have that DNA. But I've always kind of wished we did. And I kind of feel very fortunate now that now those people are a call away for me.

Demetrios Brinkmann [00:13:55]: So I understand the platform idea. I also know that in your talk when we were in London you kind of broke down this vision of we've got all of these assets with Redis and so now how can we unify them for the new generation of agents?

Simba Khadder [00:14:11]: Yeah, I've been calling IT Context Engineering 2.0, which I don't know if I love that yet, but that's what I'm going with. It's catchy. It's catchy.

Demetrios Brinkmann [00:14:19]: I'll work with it.

Simba Khadder [00:14:20]: Yeah, you gotta find the hype words, you gotta make new terms. But really what I'm seeing is. Okay, so when we started with LLMs, I think from the earliest point I realized that the only thing. Thing, not the only thing, but one of the most important things is context. And when I think of context engineering in rag, actually I always felt like we were too specific. Typically when we talked about RAG People talk about rag. They're like vector db, nearest neighbor, lookup on paragraph chunks based on the user query. That's kind of what people think when they think rag.

Simba Khadder [00:15:00]: But really the problem isn't that. The problem is I have a context window. It's yay big. I need to fill it with the most relevant and insightful information for an LLM to be able to do what it needs to do. And I need it to be also not relevant because then it can throw. It can cause hallucinations. Now it just turns out that a really, really good baseline way to do this is just grab a lot of like a tax corpus and chunk it up and kind of do rag. So RAG works naively, just like weirdly well for how kind of cookie cutter of a solution it is.

Simba Khadder [00:15:43]: It kind of just works well for everyone. But that's just a baseline and we're clearly like moving past that. And the other parts of context that have come up are like memory. It's a big one. And I think the newest frontier is structured data for agents. So it's kind of like, kind of is mcp, but not really. I think it's. It's actually a more general concept, which is how do I get.

Simba Khadder [00:16:09]: If someone asks? I mean, it could be something as simple as I'm building a support agent. Someone's like, why is my order delete? Go and find the order. Find all the information about the order, which could be in like postgres, like it could be in the structured database. Get that into context because the agent's going to need it to be able to solve the problem at hand.

Demetrios Brinkmann [00:16:27]: Yeah.

Simba Khadder [00:16:27]: So we have these three things. Unstructured data, memory, structured data. And today, if we're going to do that, you would build like a RAG pipeline here. Then you would build like a memory service, and then you would deploy like some MCP servers. And these things don't really talk to each other. They're almost like kind of disjointed context and they all kind of work differently. And so my sense is we should build relationships between this context. Like a memory can be associated with a document.

Simba Khadder [00:16:59]: There could be a row of data that kind of points to a few paragraphs in the RAG store.

Demetrios Brinkmann [00:17:07]: So almost like knowledge graph type thing.

Simba Khadder [00:17:09]: Almost like knowledge graph. I don't like using knowledge graph purely because it's so overloaded in this space. People immediately start thinking graph rag, which is not what this is and not how I would describe it. It's more just treating context as a holistic thing. There's all these indices and like different ways to, let's say, retrieve data. And you also need some sort of like semantic layer on top to understand that data. I kind of think of it, I've been thinking of it as what I wish I had was GraphQL for agents.

Demetrios Brinkmann [00:17:41]: What's GraphQL?

Simba Khadder [00:17:42]: GraphQL was kind of one of these things that was really hype, but never, I think, fully took off. And GraphQL is just a different way of building, like Rest APIs. Rest APIs are very like kind of input output. GraphQL allows you to almost create and unify a lot of data from different places and write kind of almost like ad hoc queries. And the reason why it didn't take off was it's kind of complex to do. It could be slow. And humans in general, especially engineers, we like input output. It's much clearer and cleaner to understand.

Simba Khadder [00:18:14]: The thing is, those axes of freedom a GraphQL gave us is extremely powerful for agents. Agents love having axes of freedom because they can actually take advantage of all that. And the speed to parse a GraphQL query doesn't matter anymore because the agent's the slow part. It's not typically, not the data retrieval or the query.

Demetrios Brinkmann [00:18:34]: So GraphQL I'm trying to wrap my mind around, but also, how would that even work with agents?

Simba Khadder [00:18:41]: So how I imagine it is this. You would define like, let's say you're an E commerce store. You have users, you have items, you have orders. These things are all related to. Yeah, very tabular. But they might also have documents in them. Right. Like order might have descriptions or it might have specifications or there's a mix of things.

Simba Khadder [00:19:04]: But it's like you have attributes and you have relationships, so you can define this whole thing out. You can kind of define or explain somehow how to get to that data. Then you expose this knowledge graph or semantic graph to the agent. So agent can be like, cool. I can navigate and figure out what data exists. So I can go ask for exactly what I need to be able to resolve what's at hand. The nice thing there is there's two parts, one or there's many parts, but one part is control. You as the user can really control the contextual view that the agent has, which you can't do, like text to SQL, which is like a whole nother thing.

Simba Khadder [00:19:48]: You can't really do it with API to MCP because you're just kind of stuck to every API has and the surface area there.

Demetrios Brinkmann [00:19:57]: Yeah.

Simba Khadder [00:19:57]: And so this allows you to Build a surface area that is tailored to agents, like the right kind of abstractions and semantic layer for an agent.

Demetrios Brinkmann [00:20:06]: But you see this almost being like an abstraction above all of that stuff that is behind the veil.

Simba Khadder [00:20:13]: It's a single pane of glass.

Demetrios Brinkmann [00:20:15]: Yeah.

Simba Khadder [00:20:15]: So I'm calling it today I call it, I'm calling it Context Engine, but it's a single pane of glass to all of your contexts or about structured data, unstructured data or memory. So you just connect it. It speaks mcp, but that's just implementation detail. But the agent can connect to this thing and they can go and figure out, okay, what context exists in the world that I have access to and what does it mean, how is it related, how do I get it? They can go and retrieve what it needs in that way.

Demetrios Brinkmann [00:20:42]: And you're letting the agents figure out the semantic layer on top so that they understand. Or is it humans that are kind of creating a document that says, well, this is this data. If you're getting these kind of queries, you should reference this.

Simba Khadder [00:20:55]: Yeah, it's written in Python. Um, so the, the developer would write it in Python. They could use an agent to help them do it. But I think it's really important that the humans kind of build this to control it. So I think that's a lot of what engineering becomes is, is, is that for agents, which is again, I think most of the, the biggest engineering part of agents for, for builders becomes the engine, like the context engineering.

Demetrios Brinkmann [00:21:25]: Yeah.

Simba Khadder [00:21:25]: And this is my view of what context engineering looks like, at least for the next five, ten years.

Demetrios Brinkmann [00:21:29]: Interesting. It's more the descriptions and the pointers and the understanding of the data. It's still being intimate with the data and understanding where and what you need. But it's not necessarily writing scripts or writing different functions to get to that data. It's describing that data in a way that an agent is going to be able to understand to get it when.

Simba Khadder [00:21:53]: It needs it for someone. Maybe like if you're listening to this and you want it to maybe see it more clearly, featureform had it still has open source product called Enrich mcp which is widely used for. And it really captures that. Now Enrich MCP and how it was built originally is really generic. Now when you allow Redis to become a dependency, you get. So because now you get a vector database, you get all of the abstractions that Redis provides underneath the hood and the caching that you might need for APIs, all of that stuff becomes part of it. And that's I think what Context Engine becomes is like this layer which maybe mimics or looks like Enrich mcp or is that kind of a continuation of it, but with the power of the Redis database behind it?

Demetrios Brinkmann [00:22:47]: Yeah. Almost like the Redis ecosystem because it's. I. I really like how you've shifted my thinking from Redis as this cache and just something that's super fast when you need it to. Okay. Redis has these other things though, and now you can look at it as an ecosystem and how can we use all of the. The assets that Redis has to leverage our agentic use cases?

Simba Khadder [00:23:12]: Yes. When you think of why people like Redis, there's two things. One which we talk a lot about is speed, which is like, well understood. Redis is and will remain pretty much the fastest thing you can deploy for data.

Demetrios Brinkmann [00:23:25]: And the second is the founder is kind of a legend in his own right.

Simba Khadder [00:23:30]: Yeah.

Demetrios Brinkmann [00:23:31]: What was his name?

Simba Khadder [00:23:31]: Salvatore.

Demetrios Brinkmann [00:23:32]: Salvatore.

Simba Khadder [00:23:33]: That's it. The guy's like, he's an employee of Redis again. Yeah, he's awesome.

Demetrios Brinkmann [00:23:39]: So that's the second reason, or what was the second reason?

Simba Khadder [00:23:40]: No, that would be the third reason. The second reason is the abstractions that you have. It's like the simplicity. It's almost like when you look at Redis, you're like, oh, cool. I have all these types of things I can use, like these different types of data structures that are built into Redis and I can pick and choose what I want. And it's really, really simple. It's all text. It's just nice.

Simba Khadder [00:24:03]: Like the UX of Redis has always been really nice. And so now you have a really nice UX that's super. It's like Swiss army knife. And like, it does so many things and they all kind of work really quickly. And with Redis Cloud, like now you get high availability and all this other stuff you'd need for a production setting. So you combine all that and then you build a layer on top, which is again, that kind of almost like data platform for agents. That's kind of where we're going. And I talked about Context Engine, which is a huge part of it.

Simba Khadder [00:24:33]: And I think a big. The point of Context Engine is, hey, agents give agents a single pane of glass to get to the data that they need to solve real problems. There are other things we do even to like our caching heritage. Like we have Lancash, which is. A lot of people have problems of spending too much money on tokens. Right. LLMs are very expensive and when you Go look at what people are using your customer support bot for. You'll find that a massive percentage of questions are like the same thing.

Demetrios Brinkmann [00:25:07]: 8020 principle.

Simba Khadder [00:25:08]: Again, if you were to cache naively, you're like, oh, if this exact query gets written, it's not going to work for two reasons. One is if someone changes one word and it doesn't match. Two is if someone's like, what's the weather today then? Well, you can't use the cached answer because yesterday the weather was different from today. So what lane cache is, is it uses. Actually it uses the Vector DB vectordb under the hood again, showing building these solution layers on top. And it enables us to actually see is this equivalent. And we look at a lot of different axes. There's negation, which is another thing, which is like something can be similar, but if you add the word not in it, it's actually completely the opposite.

Simba Khadder [00:25:56]: If you're like whatsoever in San Francisco versus whatsoever in San Diego, it's like it looks pretty much the same, but that Francisco or Diego completely changes the context. And so, yeah, we're just doing. I mean there's just so much happening at Redis and I think the vision is to continue. We've been the Swiss army knife for the web age, for things like sessions, rate limiting, just everything around making web apps reliable, fast at scale. I think now the vision continues to be that and I think that is still a huge part of our business. But also thinking of the future and where things are going. And I think the future of Redis is providing context to agents.

Demetrios Brinkmann [00:26:42]: What I am still caught up on is the single plane of glass and the, let's say whether it's Enrich MCP or it is this context engine. How do you basically give the data to the Enrich mcp? Is it through tools you mentioned that Enrich mcp, obviously it speaks mcp. It's kind of in the name. So that's a dead giveaway.

Simba Khadder [00:27:06]: It's a giveaway.

Demetrios Brinkmann [00:27:07]: But is that just like on your side tools going into the different databases, Is it like how does that actually work?

Simba Khadder [00:27:18]: Yeah, so there's a niche mcp, but I'll actually talk about more like the broader vision of context engine. So think of how a RAG pipeline works. You take documents in S3, let's say somewhere you transform them, which is just chunking, embedding, getting them in a vector database. That thing you have in the vector DB is actually, if you zoom out a little bit, this is a materialized view. You're projecting documents in S3 into embedding space. And so that's a weird. Historically would have been a really weird transformation to do. But it's a materialized view.

Simba Khadder [00:27:59]: Online stores, features in online stores. These are just essentially materialized views. And so one aspect is just kind of materializing views that you want your agents to be able to see into Redis, making those queryable. Redis has a query engine called RQE which powers the Vector DB and the law of the other stuff, Search, Redis search and other things. But if you project into Redis, you can then. And then you explain what are these things and how they're related, how are they associated to each other. They become accessible. But it's not just that because some things you don't want to materialize into Redis for whatever reason.

Simba Khadder [00:28:38]: You can still support APIs, calls or other MCP servers like backed.

Demetrios Brinkmann [00:28:42]: But when you're projecting into Redis and then the agent is going to get that is that just through a tool call to Redis, it's like, yes, it.

Simba Khadder [00:28:51]: Would be through context engine, but it would be backed by Redis. So it would just be like MCP in the end creates tool calls. And another way to actually think about what enriched MCP is or the I guess thing behind it. I've talked about the GraphQL thing. The other thing that I was thinking about was for a lot of use cases for mcp, it's really just data retrieval. And you find that it's actually very repetitive to write these tools sometimes because you have get user list users, get users by vid. You know, you kind of end up with like a very repetitive code and just a very kind of annoying to.

Demetrios Brinkmann [00:29:28]: Write code base and bloated.

Simba Khadder [00:29:30]: And it's bloated, but if you do it from the data first, so define the schema first and then kind of attach the MCP stuff on top, you end up, it still again becomes a set of tools. It becomes an MCP server, but it is data first. You start with the data and you define the MCP server from data first. That becomes tools as opposed to tool first and then trying to kind of map that to data.

Demetrios Brinkmann [00:29:56]: So the data first, as in you understand the data, you see what the data is almost like trying to tell you. And going back to this, maybe we can use the store example, online store as a perfect example where you've got different SKUs, you've got descriptions, but you've also got numbers and order borders and all of that. You see that you also are embedding all of these descriptions or the SKUs.

Simba Khadder [00:30:23]: Or you could embed things. But no, a lot of it is just like gets, because that's the thing is like it's structured data. Not everything has to be like. I've seen people do things like they put into JSON that they do like a near, like a, like a rag on top of JSON documents, which is for a variety of reasons. It's also just not natively how you would typically want an agent to access that kind of data. Search has its place and it's extremely powerful for agents, but it's not the answer to everything. I think we've gotten to the point where we have a hammer and everything looks like a nail. So it's not just embedding things, it's more like an orm, if anyone's ever used an orm.

Simba Khadder [00:31:03]: In fact, Inertia MCP supports SQL alchemy directly, have a SQL alchemy orm, you can actually just turn that into an enriched MCP server. And so an orm, if someone's not familiar is. You would define your database schema as classes in Python and then you define what each field is and the type and the indices that you would have on it. And then rather than writing SQL queries yourself, you kind of interact with it as you would in Python. And under the hood it always maps down to SQL queries. But when you're in the Python layer, it feels way more natural to work with classes and data objects. And so inerts is kind of the same idea. It's much more natural for mcp, which is really just data retrieval, to think at the classes and data retrieval function layer than the writing tools that just run ad hoc queries or whatever.

Demetrios Brinkmann [00:32:03]: One thing that I saw in the talks yesterday was that JSON consumes a ton of tokens.

Simba Khadder [00:32:11]: Yeah.

Demetrios Brinkmann [00:32:11]: And when you're trying to search, or you're giving agents tools to search, or just you're giving them access to data. Donay was talking about how they switched to CSV and it consumes way less tokens and therefore is faster and cheaper and all the benefits. Right.

Simba Khadder [00:32:29]: Yeah.

Demetrios Brinkmann [00:32:30]: In this case though, I'm. I'm not sure how you would look at that. It wouldn't even be that you're giving the data like that to the agent.

Simba Khadder [00:32:41]: Or you would, could, but it doesn't. It's like what some people do, that they'll dump an entire CSV to the agent. Yeah. Like I've seen this before, people will build this giant object and they'll just dump all of it because they don't know how to make the agent go find structured data, so they'll just dump all the structured data that they think is relevant just into the prompt itself. And then in that case it becomes really important to optimize those sort of things. But my sense has always been that rather than one shot queries where you just go and grab as much context as possible based on some query and just throw it into the context window, the correct way to build agents for a lot of use cases is, is actually don't do that. Let give the agent an interface, that window to go find the data it needs when it needs it. Like the agent is plenty intelligent, right.

Simba Khadder [00:33:33]: Nowadays, LLMs are so smart and it's very rare. Like you're very unlikely to have a use case that you're building an agent for where the agent isn't smart enough. The issue is almost always the system, the context. Yeah, it doesn't have the right information or data to be able to do something smart. Even code generation. Right. Like a lot of people complain like, oh, like you know, it uses too many try catches and we don't like doing that. Or our style is different and it's not like the agent's wrong.

Simba Khadder [00:34:04]: You know, the code it generates is likely good and fine, but it doesn't have all of the context about how you like your style guide and all these things that it needs to do well. Or it's like, oh, it didn't, you know, it made up a function or API that doesn't exist. It hallucinated that. And my sense is like, well, it's like pretty much skimming your code and it's guessing what functions should be there. Chances are that function probably should be in your API if it hallucinated it. And if it can't be in your API, then that's more of again, it's a context thing. There's some stuff it's missing, it doesn't really understand. It's not like if you didn't give it enough information, it wouldn't be able to reason about your stuff.

Simba Khadder [00:34:46]: It just doesn't have that. And some of the problem is like how do you get that to it in a way that's efficient without just dumping your entire code base in the context.

Demetrios Brinkmann [00:34:53]: Yeah. So when you say, hey, that function doesn't exist, you hallucinated it, it responds and says, you're welcome.

Simba Khadder [00:35:00]: Yeah, well, I'm serious. Like I do this sometimes. It will hallucinate the function. And I'll be like, oh, actually that function is actually probably Fine. To have it in our library. I just didn't think of it or didn't bother implementing it. And I'll just be like, go make it. And like, that's totally fine.

Simba Khadder [00:35:14]: Yeah.

Demetrios Brinkmann [00:35:15]: No mistakes.

Simba Khadder [00:35:15]: Yeah. Don't mess up.

Demetrios Brinkmann [00:35:17]: Okay, so I think there's a fascinating thread to pull on here with the ways that you're using search. Right. And the ways that you're exposing the context. The right context at the right time. We heard it last night a bunch how you. Just because you have access, that doesn't mean you have, like, intelligence. Yeah. To the right type of answers.

Demetrios Brinkmann [00:35:46]: And I think you're. You're hitting on that point. And that's why RAG was a disaster, because it was like, oh, well, we'll just embed everything, throw it in the vector store, and then we give access to the vector store, we give the LLM access to it, and it's off to the races. But then you're like, okay, how much do we chunk? Where do we break the chunks? And it starts getting really confusing using. And that's exactly why is that we need to, like, almost dynamically be doing this. And instead of saying, rag is the solution, let's just give agents the search tool when they need it, if they need it. And then we can also give them other tools that are going to be valuable at the same time.

Simba Khadder [00:36:33]: The first iteration of rag were LLMs that didn't run in a loop, which means that they weren't really able to make tool calls and do all the things that you might want it to do. So what you would do if you have one shot, one pass, is you're like, well, okay, I need to just put everything possible into this prompt. And so given that problem statement, brag was a good way to solve that. But now we're past that. We can really trust. I mean, yesterday when we were at the conference and there was some point where I needed to move from one library to another library, and I literally let it run and it ran for 10 minutes and it came back and it had all my tests passing. I was shocked. But this thing ran for 10 minutes and did it all for me and got it all working.

Simba Khadder [00:37:23]: So, I mean, if you get an agent to run the loop. Actually, I heard this thing from Anthropic where they're saying that the metric for tracking nowadays is more about how long can you trust an agent to work independently, autonomously without, you know, a human intervening and still making forward progress.

Demetrios Brinkmann [00:37:42]: What were they calling it? Yeah, last night they were talking about it. It Was like interruption, amount of interruptions or something like that. And so if you interrupt it, then that trust and that metric is lower, right?

Simba Khadder [00:37:56]: Yes. I think. I actually didn't see that talk yesterday, but I think that it's all referencing the same thing because. Yeah, that's a way to, to track that is that they killed the job. If they killed the job, that means that it started doing. It started going off the rails or it just got confused. And so that's a metric to track. And then if you're tracking that metric, they're saying it's doubling.

Simba Khadder [00:38:15]: I forget what they said. If it was every six months or every year, it doesn't really matter. The fact that it's doubling is nuts because let's say today, you know, it's 10 minutes. Yeah, a few minutes. Like five minutes is what I got yesterday, or I got 10. But let's just say it's five minutes. Okay, that's fine. But if we're at 30 minutes in a few years, how different is that gonna be? So.

Simba Khadder [00:38:39]: And when you start thinking like, okay, well, I need something to run for 30 minutes, it's not like you can just leave it with thoughts for 30 minutes and it's just gonna do something really smart. You kind of need to give it a lot of access to data, the Internet, all these things. Like, how does Pro work? Like the way Pro is so smart, is that like. Like GPT Pro?

Demetrios Brinkmann [00:39:00]: Yeah.

Simba Khadder [00:39:01]: Is that if you watch what it's doing, it's really just going. And like searching the web. If you didn't let search the web and just sat there and thought and kind of in a loop, it might do better, but I just don't really think it would be a lot better.

Demetrios Brinkmann [00:39:13]: Yeah, it wouldn't be that useful.

Simba Khadder [00:39:14]: Yeah, it's. It's probably. It's more powerful to just give it access to. It's just context. It's all. I mean, it's. It's so funny because like, we say the word so much, it almost sounds like it almost like doesn't sound like a real word anymore. But it's just all that matters is context.

Simba Khadder [00:39:30]: And if you can build proper context, you can get it to work. And especially now, you know, people are thinking about small models. There's a lot of things people are thinking about, but that will always be true. The better context you have, the better results you're going to end up with.

Demetrios Brinkmann [00:39:46]: So talk to me about McPay.

Simba Khadder [00:39:49]: I would love to talk to you about MCP. MCP is like McPay, McPay is a. Is a dirty word in the LinkedIn world.

Demetrios Brinkmann [00:39:59]: I think now whenever I see you online, whether it's on X or LinkedIn, people talking about MCP, I just insert that voice and I remember like, yeah, this is the hype word of the moment. So MCP it is.

Simba Khadder [00:40:18]: You know, the funny thing about MCP is really all it is is like you're moving tools remotely now, why that matters. I think the thing that people miss is like, oh, this thing's so simple. I already had tools, whatever. Like, why are we doing this new MCT thing? We already had rest. The idea is this one, like just tool discovery. There's like all these concepts that exist in MCP but don't exist in rust. Some people would argue, well, couldn't we just build this on top of rest? The answer is probably, but for better or worse, we've all kind of standardized around mcp. And that's important because getting anthropic and OpenAI to agree on something, a standard, and having all the enterprise companies kind of come and accept it as the way forward, that rarely happens.

Simba Khadder [00:41:02]: So that's a big deal. And what it's doing, and it's almost such a minor change in some ways. We're just putting the tools in a server that that's the only way you could actually make something so monumental happen on standards, what MCP is actually doing. Let's think about what it actually means at scale, at enterprise scale to do mcp. Well, what it turns into is you're separating the agent from the context layer, essentially, and you are enabling teams to build and maintain in parallel MCP servers, which are really, again, this data access or tool access that any agent can come and connect to and get value from. So it's actually like an engineering. It's almost like microservices.

Demetrios Brinkmann [00:41:52]: Yeah, it's almost like databases. Like, you now have access to my database or my. In another way, you could look at it, my knowledge, like, here's my knowledge base. I can just have this for you.

Simba Khadder [00:42:05]: Yeah, I mean, yes. And there's other piece of it which is like, how did we do this before? Well, we just put these tools directly into the agent. The problem is organizationally that doesn't scale. Like, if you're doing many agents, you don't really want to be installing libraries or maintaining libraries and then trying to kind of fit these into every single agent separately. It's much cleaner to just say, here, connect to these five things that you have access to. And when you start thinking of auth and discovery and all these other things. It just becomes a lot more scalable, especially when you have a large team to have all of these MCP endpoints. And even in a small team, it's just like you're just creating a separation of concerns.

Simba Khadder [00:42:47]: There's the agent. How I think of it is like there's the agent and like a lang graph or whatever you use. There's kind of the agent framework, which is really explaining how the agents, what should be doing, which agents exist, how they should be thinking and what they should be connecting to. There's the context engine, which is again providing the window to the data, which is typically backed by APIs, data, maybe other MCPs. And that's again the thing that we're inventing and have at redis. And then on the other side, there's the LLM itself. And so those are the three components of an agentix system. The agent workflow, the context workflow, and then the LLM workflow.

Simba Khadder [00:43:27]: And that's it. So that's it. I mean, that's all it is, all an agent is. It's a LLM running in some sort of scheduled loop or orchestrated loop that gets data. It needs to be able to solve hard problems. So it's really just taking this powerful thing, which is an LLM, and turning it into actual enterprise solutions. To do that, you need to give access to it, ability to do things and ability to learn things. And then you need something to like, suppose you have multiple agents, multiple tasks.

Simba Khadder [00:44:00]: You just need to give it a system prompt. Perhaps you have a few different agents that create separation of concerns there, which is really interesting, but maybe a conversation for another time, but that's it. It really is that simple. And so if you separate the MCP layer, or just the context layer and endpoints and APIs away from the agent layer, it makes it way easier to be able to kind of iterate on these things separately, each other. It's not like, oh, we added a new API, we want to change this. I need to go like make a PR to every single agent or change this library and have all the agents update to this new. It's just much cleaner to just maintain a service and just say, connect to this thing and we will make sure it's up to date and it provides these tools or access to this kind of data.

Demetrios Brinkmann [00:44:47]: Yeah, yeah, like APIs or like even just like separating storage and compute.

Simba Khadder [00:44:53]: Yeah, it is. Another way to think of it is like you're separating storage and compute.

Demetrios Brinkmann [00:44:57]: You're separating the action, the intelligence and the context. I like that.

Simba Khadder [00:45:01]: Yes. And I think that's. That's just how you should build agents. And I think we're just handling the context layer, amongst other things. But the context layer is personally what I'm thinking a lot about.

Demetrios Brinkmann [00:45:15]: We're good. I think we're good. Msepe.

Simba Khadder [00:45:24]: M sepe m se I'm also not surprised you have multiple voices in your head. Demetrius, knowing you as well as I do.