From Rules to Reasoning Engines

George Mathew (Insight Partners) joins Demetrios to break down how AI and ML have evolved over the past few years and where they’re headed. He reflects on the major shifts since his last chat with Demetrios, especially how models like ChatGPT have changed the game.

George dives into "generational outcomes"—building companies with lasting impact—and the move from rule-based software to AI-driven reasoning engines. He sees AI becoming a core part of all software, fundamentally changing business operations.

The chat covers the rise of agent-based systems, the importance of high-quality data, and recent breakthroughs like Deep SEQ, which push AI reasoning further. They also explore AI’s future—its role in software, enterprise adoption, and everyday life.

George Mathew [00:00:00]: Hello there. I'm George Mathew. I'm a managing director at Insight Partners, and I take my coffee as a hot or cold vanilla latte with at least 1% milk.

Demetrios [00:00:18]: Welcome to round two of this conversation that I'm having with George Mathew. This is the MLOps Community Podcast, and I'm your host, Demetrios. In the first edition, when I spoke with George so many years ago, we reminisced about what he did before he became one of the most predominant VCs in the AI and ML industry. And in this edition, we talk all about what he's been seeing on the market and his theories for the next couple years.

George Mathew [00:00:51]: Let's dive right in.

Demetrios [00:00:59]: Been two years since we chatted. I remember when we chatted last, I was in Greece, enjoying the sun, and I was totally oblivious to what was about to happen, which I think we talked even before the ChatGPT moment ate up the Internet. Now, what else has changed, though, for those last two years? What have you been focusing on and thinking about?

George Mathew [00:01:23]: Oh, my God, like, nothing happened in the last two years. Right. He was so quiet and so. Such peaceful moments and everything related to data AI ML. Demetrius. Well, wonderful to see you again. It has been too long. Thank you again for having me back on the mlops podcast.

George Mathew [00:01:40]: It's a pleasure what you've been able to do with this community for the last several years.

Demetrios [00:01:44]: Oh, thanks, man.

George Mathew [00:01:45]: Coming back to a podcast a second time around, particularly after, you know, having some perspectives even before everything got crazy when we last talked. So, yeah, this will be fun. Look forward to it.

Demetrios [00:01:55]: I remember distinctly that you talked about, I was asking, how do you look at companies and what are things that get you excited in companies? And there was something you said, and I'm paraphrasing here, but it was along the lines of, you're looking for companies that are going to really be able to build generational organizations or something.

George Mathew [00:02:17]: Generational outcomes, Yeah, I think it's. That's it.

Demetrios [00:02:19]: Generational outcomes, yeah. Which was the first time that I had thought in such long terms, and I was just like, wow, how do you even.

George Mathew [00:02:29]: Wow.

Demetrios [00:02:30]: That blew my mind. And so maybe we can talk a bit about generational outcomes and what you think are the next wave of generational outcomes.

George Mathew [00:02:39]: Sure, yeah, happy to do that. And I think it's a timely moment for me to have that conversation because I am entering my fifth year as a venture capitalist and certainly had quite a long career as a builder that has been able to scale up companies, particularly in the data AI, ML space. Prior to, of course, being a venture capitalist at Insight for four years and going on my fifth year, I mentioned that because Insight just celebrated our 30th anniversary and we just raised our 13th fund, which is a $12.5 billion fund. 13. And if you look at the arc of software for 30 years, and insight has been very focused as investors in software from 1995 to now. I graduated college in 1995 and so for me to almost think about what I did in my career as a builder for as long as I was and now being a vc, it is a dramatic shift to go from the moment in time where we started to see the introduction of client server in 1995 and the advent of the web, the emergence of mobile at scale, the cloud infrastructure that ended up defining a good decade plus of software. And now we're again at this precipice and we're well past it in terms of how AI systems are being built as we speak. So I, I can't be more excited by not only what we've been through for almost 30 plus years, if not longer, in terms of how software has become the most important substrate for how humanity continues to express itself in driving productivity around the world, but also what we can do for the next 30, 40, 50 years.

George Mathew [00:04:34]: And it's very clear to me that everything, particularly in software from this moment on is an AI driven feature.

Demetrios [00:04:42]: And what does it look like? Because there's some things that I know we were talking about where it's like AI systems are taking more than software and it's really thinking about how do we go beyond the screen. And I think probably the first step is going beyond the chat box and then we can go beyond the screen maybe. And so how are you looking at that and what are some things that you're seeing that excite you?

George Mathew [00:05:13]: Yeah, I want to set that up as well in terms of the march towards this current generation of AI systems that are being built. It goes without saying we should really highlight that we couldn't be building these modern AI systems without the emergence of the modern data stack itself. Right. Because it's very clear now that you need high quality data to be able to build your AI systems at scale. And certainly that proved itself out with the transformer architecture that really got to the GPT4 models, the L1 models, certainly the things that Anthropic has released and what many of the foundation model builders have been able to capably ascribe in terms of both knowledge and reasoning embedded into, or let's say more or less precisely imbued into the models themselves. And so you see the march from the modern data stack to now taking those models and defining the machine learning operations surrounding how you build and scale a model. And so more recently around LMS and the operations of llmops and now the agent based systems that are now possible, right? It's not just about what you can do in the foundation models themselves, but how do you orchestrate those effectively so that you have a agentic approach to being able to have multiple reasoning systems, multiple knowledge systems converge together to have a new AI system being delivered for an existing or a new problem. And so that's where I'm starting to get pretty excited right now, because I realized that most of software up to this point has been based on rules in building highly capable rules engines.

George Mathew [00:06:59]: But over time, rules get very brittle because you have to continue to encode what your business processes are into those rules. But what if you didn't? What if you had almost a reasoning system that can continuously reason what's happening? And that reasoning could be human, like perhaps even superhuman, like in terms of the reasoning that you're coming out of an AI system today. And that's exciting, right, because it starts to change a lot of where you can see the value proposition and generally how software itself is consumed, right? Because at some point up to this moment, you had to buy your software, you had to enable someone from a services standpoint to get that software working inside the enterprise. And you had some people that were operating the software as the Personas that cared about that particular piece of software. If you used a CRM software, then salespeople, sales managers, sales operations cared about that software. If you were an ERP software provider, then you had a number of folks that were in finance and operations that cared about that software. Now this future that we're barreling towards incredibly quickly is one where the AI systems are encompassing the software value proposition with the services in terms of enabling that software in the enterprise. And the first advent of digital workers, synthetics, like things that we haven't seen before, where there's a full autonomous experience of how that software is now functioning inside the enterprise without necessarily as much of the cost involved with services in terms of putting that software into work.

George Mathew [00:08:49]: And certainly now even synthetic SDRs, synthetic BDRs, synthetic developers, synthetic process automation folks that can now, you know, instead of manually processing that information, you have a full reasoning system that's processing. That's very similar to what the investment we made with CREWAI was in terms of Just being able to automate all the, what would be historically manual robotic processing with very little smart exception handling. And now you have a high reasoning function involved with how you do a lot of your back office automation and processing. So that arc of like software now transforming all the way to the emergence of these AI first systems. And these AI first systems are encompassing software and the services and labor. Like that's something we haven't really fully seen yet. It's just starting to emerge in some of these key markets and I'm pretty excited with what that opportunity looks like.

Demetrios [00:09:52]: Let's talk for a second about a few different papers. I know that you pulled up what you really liked about Dario's paper. And then there's these different almost views of where we could go with the future. And, and it's almost like one is hugging machines and the other is being strangled by machines. In a way.

George Mathew [00:10:16]: It's funny. Let's like set the context. Right. So I think it starts certainly with Leopold Aschenbrenner's paper in June of 2024, which is situational awareness. Right. He was outlining the next decade ahead of head in terms of what was necessary to build this future. And we'll talk about both papers in a moment. But what I think I saw in that paper was a very mechanistic view of the future.

George Mathew [00:10:43]: What would it take to scale out these clusters? What would it be necessary to go from AGI to superintelligence and what the timeframe was to do that? What does it mean to build a trillion dollar cluster itself? Like how much we have to lock the labs down? Which is really funny that we talk about locking labs and security of AGI, particularly the deep seq sit even six months or whatever, seven months. It's like a profoundly different way. At least the rest of the world has reacted to it than at least the paper kind of came about it. What does alignment look like? What does it mean to drive democracy in a free world? While all this is happening, which is a brilliantly thoughtful paper, I think it was also just a little heavy handed in terms of how it thought about the timing of AGI, like what would happen? I thought one of the most interesting thing was which weather and when it happened or not didn't matter. That was really exciting was this idea that if you could build a super intelligence, it could automate the research output that's necessary to create all the possible futures that you would want as humanity. And I think that that was my, my, yes, that does make sense. No matter when it Happens. That is incredible view of unlocking cognitive labor, unlocking additional opportunities for science and technology, unlocking robotics, unlocking the military edge, and more broadly just gdp.

George Mathew [00:12:14]: But it does reflect a very mechanistic view of all this. If you do these things, these outcomes will emerge. And it was pretty almost brutalistic in terms of how I thought about it. Wildly important paper though. But then Dario comes out with to your point, Demetrius, like Machines of Loving Grace. And it talks about, you know, not a completely different future, but it talks about like why these machines can actually help us as humans. Like how do we enable a better humanity because the machines are helping us get there. It, it actually reminded me of the.

George Mathew [00:12:52]: I don't know if you've came across the science fiction series called the Culture series, right? In Culture, the systems, the AI systems in particular, are just infused into how we live, right? And it's like a very post economic society that there's an infinite amount of resources and how the machines have helped us progress beyond our current constraints as humans. And what does humanity look like in that future? It's a beautiful view of what we think about the coexistence of the convergence, the singularity of humans and machines together. And both are, are definitely possible. Right. But I do find something beautiful about Daria's thinking because we have this, this tremendous angst about the future of AI and its impact on humanity. And I think there is a lot to be worried about. I have no doubt about that. Particularly if we are at a stage where we're building systems that are not aligned to human values, certainly at scale.

George Mathew [00:13:53]: But there is a possibility of reaching a future where humanity itself progresses in an exponential way for the benefit of humanity, because our AI systems are here to enable that to happen. And I think I would highly recommend anyone who hasn't read the Machines of Loving Grace to spend some time with that paper in particular, both papers, but certainly that one, if you want to believe in a more positive future for AI will transform humanity.

Demetrios [00:14:27]: And one thing I wonder about as we think about that shift, is it going to be a layer on top of legacy, or is it going to be built from the ground up? And the reason I ask is because the. I was with some friends two weeks ago and they were talking about how they tried to create a JIRA agent and it worked great in their hackathon. And then when they plugged it into their real JIRA instance, it fell flat on its face because humans in JIRA are doing the least amount possible to get across what they need to get across and another human can see it and understand it because they have all the context around it. But the agent was seeing it and it had no idea what was going on. And so I asked, would you then go and rebuild it with some kind of a graph knowledge base around it so that it could have the context? And he said something funny? He said no, I would go and I would just think about it differently. Like instead of trying to shoehorn an agent into jira, I would just think like, how can I make a better jira?

George Mathew [00:15:36]: Yeah, I think both things are going to happen. Right. I think there's going to be a number of situations where you are going to be able to take your existing systems, put that into an agentic workflow where you are combining it with some of the reasoning and knowledge that's imbued into a foundation model that might be fine tuned or you might have a retrieval, augmented generated view of just giving contextualization for how that model is going to behave with that data that's provided from that existing system. And you can get some tremendous benefits for how a compound AI system emerges there. And then to be sure, to your point, like there's going to be like a number of key new systems that are going to emerge that you're just not going to want to take that existing process. You're just going to come up with something completely new that you don't even need to wrap your, you know, complexity of the existing function towards and like just build something like radically different from what you've done before. And that, that's a, that's an interesting idea. Is there a different way of thinking about JIRA and ticketing and software development that we've been very used to, a process that could be radically reimagined.

George Mathew [00:16:58]: All right, yeah, and I think it's a fair point. There could be quite a few of those things that we're just used to doing in a certain way and we no longer have to use AI to automate what we're doing, but just completely create a new way of doing it.

Demetrios [00:17:12]: Yeah, I, for me, I feel like the, every time I see an update from the cursor folks, I think, man, they're going to eat up everything that a developer does. And so it feels like there's a world, a future where everything that a developer has to touch is in some way shape or form connected with. It doesn't necessarily have to be cursor because cursor may not perform if we see another UI UX change or shift as we saw with with copilot. Sure. Like losing the lead. And so you think about it that way. If something is aware of what you're doing and what you're trying to do, then wouldn't it be able to update tickets for you too?

George Mathew [00:18:01]: Yep, that's right. Yeah, yeah. Go further downstream in terms of what the responsibility that user, that Persona is and what they're doing versus just the task at hand. Right, yeah, yeah. This is where. Look, I've shifted my thinking on this quite a bit even this past year because I think about a year and a half ago a lot of my thought process was that most of what this AI based emergence in software is going to look like is like a bunch of, call it co pilots. Right. You have work and this thing is going to just help you do that work better.

George Mathew [00:18:35]: And that's generally the mindset that a lot of folks have had on copilots. Now I've really embrace the idea that you can see full autonomous autopilots and this full autonomous autopilots don't have to even do the work the same way you've been doing it as a human.

Demetrios [00:18:56]: Wait, click into that a little bit more because it makes me think about how when they let two chatbots loose, they created their own language.

George Mathew [00:19:04]: Yeah, that's the beauty of some of the probabilistic capabilities of the models that we can now see these emergent behaviors that we were not expecting just because of the constraints of like how we think about it as humans. But these probabilistic models can interface with each other in ways that we probably haven't even thought of. And so I do think there is this value in. We worry about confabulations and hallucinations, but I actually think there is some significant value in the systems that are emerging here, the probabilistic systems that are emerging here to have different conversations with each other than the ones that they would typically have with existing business processes and existing people that are interfacing with. So I think there is a lot to be said there. Now look, there's a lot more to go because it does feel like the. The vector of reasoning. There's certainly quite a bit of advancements that have been made with the emergence of one model.

George Mathew [00:20:14]: Certainly we can talk a lot about where Deep SEQ is, but you can sense that the reasoning vector has been really kicked off this past year. Even as the knowledge vector, the accumulation of knowledge, we're probably slowing down because there's less and less data that is pristine to consume from the public Internet and Therefore, we're not using private data sources, et cetera, to be able to train this next generation of models that are coming to market.

Demetrios [00:20:44]: Yeah, there's that vector. Do you feel like there's other vectors that we are going to pull on too? Because I look at the idea just how humans now are okay with waiting 20 minutes for an answer. And when ChatGPT first came out, that was unthinkable. We wouldn't wait around for 20 minutes. But now I use deep research all the time and I'm like, yeah, just go and do it, whatever. And so that's another vector of the patience of a human being. And then you have the actual like tech vectors that you're looking at. Like you're saying there's the reasoning or maybe there is some kind of, there's other things that you've been thinking about that we could augment.

George Mathew [00:21:27]: Keep in mind, a lot of ways the reasoning vector encompasses some of the things that you just mentioned, right? Because if you think about the notion of the chat itself, reasoning inside the chat is instantaneous, right? It's synchronous, it's immediate. You want that answer right away. But you can also think of as you advance into more complicated longer form thinking, the short and long form thinking, and so you get into some of the longer form thinking, then the reasoning can take longer, right? You can think about it and come back with an answer that doesn't have to be synchronous to the chat. It can be quite asynchronous to go to your point, do the research and come back. And we're patient about research being done as long as their thinking is higher quality at that point with a longer form response time. So I do think that it's been proven now that they're just like humans have short form and long form thinking. There's no reason why our AI systems shouldn't also have more immediate short form as well as longer form reasoning and thinking.

Demetrios [00:22:36]: It's funny to think about the idea of hallucinations as a feature and allowing those to happen in a way or almost like you, you're giving the space for models to go and get creative and they can do that over here, maybe in a little sandbox and then they can come back with some wild ideas and you have the almost like the judge that is going to say does this work or not? And then reason with that idea and might be total trash, might not work at all. But in a way, like giving it the directive to think outside the box. And almost like hallucinate on. This is a funny way of looking at it.

George Mathew [00:23:24]: Yeah. And when you think about the type of work we're either augmenting or fully embracing as an autonomous system, you can see vast quantities of creative work where you do want, you know, a highly, call it diverse output that you can iterate through and just have the models, you know, to lesser or greater degree genuinely hallucinate and come back with output that you wouldn't have thought of and leverage out quite effectively, particularly in many of these creative domains.

Demetrios [00:23:59]: Right.

George Mathew [00:23:59]: You do want that constraint to be relaxed to get more, more possible creative things out of the model that you might be working with. And then in other cases, if you're doing a trade settlement process or straight through process in a back office, that's the last thing you need. You just need a very tightly wound model that doesn't go off its guardrails. It has the human like reasoning to handle and understand, like when an exception occurs, like how to process that. But then everything else, it literally does what it needs to complete that process. And I think that's where I'm really excited honestly with where the recent developments around like something like Deep SEQ is, has really opened everyone's eyes. There's a lot of things that we could talk about with Deep SEQ which happy to do it. But one of the things that really caught my eye was like you can take a very large model and drive model distillation to a point and using some really compelling techniques to just be able to hyper optimize the efficiency of how a model is not fully trained.

George Mathew [00:25:10]: But also inference using reinforcement learning, transfer learning techniques, you can build very interesting domain specific models that really stay on the rails, do exactly what they need to go, do have high quality human reasoning involved in it, and doesn't go off and think about 14th century British history. So I think that is a tremendously valuable element of what we are going to likely see. That it's not just this idea that you need a go from a trillion parameter GPT4 style of large language model to whatever 10 or 20 trillion parameter model or maybe even bigger. There's a lot of questions about how that's going to emerge. And let's see, I'm sure the researchers are working through that, but there's something to be said that these elegant, compact domain specific models, and I said this years ago, I just never saw a manifestation of it like we see with deep sea, are going to be just as important, if not more important for how the proliferation of AI systems function in not Only the enterprise, but probably most of humanity.

Demetrios [00:26:28]: Yeah, let's keep pulling on the thread of Deep SEQ right now because it is so fascinating. It's captivated the world. And one thing that I am wondering about is your thoughts on the other model providers and what that's done to their positions.

George Mathew [00:26:48]: It's gotta raise a bunch of questions. Look, I'm not inside of any one of the model providers and just to state it out loud, Insight never invested into any of the foundation model builders, which not because we didn't love what the foundation model builders were doing. We didn't have deep faith in what was coming out of the model builders factories and research shops. We were profound believers in a lot of ways. One was just the valuations never felt right at any moment in time. And hindsight B20, there's probably a few that we should have been in right at certain valuations. Like I'm pretty sure like the $1 billion valuation round at Pat Anthropic was a good one and we had a chance to be in that one. So you think about these anti portfolios that you build over time.

George Mathew [00:27:38]: That's another one in my anti portfolio for sure. But, but we just had a very strong belief that the data was the most important substrate. And I'll talk about some of the sort of thematic investments that we've made, particularly on the idea that the data was the most important substrate for building these AI systems. And we also just knew that with how much was continuing to popularize in terms of open source models, we're seeing a hugging face like there was always a moment where the best model in the world from a knowledge and or reasoning standpoint continuously had, call it the near equivalent emerge within six to nine months afterwards. The open source community and Deep SEQ is probably one of the best examples of that, at least for the reasoning vector. Like this is equivalent to what the Olin models, if not as good or better than what the Olin models can do and done with hypothetically a lot less resources. So if you're a big model builder, you got some work cut out for you at this moment, right? Because you got to show this vastly incremental traversal of either the reasoning vector or the knowledge vector, which again the model builders are really thinking about as they're training these next foundation models, frontier models that, that they've been working on. But when you do that, you're spending hundreds of billions of dollars across the network, the GPUs, the power required, the resources required to make it happen, up to this point, every time a significant model evolution has occurred, it does seem there is a more elegant, efficient way to catch up.

George Mathew [00:29:27]: And certainly that's what should worry any of the model builders, right? Particularly in the large scale research shops that are these Frontier foundation models, like when do they actually have a true moat that is sustainable for more than six to nine months? And that, that should worry everybody. I, I heard, I heard rumor that even one of the more efficient model builders that were open sourcing like Meta and on the lava models, like I think there was a moment where Zuckerberg was probably yelling at his team about okay, how is it possible that you're asking for y amount of dollars to go build this next generation of Llama 4 and beyond models and this is out here doing it at a fraction of the cost with some very clever methodologies and distillation techniques. So yeah, everyone's gotta be thinking about this, right? Everyone who has spent hundreds of billions of dollars up to this point have to be thinking probably important, right? Important like the. I used to talk about this as you needed the space program to build the aerospace industry. And yes, we do need to put the fundamental dollars into the space program, but now we're at a point where it does feel like we should be able to build the aerospace industry. And so at least in the analogy, what I look at is like the AI research versus the industry that's forming around AI systems. So when we're doing that, should we be spending the dollars that we had to spend on the research to get there and how much more is really necessary? So there's gotta be a lot of questions going on and the big research about this exact topic.

Demetrios [00:31:14]: That's a great analogy. Bringing the aerospace industry to the 2025 basically and making sure that we're building on top of what has already been established in a way and now it's just time to take off. And so there, there is another thing that is interesting when it comes to these models being built, in my eyes that is, it's a bit of a crapshoot and you don't really know what you're going to come out with after you spend all that money. And it could be good, it could just be mediocre. And I think we've seen a lot of models that come out, they don't really make a splash because they're, it's cool. Yeah, there's another foundational model that's out or, and almost like this base model that you can then go and fine tune, but you quickly see them Fade away into history because they're not that good.

George Mathew [00:32:18]: Yeah.

Demetrios [00:32:19]: Compared to what's out there.

George Mathew [00:32:20]: Yeah. Yeah. No, I think the long tail of models littered on Hugging Face is probably as much an indicator of some incredible things clearly in the leaderboard. But also there is a. Quite a bit of a long tail there and I think it just shows that there is a. There is definitely room to continue to create these, these beyond expected outcomes like we just saw with Deep Seq. But to get there, there's a lot of failure that you need to iterate through. And sometimes that's happening inside of a specific research shop in their work, sometimes it's happening across research shops.

George Mathew [00:33:07]: And certainly it shows in the long tail of what we see on Hugging Face today.

Demetrios [00:33:11]: Yeah. The interesting thing too from my side, I guess not as interesting, but I don't know if you saw that Mistral dropped a new model a few days ago.

George Mathew [00:33:25]: Yeah.

Demetrios [00:33:25]: And for me it was like bad timing, man.

George Mathew [00:33:29]: It's like just a complete. Yeah. Tough, tough choice to have to put a new model out there this week.

Demetrios [00:33:34]: Yeah, yeah. But Mistral was that in the past. Like Mistral had their moment of they put out an open source model. It was all the rage. Everyone was talking about it and I think they got a little confused that, oh, we could do it again and we'll put it out. But it wasn't. Again, going back to the quality of the model, it's not this groundbreaking thing. It doesn't have this whole narrative around it.

Demetrios [00:33:59]: It's not capturing and captivating everyone's eyes. And in my eyes, it's like you do have the Mistrals of the world that are in a bit of a tough spot right now because of how they're trying to do things. And somebody sent me a article that said that Mistral's trying to go public. And it was like out of all of the model builders, I was not expecting Mistral to be the first one that goes public. I'm going to be honest that it wasn't on my dingo card.

George Mathew [00:34:31]: Yeah, yeah. The race is continuing across the existing model providers. And yeah, I would agree that, you know, you'd expect a little bit more scale. Right. For the few that would end up going public, you certainly would expect a bit more scale in terms of where they are in their respective journeys. We tend to see a little bit of this bit from the view of weights and biases being an investment that my first investment, in fact, that I made at Insight and being the experiment, tracking parameters, tuning version control, de facto Solution. It turns out that OpenAI, Anthropic, Cohere, Mistral, Meta, Nvidia are all customers using weights and biases for building high quality models. And because of that, you get a pretty interesting sensibility, even just in terms of the amount of users there are in these respective organizations and how much leverage they're getting from a tool like WB to converge and build their models.

George Mathew [00:35:39]: It is, it is fascinating to see that, you know, there's definitely like this, call it core of five or 10 research shops that are doing the, the work that's leading the market. But then there's these moments where everything gets disrupted or reinterpreted when something like deepsea comes along and then you have to think about what just happened. But just seems like there's still great work coming out of OpenAI and Anthropic and Cohere and Wishtrail and Meta and Nvidia for sure.

Demetrios [00:36:11]: Yeah, yeah. And you need these moments to raise the bar and keep people on their toes. And also it's, it changes the landscape. It makes things a little bit more, oh, cool, I can go use Deep Seek now. I'm going to think twice about a different model if I can host it on my own. But it's not like you just grab the model and then you're good and you can be all right with it. I think the other pieces are. We just had this thread in the community that was fascinating to me because there was someone asking about the cost differences that people have seen between using SageMaker and not using SageMaker.

Demetrios [00:36:55]: And somebody chimed in on the thread and said, yeah, we saved five grand a month from switching from SageMaker AI to just regular. I think they were using, what is it? ECS and EC2. Yeah, EC2.

George Mathew [00:37:13]: That was it.

Demetrios [00:37:14]: EC2 instances, yeah. Oh.

George Mathew [00:37:16]: So they, they didn't have a call it manage SageMaker experience. They just rolled their own tools into AWS Infra now and they saw that.

Demetrios [00:37:27]: Wow, okay, now we're saving five grand a month. But what you don't see, and they were very transparent about this in the thread, was how much is the cost of having engineers that know how to do this? What are you really trying to optimize for here? How big is your team? How mature is your team? All of that stuff comes into play and it's that classic build versus buy decision. And so you can't just think, all right, I'm going to just grab Deep Seek now and I'm good. I don't have to worry about anything. Because as another friend told Me, he said, whenever possible. At my job as a ML platform engineer, I am trying to outsource the ML platform to one of the big model providers because I don't want to have to deal with that.

George Mathew [00:38:12]: It's such a headache. Yeah, no, it certainly makes sense. Right, because if you're one of those ML teams, your job is to introduce the highest quality AI system that incorporates that model, that brings in the data that's necessary to train a domain specific model for your needs and get that into a production workload that is actually benefiting that organization. And yeah, the more you can get platforms and tools that help you support that, it just makes sense. If anything, that was, for lack of a better expression, the picks and shovels thesis that we had at Insight for everything in the space. We just definitely leaned in on most of the picks and shovels in this area.

Demetrios [00:39:06]: Yeah. And he's even in a regulated space and he was saying unless you have data issues and regulations, just outsource it, picks and shovels it and get rid of it. And then you can spend your time on higher priority issues. And so it's fascinating to think about that. The, the other.

George Mathew [00:39:25]: Yeah. By the way, I've come across certain enterprises where I've seen, oh yeah, we built in with a lot of pride. Like we've built very custom bespoke ML ops pipelines. And I was like, oh you're. You poor thing. Like why would you put all that energy into that when you could? And some of it was just decisions were made a certain time. There's like a belief that you couldn't take everything from what's available in the market and coerce it and harness it together to make it make sense. But I gotta imagine like that's not.

George Mathew [00:40:00]: Gotta be a great use of resources, particularly in an organization.

Demetrios [00:40:05]: It's. And you're always having the conversations, every quarter, I imagine, or every year you're having a conversation like, do we continue to support this or do we try and migrate to something newer? Because in a way when you choose your stack, it's a snapshot in time of what the best capabilities are at that moment in time. And if you're really going all in on it. I remember we had a guy from Pinterest on here and a lot of the stuff that they are using for their ads platform was chosen in 2018 or 2017 and there's been so much advancement and they have to decide, do we manually. Maybe you take a piece off here and then upgrade it here. But maybe it doesn't Fit. And so you can't add that new shiny thing which can be good, it can be bad. You don't always necessarily need the new shiny things, but, but really thinking about like how if you decide to go down that route, you're deciding to take what is out there best right now and have that snapshot be what you're.

George Mathew [00:41:24]: Using and then forego all the future innovation that could come a little bit.

Demetrios [00:41:29]: Huh?

George Mathew [00:41:29]: That's the things that are outside your four walls. Yeah, that's the thing. You gotta be careful about these build versus buy decisions. And this is not just in your as and put in just if you're going to lock down something now, you're going to get likely something that is very highly specific to your needs. But then eventually there's just going to be this external innovation that's going to just keep moving ahead. One of the things that I've been fascinating to see by the way, since we did talk about building buy, is also this wave in AI based software systems, or AI systems that are empowering and enabling this next generator software to be more precise, is how much more you can now do custom software development better. And the reason I just mentioned this is that before you had to almost rely on your package software provider to do the things that you needed them to do in the product and you're waiting for those releases to happen and then you were implementing that package software. But we're not a world where you can build taking some of the componentry that we're talking about and the componentry, the data, the pipelines, the building blocks can continue to evolve.

George Mathew [00:42:47]: But then how you harness that can be very much highly customized software experiences that are for all intents and purposes custom software with a number of really upgradable underpinnings like the lifecycle management on it becomes much more straightforward. So I do think there is a moment here that you can just deliver highly specified custom software. But the advancement of all the underlying componentry to do it continues to have its own upgradability lifecycle. That's not where you have to build bespoke stuff. It's actually in the layer, the abstraction layer above where you can continue to iterate that faster because you have these building blocks that you're working from and it's. It is really fascinating to see how much you can get done today from a custom software development than ever before.

Demetrios [00:43:38]: Is this because you are thinking about it as I can prompt my software or my software, I'm not interacting it always or I'm not interacting with it always through a gui. It's also just through language.

George Mathew [00:43:56]: There's that just the way that the UI falls out in this experience. But it's also just a when you have to now take your existing software packages and almost bolt on the AI capabilities versus like natively building it up from the bottom up and then natively having a very enterprise specific functional experience with your software that you're not waiting for someone to develop a feature that you need versus you're capable of building those higher level features yourself. I think you're getting much more specific curated software built for your business. Right. And that, that. And by the way, like there's moments you don't want to do that. Right. When everything's settled and there isn't a lot of innovation that's going on and you can just take something off the shelf and it's as good or better than anything else that you can build.

George Mathew [00:44:51]: But we're at a moment where you can build stuff better than you can take anything off the shelf. And so that's why I think there's a really compelling moment for custom software development in the middle of all this.

Demetrios [00:45:03]: Yeah. And it is looking at that abstraction layer and saying that we're going to benefit from things as it continuously moves underneath us, but we can have what is needed above for the users. And yeah, it is interesting to think about because it does feel like it is more Lego blocks now. And since it is so new and there isn't really much settled and every day you get something new, you are more inclined to build quicker or build on your own. And I imagine I'm just thinking back to the Pinterest story and how I bet in those days they went out and they were like, look, there's not really anything out there that's good enough. We can create the abstraction layer we need and then we'll have our custom stuff underneath that is getting us to where we want to go. But inevitably, like you said, it changes. There's that right now we're in that phase of it's easier to build for this type of stuff and create our own custom software.

Demetrios [00:46:23]: And then you probably will get over the hump and you'll start to realize, all right now there's offerings that are pretty nifty and we're going to want to go with those instead.

George Mathew [00:46:33]: Yep. And there's another nuance to add in. Since you mentioned Pinterest, when you look at some of the more compelling open source that has actually emerged this past decade, it actually started as custom functional work that was needed inside a civic organization like oh, other people might actually need. Airflow came from Airbnb.

Demetrios [00:46:55]: Yeah.

George Mathew [00:46:55]: Right. Kafka came from LinkedIn. And so you look at these like now modern OSS components, they actually emerge from a very domain specific need that happened to be in a very specific domain and then generalized into the rest of the world by just releasing it as open source. So that there is some interesting benefit to even building these things in a way that you're getting what you need out of it through that customer experience. And then when you realize there are some things that other people will also need that are reusable and there's a benefit to then open sourcing it. And we certainly saw that with a number of open source capabilities, particularly this last decade.

Demetrios [00:47:39]: Yeah. What a great example. Speaking of open source, there's a little open source project called Spark.

George Mathew [00:47:47]: Oh that one. Is that still around? Is that still doing anything out there?

Demetrios [00:47:51]: People are trying to kill it. Yeah. But it's still around. Yeah.

George Mathew [00:47:56]: I got a fun story about that by the way.

Demetrios [00:47:58]: About Spark.

George Mathew [00:47:59]: Yeah. I was at Spark Summit 2015 and my team at Alteryx, including myself, worked on shipping Spark R. And our co developers on it was a very clever VP of engineering at databricks named Ali Godsey and his team.

Demetrios [00:48:24]: No way.

George Mathew [00:48:25]: And yeah, that's how we got Spark R into the market. No way. Yeah, that was.

Demetrios [00:48:30]: So you were working with him back in the day and now you're working with him again because I saw recently you posted you were part of their massive Series J, I think.

George Mathew [00:48:40]: Series J, Yeah. One would not have imagined that databricks back then would have arrived at its Series J, but here we are a decade later. Yeah.

Demetrios [00:48:51]: How is it even possible that a Series J exists? That's one thing. And then why now? What is it about you? And you've obviously been in the databricks Spark ecosystem for a decade plus.

George Mathew [00:49:11]: Yeah.

Demetrios [00:49:13]: Why all of a sudden you do you want to become a shareholder?

George Mathew [00:49:18]: Yeah. Sometimes it is okay to be late to a party, right?

Demetrios [00:49:20]: Yeah.

George Mathew [00:49:21]: But in this case it's a little bit more methodical than just being late to the party. We were investors about two rounds ago with a much smaller position and we were continuously excited by how much the evolution from Spark to databricks to really what databricks stands for today as a unified data management platform using the lake house as the construct for all the pre processing, all the data prep at petabyte scale and then this little capability that they've introduced called data warehousing inside of databricks, which is now the data warehouse product, is a $600 million business for Databricks. Which is incredible. Right. When you think about it, as you sum all of these things up, the Mosaic acquisition, the in house built capability around data warehousing, the Unity catalog itself becomes the unifying layer for the enterprise. The tabular acquisition and now owning the iceberg format. And that's just like the product strategy. We're not even talking about how much the go to market and industry strategies have really evolved from even its early inception.

George Mathew [00:50:43]: It just feels is a generational company. I don't think I'm saying anything out of turn by saying that out loud. And it does feel like there's many more miles to go. And so that was our thought process there. The opportunity came up to be part of a very large 10 billion Series J, which is I think the largest venture capital deal ever. Our friends at Thrive and Josh Kirschner led it, we co led it with a pretty sizable check that, you know, was really helping databricks continue its journey. And for them it was just making sure that there was tremendous amount of just liquidity for everyone who had been on this journey for a long period of time to consider their needs as they continue to grow and scale the company all the way to the next levels and stages of growth. Yeah, we couldn't have been more excited.

George Mathew [00:51:41]: And it certainly was a, was a big investment for Insight. It was a very large round for everyone that was involved. And it does appear like there's many more models for databricks to go in the market and we're just delighted to be a part of that journey. And I guess one would say it's never too late to come to a company like that and just help them along their journey.

Demetrios [00:52:07]: Yeah, you say like generational and it really does feel like that. And I instantly thought, do you feel like there are any other generational companies in this space as you look around in the data AI space? Because for me it's like databricks is just performing top notch right now. But I don't know, maybe you're looking at other companies and thinking yeah, they're generational companies too. And. Or maybe it's if some things go right, there's going to be another one of those.

George Mathew [00:52:53]: Yeah, look there, there should be by definition the next generation of companies that emerge. And so they're coming and I just want to be clear about that. But it does feel like there's something special this moment. Certainly with databricks, you know, we shouldn't not mention what Snowflake has been able to accomplish as a company in that same cohort and period. If you look at Palantir as another great example of a company that has in a very different way built something generational in nature, yeah, it is exciting to see those folks who started 10, 15 years ago build the companies that are now the sort of behemoths, the foundations of how at least the data AIML space continues to progress. I think there is a clear view of what's next. Right. Without stretching too much, you can certainly say that the foundation model builders and the growth and scale that they've respectively had have been quite impressive.

George Mathew [00:53:53]: The enablers of that growth and scale, like a company like Scale AI for instance, is pretty impressive as well. And just being able to initially provide a lot of the necessary efforts to be able to enrich data that's necessary to build a model and now providing things like RLHF for the model providers themselves as they're coming to market. So it does feel like there is something surrounding, you know, these current model builders and the capabilities underlying the model builders that have gotten pretty big. It is a little trickier, right? Because when you look at something like databricks, it hugely is benefited by Deep seq. Right? And the reason I mentioned that is if you think about what Deep SEQ is a manifestation of is, and I think everyone's been talking about this past week, it's like this idea of Jevons paradox, right? As the cost of a resource comes down, you consume that resource more and it takes up all the capacity that's available to you at that moment. So I think that should be the case for everything related to the GPUs, the compute infrastructure that we're talking about over time. But we have a moment here where no matter what happens, you just need high quality data to be able to build this next generation of models, whether they hyper efficient domain specific ones that you can see emerge from like the likes of Deep SEQ and its cousins that will literally come out within the next weeks and months, as well as some of the largest foundation models in the world. So we feel pretty good that we're sitting in a good place with the databricks investment.

George Mathew [00:55:36]: It feels like it's going to be a bit of a roller coaster for a little while for the foundation models themselves. And look, the next generation of companies are coming about and getting to scale. I mentioned the weights and biases example, right? There's 900,000 machine learning practitioners that are now standardized on Weights and biases. There's a lot more miles to go for a company like weights, biases now. So I do think it comes in these waves. Right. And so the wave of like full scale maturity is out there and we see them as public and private companies now. This next wave of particularly around the foundation models and the support infrastructure around those foundation models have come to fruition as we speak.

George Mathew [00:56:21]: And then you're now starting to see this next, you know, much smaller companies, incredible founders starting to build that will get to the next level of scale in the 5, 10 year timeframe.

Demetrios [00:56:35]: Yeah, that is one thing that you have a nice position of being in is the almost like round agnostic ability that I know you don't do super early pre seed, but you still see really early companies and you get to see and participate in the later companies too.

George Mathew [00:56:54]: We've been very specific and insight about this, so we've even said this out loud. We don't think that stage is a strategy. Right. So we don't believe that this idea as investor stage is a strategy. We should be able to find the generational growth opportunities at whatever stage they might exist. And we tend to be a little bit more focused on that middle stage of growth and certainly have done quite a bit of work, particularly in our teams that do a lot of the scale network in terms of buyouts and some of the larger checks that we've written. But that's not a strategy. We look for growth everywhere.

George Mathew [00:57:32]: And one of the wonderful things I had a chance to do working with my partners for the last four years is really build out our early stage Right where we are now very capable at not only the series age check, but occasionally even the seed checks. Right. Because we see these opportunities and we see that profound growth happening with a founder that just really has a maniacal view of what they want to deliver and build in the future and thematically want to be part of those conversations. So that certainly has been part of our strategy to work with founders whenever and wherever that growth is existing independent of sage.

Demetrios [00:58:13]: How are you thinking about when it comes to the way that we are interacting with computers right now? We have very compartmentalized applications, but it almost feels like some of the stuff that you're insinuating at. It would be better to just have the one AI application that can know and be with us. Right. And you mentioned it about the Sci Fi series. Culture, how it's called Culture.

George Mathew [00:58:45]: Yeah, the culture series. Yeah.

Demetrios [00:58:48]: And in order to enable something like that, we need this. What I think Microsoft is Trying to do it right now with recall where it's like this always on AI that can come and it can help you and be there with you when you need it and then just be in the background when you don't. And for me, the way that I'm trying to wrap my head around it is yeah, it's much better if you have all of the context to give to the language model and then it can reason about it and it can understand. When we go back to that JIRA example, if it doesn't have the context, then it gets lost. And so in striving to get to that point where you want to give all of the context, I think what you end up doing is just saying come with me everywhere, be with, be like my shadow in a way. And so have you thought about that and how that could potentially play out in the future?

George Mathew [00:59:49]: I have a little bit, yeah, I have. And by the way, interestingly, that description of a almost ever present, how do I want to say it? Like just like completely in the moment available AI system that's doing things that you need it to do, it doesn't have to be single and monolithic either when you think about it, right, there could be a view that it's like just hundreds of agents that are just continuing to marshal out to do many things and then converging to your needs. So I don't know if I necessarily see this sort of singular thing that just is always constantly doing the bidding of what you want it or other folks would want it to do. I actually see a lot more, at least in the near term, this emergence of massive amounts of personal enterprise and almost hybrid agents that are continuously doing little things, medium sized things, big things on our behalf. While we're talking here, wouldn't it have been nice for an agent to go figure out how to book that trip that I need to make to the Middle east and give me all the optimum answers to do that and book the ticket. That's the kind of things that we're starting to see with these automations that are embedded into the foundation models, but also just in generalized sort of agentic systems that are coming about. So I do think that we're going to end up in a world, at least in the near future, that it's about these really well meaning agents that are doing smaller things, tasks that are rote, repeatable, that should not have to be done by any other human on our behalf. And I think that should be how at least the near term future is.

George Mathew [01:02:00]: And then we'll see where the more science fictiony future of a complete system that's always with you, that's just part of your being. It seems like it's a little further out, but certainly that is a possibility that could emerge when we're all said and done.

Demetrios [01:02:18]: Yeah. And I've thought about it as the what you're talking about there is like where is it going to live? Is it going to be in on the web or is it going to be something that you're starting to see more is these ways that the products are being created is directly in Slack, like Devin for example. And then you just deal with these agents in Slack and I think it would be pretty funny if all of a sudden Slack becomes your control center too and you just send out agents from Slack to go do different things that you needed to do.

George Mathew [01:02:55]: Yeah, it's actually I was searching for the word because it was escaping me, but it's the nature of how AI systems are going to be ambient or not. Right. I think a lot about that. Like the ambience of our AI systems of now and the future and how could they be so embedded, so ambient. So just an extension of how our minds work that like it just like we think about it and things get done that, that does feel like not a science fictiony future. I think actually that we're not that far away from getting to more of these sort of ambient AI systems that are doing important work at home and at in the Enterprise and everywhere in between.

Demetrios [01:03:41]: Yeah. Because how many times I don't know if it's happened to you, you might have more willpower and focus than me, but I have something in my mind that needs to get done. And then I pick up my phone with the intention to do that thing in the moment and some kind of a notification distracts me of course. And then 30 minutes later I'm like oh, what was I? What did I pick up my phone for? I can't even remember anymore. And that task that I wanted to do is in the ether and maybe it comes back to me, maybe it doesn't.

George Mathew [01:04:16]: But what if you had an ambient agent that the moment that task came up fires off and goes and does that task in a highly reasoning long form way. 20, 30 minutes later or even longer if needed.

Demetrios [01:04:28]: Yeah, I'm just excited for a Siri that actually works and then I can give like tasks to because right now the Siri that I use, it's very hit or miss. And so you do see the world where. Yeah. Typing into Slack chat is great, but I also have a million notifications on Slack, so it would just be cool if I could talk to the phone directly.

George Mathew [01:04:56]: Or talk to you. It does seem like a more obvious opportunity for Apple intelligence, right? Just to be that conversational layer, but doesn't seem like it's a merch up. No, we'll see.

Demetrios [01:05:09]: Something's going on there. I don't know what it is, but something, yeah.