Bridging the Gap Between AI and Business Data

Deepti Srivastava [00:00:00]: What if you could put a box right in between all of your data systems on one side, all of the data like databases, data warehouses, whatever data systems and your LLM systems on the other side and just have straight lines. Oh, you need data from Salesforce, we'll go get it for you. You need data from Postgres. Oh, we'll just go get it from you. You need data from Snowflake or Databricks, we'll just go get it from you for you. My name is Deepti Srivastav. I CEO, founder of Snow Leopard AI, which is a intelligent data retrieval platform that we're building. And I take my coffee with milk and sugar and you can come at me for that.

Demetrios [00:00:49]: You just said to me that you, back in the pandemic days went to some virtual online meetup that we did and it was on ML Flow. And I think I know exactly the one that you are talking about because it ended up being one of the most watched episodes later when we put it on YouTube. And I really think it's because it hit a nerve and the episode was the difference between kubeflow and ML Flow.

Deepti Srivastava [00:01:18]: Oh, interesting. Yes, maybe.

Demetrios [00:01:21]: And it's super confusing, especially back in Those days in 2020, folks didn't really understand either one or how when to use one and when to use the other. And Byron came on and he explained it like, look, kubeflow is a sledgehammer and mlflow is like a little pickaxe. So you, they're both great tools. It just depends on what you're trying to get done. And so I, I thought that was, that was awesome. And, and it's really cool to hear that all the way back in those days you had seen the ML Ops.

Deepti Srivastava [00:01:57]: Community when I wasn't sort of when I wasn't as excited or when I wasn't in it. Actually, I was excited about the community, but I wasn't even in it at the time. And I was like, oh, this is interesting. At some point I should look it up and lo and behold, like, what is it? Five years later, four and a half. Here we are. So that's very exciting.

Demetrios [00:02:16]: Full circle. And so the, the thing that I think is, is really cool, as you were saying. Yeah. Back in those days even we were talking about this problem that you've been hearing and talking with folks about a lot. Can you just break down what the problem is and then we can get into different thesises around it?

Deepti Srivastava [00:02:40]: Yeah. So there are many ways to talk about the problem, but the crux of it is in this new world of LLMs, like we expect AI to like be this trillion dollar like new industry, new platform shift. By the way, I bought into it, like this is why I'm here with Snow Leopard, right? Like I bought into it whatever 22ish. When the, when chatgpt really broke. Yeah. When the hype happened I was like, because I'm not into hype, I'm a, so, so backing up. I'm a distributed systems person. I've been in infrastructure my entire life.

Deepti Srivastava [00:03:16]: We are anti hype. Just, you know, we are like, we are the opposite of early adopters. Let's just put it. Because we care about production and five nines of availability and all that stuff and you can't really just be like, oh yeah, let's try the new thing. And like, yeah, so, so I come from that. And then I think that's important because as I said, I don't believe in hype. But when I saw it and played with it, I was just like, no, this is truly a platform shift. I also did a minor in AI back in the day and so in undergrad and I was like, this is hype.

Demetrios [00:03:48]: Yeah.

Deepti Srivastava [00:03:49]: So the issue is, I mean this is truly a platform shift. LLMs enable so many things that we didn't even think was possible. But and this is the crux of what I believe the problem is, right? Like for LLMs to be the platform shift that they are for gen AI, for AI applications to truly change the way people live, work, behave, you need to connect them to the crown jewels of any business or any sort of user situation. Right. And that is operational data. And by operational data I mean structured data. So I mean SQL, NoSQL, API based information, Right. Today there is a huge chasm still between operational data on the one side and LLM and LLM based apps on the other side.

Deepti Srivastava [00:04:50]: And that's precisely what Snow Leopard and I set out to do when I started Snow Leopard.

Demetrios [00:04:58]: So it's a. I 100% agree with this idea of for AI to live up to the hype.

Deepti Srivastava [00:05:06]: Yes.

Demetrios [00:05:07]: We're going to need to plug into all of this data and I don't think that's necessarily the most spicy take. Yes, everyone is saying that and they're almost saying it like incessantly on all of these different social networks we gotta have. You want to have your own data and you want to have your own models. That's kind of the theory that we see. And on the other hand though, what I do think is, is fascinating here with what you're saying is we need to be able to connect it to data that right now we are not necessarily connecting it to unless it is. We've seen those use cases where it's like agents that are text to SQL agents or you have your data analyst and so you're using LLMs to query your database, that kind of thing. But I feel like what we are seeing more often than not is a whole new building of gen AI apps that are off to the side of your structured data. So it's taking, it's connecting the gen AI and the LLMs to your notion or to your docs or trying to make Jira work with it and it's not really thinking about connecting it to the databases or the APIs that you're talking about.

Deepti Srivastava [00:06:30]: Yes, that's right. So you're right that everybody talks about your data should be ingested, right. And like we should connect it, et cetera. But I think where exactly to your point where the focus has been has been like chatting with your PDF, right? Yeah, and that's cool. It is something that was, you know, wasn't easily possible before this but. But I think you mentioned something very important which is that building apps to the side of your stack of your critical path is never going to really generate those new use cases, user value, business value, et cetera. And so you have to make it part of your critical path in order for it really to your point, live up to the hype. So today everything is around the fringes, right? One of the things that you know, when I started talking to.

Deepti Srivastava [00:07:22]: So I validated the problem before I even started Snow leopard it. I know sort of counterintuitive, but just that's just how I am. But when I started talking to people like VPs of platform or like head of AI at like all these companies, enterprise companies, SMBs, you know, people in my network, all of them were like, you know, we don't have a way, good way of, of putting it in our sort of true stack. Right. And there's a lot of talk to your point about how there's a new stack emerging like the anytime there's a new stack like that's outside of what the existing stack is, to be honest. Right. And in my experience of 20 years of doing this, building data infrastructure and systems for enterprises, right, Production grade data systems, you have to like everything exists like the tech stack exists in an ecosystem, it exists in a context. And if you want to derive value or create new value, whether it's for users or for business, right.

Deepti Srivastava [00:08:26]: You actually have to be part of that unique ecosystem. You have to fit into it. You can't just be like this cool new thing and adopt us here, but like the rest of your whole world is here. Yeah, I think that's really the key. Right. And we can get into this. But I do believe just one more thing here, that while people say that you should connect your LLMs and your agents and your assistants to this data, I just fundamentally believe that the way that people are going about it isn't going to solve the problem.

Demetrios [00:09:02]: It's funny because I instantly think of this visual of I live in Germany and I was driving down the street the other day and what do you know, there's a caravan of motorcycles with sidecars on them. And it feels like what we're doing with Gen AI is the sidecar to the motorcycle, which is the production system.

Deepti Srivastava [00:09:27]: Yeah. Yes. And you know, I. Let's. Let's get into the spicy takes. I have quite a few, but there we go. But I think the way people do it today is, you know, rag, which is retrieval augmented generation, which the concept sounds good, Right. There is the other thing which you're saying which, you know, you train your own LLMs and you do all this stuff, right? Both those options actually don't get you to where you need to be.

Deepti Srivastava [00:10:01]: Right? So like, spicy take number one is that, you know, doing ETL and putting all your data into a lake house, ocean, like, whatever. Yeah, right. Isn't actually solving the problem right? Spicy take number two is. And this is the more important spicy take, which is that, you know, we're talking about intelligence will solve this, agents will become self. Self aware, et cetera, and like, we'll go out and do stuff and that's cool. I'm not against that. Right. But the problem is the most intelligent machines today, human beings don't make the right decision regardless of how smart they are, regardless of how cool their reasoning is.

Deepti Srivastava [00:10:49]: Right. Like how good they are at reasoning at like all of that stuff. The intelligence today doesn't know how to make good decisions with bad data. Bad data is equal to stale data also, by the way. Yeah, right. Like if I ask you, if you've been in a coma for six months and I ask you, right, who is the President of the United States or who is the Prime Minister of another country? Until I give you that information, until you Google it, perplexity at OpenAI, at whatever, you don't know that answer. You cannot make the decision. Right? So if.

Deepti Srivastava [00:11:23]: If intelligence today, best intelligence today can't make that right. Can't make right decisions without the right data at the right time. Those words are important. Right data, right place, right time, right. Until you do that, like there is no way that artificial intelligence is going to be able to make the right decisions. Right. Like we just have to. Reasoning will not solve it all.

Deepti Srivastava [00:11:46]: Is, is the point I'm trying to make. I know this is a, this is a counterculture take, especially for this audience, but, but I'm really honestly trying to help. I'm honestly trying to make that intelligence more intelligent. Right. Like more useful.

Demetrios [00:12:01]: Yeah. And I, I don't necessarily think that it is so counter intuitive to say that reasoning is not going to do it. Because the thing that we've seen time and time again is the more context you provide to a model, the higher likelihood that you're going to get to either the task being done or the answer that you're looking for. And so again, if you're not able to provide it the correct data in the form of the context, even if you are providing it a lot of data, but it's not the right stuff, then you're going to, it's going to be a shitty answer or outcome in that regard.

Deepti Srivastava [00:12:45]: We talk a lot about hallucination and yes, there are inherent lesions in the way the alarms are built that can cause hallucination. I agree with that. But honestly, a lot of the hallucination is because you're not getting the right data. It's just trying to answer like human beings hallucinate. If, if we get the wrong information at the wrong time, we will hallucinate. Right. And the other thing I will say here is you were saying, right, they're giving you like, the more context you give it, there is an inflection point on which if you give it too much data, it will also come to the wrong conclusion. Right.

Deepti Srivastava [00:13:18]: Because it doesn't know what to focus on, which is very intuitive as human beings. Right. Like if you throw a bunch of data at me and I, or information at me and I don't know where to focus, I mean, I may pick up the wrong thing. Right. And start going down the wrong rabbit hole. So again, these are like very sort of to your point. They're not sexy things, but they are very important in my opinion, again, as a boring infrastructure person, to enable all the cool stuff to be built. Right.

Deepti Srivastava [00:13:45]: Like we're, we're still nibbling at the edges and until we like really like bring the right data from the right place at the right time to these systems, whether they're assistants, agents, the next big thing, right? In reasoning and AI like, we're not going to, we're not really. We, we're only scratching the surface here.

Demetrios [00:14:05]: Yeah, it's true. We had Donay on here probably three months ago now, and she was talking about how difficult it was to get an agent to say no or to say that it did not have the right information that it needed or that it did not understand the task that you are asking for. And it plays right into what you're talking about with hallucinating. But because you ask it to do something and then if it doesn't fully understand or there's a fuzziness to the terms that you're using, then it comes back with one interpretation that may or may not be what you are trying to get. And then you're like this just hallucinated the shit out of the answer. And so you have to figure out a how to troubleshoot that. And what they did is they said, we are going to create a glossary of all of the terms that we're using and we need to make sure that there's no vagueness in any of these terms. The definitions of these words and these key terms that we are creating, they can't be fuzzy, it can't be vague at all.

Demetrios [00:15:19]: Which when a human reads it, they think, oh, that's fine. And then when you go and you try and evaluate the output on it, you see, oh, yeah, I guess it didn't really understand what I meant by create a good summary. It's like, what is good?

Deepti Srivastava [00:15:34]: That's right.

Demetrios [00:15:34]: Subjective.

Deepti Srivastava [00:15:35]: Right. And I like, I think there's two things here that I, I believe are key. Right. One is like, imagine, like just think about how much work they had to do and it still wasn't producing the right answers. Right? Because they are inherently sort of. So here's the other reason that I think what we're doing is interesting to me, because LLMs are actually really good at the sort of fuzzy interpretation of stuff, right? They're actually really good at summarization, classification, those kinds of things that inherently require a little bit of extrapolation, if you will. But they're not good at point, lookup point, solutions, specific. Yes, no answers in all cases right now, yes, you can, you can tune them, you can do reinforcement learning, and then they'll get closer and closer and closer to what you wanted to do.

Deepti Srivastava [00:16:28]: Right? But. But not everybody has the time, money, expertise to do that, first of all. And secondly, they're actually really good if you just give Them again, I'm going to keep going back to this, the right information for them to make the decision, right? And this is where I think, I think honestly LLMs get a bad rap in some way because ultimately everybody's using rag, right, to do it. And the way RAG is done today is ETL a bunch of data into a vector store, right? And then use the vector store to build context at query time. But vector stores themselves are fuzzy matching systems. They are not point lookup systems, they are not precise answer systems, right? They're meant to do what they're doing which is be fuzzy matching, right? Like whether you spell Demetrius with an, with an S or a Z, they're going to come up with the same, like they may come up with both answers because they, you know, to them they're like, oh, these are similar. You wanted similarity search like here, these two are similar. Now you figure it out, right? And so like we're taking first of all, all the, all the information that truly can help make the this happen is you know, the crown jewels, as I keep saying, are in your databases, data warehouses in your like Salesforce, HubSpot, those kinds of API based systems, right? Which if you extract them, you change their nature, you lose business context and you put it in this vector store, right? So now they've actually lost all the structure, the information that was very carefully designed to be in those systems.

Deepti Srivastava [00:18:05]: Like you've completely lost it, right? It's in some blob, in some essential fuzzy matching system, right? And then you pick it up from there. So not only is it stale, not only does it suffer from all the ETL and reverse ETL problems, it also suffers from you took it out from a context in which it exists, you put it in a different context with a bunch of other data, right? And then you serve it to the LLM and then you're like, okay, the definition of good aside, it won't just know whether to how and how to answer the question, right? So I think there is something to be said for fine tuning around. Like give me a yes or no answer. But you know the way the models are going, you, if you just tell them not to. Like, like if you don't know, just tell me, don't know. Like they will tell you that today like the, the models are getting smarter in this regard, right? So, so then what you need to do is no matter, again, no matter how smart they get, like without them knowing what you're asking about, they can't give you the answer.

Demetrios [00:19:04]: And I, I do like this almost. I look at it as compression loss. When you're taking it, you're ETL ing it and doing all of this stuff. And, and I also. It makes me think about something that I've been pondering for a while, which is you don't hear as much about RAG these days because agents are all the rage.

Deepti Srivastava [00:19:25]: Yes.

Demetrios [00:19:26]: Yeah.

Deepti Srivastava [00:19:26]: It's the next wave of. Of hype.

Demetrios [00:19:28]: Of hype, exactly. And. And you just connect them to your MCP server and you're good.

Deepti Srivastava [00:19:33]: Let's talk about that.

Demetrios [00:19:34]: We're all happy about the outcome, then it changes the world and the way we work. But the thing that I am constantly going back to when it comes to RAG or when it comes to how we are doing things, and we have been doing things with LLMs to try and create products that are valuable products with them, and new products, inherently new products. It's not like we're grabbing an LLM and sticking it into a fraud detection system that we've traditionally used ML for.

Deepti Srivastava [00:20:06]: Yep.

Demetrios [00:20:07]: And with rag, I always kind of laughed because I would think, are we just over engineering the shit out of this?

Deepti Srivastava [00:20:17]: Are we just.

Demetrios [00:20:18]: Straight up, we're going, and we're creating all of these different hacks. And so in the beginning it's naive rag, and then it goes to advanced rag, and then we're like, no, we gotta do graph rag. Because at each stage it feels to me like we get to the stage we think this is going to help us create a better system and have more reliability, have more consistency in the output. And then at some point you see the outcome and you recognize. I mean, it's good. Yeah, it's good. But is it what we really need? Is it. Well, maybe let's try the new technique.

Demetrios [00:20:55]: Maybe there's a better technique out of there and. Or maybe we can engineer this a little bit more to try to get better results. And it's, it's inherent because of the stage that the models were at. I do think that with the new reasoning models, potentially we're going to see these huge leaps due to that. But that always, it always makes me laugh, like, oh, it feels like we're having to do a lot of bending over backwards to make this actually work. And I also would laugh because all of that work that we're doing, all that bending over backwards to create some chatbot that can tell you about your company's HR policy. HR policy is not worth it. And that is very clear.

Deepti Srivastava [00:21:52]: I am so happy that you brought this up because. Happy. But just like it just resonates with me. That's, that's sort of what I mean. Because yeah, I, I have been in that world for so long. Forget AI, right? Like just this world of like, oh, we need this data now. So now we're going to create this complex pipeline to go from point A to point B. But oh, we forgot about this N plus one silo, right? And now we have to like create a whole new sort of pipeline, right? And like a whole new way to do etl, et cetera.

Deepti Srivastava [00:22:27]: And that takes months, first of all, if not right, many months. It takes a few months for sure. And then, oh, by the way, we changed the dashboard or the use case or now in the AI world, the question a little bit, which requires a mobilization of entire teams to like rejig the whole thing. Like you're right. I'm so sad to see these like complicated wires, pipes going everywhere. Like if you look at a data architecture, like a real data architecture diagram, not one put forth like to present some kind of decision maker. Like it's like this, it's squiggly lines. I know there's a podcast, you can't see me like gesticulating like crazy, but it's squiggly lines.

Deepti Srivastava [00:23:11]: And no, sometimes you can't even tell why the line exists, where it goes to, what's going on, right? It's just so over engineered, which honestly my heart goes out to the developers that are like trying to maintain these systems and build them and like enhance them. Right? Because you need an answer to a new report instantaneously. And the way these systems are crafted precisely for the, for many of the reasons that you called out, right? Like the, the models were not as good, the data systems were not as good, they're not as scalable, they're not as like sort of any to any connections. So yeah, what if we just took an eraser? That's literally what I tried to do when I was initially coming up with the architecture for Snorro is like erase, just erase. What if you could put a box right in between all of your data systems on one side, all of the data like databases, data warehouses, whatever data systems and your LLM systems on the other side and just have straight lines. Oh, you need data from Salesforce, we'll go get it for you. You need data from Postgres. Oh, we'll just go get it from you.

Deepti Srivastava [00:24:12]: You need data from Snowflake or Databricks, we'll just go get it from you for you. Right? Like no need for this craziness because like in my experience again having seen, you know, Google, Oracle, you know, even at observable, like just looking at how people are building applications on top of these stacks, they spend developers like data analysts, business analysts, all, all of that. The whole data engineers are spending upwards of 70 to 80% of their time just doing this part. Which means you're not spending as much time building those new value creation applications, new business creation, new like you know, user value opportunities because you're spending all your time doing this. So what if you just didn't have to do that and you could spend time like coming up with the new creative ways of doing stuff.

Demetrios [00:25:06]: I was talking to a data engineering friend of mine a few weeks ago and he mentioned the predicament that he is in right now, which is for the last three years their company made this big push to self serve analytics. And so that was incredible, you know, the whole digital transformation thing. And he's a data engineer and so he said, so now I've come into this company and We've got over 11,000 data assets that the engineering team has to keep and or has to service. So he's saying we're kind of rethinking this whole self serve thing and wondering what can we blow up or what can we take the eraser to and what actually is valuable or it crosses a certain threshold of value and is still being used because a lot of these are built by someone, that person leaves. Then there's no knowledge sharing on how those dashboards are actually created. It breaks one day and boom, that person's out of there. And you know, you don't really get it again. And why would you if it's self serve? So it's almost like, well I didn't really even like that dashboard that much.

Demetrios [00:26:25]: So I'm going to create a new one and I have a better way of doing it. Right, of course. And so you get into those scenarios. I do wonder though, in this world that you're mapping out where you have the abstraction on top of all the data sources, what would keep it from also being in that sprawl?

Deepti Srivastava [00:26:49]: Because the developers are not building that sprawl. Right. Like I think there's a couple of things here. One is we're not saying it's magic, but we're saying instead of each developer and data engineer team, et cetera, doing this over and over again, like we just um, we do it in a generic way for them so that they're not having to engineer pipelines at all or maintain them or like build them at all. So the what we are saying actually. So this is again, I think a spicy take maybe. I just think it's the, like you have to imagine the world differently and then see if you can get there in a way, right? Because a lot of this sprawling situation comes from frankly the data world I've been in. And like, you know, data grew faster than the systems that could support it, right? So then there was all this like, put it in a lake, then put it in an ocean, put it in a house, right? All those things happened.

Deepti Srivastava [00:27:41]: And to be honest with you, like ETL is helpful especially in the sort of business analytics, like historical trend analysis, those kinds of things. But now that people had a place to do that, right? Like round peg, square hole, everything goes in there and everything needs to get out of there, right? Like that's one way to go about it, which causes this, like do we have this like nth silo? Have we put it and put it into this lakehouse ocean? Like if we haven't, then we have to go build it, right? And then to your point, like there are these, all these random dashboards. But what we're trying to say is, specifically what I'm trying to do here is what if you didn't have to move the data? What if you could just go get the data right from the source directly when you ask a question. So there are no pipelines. It's not that we are building the pipelines. There are no pipelines. There is just connection to the data source and you go fetch data from there when you need it. It's kind of like when I ask you a question and you go look up your favorite search engine and find the answer.

Deepti Srivastava [00:28:44]: You're not keeping all that data, downloading it, like sifting through it, filtering. None of that is happening. You're just saying, hey, what's my date of birth? Oh, go to the database that you Social Security database or something and go get it.

Demetrios [00:29:00]: Yeah. So how do you deal with things such as, okay, I need a lot of data points on, I need to know how many customers did X, Y, Z. But in the source data there, you need to apply a few different kinds of transformations on top of that to get that answer.

Deepti Srivastava [00:29:21]: I mean that's a classic like data warehousing problem. And so those data warehouses are already built. I am just saying you don't need to build new pipelines and new data sources and new like connections, Right. So if you're doing something that is important to your business, which is where the original Data warehousing concept comes from, right? We can go fetch it from that data warehouse. But what we can also do, and this is what's done through complex software or complex pipelining today, we can also do things like, or, or we should be able to do things like, hey, you know, for example, if you ask me, I say this often, like I just, I don't care about Terminator happening, right. I care about where my order is. If I ordered something online, for example, right?

Demetrios [00:30:07]: Speaking of which, I ordered some fucking coffee the other day. Turns out for some reason it was in Google my address for my old house in Spain. So now somebody in Spain has got some nice ass coffee.

Deepti Srivastava [00:30:21]: That's hilarious.

Demetrios [00:30:22]: BY like the 10th day, I'm like, where is that coffee? I could really use it right now. But anyway, sorry, I, I digress.

Deepti Srivastava [00:30:28]: But that's a really good example. Like somewhere something should have updated your data.

Demetrios [00:30:33]: Yeah.

Deepti Srivastava [00:30:33]: Like it didn't happen, right? Because some pipeline somewhere failed ultimately. But the point is like we're saying what if you had to, for, for you to look up your order? Like just ask, right? Your favorite assistant, where is my coffee? It needs to do two things. You need to look up where your coffee order was made, right?

Demetrios [00:30:55]: Yeah.

Deepti Srivastava [00:30:56]: Whether it was Nespresso or some other fancy stuff, right. And then where was it shipped? Which means now you're looking at basically a postgres database or some sort of CRM system. Right? And then you're also looking at your tracking system like UPS or something like that. That means you're supposed to do a join between two data sources that are outside. So databases, data warehouses, data lakes are all great at joining within. And if you need to join two sources today, you have to dump the data in one place so that they can do a cross door join. Right? But what I'm saying is you just need, what you actually need to do, what a human would do is look up your tracking number basically from your order management system and then look up the tracking history in the delivery system. Yep.

Deepti Srivastava [00:31:45]: Snow Leopard can, can do that. Like the aim with Snow Leopard is you can pull from both of those places and give you an answer. So you don't have to build any pipeline, you don't have to like create any new way of answering this question. If you're building an assistant or an agentic flow, you just asked Snow Leopard the question and Snow Leopard said, oh, I need to. So it's doing intelligent routing, it's saying, oh, I need to go to these two different sources, right? And then it's doing intelligent querying, query building, which is like, oh, for ups, I need to write this kind of query because it's an API. And for, you know, Salesforce, I need to write a SoQL query. And for Postgres, I need to like a PostgreSQL query. Right.

Deepti Srivastava [00:32:24]: Which is different from a Snowflake SQL query, which is different from a database query. Databricks query. Yep. So it built that in real time, then it went and just fetched directly from those sources. So that means the data is fresh, which means if, you know, if the coffee was delivered and to Spain in the last half an hour, you should be able to get that information instead of it's still in transit, then you go back and look at it next day because, you know, data dumping is stale. That's sort of what we're. That's the world we are talking about. That's the world we're imagining.

Deepti Srivastava [00:32:55]: And that's sort of where, you know, I've talked to recently to a CTO at Salesforce, I talked to like, you know, head of Data and AI in Asia for P and G. Like the stuff you were talking about, they're dealing with that same stuff, right? Like 8,000 silos of data. Which dashboard is powering? What? Right. Like, let's just pare it down because we need to figure out how to maintain it. Right? Like our data engineers are, are drowning in that. Our data sort of like data scientists are drowning and all of that. So yeah, yeah.

Demetrios [00:33:30]: And we didn't even touch on the data quality part where it's like, oh, something changed upstream and now the data is absolutely worthless because of the way that these dashboards have been built. One thing that I want to know about this vision of how things work. Do dashboards still play a part in this world or is it just questions? And I have to know what kind of question I'm asking before I can get the answer. Because I think sometimes when I look at a dashboard, it provokes questions inside of me.

Deepti Srivastava [00:34:08]: I think this is a very interesting philosophical question and I think there's different camps on this. Again, as an infrastructure person, I don't like predicting the future. But you know, my work at Observable, which is a data visualization platform, like, we are all visual people. So. Right. Like data points on a graph is much easier to grok than like a thousand data points. Like, we just know that. So are dashboards going away? No, I don't think so.

Deepti Srivastava [00:34:40]: I think data visualization will always exist because the easiest way for humans to Grok information. But what if you could augment that with asking all the questions that came up right in your mind when you looked at a dashboard could be answered instantaneously because you could go get the data if the data exists, right? So you didn't have to. I will give you an example. When I was looking at Churn for my product at Google, right? I have to first, because first of all, I didn't have, I didn't have the right dashboards. I had to build the dashboard. So I came up with sort of a solution for what kind of data we need, et cetera. Then I had to talk to my business analyst friend who was by the way, stretched thin because he was actually, you know, looking after 10 different products. I had to queue into his thing, into his workflow, like, hey, I have, I want to build this Churn dashboard needs to pick data from here, here, here, right? Can you build me that right now? Once, once he built it for me, like three weeks later I looked at it, I'm like, oh, I actually need to know like Churn by geographies.

Deepti Srivastava [00:35:42]: So churn in Southeast Asia, churn in the US churn in Canada markets and in fact one level deeper, Churn in, you know, industry as well. Right. So each time that answer question came up, it was a few weeks to get that answer. Either he had to build that into the, into the data warehouse or he had to take the time to pull it out and give it to me. But what if I could just, you know, once you have the dashboard, what I forgot to just chat with, let's say it was BigQuery, right? That where all the data existed, I could just chat with my bigQuery and I would also be able to like join it to like, you know, all the salesforce data that had the latest customer information. What if I could just do that and the system would take care of that for me? Right. So we're talking about agentic flows here too, by the way, right? Like agentic flows have the same problem. And spicy take number three is while MCP is amazing and has, it's a great start, open source start to the, to the connector problem.

Deepti Srivastava [00:36:44]: It doesn't solve it. Right. It also, I believe it has all the same problems that previous generations of solutions have had, which is that it does it. It's not really tackling the hardest part of the problem. And the hardest part of the problem always is intelligent routing and more importantly, business logic.

Demetrios [00:37:06]: Now I mentioned before we had Donay on here and she had built a data analyst Agent. And the way that they did it to keep the agent in line, we could say is that they had different slack channels for different agents with access to certain databases. So you would have a marketing slack channel that had access to the marketing databases and the marketing type of queries were happening in there. And so that was almost the way that they were able to hard code.

Deepti Srivastava [00:37:46]: Yes.

Demetrios [00:37:46]: Hey agent, you speak this dialect of SQL. You have access to this type of database right here. And here is your glossary as we mentioned before. So if any of these key terms come up, you know what they are and you know where to get them and grab that information. And so they were able to do cool stuff and the, the agent could then go and create SQL statements and pretty much do a lot of the menial work. We could say. I, I think I remember them saying it was like a barbell. The majority of the questions that were asked were those questions that are not super complex.

Demetrios [00:38:34]: Yeah, but they're not something that someone who has no idea of data and is not a data analyst would be asking. It's like those kind of middle of the road questions. And anyway, this, all this whole story is because I would love to know how are you making sure that, that the agents or your system speaks the right language to the right database?

Deepti Srivastava [00:39:03]: Yes, that is a wonderful question. In fact, everything you just described is sort of how people are trying to do it, right? The hard coding piece is so like that's just how it's done today, right? You have to hard code and you have to separate out because otherwise mixing questions that go to MySQL dialect versus going to Snowflake dialect, even though they're all SQL and they all follow ANSI SQL standards, will confuse the heck out of everything. The system, entire system, including LLM. Right. This is actually why text to SQL doesn't work either, right? Apart from the fact that it hallucinates and all that stuff. It's just which dialect and what is allowed in which dialect. And this works better for open source databases because the LLMs have gone through that. But for closed source databases it's like a nightmare.

Deepti Srivastava [00:39:55]: I know because we have been doing it, right? So what we're doing is we're using known data infrastructure techniques to do things that are essentially systems problems. We are not trying to shove systems problems into the reasoning AI, you know, predictive world, as I said earlier, right. Like LLMs are great at classification, at summarization and, and those types of tasks, right. Deterministic systems are great at if you, if you code them the right way. They're great at, you know, point lookups and, and being able to do precise and deterministic way work. When you are building a SQL query for the right dialect, you are doing precise and deterministic work. So we are using essentially data retrieval techniques, not in ir, but data retrieval techniques. So we are building SQL, for example, if you're talking about SQL, right? Like we're building the SQL for the right dialect, right, Based on where that query needs to go to.

Deepti Srivastava [00:41:13]: And we're doing it in a deterministic way, which means we're using like data or sort of query building techniques. Query building techniques exist like databases. Use query builders. Right? Data warehouses using use query builders. A bunch of data retrieval systems use query builders. We are doing it using those techniques and like using existing sort of methodology to like for example, ORMs are great at doing this. Right. So we use sort of that existing methodology for building like a PostgreSQL query versus a snowflake query.

Deepti Srivastava [00:41:51]: We're doing this today. We have a design partner that has data in sort of one of the proprietary data warehouses and we just build like queries for that. Right on the fly. Right. So where do you need to do sort of that intelligence in retrieval? Like, oh, you know, based on this question and based on my understanding of business logic, we know that the query needs to go to postgres and also there's a separate query that needs to go to, let's say, BigQuery.

Demetrios [00:42:25]: Interesting.

Deepti Srivastava [00:42:26]: We build both of those. Once we know that, right. Then the rest of it is deterministic. Right? Like the rest of it is, oh, build a PostgreSQL query, build a BigQuery SQL query. So we do that and that is just sort of like we're using. This is what I think is exciting is we're using data system and deterministic programming techniques for that piece and we're using AI and agentic work for the sort of intelligence in the routing and intelligence and what data we need to fetch the understanding.

Demetrios [00:43:01]: So if I'm understanding this correctly, it is AI gets to a certain point, it takes the query and then it says, all right, cool. To answer that I'm going to need to use these tools. And then boom, when it fires it off to the tools, then it steps back. It's not actually creating the SQL. And I think that is a really good way doing it because you're taking a lot of the responsibility off of the LLM. And that's right. Anyone who's dealt with LLMs know that the less responsibility you give them, the better. Unless you do something where it's very siloed and it is only you.

Demetrios [00:43:45]: As I was saying, donate created where you're a market, you're in the marketing channel. I know this agent only has access to the marketing database. It's going directly there. They opted to let the LLM do the SQL query, but they also have a lot of calls to the LLM to like double check the query and do this. So it gets back to that hackiness a little bit like all right, cool. Yeah, we do that. And then when something gets returned, they're also double checking it there to make sure hey, judging by this question, is this SQL query and the data retrieved doesn't make sense. And so you'll have like a critiquing agent that is, is double checking all of that.

Demetrios [00:44:30]: But in your case you're saying the LLM is only used to understand the query and then fire off the different tools that it needs.

Deepti Srivastava [00:44:41]: Right. And that's sort of like how you would describe it in the sort of, you know, these days, the MCP agentic world. Right. But we're take, we're not just firing off a tool call, we're taking responsibility for how that tool is created and the accuracy of that tool as well. Right. So when we ran our benchmarks, for example, internally, we did it with an older sort of rag ish system, which is how most of these things work. Like we had a 99, 98% accuracy. This is without us doing any fine tuning of our own intelligence.

Deepti Srivastava [00:45:15]: And the system we were comparing against had like 6% accuracy for this type of SQL based lookup. Right. Because either it doesn't know how to build the right SQL or it's not getting the right data or it's hallucinating. All of those things just don't like isn't what you would do. But that aside, I think back to your use case that Dune was building. Right. What we are doing for sort of our design partner here is like that Slack channel can actually be used by sales and marketing people, right?

Demetrios [00:45:46]: Yeah.

Deepti Srivastava [00:45:46]: What if you could have that? Because turns out there's a lot of overlap in the kinds of questions that sales and marketing people have around, you know, the, the pipeline piece and all the way to the sale and post sale piece. Right around customer data and around like all that kind of stuff. So if both of them can ask questions, then we didn't have to and you didn't have to hard code anything. Right. Then you actually open up this whole New world where they could probably even talk to each other and come up with better campaigns and better end to end workflows for making a sale happen. Right. So I think what I'm imagining is in this world of ad hoc questioning and ad hoc information needs, why not make the information like retrieval ad hoc? Why not make the data retrieval ad hoc, right? So what we're doing is we're having like a natural, you know, there's in the data world, right? There's all this like single API to fetch everything, which has never worked because. Right.

Deepti Srivastava [00:46:50]: You just talked about one of the many problems, which is which SQL do you build at the end of this single API? But what if the API is natural language, which is what it's become, but then it translates to the precise like other end, what you're calling tools, right? Like it translates to the precise dialect that those tools talk. So the tool is not doing any guessing, right? The API is not doing any guessing, right. You use the intelligence for what needs to be used for the understanding and the, oh, I think it needs to go here and do this and then you leave the rest of it to the parts that know how to do it, which again leads to better accuracy.

Demetrios [00:47:34]: I didn't quite understand. How are you differentiating on that tool call part and taking ownership for the outcome that. All right, we're going into whatever database, whether it be airtable or your CRM of choice, and we're ensuring that it's going to gather that data properly. Or I think you, you said something like we authenticated or we authorized that it, it will be correct.

Deepti Srivastava [00:48:07]: Oh yeah, we're just taking responsibility for the accuracy, I think. Right, so, so think so. Snow Leopard is a three part sort of workflow. The first part to be more, more like. Let's dig into it. Right. Like the first part is just what you said. Like you have a question? We do natural language understanding.

Deepti Srivastava [00:48:24]: No rocket science here, right. Mostly AI here. And then the next part is we are combining data infrastructure logic, your business logic, to figure out which API call needs to be made. This is our proprietary sort of way of doing, or rather this is the intelligence we're building. Right. And then once we figure out, or it needs to go fetch user id, you know, in your case, for example, it needs to fetch your name, your address and the order number, right. From one place, right. Which is your, let's say CRM system.

Deepti Srivastava [00:49:02]: And it needs to fetch the order information and shipping information from the delivery system. Right. Once we know those two things, this is what the Intelligence is telling us then building the right query. Like we're not doing text to SQL, right? So there is no fuzziness there. We're literally saying build the SQL statement, which is like it's a known thing, like build the SQL statement directly for this particular dialect and we go build it, right? So we're basically using like regular programming, like systems programming to do that and then we go run it on that system. So you know, if you run a PG SQL query on a PG SQL database, it will give you the information, right? That's known. And then yes, we can run it through an evaluation system and all that stuff, right? Which is again, these are known things that people have done, right? But there is no like, oh, did it get the right data? Like did it, you know, we had this column, but does that column exist? Like, I don't know, like that's what you do when you do text to SQL or sort of these agentic flows, right? The tools have to. You hand off a lot of responsibility to the tool and the tool developer has to take on the responsibility, which.

Deepti Srivastava [00:50:15]: And you don't know who the tool developer is, right? Like yeah, number one, number two, different tool developers. Like so MCP servers for postgres can be built by anyone, right? And so because there's no, like you have to build it this way. MCP the protocol and MCP the sort of open source framework doesn't tell you that you have to do it in this way, which means different people can interpret that framework framework and build different MCP servers, right? So either you just like with any other open source software, either you take somebody's MCP server, then you like harden it, you test it, you fix things in it the way you want them to be fixed, right? Or you just rely on them to do the right thing, which anybody's that been in the open source world like, right? It's just the last 20% is sort of where all the 80% of the work goes, right? In hardening it. And the one additional thing here is that we are taking the sort of responsibility of understanding business logic that you tell us, right? Which the NCP server is just a connector, right?

Demetrios [00:51:21]: Yeah.

Deepti Srivastava [00:51:22]: So it's just going to do. If you give me like a SQL query, I'll go run it on postgres. But like which SQL query to run is that the right SQL query to run? Like does it understand whether column order ID with a capital O is different from column order number with a small O? Like it. It goes down to that level of confusion and Complexity. Right. This is what we are trying to do.

Demetrios [00:51:46]: Yeah. The downfall of the MCP servers are very much that. And I appreciate you calling that out real fast because it is very easy to put up a MCP server and that's like the blessing and a curse.

Deepti Srivastava [00:52:02]: Yes, 100%.

Demetrios [00:52:04]: That's, that's probably why it is so incredibly popular right now. Because folks can stand up an MCP server in a few hours.

Deepti Srivastava [00:52:13]: That's right.

Demetrios [00:52:14]: But as you said, if you don't know and really know intimately the ins and outs of your postgres database, you throw up a server with a lot of tools and those are your opinions on how the tools work. You could get yourself into trouble. Or if I come and I grab your MCP server and I think, all right, cool, it's a postgres MCP server, I'm going to just throw this into my postgres instance then. Yeah, there's, there's a little bit of wire crossing that can happen.

Deepti Srivastava [00:52:49]: Yeah. And I, I agree with you. Like, I think it is a blessing on the course. Right. Like, I, I'm not trying to poo poo on it. I'm just trying to point out because people attach themselves emotionally to the next big thing, to the next hype, and then they get super disappointed. Right. And I want people to just understand and go wise, wide open.

Deepti Srivastava [00:53:05]: Right. Like, it's actually great for Snow Leopard if there's MCP service because if one gets popular, it actually helps us because we don't have to build a connector then. So it accelerates, you know, user delivery for us, honestly. Right. But I just want people to like, know that there are sort of, you know, there are blindsides to this that you want to be aware of. You want to go in eyes wide open, like it's fun to play with. But in my experience, building those high availability, high reliability production systems requires a lot more than putting something together in a few hours. That's not the hard part.

Deepti Srivastava [00:53:44]: The hard part is again, going from POC to production. This is still a problem, I think, with AI systems from the engineering leaders CTOs that I talk to that we built this cool thing. We couldn't put it in production because reliability and accuracy and you know, all performance isn't even a thing. Right. Right now it's just reliability and accuracy. Like, I needed to answer the question when I needed to answer the question in a reasonably accurate way.

Demetrios [00:54:14]: Yeah, we're, we're okay with waiting 15 minutes to get this answer back as long as it's the right answer. And so. Well, anything else you want to touch on before we go?

Deepti Srivastava [00:54:30]: I just think it's a really exciting world that we live in right now. I think the fact that there is MCP and the fact that there is all this discussion around tools and things like that means that people are finally really starting to understand the thing that I was saying. I went blue in the face. If you look at my LinkedIn from last year going like, but you need your structured data systems. You need them. And so I'm personally really excited that people are starting notice this and starting to pay attention to this problem because that is what I believe will cause the true wave of AI to like, make people's lives better. But yeah, I'm really excited to talk to the people who are facing this problem. I want to, you know, even Danae, for example, it's been really exciting to hear from you about what, you know, they've been doing there because I just want to understand honestly how people are tackling this.

Deepti Srivastava [00:55:20]: Do they have this problem? How are they tackling it? Right. Can we help? Like, it's a really exciting time to be in this space. And I keep saying, the last thing I'll say here is, you know, I, I've been telling my, my systems friends, like, hey, come over on the AI side, it's okay. I know it's non deterministic and I know you hate that.

Demetrios [00:55:37]: Like, oh, hell no.

Deepti Srivastava [00:55:38]: Come on, it's, it's fun out here. And I really, I feel like some people are sitting up to pay attention, which I'm, which I'm excited about because I want the AI world and the systems world to come together because that's how we're gonna, we're going to solve these problems.

Demetrios [00:55:52]: Yeah. I'll have to introduce you to Donay and for anybody that wants to check it out, it's Snow Leopard AI And I, I love what you're doing. I think it is super cool and I'm excited to see how it progresses and live in a world where you have infinite connectors to all of these different databases. So I can. I actually have a use case right now that I'm thinking about where I was asking myself so many different questions yesterday. And I have to go and gather the data between Airtable and Salesforce and I am really bad at Salesforce and so it took me way longer than it should have. And. And then you get into, oh, I don't know if I have access to that data.

Demetrios [00:56:36]: And that's not going to be solved by Snow Leopard. I'm sure, but I have to go and talk to somebody and say, hey, I need this for my report. Blah, blah, blah. So, yeah, data fun.

Deepti Srivastava [00:56:46]: Data fun.

Demetrios [00:56:47]: I'm excited for a world where snow leopard can hopefully get rid of the majority of that pain.

Deepti Srivastava [00:56:53]: Yeah. Thank you. It was so fun to talk to you, Demetrios. Thank you so much for having me. And, yeah, this is the kind of thing that I think people should be talking about. It's very exciting.