MCP Servers Are Becoming the UI for AI Agents

Demetrios: [00:00:00] All right, so at the beginning of April, I went to New York for the MCP Dev Summit, and while I was there, I got the chance to record a few podcasts with attendees that were at the event. This is one of those conversations. Hope you enjoy.

Naseem Al-Naji: Gateways are very interesting, I think because they're solving, like, a very similar issue-

Demetrios: Mm-hmm

Naseem Al-Naji: where, hey, there's now this new way to do things. It's called MCP. It's how you connect external services to your AI. Yeah. And with great power comes great responsibility. If you're a security product, focus on the security aspect. Don't also focus on, like, productivity.

Demetrios: Naseem Al-Najji is the co-founder of MCP Cat, a platform that helps developers debug and understand how AI agents

Naseem Al-Naji: interact with their MCP server.

Naseem Al-Naji: Wait,

Demetrios: wait, before you jump forward - Yeah ... the goal idea? Yeah. Explain more of that, because that feels like something novel I haven't really heard.

Naseem Al-Naji: Yeah, yeah, yeah. When you're building an MCP server, you're actually a bit [00:01:00] disconnected from your end customer, right? You might have your MCP server, but the person using that is interacting through it through Claude or Cursor or Claude Desktop, and those companies own a big part of the experience.

Naseem Al-Naji: "Oh, this guy says there's a problem. I don't know why. Oh, he's using Cursor? What version is he using?" Works on my

Demetrios: machine, dude. Like, yeah.

Naseem Al-Naji: What are you talking about? Works on my machine. Yeah, exactly. Works on my machine.

Demetrios: Yeah. Oh, well, I like this. Let's pull on that thread a little more because-

Naseem Al-Naji: Gateways are very interesting. I mean, I think they're, they're very valuable. Um, I'm surprised by how many there are.

Demetrios: Yeah.

Naseem Al-Naji: Uh, but I guess that's like good, you know, good, good competition.

Demetrios: I think a lot of people pivoted into the space.

Naseem Al-Naji: Mm-hmm.

Demetrios: And the most logical first step [00:02:00] is a gateway.

Naseem Al-Naji: Mm-hmm.

Demetrios: And especially if you're a security company already and you're doing like package scanning.

Naseem Al-Naji: Yeah. Yeah.

Demetrios: Okay, what else should we scan? Yeah. MCP servers.

Naseem Al-Naji: Yeah. I think also it's a repeated model-

Demetrios: Mm.

Naseem Al-Naji: Because when LLMs came out-

Demetrios: Yeah ...

Naseem Al-Naji: uh, LLM proxies were... I mean, everybody, I mean- Yeah ... so many established companies use LLM proxies. Yeah. So it's kind of like the next step. Oh, okay, well, like let's just get another proxy

Demetrios: in place.

Demetrios: Yeah, we should have MCP proxies- Yeah ... and LLM proxies and- So ... we need our gateway to check the cost.

Naseem Al-Naji: Yeah, yeah. Um-

Demetrios: Yeah, actually now that I think about it, is it not the same thing? Like, they just proxy. They just are the gateway, and then if it goes through an MCP server through the LLM, then okay.

Naseem Al-Naji: I mean, it's, uh, it's very similar.

Naseem Al-Naji: Um, I think that a proxy is very good because it can catch what people are [00:03:00] chatting with LLMs. Yeah. And if it doesn't like something, it will, like, kill the user's session.

Demetrios: Yeah.

Naseem Al-Naji: Right? So you can actually, like, filter out what's getting sent to an A- uh, to an LLM. Uh, you can also, you know, block all network traffic to other LLMs.

Naseem Al-Naji: Um, gateways are very interesting, I think, because they're solving, like, a very similar issue-

Demetrios: Mm-hmm ...

Naseem Al-Naji: where, hey, there's now this new way to do things. It's called MCP. It's how you connect external services to your AI. Yeah. And with great power comes great responsibility. Um, so here's a ton of, like, gateway solutions.

Demetrios: Yeah, totally. And I, I guess a lot of the gateway value prop is hit our gateway and you don't have to load up 5,000 servers. You load up our gateway, it's the one server to rule them all, and we will figure out the most efficient way to grab whatever tool you need from whatever server you need.

Naseem Al-Naji: Mm-hmm. Yeah, I personally don't...

Naseem Al-Naji: I, I personally wouldn't use them for [00:04:00] that.

Demetrios: No.

Naseem Al-Naji: Like, I, I, I, I think having, um, as a customer of an MCP- Right ... I would prefer that the MCPs I use are, like, the ones I use kind of... Like, it's okay if they're proxied through a gateway-

Demetrios: Mm-hmm ...

Naseem Al-Naji: but I don't want the gateway to do the routing for me-

Demetrios: Oh ...

Naseem Al-Naji: I think, personally.

Naseem Al-Naji: Why

Demetrios: not?

Naseem Al-Naji: Um, I think it's... I mean, I think that LLMs are good enough, or the, the MCP clients are good enough at picking the right tools. At least that's what we see at MCPCAT. Like-

Demetrios: Yeah ...

Naseem Al-Naji: um, that I think I do see some gateways try to say, like, "Hey, we'll give you all the tools and, uh, use our gateway to, to, like, navigate them."

Naseem Al-Naji: And I... It just feels like an unnecessary extra step. Like- It's

Demetrios: an anti-pattern?

Naseem Al-Naji: Well, I think it's like... I think, like, it's important to focus the solutions, right? If you're a security product, focus on the security aspect. Don't also focus on, like, productivity. Yeah. I started a cybersecurity company, um, eight years ago, and I sold it.

Naseem Al-Naji: No way. Um, [00:05:00] so, uh, I think, like, that's- It's, like, a more important, uh, it's more important to understand your ICP. Um, and so, like, I think as, like, a user I'd, I'd want just like, "Hey, this is, like, how we do security, and usability is handled by different solutions." But that's my personal take.

Demetrios: Yeah. Oh, well, I like this.

Demetrios: Let's pull on that thread a little more because- Yeah ... you're not going to have five different gateways, right? Like, so if there is a usability feature-

Naseem Al-Naji: Yeah,

Demetrios: yeah ... but it doesn't have the security, you're not gonna, like, go and then also have a-

Naseem Al-Naji: Yeah, of

Demetrios: course ... security gateway, and then you, like, have- Yeah

Demetrios: two gateways that it's going through.

Naseem Al-Naji: I, I think, like, what I, what I just mean is, like, if I'm going to use a gateway, I don't want it to be opinionated on which version of a customer's... Uh, oh, on which version of MCP server I use. Like, I wanna pick- Oh ... and choose the MCP servers that are loaded up into it.

Naseem Al-Naji: Um- Oh, okay ... if that makes sense. Yeah. That's like a, like, yeah.

Demetrios: I see that.

Naseem Al-Naji: Um, 'cause what if, you know, a new version comes out and it enables [00:06:00] different tools? Um, I guess, for example, Webflow is one of our customers. Like, if I wanna use, like, a new update of their MCP server, I don't want my- gateway to be the reason we can't use it.

Demetrios: Yeah.

Naseem Al-Naji: Yeah.

Demetrios: Yeah, especially 'cause it's like you're constantly, as that gateway, having to stay up to date with- Yeah ... whatever the external servers are doing. Yeah. And which you, you kind of in a way, if you're doing the security thing, you also have to do, you have to scan it and make sure that- Yeah, yeah

Demetrios: there's no malicious stuff. But- Yeah ... I, I see what you mean. It could be a potential blocker.

Naseem Al-Naji: Yeah.

Demetrios: The thing that this makes me think about is how you're seeing success-

Naseem Al-Naji: Yeah ...

Demetrios: with folks that are using MCP.

Naseem Al-Naji: Yeah, yeah. Totally. So I guess for context, uh, if people don't know what mcpcat is, um, we are a, like, product analytics layer for MCP servers.

Naseem Al-Naji: So if you're making an MCP server and you wanna make sure it's good, you use us to tell you that, right? [00:07:00] So, um, what we do is we get feedback from the agents using your MCP servers and share that with you in a way that's statistical. We find their goals with your MCP server, and we help you understand if those goals are succeeding or failing.

Naseem Al-Naji: Are agents hallucinating on certain tool calls? Mm-hmm. Um, and so, you know, you t- you see a lot of the, the problems people have with MCP. A big reason is that most of those MCP servers are half-baked, uh, hackathon projects- Yeah ... that never got any real, uh, maturity from the organization. Like, like the main way they hear about problems are through GitHub issues.

Naseem Al-Naji: Like that's, that's not how we find problems in mature software today. Yeah. So how do we fix that? And, and so, um-

Demetrios: Wait, wait. Before you jump forward- Yeah ... the goal idea.

Naseem Al-Naji: Yeah.

Demetrios: Explain more of that because that feels like something novel I haven't really heard.

Naseem Al-Naji: Yeah, yeah, yeah. So, um, when you're building an MCP server, um, you're actually a bit disconnected from your [00:08:00] end customer, right?

Naseem Al-Naji: You might have your MCP server, but the person using that is interacting through it through Claude or Cursor or Claude Desktop, and those companies own a big part of the experience. But at the end of the day, if your MCP server cannot enable your customer, then they're having a bad experience. Yeah. Right?

Naseem Al-Naji: Um, and so because there's these third parties in the way, you actually don't know what your user is prompting at the, like, to get to the, like, the MCP server. So if I wanna use, like, Webflow's MCP server to update my blog on my website, um, it's going through Claude Code and Webflow's MCP server actually has no idea, uh, that that's, that was my goal.

Naseem Al-Naji: Or if I wanted to translate it into Spanish and have a Spanish version of it- They have no idea that that's the goal. They just see, "Oh, it's calling this tool."

Demetrios: Uh-huh.

Naseem Al-Naji: Yeah. And so what we do is we actually extract the user's overall [00:09:00] goal with the MCP server, and we do that by, uh, augmenting any MCP server's tool list with required fields that tell the agent, "Hey, tell us a little bit about why you're calling this tool and how it fits into your overall goal."

Naseem Al-Naji: It's kind of like a micro- No way ... micro user interview.

Demetrios: Yeah.

Naseem Al-Naji: Um, and, uh-

Demetrios: And so the agent has to say that when it sends a request?

Naseem Al-Naji: Yeah. Yeah. And- Oh,

Demetrios: that's cool ...

Naseem Al-Naji: and so that's a lot of data. Yeah. So that's like a ton of tool calls, right? So how do you stitch that in together, uh, into a reasonable way? And so we'll, like, take all of those, stitch them together, and you have, like, a play-by-play of what the agent was doing, why it was doing it, um, and you can even see, uh, extract, like, higher level goals with your MCP server, like what is, what are the top 10 use cases for this MCP server?

Demetrios: Does it not add to the cost?

Naseem Al-Naji: Yeah, it does. It adds, uh, like a fraction of pennies to the cost. Okay. Right? Um, I think [00:10:00] that what's crazy is we have over 100 million events logged every month, and no complaints from customers because that's just, like, the way people are paying for these, I think tokens are actually quite abundant.

Demetrios: Mm-hmm.

Naseem Al-Naji: Um, and so, uh, many tokens are used in the process of even deciding to make a tool call that we add 0.05% to that cost. And so-

Demetrios: And the value you get from those 0.05 tokens or that, that cost increase is so much more. If I can understand how my customers are using my MCP server- Mm-hmm ... and why-

Naseem Al-Naji: Yeah

Demetrios: that's gonna enable me to- Yeah ... build a better MCP server.

Naseem Al-Naji: Yeah, and it's also opt-in. Like, if you're, you could ask, like, you can flip a switch in MCPcat and say, "Also ask my users if they are cool with it." Um, and so it's not, doesn't even have to be, uh, by default enabled for your MCP server. You can sort of let users decide if they are giving you user, [00:11:00] like, usage data.

Naseem Al-Naji: Mm-hmm. Like we do in traditional software.

Demetrios: And you're also tracking where it's coming from. And I say this because one of the big use cases I heard yesterday is internal MCP servers-

Naseem Al-Naji: Mm-hmm ...

Demetrios: in large companies. And so if I am building an MCP server and I have no visibility into it, I don't know who's using it in the company, what they're using it for.

Demetrios: Yeah. Yeah. But then if I do know that, hey, it's coming from legal and they're using it for this reason- I- is that, like, something you can do to, like- Yeah ... get that traceability as to where in the-

Naseem Al-Naji: Uh, yeah, totally. So you can have, uh, what you're talking about is sort of identifying the session of a user- Yeah

Naseem Al-Naji: and who they are and where they're coming from, right? Yeah. Like, we will get you information that it is, you know, Kevin Smith- Whoa ... who belongs at this company, and is part of this team and this organization, and they're using Cloud Code. [00:12:00] Whoa. Right? All of this stuff. So, you know, when an issue happens, you might actually be able to tell through our products that, oh, this is affecting Cursor users only.

Naseem Al-Naji: Oh. Um, because Cursor interprets the protocol differently than other clients do. And so otherwise, you, you would have no idea what- That's so cool ... oh, this guy says there's a problem. I don't know why. Oh, he's using Cursor? What version is he using? Works on my

Demetrios: machine, dude. Like,

Naseem Al-Naji: yeah. Works on my machine.

Naseem Al-Naji: Have a buggy truck. Yeah. Exactly. Works on my machine.

Demetrios: Yeah.

Naseem Al-Naji: I don't know. Uh-

Demetrios: So you can get that in-depth Uh, analytics and, like, telemetry

Naseem Al-Naji: Yeah, yeah. I mean, I think you need it to-

Demetrios: Yeah ...

Naseem Al-Naji: you need it to run a mature MCP server. At, at, at the end of the day, like, I mean, it's-- This is, like, par for the course for web analytics.

Naseem Al-Naji: Um, you need these answers in order to know how to fix problems before your customer even gets angry.

Demetrios: Yeah.

Naseem Al-Naji: Right? And for example, at mcpcat, we have tons of analytics where I get paged if the page loads slowly.

Demetrios: Yeah.

Naseem Al-Naji: [00:13:00] And I'll message our customer like, "Hey, sorry about the, like, delay, like, the slow- the slowdowns.

Naseem Al-Naji: Um, we're on it."

Demetrios: Yeah.

Naseem Al-Naji: And one of our customers, she was like, "Oh." I, I was like, "I was gonna message you, and you m- messaged me before I even could message you." I'm like

Demetrios: Because you can then set up events, so if X happens, then we do Y, and-

Naseem Al-Naji: Yeah ... it

Demetrios: doesn't... It can be very deterministic. Like, if we're seeing a spike in latency, tell the agent to reply or tell, or tell the end user and message them.

Demetrios: So-

Naseem Al-Naji: Yes. Yeah, you could even-- Yeah, if a tool call is slow, the rep- the response could say, "Hey, sorry for the slow response." Yeah. "Apologize to the user on our behalf-" Yeah "... if you are cool with that." And then the LLM will say, "Hey, this c- this company's MCP is slow today. They wanted to apologize."

Demetrios: Wild, man.

Naseem Al-Naji: I mean, you don't have to do that.

Naseem Al-Naji: Yeah But-

Demetrios: But it's possible ...

Naseem Al-Naji: it is possible. Yeah,

Demetrios: yeah, yeah. Yeah, you can do a lot of stuff, and [00:14:00] it is true, like, for a mature product- Yeah ... you need that kind of analytics. Yes. I don't know anybody who's not running something like PostHog-

Naseem Al-Naji: Yeah ...

Demetrios: in their products or a variation of PostHog anyways.

Naseem Al-Naji: Yes, yes.

Demetrios: Yes. PostHog's very startup-y stuff. In PostHog and all these, like LaunchDarkly, you have a lot of ways to update features. Do you, within mcpcat, also have that route where it's like, "Hey, we're gonna A/B test some stuff," or, "We're gonna do, like, a little bit of a rollout, feature flags type of idea"?

Naseem Al-Naji: Yeah, very interest- Uh, we don't currently do that.

Naseem Al-Naji: Like, um, I think, you know, feature management is probably an interesting space. Uh, but I don't think MCP servers need that yet.

Demetrios: Yeah.

Naseem Al-Naji: I, I don't see very many people wanting to A/B test their MCP. Um, and so, you know, I think we're already quite early to the analytics game- Yeah ... for MCPs. Uh, and so I think, like, just sharing, like, [00:15:00] what users are up to and- When they're running into problems is the core focus of what we're doing.

Naseem Al-Naji: Um, and maybe one day we'll do, like, feature management, but I think there's also, like, a ton of solutions out there that are not unique to MCP that might do it better, right? Grip for it. Yeah. So, I mean, I think, like, we solve problems unique to MCP and probably stay, stay- Goes

Demetrios: back to that focus idea we were talking about just a few minutes ago.

Naseem Al-Naji: Yeah. I mean, I think what's really interesting is, um, what we see makes MCP servers better. Like, has nothing to do with our product. It's, it's purely just, like, uh, we'll flag when issues are happening, and the solutions to them are quite creative. Um-

Demetrios: Mm.

Naseem Al-Naji: For example, one of the biggest gaps I see in MCP servers is many people do not use errors on tool calls as a way to guide an agent into the right way of doing things.

Naseem Al-Naji: So, [00:16:00] like, you might hit a webpage and it errors and it says F- 500. Okay, sure, you're a human. There's nothing you can do to fix that. Um, but, uh, what's nicer for agent experiences is when they mess up, the tool call tells them, "Hey, you called this error. Maybe you were trying to call this or this or this instead.

Naseem Al-Naji: Go do that." And that makes agents a lot more recoverable when they make mistakes. Um, and so there's tons of different techniques that we've seen customers use to, uh, actually improve, uh, the success rates of their MCP server.

Demetrios: And so this is just when you have errors, you're guiding them in different directions, and these are hard-coded, or this is something that you're s- telling an agent to make sure when you're creating-

Naseem Al-Naji: Mm-hmm

Demetrios: if, you know, there's an error, you give three other options.

Naseem Al-Naji: I think it's more just a opportunity. There's no... I, I don't have, like- Description ... [00:17:00] whether it's hard-coded or not, right? I think that's kind of up to how much time you guys wanna spend on it.

Demetrios: Yeah.

Naseem Al-Naji: Right? But, uh, the fact of the matter is that errors propose a unique opportunity to agents, uh, to kind of tell them what they're doing wrong with your MCP server, and course correct.

Naseem Al-Naji: And if you're not doing that, if you're just saying, "An error happened, sorry," you know-

Demetrios: Yeah, good luck. Good luck. You should try again.

Naseem Al-Naji: Try again. Yeah. Like, uh, it's, it's not constructive. The agent doesn't know what to do, so why wouldn't you use that as an opportunity to, to guide them?

Demetrios: It is so wild because as you're saying this, this is, like, the first time that I can think of- Where you get that ability to, like, reroute in flight- Yeah

Demetrios: and you get the, the knowledge and that, uh, the intelligence that the agent has to understand the error codes- Mm-hmm ... and then redirect its intention.

Naseem Al-Naji: Yes, exactly. And actually, not that many companies do this, um, just because they haven't thought about it or, you know, previously, error codes are something you might, like, [00:18:00] not want to show your user.

Naseem Al-Naji: Like, what if it's, uh, you know, too cryptic, you know? Or, like, they don't know what the, like, the parameters... Like, they don't need to know the parameters of the API that detailed. Like, you maybe you wanna hide it. But actually with agents, you, you want to be clear and communicative with them, kind of the same way we are with humans.

Naseem Al-Naji: Like, if you filled out a form to purchase a flight and you forgot your last name, you'd see, "Hey, your last name's required."

Demetrios: Oh.

Naseem Al-Naji: That's a better thing to tell a human, so why wouldn't you tell an agent, like, "Hey, you got the parameters wrong. You didn't just get them wrong, you forgot the last name."

Demetrios: Yeah.

Naseem Al-Naji: So fill out it, like red- retry your request with last name Like, as an

Demetrios: example of, of a proper way, yeah And it's such an easy fix.

Demetrios: But if you don't think about it, this is one of those ones where it's like no, or not that much overhead is needed to put this into practice. Yeah. And you'll probably see such huge gains from it.

Naseem Al-Naji: Yes. Yes, and, and we've seen customers have huge gains- Yeah ... [00:19:00] from that. Yeah, for sure.

Demetrios: All right, what other tricks you got for me?

Demetrios: Other tricks? Successful MCP servers.

Naseem Al-Naji: So this, this is probably, uh, unique to, you know, different use cases. We have a logging provider as a customer, um, and so an MCP could query the logs, right? Um, but those logs could be like t- if you, if it's like a broad query, you'll get thousands and thousands of results and- Yeah

Naseem Al-Naji: you don't want that to send to the MCP, or you don't want that to go into the context window. Like, that would fill up the context window, and it's not useful. Something that we've seen some MCP servers do is they'll have as a parameter to tool calls, token limits.

Demetrios: Mm-hmm.

Naseem Al-Naji: So when they're making the to- the tool call, the LLM can say, "I only want you to return 100 tokens."

Demetrios: Mm. "

Naseem Al-Naji: So don't send me more than 100 tokens, 'cause I don't need that many." Does this make sense, uh-

Demetrios: Yeah. Yeah, yeah Yeah, yeah, yeah Okay. So I'm pinging an [00:20:00] MCP server, and then what I get back is very within a certain bounds that I define.

Naseem Al-Naji: Yeah.

Demetrios: So it's like it can't be more than... You, you set the parameters. It can't be more than X tokens.

Demetrios: It can't ... I'm sure you can set a lot of other parameters too, but is the token length the one that gives you the most bang for the buck?

Naseem Al-Naji: It's, it's unique, right? 'Cause normally you think of to- your, you think of your tool calls mapping one-to-one to API parameters. Ah. Right? You're thinking like, "Okay, I want you to search for my users in this country."

Naseem Al-Naji: The agent might not know that you have millions of users,

Demetrios: right? Yeah.

Naseem Al-Naji: And so when it does that query, it might get millions of results. Um-

Demetrios: And it might try and give you back all of

Naseem Al-Naji: those results. Yeah. And maybe some, some APIs have limits that you can set on results. Uh, but sort of what's nice is to give the agent, the agent has more context on how many results it wants or expects to get its [00:21:00] problem, um, solved.

Naseem Al-Naji: So having a token limit on every tool call, uh, that could be lengthy is a really nice way to let the agent decide how many tokens it's filling into the context window with a tool call.

Demetrios: And so when the agent comes and submits whatever its request is- Yeah, yeah ... it is always saying that's one of the parameters that it's throwing in.

Naseem Al-Naji: Yes, exactly. Exactly. For, for lengthy tool calls

Demetrios: Yeah. So y- yeah, it doesn't need to be every time. Yep. But when you know that there are certain tool calls that can get a little bit-

Naseem Al-Naji: Yeah ...

Demetrios: unruly, you say, "All right, cool." Yep. "Make sure to submit how many you want as one of these parameters-"

Naseem Al-Naji: Yeah, exactly "...

Demetrios: in that form that you make them fill out before they are hitting it."

Demetrios: That's, uh, that is super cool, and it saves a lot of money, I'm sure.

Naseem Al-Naji: I, I think it saves a lot of money and, and, you know, people talk about MCPs. I mean, the goal is to save money and be more reliable and more efficient, right? And so this is just one way that [00:22:00] agents can, uh, sort of not get surprised with massive context window, uh-

Demetrios: Yeah, because then it, it ends up, at first it fills it up- Yeah

Demetrios: and then it ends up not being useful.

Naseem Al-Naji: The other fun one that I have for you is, um, something that we do automatically, but any, any, anybody with an MCP server can do, right? Um, but we'll do it automatically, uh, for, for our customers, is, um, you add an extra tool to your tool list that says, "Tell me what we're missing to get your job done."

Demetrios: Mm.

Naseem Al-Naji: Okay? So, you know, let's say I, uh... Is there an MCP server you use much, like, that you like to use?

Demetrios: Uh, of course Playwright.

Naseem Al-Naji: Okay. Playwright. Yeah. So, um, let's say you asked an agent to do something with Playwright, and that might be involving a Chrome extension. Like, you want it to also click a Chrome extension, right?

Naseem Al-Naji: Yeah. But Playwright's tools don't let you access Chrome extensions. [00:23:00] Um, Playwright could expose a tool on their tool list that says, "If you don't have what you need, report it." So report missing functionality. Yeah,

Demetrios: it's like, "Give us feedback."

Naseem Al-Naji: Yeah, yeah. So instead, your agent will report to Playwright, "Hey, the user wants to access Chrome extensions-" Wow

Naseem Al-Naji: "but I can't do that with the current available set of tools," and that gets sent straight to the devel- developer team of Playwright, and they can say, "Oh, okay. Well, this has been asked, you know, 1,000 times. Maybe we should do it by now." We should add it. Yeah. Right? And then they'll, they'll know that you're trying to do something that can't be done, and they never even had to talk to you.

Demetrios: Or, or it just automatically creates an issue and starts trying to create that-

Naseem Al-Naji: Yeah ...

Demetrios: right away.

Naseem Al-Naji: Yeah. If they, if they have that, um, set

Demetrios: up. The developer gets the PR. Yeah. They don't get the feature request. Uh, that is wild.

Naseem Al-Naji: Yeah. Couldn't agree more.

Demetrios: That is so logical, and it makes so much sense. And again, it's something that if I didn't think about it, [00:24:00] I probably wouldn't do it.

Demetrios: But now that I'm thinking about it- Yeah ... it's just adding an extra tool.

Naseem Al-Naji: Yeah, just one extra tool.

Demetrios: And you get so much insight.

Naseem Al-Naji: Yeah. I mean, I, I... It's, it's business critical. And so when you see these discussions about, "Oh, MCPs suck," well, candidly, they've been out a year and no one has put in tons of effort into making them better.

Naseem Al-Naji: Yeah. Right? Um, and they are now, and they're getting better, and they're... And some of them are great. Like, I, I love some of them. Um, so-

Demetrios: Have you seen, like, good practices on how to build tools and how to properly, like, abstract the tool calls? Because the classic gripe you s- you hear is, "Well, if you just map it one-to-one with your API, then it's a horrible experience, and you've got a million tools, and you load up all those tools into the context window, and it blows everything up."

Demetrios: And so then I've heard, "Well, you wanna abstract it away, and maybe you just have three tools that you can [00:25:00] go to first- Yeah ... and that will bucket it, and you, then you can tree search down on the different tools." Or it's a clear workflow. You know the intention.

Naseem Al-Naji: Yeah.

Demetrios: So you don't need to have each minuscule detailed tool if you know the intention.

Naseem Al-Naji: Yeah. Yeah, yeah, yeah. Um, so I think that the discussion when, when people are talking about mapping tool calls one-to-one to APIs and, and extracting them out into, like, more usable things, like, I think there's obvious gains, right? Like, you could turn, instead of making multiple tool calls for the same thing, yeah, you're saving round trips if you're able to combine three tool calls into, like, what they actually do together.

Naseem Al-Naji: Yeah. That's, like, obvious. But I think the hard thing is giving people recommendations for, well, like, what is that? Like, how do I know what those are? And what I like to tell people is base it off your UI. Like, do you have a UI today? Like, your UI [00:26:00] is probably using 20 API calls to solve a human need, right?

Naseem Al-Naji: And so that can be a good place of inspiration for where you would normally combine tool calls, you can actually expose, like, oh, well, this page of our product is, um, like a review page, and it uses like 10 different API calls, but we can just tell the agent, "Oh, you want, like, us to do that for you? Like, here's like...

Naseem Al-Naji: We'll do one, like just expose one tool call called review, and it will, like, do that," and it does everything in behind the scenes that... Yeah. So like- Yeah ... kind of taking inspiration from your UI, 'cause you've done a lot of thinking on, on what the human wants, and- Yeah ... agents kinda want the same thing sometimes.

Demetrios: Yeah, you don't have to overcomplicate it.

Naseem Al-Naji: Yeah, yeah.

Demetrios: Like, I'm thinking about how you probably have a lot of extra stuff in your UI that you need to do because a human is doing it, but you potentially don't [00:27:00] need as much, but you still know the outcome.

Naseem Al-Naji: Yeah.

Demetrios: So at the end of the day, you know the outcome, and if you know that outcome, that can be the tool that you're exposing.

Demetrios: And then along the way, if you need to get information from the human or the agent, then you get it.

Naseem Al-Naji: Yeah.

Demetrios: And that is part of that form or those, those pieces of extra information that you wanna get from the agent.

Naseem Al-Naji: Exactly, yeah. Um, I think, like- The UI example probably has some things that aren't even relevant to an agent, so you don't like, you know, you would cut those out, but there are hidden gems in there of like core, like use cases that you could put into one, uh, nice tool call and maybe combine like three or four, and then your agent has less tools.

Naseem Al-Naji: 'Cause once you hit over 30, uh, we see like most agents start to crumble- Mm-hmm ... in terms of like success rates. Uh, and so you kinda wanna have less than 30, and if you have tons of API calls, you can condense them down into like more appropriate use cases. Um, and [00:28:00] if the use cases are not what the agent wants, you can always expand your tool list because the use cases aren't what the, what it wants.

Naseem Al-Naji: So

Demetrios: have you seen much success with just having like two tools, which is like search tools-

Naseem Al-Naji: Hmm ...

Demetrios: and then execute?

Naseem Al-Naji: Yeah, yeah, yeah. Uh, Cloudflare does this, right? Yeah, yeah, yeah. Um, we don't have any, uh... I, I think like we've only seen Cloudflare do, do it. We don't see... None of our customers have done that. Um, I think it's, uh, it's interesting.

Naseem Al-Naji: Um, I think it could work, but I think that it was a solution... It's like changing your product for, you know, a solution that needs to happen more on the downstream. Um, and so I think it's like less necessary now because Claude and some more intelligent agents are doing searching on the tool calls already.

Naseem Al-Naji: Um, and so like before then, I thought maybe like, okay, if you have tons [00:29:00] of tools, sure, that makes sense. Like Cloudflare has so many different services. I, I'm sure like that they need it that way. Um, but now, uh, Claude can actually search across tools for what it needs and, um, I don't think you really need those as much anymore.

Demetrios: What about going back to this idea of a goal?

Naseem Al-Naji: Yeah, yeah.

Demetrios: Have you thought about how granular you have MCP servers? Like, do you have one overarching MCP server, or do you break it up and you say, "Well, I know a clear set of goals, and so I'm gonna have one MCP server that's for those goals, and maybe that means I make five different servers"?

Naseem Al-Naji: Yeah, I mean, I think if you're a large organization, you need separate servers, uh, just because developers are deploying and you don't want, like, their deployments to break other services' deployments, and maybe certain teams move at a different pace. Um, and so I think you'll always want multiple MCP servers if you have, like, multiple [00:30:00] products.

Naseem Al-Naji: And that way you can, like, enable or disable them. Um, like the user can choose to, like, disable or- Mm-hmm ... enable them. Um, I think that, uh, Cloudflare's solution is very interesting in that they probably have tons of MCP servers that they're, like, routing the execute calls to.

Demetrios: Yeah.

Naseem Al-Naji: Um, so I'm pretty sure that what Cloudflare is doing is it's actually an agent behind the search and execute.

Naseem Al-Naji: Oh. And it's using... It has, like... It has this, like, very Cloudflare specific knowledge- Mm-hmm ... under the hood, but I think that's very cool. Yeah.

Demetrios: Yeah. Yeah, and that's something that, like, as you said the goals idea earlier, it made me think about, okay, well, how granular do we want our goals to be?

Naseem Al-Naji: Yeah.

Demetrios: And I didn't even think about how you want teams to own certain servers, and you wanna have that type of flexibility- Yeah

Demetrios: within the servers, yeah, and the deployments, and all that good stuff that microservices bring- Yeah ... to the table.

Naseem Al-Naji: Yeah. So it's- [00:31:00] Well, yeah, good things. I, I mean, I think like, um, yes, like microservices will, like... are definitely enabled, that will enable large, large companies. I don't know that, like, a smaller team would want them, per

Demetrios: se.

Naseem Al-Naji: Yeah. But yeah.

Demetrios: Yeah, and, and maybe your, like you're saying, like your product- Mm-hmm ... is not so-

Naseem Al-Naji: Yeah ...

Demetrios: big like Cloudflare, where you- Yeah ... need 50-

Naseem Al-Naji: Yeah ...

Demetrios: million different tools.

Naseem Al-Naji: I will say what's really interesting is last year, um, we saw- Like, kind of a shift in organizations. Mm-hmm. Um, we saw companies, their MCP servers were all one team, and, like, companies had, like, an AI team- Yeah

Naseem Al-Naji: to do the cool new stuff- Yeah ... that is coming out. And so you'd see that, oh, one team of five built the MCP servers for, like, all of Cloudflare or all of these companies, right? Um, and then what ended up happening is their MCPs took [00:32:00] off, and they realized, oh, wait. Okay, like, organizationally, this MCP server is an interface for this product team's product.

Demetrios: Yeah.

Naseem Al-Naji: Um, but the AI team made it and is main- managing it. And so now we've seen MCP servers shift in responsibility away from AI dedicated teams- Wow ... to product dedicated teams that now just need to think about AI for their product as a f- like, first principle. And so now we've seen the ownership of MCP servers shift internally into the dedicated team.

Demetrios: Yeah, it's getting closer to the source.

Naseem Al-Naji: Yeah, yeah, yeah. And so we'll talk to some teams now, and we'll say, like, "Hey," like, "Are you guys managing this MCP?" And they'll be like, "Oh, no, that's actually those guys. They manage that MCP."

Demetrios: And that's where you get more granularity and it breaks up into

Naseem Al-Naji: different- Yeah, yeah.

Naseem Al-Naji: And so now what we see in organizations is, like, there's a platform developer experience team that's in charge of making sure all the MCP servers kind of use the same properties. [00:33:00] Mm-hmm. But product teams are owning their own MCP servers, uh, as almost like the- their, like, UI for agents.

Demetrios: Yeah. And I imagine data teams have their MCP servers- Yeah

Demetrios: to sync in with the databases or whatever. Yeah. So you let the teams who know the most about the product that they're- Yeah ... exposing to the agents be the owner of the MCP server. But I, I do like the, you have a platform team, in a way, to make sure that things are a little more standardized.

Naseem Al-Naji: Yes, exactly, right?

Naseem Al-Naji: Like, they're all TypeScript MCP servers, and they all use this form of authentication for the user, and like, 'cause there are things that you wanna standardize the developer ex- like, or, like, the user experience on-

Demetrios: Wow ...

Naseem Al-Naji: uh, while enabling product teams to handle their core business logic, for sure. Yeah.

Naseem Al-Naji: Yeah.

Demetrios: Wow, okay. Makes complete sense. What el- what else is there to chat about?

Naseem Al-Naji: Yeah. I think, um, something that I think we came [00:34:00] to this conference for-

Demetrios: Yeah ...

Naseem Al-Naji: is mainly There are protocol-level discussions happening that I think affect user analytics-

Demetrios: Mm-hmm ...

Naseem Al-Naji: pretty significantly. Um, and so what I mean by that is, um, the protocol is developing ways to make, uh, running MCP servers easier for developers.

Naseem Al-Naji: Um, but there's consequences to understanding a user's journey. Mm. And so what I mean is, like As a company, you wanna know that the user came from Claude Code and, uh, asked these, like, 10 different, like... Like, tried these different tool calls in that session. But what does that mean, and how do you know that that's a user session when they have five Claude Codes and all five of them hit your MCP server, and they're [00:35:00] asking questions, uh, to your tool, uh, to your MCP ser- With different goals.

Naseem Al-Naji: With different goals. And then maybe even one of them spawns sub-agents. Yeah. Right? And, and so, um, I think that there are a lot of protocol conversations happening to understand that on, like, a network level or, like, a authentication level. Like, how do you know they're all authenticated the right way? But, um, we wanted to see how much of the protocol is actually thinking about this usability, um, and traceability for knowing that the experience is good.

Naseem Al-Naji: Um, and so there's conversations happening actually, like, right now, um, where they're, like, talking about MCP version two, uh, for the SDKs. I think, like, uh, someone from Anthropic's giving that talk. Dan,

Demetrios: yeah.

Naseem Al-Naji: I think so. Uh, I, I think, yeah, it might be Dan. And, uh, it has real implications for how that's done on a protocol level, and I am, I'm curious to see what, what they come up with.

Demetrios: They're not thinking about it as the product analytics point of [00:36:00] view.

Naseem Al-Naji: Yeah, I, I think rightfully so. They're thinking about it more from a networking and security standpoint. Um-

Demetrios: Very deep technical- Very deep technical ... like, how can we make this the best technical experience?

Naseem Al-Naji: Yes, and, like, infrastructure experience, et cetera.

Naseem Al-Naji: And, and that, that's totally where we should, that the protocol should be focused on. Yeah. Um, I just think that there are repercussions to product analytics that we just wanna make sure that whatever they're solving also makes, uh, session tracking easier and, and, um-

Demetrios: But are you afraid that it's going to totally forget about

Naseem Al-Naji: that?

Naseem Al-Naji: Or- No, I don't think so, because I think, um, even for a good Claude Code experience, it needs to know, uh, that the ways it's using the MCP server are consistent with the session itself. And so I don't think it will, like, be completely discarded, but I do think it's useful for companies to know that a MCP [00:37:00] tool call came after these tool calls f- and was spawned by a sub-agent, um, with different goals.

Naseem Al-Naji: And-

Demetrios: Hmm ...

Naseem Al-Naji: yeah, I think, like, if you want to make a good experience, you need to know the experience's structure

Demetrios: Because right now you can't tell if it was spawned by a subagent to then execute

Naseem Al-Naji: more. Yeah, right now

Demetrios: you can't.

Naseem Al-Naji: Yeah. Right now you cannot tell that it was spawned by a subagent. You can tell that it's a different, like, goal and purpose and session, um, but-

Demetrios: From the same user

Naseem Al-Naji: From the same user, but you can't tell that it's a subagent.

Demetrios: But you don't know that it was like, "All right, we hit this MCP server, we got this information, now we realized we need more."

Naseem Al-Naji: Yeah.

Demetrios: And so we spawned subagents, and then we hit it again.

Naseem Al-Naji: Yeah. To, to a company even using mcpcat, right? Uh, it would just look like that's a new goal and new tool calls were made for the goal, which is not, like, necessarily wrong, right?

Naseem Al-Naji: Yeah. But they tie into overarching goals of, of your-

Demetrios: Yeah, what that [00:38:00] intention is.

Naseem Al-Naji: Yeah.

Demetrios: And also the same... Ooh.

Naseem Al-Naji: So tracking intention becomes more complicated- Yeah ... if these network primitives change a lot or, or, or don't, like, really think about user analytics at the higher level. Yeah.

Demetrios: Okay.

Naseem Al-Naji: Yeah.

Demetrios: Man, that is so wild.

Demetrios: That is so-- It is so cool that you're working on this, and, like, it is very novel in that I've seen a lot of MCP gateways. I haven't seen MCP analytics. Well,

Naseem Al-Naji: I'm glad to be a breath of fresh air. Yeah. Uh, I mean, like, l- I think the gateways are solving a good problem. Um- Yeah ... but I, I would say, yeah, I don't know anyone else doing what we're doing.

Demetrios: Oh.

Naseem Al-Naji: Um-

Demetrios: It feels like people would resonate with it pretty easy. Like, if they have more than three MCB servers set up, have you found that it's, it's quite easy for people to understand it? Because you're effectively creating a new Like a whole new, what do they call it? Like the blue ocean. You've got the blue ocean strategy.

Demetrios: You're cr- you're creating a new product [00:39:00] category.

Naseem Al-Naji: Uh, yeah. I mean, that's a very nice, uh, visionary way to put it. I got the, got the hair and- Yeah, exactly. Uh, yeah.

Demetrios: But I- We just do hard things that nobody else is willing to do. Yeah.

Naseem Al-Naji: I mean, um, it's a very, like ... I think that it depends on the organ- organization, right?

Naseem Al-Naji: If they are, if they are viewing MCP as a core strategy, it clicks. If they are not yet and they're still experimenting, it doesn't click. Oh. Like, they are like, "Ah, I don't know."

Demetrios: Yeah.

Naseem Al-Naji: Getting our user- our users emailing us that it's broken is fine right now.

Demetrios: Oh.

Naseem Al-Naji: Right? But it change, like it's like, it's a timeline thing.

Naseem Al-Naji: Uh, for most people we talk to it's, it's like, "Yeah, we need this, it's just when we, we need it."

Demetrios: Is it easier to talk with enterprises? Because, like we were saying, you know, they have a lot of these internal MCP servers.

Naseem Al-Naji: Oh, yeah. I think internal MCP servers are, are, are like less of a value add for us. Like- Oh, really?

Naseem Al-Naji: like, well, [00:40:00] they already have, like they could talk to their employees more readily on, on what's wrong, what's broken. I mean, we're better for like, like if you have a customer facing MCP server, I mean, that's like a dif- like you can't let that experience be bad, right? Um, that's like, you know, one day a AI native product could come around that has an incredible MCP experience and a cr- an incredible agent experience, and yours sucks so you need, you, you need the data to make it better.

Naseem Al-Naji: Um, and so yeah, I mean, bigger companies for sure are, are better fits for- Mm-hmm ... for our products. But, you know, anyone that it's a core part of their strategy, it's, it makes sense.

Demetrios: It feels like you are very orthogonal to the whole movement of agent experience. Mm-hmm.

Naseem Al-Naji: Mm-hmm.

Demetrios: That I think is gaining a lot of popularity and-

Naseem Al-Naji: Yeah, it is a thing.

Naseem Al-Naji: Yeah ...

Demetrios: a lot of us wanna know, like, how can I use a product that my agent is going to be able to grok really [00:41:00] easy?

Naseem Al-Naji: Yeah. Yeah. It's so interesting 'cause it, it's kind of like now when I'm picking a vendor I kinda wanna know how good it is with Claude, right? And there's no, uh, there's no way for me to find that out.

Naseem Al-Naji: Like, there's no registry where I could see like, oh, this MCP-

Demetrios: Actually-

Naseem Al-Naji: Yeah ...

Demetrios: there is. I found one.

Naseem Al-Naji: Oh, really?

Demetrios: Oh. Um-

Naseem Al-Naji: Oh,

Demetrios: yeah. Please do- I think it's 2027.dev. They gave a talk at one of our virtual events.

Naseem Al-Naji: It's a number.dev? Yeah. Okay.

Demetrios: Yeah, 2027-

Naseem Al-Naji: Okay, okay ... .

Demetrios: dev, and they have a whole, uh, they did a bunch of research on agent usability-

Naseem Al-Naji: Mm

Demetrios: products.

Naseem Al-Naji: Yeah.

Demetrios: And so there's all these different products that are like the best ones, you know, an agent can use it within the first four seconds. The worst ones, it takes like 10 minutes.

Naseem Al-Naji: Oh, that's cool.

Demetrios: So, and they have the whole spectrum- Wow ... on how different products are, are shaping up against it.

Naseem Al-Naji: Wow, very cool.

Naseem Al-Naji: Yeah. Yeah,

Demetrios: yeah. So but I, [00:42:00] I understand what you're saying because if your product is not open source- Yeah ... it's very hard for you to just tell Claude to go and test it out.

Naseem Al-Naji: Yeah, completely. I mean, uh, like I need to sign up for an enterprise account to get access to your MCP server. I mean- I don't know.

Naseem Al-Naji: That's, that's bullshit. Yeah. Oh my God, I don't want to do that.

Demetrios: Especially if you're testing out, like, the agent usability of it.

Naseem Al-Naji: Yeah, exactly. Like I, I haven't even decided I... It's, it does a good job yet. Yeah. Right? Um, and so yeah, I think it's, it's very interesting. I'll check out this, uh, this registry.

Naseem Al-Naji: Yeah. I mean-

Demetrios: You'll like it ...

Naseem Al-Naji: you know, I don't know if we need another registry, but

Demetrios: yeah. Well, it's

Naseem Al-Naji: not,

Demetrios: it's not a registry, it's just research that was done though. Oh, it's just research. Yeah. Oh, cool. Okay, yeah. So it's not like, "Oh, hey, here's the registry." Yeah, yeah. "Use it. Use us," and then- Yeah,

Naseem Al-Naji: yeah, yeah

Demetrios: it's just, "Hey, we tested all these with these coding agents. Here's what we saw." And they did it with different coding agents- I see ... so it's not like biased.

Naseem Al-Naji: Okay. Yeah, yeah.

Demetrios: And they're, they gave a whole report on it.

Naseem Al-Naji: I see. Well, what I was thinking is, is something that is a bit more like... I [00:43:00] mean, it's something we could do, honestly.

Naseem Al-Naji: But, um, it's a, like a review system for agents and these MCP servers, where the agents report to the MCP.

Demetrios: Yeah.

Naseem Al-Naji: Like the, like the agents report to a reviewing service like, "Hey, like- This

Demetrios: is

Naseem Al-Naji: server is- ... my experience with all the tools I used and all the skills I used were like X, and I like-

Demetrios: It was mediocre at best.

Naseem Al-Naji: Yeah, yeah.

Demetrios: Imagine you have the, like, comments on there too. Yeah. It's like Yelp except-

Naseem Al-Naji: Yeah ...

Demetrios: yeah, it's like the Mope book of... the Yelp Mope book

Naseem Al-Naji: cross. Yeah. And now we look at like Yelp for what services we want to use, 'cause agents are willing to actually take the time and probably review all of them. I think it's just like what's the benefit to the user?

Naseem Al-Naji: I'm not sure.

Demetrios: Yeah. Well, if it's a more reliable way of using a tool. Like if an agent comes to me and says, "Hey, we can use Railway or Vercel," but, you know, "Railway's really cool. It does all these things." Yeah. "And my confidence score that I'm going to be able to [00:44:00] properly execute is X."

Naseem Al-Naji: Yeah, yeah,

Demetrios: yeah. "Vercel, my confidence score that I'm gonna be able to properly execute is Y."

Naseem Al-Naji: Mm. Mm-hmm.

Demetrios: Because I read on- Yeah ... you know, the Mope book Yelp. I,

Naseem Al-Naji: I do wonder if this is something that Cursor or Claude would do for their own

Demetrios: tools.

Naseem Al-Naji: Internally. Well, not, no, no, not necessarily. Oh. Like, like, like, so like let's say I'm Cursor as a company. And Cursor can have its Cursor agents do public reviews of MCPs if their users, like, adopt them.

Naseem Al-Naji: And then you know, oh, for Cursor, these MCPs- Are the best ... perform the best. And so, like, you know, that makes me... Like, that's something Cursor could do.

Demetrios: Yeah.

Naseem Al-Naji: Right? And then I know which ones are best with Cursor. Maybe Anthropic also has its own reviewing system.

Demetrios: Yeah, and it almost, like, biases towards using those tools because it knows it uses them the best.

Naseem Al-Naji: Yeah. Yeah, exactly.

Demetrios: I saw... Did you see... I thought it was super novel, but I don't think they've done anything with it. Uh, because a lot of ways that I find out about new tools is I get suggested in my coding sessions. Like, "Hey, I wanna do [00:45:00] XYZ." Mm. And it says, "Well, you can do that with these different tools."

Demetrios: And, uh, Amp was having ads be put in, so you could pay so that you- Wow ... would be suggested in those moments.

Naseem Al-Naji: I hate that. But I, I mean, like, I get it. I, I, I think it's, like, a fair thing. I think it's just, um-

Demetrios: I don't think it went anywhere.

Naseem Al-Naji: Yeah, I'm sure. Yeah, yeah, yeah. I think, like, I want to make sure that the recommendations are unbiased, right?

Naseem Al-Naji: And so if there's, like, a maybe, like, central reviewing system where, like, I as a user of a agent can, like, have a MCP server that, like, after every session, like, reviews every MCP server it used for how good it was, and that all goes somewhere, and everybody uses this, and it all goes to the same place, then it's like a...

Naseem Al-Naji: It is like a Yelp for-

Demetrios: Yeah ...

Naseem Al-Naji: for agent experiences.

Demetrios: Uh, yeah. I imagine you would have it in, like, the Google sponsored... These first [00:46:00] two links are sponsored.

Naseem Al-Naji: Yeah,

Demetrios: yeah. And they have this review that you're talking about.

Naseem Al-Naji: Yeah,

Demetrios: yeah. And then the next ones are, are internal-

Naseem Al-Naji: Yeah,

Demetrios: yeah ... SEO surfaced ones.

Naseem Al-Naji: Yeah, that'd be cool.

Demetrios: But that is... Man, the agent usability on it, I, I do like that, and the whole agent experience. To have an agent be able to go... I mean, it, it goes back to what you're saying with the what are we doing or what are we not doing-

Naseem Al-Naji: Yeah ...

Demetrios: that you want us to be doing- Yeah ... so that the experience can be better?

Naseem Al-Naji: I, it needs to be better, right?

Naseem Al-Naji: I think we have been, as an industry, like, kind of lazy. Like, we have been relying on the model labs. Like, we've been relying on Anthropic and OpenAI to just make AI smarter- Yeah ... when there's so many things we can be doing to just make their jobs easier. Yeah. Right? Like, like to make, like, the model's job easier.

Naseem Al-Naji: Yeah. Um, so I think that it's a natural... Like, we've seen in the last, like, s- six years, like, models come to here, [00:47:00] and no effort from, like, existing software- Existing, yeah ... to meet them there. And so now I think we're shifting into a world where obviously the models will improve, but we're also improving interfaces for them to do better.

Naseem Al-Naji: Mm. Do better, right? So I think MCP is like the biggest step we've done to, to, to get them there.