What an Anthropic Engineer Thinks About MCP

David Soria Parra: [00:00:00] Now we're in a state where people rely on the SDKs, where the SDKs have hundreds of millions of downloads a year or a month. Um, and that gives us a different lever that we didn't have 12 months ago. And you need to be aware of these and then use them to your advantage and ma- and be mindful around the limitations.

Demetrios: It does feel like a maturing and this is something- It definitely is

David Soria Parra: maturing, yeah.

Demetrios: Yeah. It, it's something that you're like, oh, very clear value prop there.

David Soria Parra: I need to do the minimal thing that I know I can extend so I can solve future problems additively rather than, um, subtract, like with a subtraction or removing things.

Demetrios: What do you think people don't understand about tasks?

Demetrios: All right, probably a good place to start this one, dude, and what I'm most stoked about is hearing the context [00:01:00] around the new release. What is it? What was the inspiration behind it?

David Soria Parra: Yeah, like, the... I think the new release is honestly probably one of the biggest release we have ever done around mCP. It has, um, one big change to how we are using mCP in, in the sense that, um, when we originally developed mCP over the last year, the, the protocol ha- was somewhat stateful, so you needed to keep, you know, a session open between the server and the client, and that did not work very well for some of the particularly big hyperscalers that now use mCP pretty much everywhere, that being the Microsofts of the world, the Googles of the world, the Anthropics of the world.

David Soria Parra: And so this release is really about moving away from the statefulness of the protocol to a fully stateless protocol, and we have worked really, really hard to make this, um, work in the, in the way mCP, um, operates, um, and have put a lot of work into this, and this is probably the biggest change we've ever done to the protocol, so I'm, I'm super hyped about it.

Demetrios: So [00:02:00] no more handshakes. What-- Can you go a little bit deeper into this inspiration? Because you probably saw this coming up, and then there was a point where you were like, "We need to figure out what's going on here."

David Soria Parra: Yeah. Like, the, the original version of the protocol, right, it was, like, all standard IO. It was, like...

David Soria Parra: It was a very natural thing that you have this notion of, you know, a program that you can always interact with, um, that's this mCP server. And then as we went over trying to twist over HDP, we tried to retain this because we fundamentally believed that agents are somewhat stateful. Mm-hmm. But it turned out operationally over the year, we learned that there are other methods how you can still get that without having to rely on a stateful protocol or a stateful transport, and the, the cracks kind of showed particularly when we saw, again, the really big hyperscaler try to do this with, like- At, like, massive scale, like thousands of tens of thousands of servers serving, like, millions of requests.

David Soria Parra: And then you, like, you're getting into [00:03:00] really hard difficulty problems of, like, how do you scale this? How do you, you know, move the state around? Um, there's a particular problem where, you know, you hit one server and you kind of wanna have a load balancer hit the next server, but if you have some form of state in this one server, you kind of need to communicate it to the other server.

David Soria Parra: Mm-hmm. And that's hard to do, and you can do it, uh, but it's painful. It's, um, it causes, uh, latencies, and it causes CPU usage, and it's just, like, very unpleasant to serve. Um, and so a lot of the hyperscalers really want stateless workloads so that they can go to one server, um, make a request, take the request, go to the next server, um, do, you know, something else, and everything, you know, can go through a load balancer.

David Soria Parra: Everything can... You know, some of the things can be cached. Just the standard things we kind of know how to do. Um, we kind of went there and we, again, over time figured out what are the pieces of MTP that need to be stateful, what are the pieces that don't need to be stateful- Mm ... and we decided the transport part doesn't need to be.

David Soria Parra: And so it's, I think it's a very natural, um, [00:04:00] you know, progression and evolution of the protocol. Um, maybe we could have seen some of it a bit earlier, but I think now I think it's a great time, and we have worked really hard with, again, some of the, the industry giants, honestly, to, like, make this work for them and also make it work for us.

Demetrios: It does feel like a maturing, and this

David Soria Parra: is something- It, uh, it certainly is maturing, yeah.

Demetrios: Yeah. It, it's something that you're like, oh, uh, very clear value prop there. Are there any trade-offs that you had to grapple with when you were deciding this?

David Soria Parra: Yeah, some of the hardest part is that some things inside the protocol are somewhat inherently stateful.

David Soria Parra: Uh-huh. So for example, if you send a tool request and you wanna do an elicitation, so you ask the user for explicitly for input, that is kind of bound to that tool call. Um, in the original specification, it was actually could a- at any time send this elicitation even if there is no tool call. Hmm. So we had to make constraints of like now you need to do it within a tool call, which is...

David Soria Parra: We, we did a survey, [00:05:00] um, of all GitHub proposals we could find, and basically 99.9% use it exactly the way via a tool call. Um, but it means that when you do this, um, you still need a way for like there's some form of state being held on a server while this tool call is ongoing and you are asking for input from the user.

David Soria Parra: And so we needed to build a little bit of a machinery on top of it, um, of this like statefulness approach, uh, statelessness approach, um, which we call multi-round trip, um, responses, um, which... Or requests which require you to basically go there, do a tool call, get an elicitation back, put all of this together, tool call plus elicitation result, and send it again.

Demetrios: Hmm.

David Soria Parra: So you're basically appending to the context, and you're transporting the whole context over the wire, and that just allows the, like another server can now send like response, um, without having, you know, while having the full context. So [00:06:00] there was some of that machinery we needed to build into that is less, that is a bit more unique to MCP because we have this like, there's inherent statefulness in some of the semantics that we, that are in the protocol.

Demetrios: Well, it does feel like these agents and the assumption at the beginning of this needs to be stateful because agents potentially- Yeah ... have these unique characteristics, and that got disproven in a way, if I'm understanding you correctly. Can you talk a little bit more about how that got disproven?

David Soria Parra: I, I think the...

David Soria Parra: it, it's less about it got disproven as a, as a concept. I think what we learned is that it doesn't have to be on a transport level. Mm. So it doesn't have to be on the HTTP channel that the state is open. So for example, um, the, that there is state between a tool call and elicitation. When you make a tool call and you ask the user for input, there is some form of state.

David Soria Parra: You know, the tool call waits for that elicitation to, to be done and get this additional input. So there is some form of [00:07:00] state. But what we have now done is we have taken all this state that is required and basically move it over the wire over and over again, so effectively making the, the transport doesn't have to be open all the time, but the data, um, still encapsulates that kind of state.

David Soria Parra: And so what we have learned is just to be more efficient about encapsulating and how we encode the state, um, and we have learned areas where we don't need that state. So for example, we have learned that people always want to ask for user input during a tool call and never outside of that, and that's some of the things that we have learned, and that learning over the years, that has led to this, like, new stateless mode that we call, like, multi-round trip responses or replies.

David Soria Parra: Um, and I think that's, that's really the thing. Fundamentally, there's still statefulness in agents, I think.

Demetrios: I, I do like that you're starting to see the patterns and you're starting to help MCP protocol work together with the patterns, as opposed to- Yeah ... all of us [00:08:00] figure out how MCP works and then create patterns around it.

David Soria Parra: Exactly. A second part of that learning, for example, is we have learned that when you do need some form of session of, like, oh, you call, you know, you call the tool to add something to a basket, only then you have a new tool that has, like, a bask- you know, I don't know, list my basket or whatever. Mm-hmm. Um, we have learned that nowadays, a year and a half, and part of that is the models just got better.

David Soria Parra: We-- Is that you can just give the model a s- like an ID that it just carries around between the tool calls, and so it's like an implicit session. You don't have to put it on the protocol level anymore because the m- the model will just perfectly happily take a, like an ID that you give it and then move it into every other tool call so you can piece them together that they belong to each other.

David Soria Parra: Um, and we have proven this with basically all major models, and all of them do this now well. Mm-hmm. A year and a half ago, I s- it would've been a very different story. So part [00:09:00] of it is also that the models just got better.

Demetrios: Yeah, and they're happy to carry that ID.

David Soria Parra: Yeah, they're happy to carry the ID now, yeah.

Demetrios: Uh, and that makes it much easier on your side. I could see how that will help with the, the whole designing. You go, "Okay, well, if this is true, then that makes our lives a little bit easier," and you feel more confident removing some of these pieces.

David Soria Parra: Yeah, exactly.

Demetrios: And now paint me the picture of someone who is currently using the MCP version and wants to go to the new version, what does that look like?

David Soria Parra: Uh, luckily it looks very, somewhat very easy to you as long as you use one of the SDKs. So we put a lot of effort at the moment, uh, into changing the SDK such that the, the lift is somewhat minimal, at least minimal with the help of a model. Um, there, the TypeScript SDK and the Python SDK, for example, they come out with a version two that will support this, and the changes between version one and version two are very easily done by, [00:10:00] like, an Opus model or something like that.

David Soria Parra: And so it's maybe a, a minor update, but it's not a hard lift. It's not a fundamental change. A lot of the hard pieces are abstracted away by, um, by the SDKs. If you're not-- But if you're running your own SDK, the lift is significantly higher, and I would highly su- um, suggest that you gonna go and use some of the SDKs.

David Soria Parra: Um, all of these SDKs currently kind of work towards a version two. I think the C# SDK also works towards a version two. Not sure about the Go SDK. I think they stick to a one, to still to that one zero. But most of them, again, um, have put a lot of thought and effort into making sure the upgrade is very, very minimal and basically one-shot-able with a model.

David Soria Parra: That was, like, the base requirement, um, that it's not a hard lift. So it should be fairly easy. And then similarly, if you have existing soft- if you have existing servers, um They should be able to, to, uh, to continue [00:11:00] to work, right? The protocol has versions.

Demetrios: Mm-hmm.

David Soria Parra: Um, there's a, there's a lot of sessions and a lot of, um, documentation and work in the SDKs to make sure they fall gr- gracefully back to, um, the version that your server is gonna use.

David Soria Parra: So it should just continue to, to use, to work. So

Demetrios: this can be, in a way, a forcing function to use those SDKs. If you had been thinking about bringing over some ideas and it's, uh- You should really use the SDKs. Yeah. Just use it. Make your life easier. Yeah. You don't need to impress anyone by-

David Soria Parra: Exactly.

Demetrios: Exactly ... using. Oh, and- Yeah ... the other PSA is don't forget to port everything to Rust before you change it to the SDK- Of course ... because apparently.

David Soria Parra: Of course. Always. Always.

Demetrios: It-- Everything runs better after it's been ported to Rust, apparently, from the research. Yeah.

David Soria Parra: I-- But, uh, you just, uh, just give it to, to Fable, and it will just do all of it for you-

Demetrios: Exactly

David Soria Parra: all the conversions. Yeah.

Demetrios: That's what I've heard. Uh, and it's so wild to think like, "Oh, take it, port it to Rust, and [00:12:00] then you can port it back to the same language, and it will perform better." So.

David Soria Parra: Yeah. Maybe. I don't know. I have to try.

Demetrios: Yeah, yeah. There, um, there's the rumor going around, so we need some hard details and science-backed proof on this.

Demetrios: But until I see that, I'm still gonna be per- perpetuating the rumor. Uh-

David Soria Parra: Okay ...

Demetrios: anyway, there's a few things that I wanted to go into that are more theoretical and less based in the protocol, and that's, like, what you've learned. You've been manning the ship, and there's a great maintainer team that is also working on this.

Demetrios: But there's a lot of precedence for open source projects over the years.

David Soria Parra: Yeah.

Demetrios: Are there things that you feel like the MCP protocol can learn from all of our past open source winnings? And this one comes specifically, I asked the community for questions, and we got some incredible questions. This one's coming from a question from the community.

Demetrios: They're wondering, like, [00:13:00] what are you gonna take from HTTP and all these other protocols that we've seen in the wild?

David Soria Parra: I think there, I think there's always something to learn. I think you can always take a look, um, and sometimes you learn things you wanna repeat, and sometimes you learn things you do not wanna repeat, or they don't apply to you.

David Soria Parra: It's less often that they're wrong. It's mostly that they might not apply to you. Um, so there- I think there's a few things. The, the first thing is on the standards, some of the big challenges is the, the standards. Most successful standards work, um, in a way that they take a long time to be built. Mm. Um, and they're mostly consensus-driven.

David Soria Parra: That's like HTTP is some of that, a lot of the, uh, authorization work. Things that come out of the IETF in general, for example, are very consensus-based. Consensus-based mechanisms are very, very slow. They, they take a long time. [00:14:00] Um, but at the same time, they often... Depending how you do it, they result in usually solutions that are quite good.

David Soria Parra: They're not perfect- Yeah ... but they're quite good. Um, HTTP 1.1. HTTP 1.0, interestingly enough, did not work well. 1.1 is the one everyone's using, right? Yeah. And I think we keep forgetting that quickly. 2.0 is also quite well done, um, but then even they already had shortcut, uh, comings, and now they're at 3, right?

David Soria Parra: And you can learn some of the carefulness and considerations and being sometimes more on the conservative side and saying no to things because you know that two things can happen. First of all, you can change things and add things later. That's happened to, you know, HTTP 1.1. I am sure during the process of HTTP/2 people have known that there will be shortcomings that they are currently putting into the protocol, and yet they continue because it is good enough, [00:15:00] and it is within the timeframe-

Demetrios: Mm

David Soria Parra: that they have, and they know that version 3 a few years later might, might change it, and that's okay. Standards are not there to solve everything. Just they're there to solve most problems. Um- And so I think there's a lot lessons to learn in that regard. There's a lot lessons to learn in, like, how you run a successful open source project and, like, in a governance and, like, a, a general sense, and there are many, many ways to do this.

David Soria Parra: You can run it in a BDFL style, like, like- Mm-hmm ... the Linux project. You can run it in a steering committee style, like the Python community. And they all have their drawbacks and benefits, um, and I think what we have learned here is that I think for us, like having a, a strong core maintainer foundation, so a set of, like a more of a steering committee type, um, plus a BDFL on top, has worked for the most part quite well.

David Soria Parra: I think there are things that we will change this year to make it even more democratic and allow more as the m- the project is [00:16:00] maturing and, uh, the structures are maturing, um, like being able to be a bit more, uh, be even a bit more democratic.

Demetrios: Mm-hmm.

David Soria Parra: And so it's always, like, a matter of, like, you look at these projects and like, okay, you know, an ITF project, um, chooses democracy over everything else and consensus, and as a result it's slow.

David Soria Parra: Um, a project like, um, I don't know, uh, take something like, um, I don't know, like a, like a famous new open source project like Ghosty, like with a, with a clear one leading person was Mitchell, right? Is very fast in, in certain ways because it's just one person making a decision, right?

Demetrios: Mm-hmm.

David Soria Parra: Um, and then there are mixes, like the Python project, and you choose the thing for the stage you're in, and I think that's the, there's a good lesson to be learned there.

David Soria Parra: Um, so for us, the interesting part is finding that middle ground of, like, we wanna be fast, we wanna be somewhat democratic, but we also wanna m- we also wanna be considerate, uh, and be somewhat conservative. But at the same time, you know, we also work in a, in a, in a slightly different environment because we're working [00:17:00] in software where, um, things can change quicker than, for example, if you like pr- creating, like, a hardware standard or a telecommunication standard where someone puts the standard on software onto a box, puts the box 200 kilometers into the north of Sweden, and it's gonna sit there for the next 10 years.

David Soria Parra: You have a very different set of problems than if you build MCP, for example.

Demetrios: I like that juxtaposition, uh, because in the real world, yeah, the atoms take a whole different route for the standards. And it also is cool to hear you say, like, you're choosing this path right now because it is what you feel the most optimal route, but in the future once things stabilize even more- Yeah

Demetrios: than they already are, you're open to taking on new ways and trying those out and seeing, like a new pair of pants. Do they fit? Do they, do I look good in them? All right. Yeah. Cool

David Soria Parra: I, I, I have a very clear picture in my mind of, like, where this is gonna go in a year or two years, and how this needs to evolve on a governing [00:18:00] perspective.

David Soria Parra: Mm. Because it's pretty clear that I think this model is, is, is a, is a temporary artifact in time for the current situation the protocol is in.

Demetrios: Mm.

David Soria Parra: But that will not be the situation the protocol is in in a year from now.

Demetrios: Yeah.

David Soria Parra: And then it requires different governance.

Demetrios: Yeah. And I did wanna hit on something else that you mentioned, which I have found to be true, especially back in the MLOps days, is the sooner you can get something out there and get folks using it and testing it, the sooner you can realize where you had gaps in your thinking.

Demetrios: And so it is, like, that speed that you're looking for is a feature, not necessarily a bug in that regard.

David Soria Parra: That makes sense. I, I agree. With some constraints, right? The constraint is, like, how flexible is the environment? Mm. And again, I think the environment in which MCP is at is, um, exceptionally flexible, because people around AI do two things.

David Soria Parra: They first of all assume an [00:19:00] exceptionally high velocity of change, and they're using models that help them to retain that high velocity. So some of the burdensome work is less burden- burdensome. And so those are things that are cultural pieces that you need to take into account when you design the, the speed that you're, you're going at.

David Soria Parra: Um, this is, for example, not true for programming languages per se. In programming languages, um, you know, particularly older programming languages or certain niche programming languages, you might have less adoption of AI. You, you might have, um... You might not want to change your code constantly, um, because, uh, of changes and, and because you just wanna get stuff done.

David Soria Parra: And so there, there's, mm, um, there's carefulness to be considered here, and even we, I think, have limits, right? Like, when we did the protocol changes, and now where we're doing the SDK changes in particular, [00:20:00] we're paying a lot of attention that the delta is small enough to be done- Mm ... because we don't want to jump into the, um, problems that, let's say, like, a Python 2 to Python 3, um, change had or- That

Demetrios: was

David Soria Parra: painful

David Soria Parra: there's, there's other aspects or, you know, IPv4 to IPv6. There's a bunch of these type of things where they are very cumbersome and difficult to do, and you need to be careful that the delta is small enough that, that the net benefit outweighs the friction.

Demetrios: Are there breaking changes that you feel like may happen in the future?

David Soria Parra: I never know. I think we are getting into a shape where it's, a lot of the things are additive. Um, the, the problem at the moment, and the thing we're gonna see now, is that the complexity of the protocol will increase.

Demetrios: Mm.

David Soria Parra: Um, by virtue of we're trying to We are probably at like an 80% solution space, so like 80/20, where 80% of the problems people have in that space are solved and 20% are [00:21:00] not.

David Soria Parra: And if you-- it turns out if you push the boundary beyond these 80%, the last 20% are actually like 80, like 80% more complexity added, right? Yeah. And so- 80 times. Yeah ... it's, it's disproportionately more complexity, right? We're adding caching, we're adding caching scopes, things that look very similar to things that exist in HTTP.

David Soria Parra: Mm-hmm. At which point you need to ask yourself, okay, at which point are you just gonna rely on HTTP, right? Um, and we have made some choices a year and a half ago that make this a bit more tricky, and so there are, there are decisions to be made, um, down the road. But most importantly it is, it means added complexity.

David Soria Parra: When we originally designed the protocol, we were mostly concerned and mostly obsessed with adoption because that's the thing that makes or breaks a protocol. And the key principle for the very first version or the first six months of MCP was that it's very simple to implement to the point where we constantly tested that if you just give it an SDK or if you just give it even the specification, uh, a model will write you happily [00:22:00] an SDK in that language.

Demetrios: Mm-hmm.

David Soria Parra: I mean that was true six-- and in the first six months. I don't think that is necessarily true anymore because the complexity has increased. Um, but that's okay because now we're in a different state. Now we're in a state where people rely on the SDKs, where the SDKs have hundreds of millions of downloads a year or a month.

David Soria Parra: Um, and that gives us a different lever that we didn't have 12 months ago. And you need to be aware of these and then use them to your advantage and ma- and be mindful around their limitations.

Demetrios: Hundreds of millions of downloads a month there is unreal just as a stat that you kinda glossed over. And that is a great segue into I think this philosophical battle that I'm sure you're going through day by day as you're looking at what to add as an extension versus bring on as a core.

Demetrios: And one of the questions that the community asked me too to ask you f- well, it comes from Manvika, and she was asking: Where do you draw the [00:23:00] line between something that is core versus extension, especially with the idea of simplicity in, like, that you wanna touch on?

David Soria Parra: There's a few things. I think, I think it's a very good question.

David Soria Parra: Um, it's an aspect that we very rarely explain, but I think make, uh, there is some very thought- there's some thoughtful thinking behind this. Um, what the c- the, the, the principles that need to apply to core are it needs to be composable, it needs to be so that, for example, like a res- a resource, uh, in MCP is a fairly general, like remote file system approach, um, that perfectly composes nicely with tools as an execution approach and so on.

David Soria Parra: Um, that's one aspect of that. The second aspect of that is it needs to solve, like serve all clients and all servers at the same time.

Demetrios: Mm.

David Soria Parra: I'll give you an example. MCP apps is a great thing, but it only, uh, works in surfaces that have [00:24:00] browser or browser rendering it, which is very constrained, right? Mm.

David Soria Parra: Because it re- an MCP app renders HTML, right? Yeah. And so a terminal will never be able to look at an MCP app, and that's okay, but it means that it's not part of core because you're leaving out a large chunk of the MCP client, uh, clients out there. And so there's these kind of consideration that we're having that is, that is for what belongs into core.

David Soria Parra: The second thing that belongs into core is concepts that we are very confident will stay very stable.

Demetrios: Mm.

David Soria Parra: Um, so one of the best example of this is somewhere we maybe made a, maybe not a mistake, but like we were, it was as a sequencing issue, um, is MCP tasks. MCP tasks is this idea of long-running tasks or long-running tool calls, which allows you effectively to do agent to agent communication over MCP, which I think is a massive unlock.

David Soria Parra: I think it's a huge thing that will be super good, super widely [00:25:00] used in the long run, um, and will have actually in the long run a lot of client support. But the first iteration of it had issues, and it was part of the c- as an experimental change to the core, and it turned out we are much better separating it from the core into its own extension where it can be developed at its own pace independently from the release cycle of the core.

David Soria Parra: In there, like basically hit rate fast enough to the point where we are back being able to move it into the core because then we have confidence that it works the way it works. And the extension is already the now MCP task extension is significantly, um, different from the original experimental core piece that we have now moved out.

David Soria Parra: Um, and so we wanna basically just like let it cook. Like let it cook in the extension space. Let it, let people iterate, let people play around with it, and once we were confidence in the design, we're gonna put it back into the core because that is one of the pieces [00:26:00] that- Does fit nicely into the like, um, compo- like it composes nicely with the rest of the, the primitives

Demetrios: Do you feel like that's a nice pattern?

Demetrios: Let it be an extension, kind of prove out its value, and then merge it into core? Do you see that being- For

David Soria Parra: potential core pieces, yes. Yeah. There are pieces like, um, MCP apps or skills around MCP that will always stay extensions.

Demetrios: Yeah. But then there's other pieces that potentially could be core, but instead of doing it like you did with- Yeah

Demetrios: tasks, you can let it stand- Yeah ... on its own two legs first, and then go Yeah, that makes a ton of sense. So I wanna go into a few more of these questions that are coming through. By far, the most asked question is about the progressive disclosure. Can you talk to us a bit about that?

David Soria Parra: Uh, yeah, progressive disclosure I think is an interesting case.

David Soria Parra: So what people... I'm curious what your take is of what people actually mean by that, because, uh, first and [00:27:00] foremost is like what do people, what do people actually think about what, what does this mean? Um, progressive disclosure, most of the time I've seen it being mentioned, it refers to this notion of making sure that the model learns about specific of a tool only when it needs it- Mm-hmm

David Soria Parra: because otherwise if you put all the tools into, um, into the context window, you get context bloat very quickly. And so it's, it's less that the people are worried about like they want progressive disclosure, they just think it's one solution to the problem of context bloat.

Demetrios: Exactly. I wholeheartedly agree with that.

David Soria Parra: Right. And so context bloat is, um... context bloat is a, is a, in my mind, an interesting problem because there's a separation in my mind what does the protocol do versus what does the client do. Hmm. The protocol transports information. It transports information about the tools [00:28:00] available. The problem we have seen is that naively most client implementers take whatever comes over the protocol and just slap it into the consti- con- context window, and that is not necessarily, uh, the right approach.

David Soria Parra: What you should do is you take the protocol as an informative piece of like these are the tools available, but I then choose techniques on my side, on the client side, that are avoiding this context bloat. And the two mechanisms that we found exceptionally useful and basically use everywhere, Cloud Code does this and basically all, um, Anthropic products do this, is using tool search.

David Soria Parra: Okay. So like instead of putting the tools into the context window right away, we are in, um, we are putting it into a effectively an index, uh, and the model has train- it's been trained to like search for certain tools when it thinks it pro- [00:29:00] probably needs a tool Um, and that works quite well. Um, and it basically reduces your token usage as you would expect, right?

David Soria Parra: Like it's the exact... It's basically effectively some form of progressive disclosure, um, or discovery from the model side.

Demetrios: Mm-hmm.

David Soria Parra: The second part of that is programmatic tool calling, which is like how you combine or compose tools together.

Demetrios: Mm-hmm.

David Soria Parra: Um, norma- normally the traditional old school way is the model runs inference, it spits something out, it tells you to recall a tool, you write, you call the tool, you give back the tool results to the model.

David Soria Parra: Now the model says, "Oh, thank you for the input. I'll call this other tool." And so what the model effectively doing is it's composing tools together, um, in a quite inefficient manner because it does it via inference. And what you can do instead is you can have the model, um, optimistically write a piece of code that says, "Take this," you know, a JavaScript block that says, "Call this tool, take the output, parse it into JSON, put this over there into this other tool, and then give me the output," right?

David Soria Parra: And so you have two tool calls put together in a [00:30:00] script, and you execute that. Um, and that saves tools as well. And these two things combined are very, very, very effective, like tool search plus, um, programmatic tool calling-

Demetrios: Hmm ...

David Soria Parra: uh, to basically get rid of tool, um- T- uh, bloat. And so that's a long-winded way to say that I think n- 80% of the problems are client implementer problems, that they're not doing a good enough job.

Demetrios: Skills issue.

David Soria Parra: That being said, I think there is an argument that we're slowly coming around where it might be useful in certain situations where the server offers, or the server authors knows better about the shape of the problem, so it might wanna offer some form of search or some form of progressive discovery or disclosure.

David Soria Parra: And so we're thinking about some of the primitive. But fundamentally, it's pr- ma- to some degree a client problem.

Demetrios: I am very proud of myself because in my head that's kind of what I was thinking. I... Especially on those two where it's like, yeah, well tool search, I [00:31:00] think Cloudflare was the first to popularize that, right?

Demetrios: I, I can't remember who exactly-

David Soria Parra: Uh, tool search, not that. They did something called, uh, code mode, which is what the other word for programmatic tool calling.

Demetrios: Okay.

David Soria Parra: That was- Um, we did it, we did it around the same time. I think Cloudflare did the popular blog post about it.

Demetrios: Yeah. I knew- And thing, yeah ... one of those w- I s- remember seeing that and thinking, "Yeah, that makes sense," right?

Demetrios: And- Yeah ... especially because I had h- been hearing murmurs in the community of folks saying, well, you wanna just bring the abstraction layer up so you don't give them 90 tools and say, "Here is Git-" Yeah ... "Slack ID, here is send Slack message." It, it's like you wanna just figure out what's the intent and create that from the intent as opposed to- Yeah

Demetrios: a million tools, and then you gotta choose one by one. You have one tool that can cascade down. And the search is... It's kind of obvious because that, in a way, is very much progressive disclosure, just a different way of doing progressive disclosure, right?

David Soria Parra: And we have seen, without going into too much details, we have, [00:32:00] we have seen that tool search plus programmatic tool calling can be as good, if not even better than, for example, CLIs.

Demetrios: Yeah, I believe it. Uh, I believe it 100%. And, you know, I think the reason that folks felt like they were getting this bloat in the beginning was because there was so many community-built servers that just went up so fast, and you would go and try and use them, but it was, they were built so poorly without any idea of how to do any of this.

Demetrios: And so again- Right ... it goes back to that maturity. If I used an MCP server that was built poorly back in the day, of course I'm gonna be complaining. And GitHub was the notorious one, uh, who did this the worst because they just- Right ... threw all the tools into the tool, and it was, so that was what you were in.

Demetrios: Exactly.

David Soria Parra: Yeah. And one part of that is like, uh, the, the classic is like take an OpenAPI spec and like transport it, like transform it one-to-one to-

Demetrios: Yeah ...

David Soria Parra: MCP, and that's, that's [00:33:00] fundamentally an anti-pattern- Yeah ... um, because it just leads to poor, and to, to very poorly performing MCP servers that also have the problem that the tools look similar enough that the model gets very easily very confused about- Yeah

David Soria Parra: which tool to use. So it's a bit of a, it's a bit of a di-these, uh, you 100% right, like there, um, it was amplified by poor servers, and it was mostly amplified by poor clients.

Demetrios: Yeah. Yeah. It got, it-- You just didn't realize it until later that, oh, this is a skills issue. This is not the protocol necessarily that's causing these problems.

Demetrios: It's just that folks who didn't really go deep into the protocol and figure out how to build it properly or leverage what is there, like this search.

David Soria Parra: And part of it is also learning, honestly. Like, everybody learns on it, right? Like, when MCP came out, like, how many tools would you, were you using in a session?

David Soria Parra: Like, 10?

Demetrios: Hmm.

David Soria Parra: Right? Yeah. Six months later, 12 months later, MCP led to pro- tool proliferation in the ecosystem, right? And so some of the solutions are needed because of [00:34:00] MCP being part of the amplifier of tools, and so it's a bit of a chicken, egg type of problem, so I'm not worri- I'm not, I, I don't, I'm less worried about that.

David Soria Parra: I think the client author didn't do it right from the beginning. Completely normal. We all learn. We all do iterative work. I think what I'm, what I wanna make sure is I think a lot of people don't realize that the situation gets significantly better and most clients don't have this issue. At least most major clients don't have this issue anymore.

Demetrios: Speaking of which, we talk a lot about, like, this forward compatibility. You know, in software you have backwards compatibility. That's a thing, obviously, but I've heard people talk about forward compatibility, especially as you're building and knowing, all right, there's certain things that I can build this right now or I can wait six months and the model's probably gonna be able to do it on its own.

Demetrios: And the classic examples of this are back in the day when folks would try and do all these hacky moves to make the context window bigger. Or I know people in, uh, [00:35:00] certain e-commerce stores who built their own MCP because they didn't have that, and it's like, wow, if you just waited a few months, MCP came out, it became popular.

Demetrios: So as you're looking at ways that things could now be forward compatible or as you're thinking through, like, what's going to get better Where do you plan on, okay, this can get better. Maybe you can build it right now or maybe you can wait six months and y- at the rate of change, it wouldn't be a far stretch to say that this is going to be done.

David Soria Parra: I, that's such a good question. I think there's a lot of these kind of scenarios where you need to be so careful, and some of them you will get right, some of them you will get wrong, right? Um, like the session piece we kind of got wrong, I guess, in retrospective- Hmm ... right? Or there, it was a temporary thing.

David Soria Parra: Um, but another example is like people often ask us to do something what we, what they originally called primitive grouping, um, which is like add, allow tools to be grouped, right? And that's super sensible. It's a, it's a very good [00:36:00] use case. There's lots of use cases around this. But if you think forward you're like, okay, first of all, there's two things to this.

David Soria Parra: First of all, the models will get really good and people will not select tools as a group in a, in a session anymore. Six months ago, 12 months ago, you had to select tools in, you know, UI. Nobody does this anymore. Nobody expects us to do this anymore. And so these things will just go away to some degree, right?

David Soria Parra: At the same time, the models will get better to reason based on the tool description about the grouping that's there in the first place, that you don't have to spell it out as a, as a human. Um, or that the group might not be relevant to the, the problem that the user, that the user has, and the, the model might find a different grouping mechanism.

David Soria Parra: Um, and then the second part of that is like, um, a group is a, is nothing but like a tree with a, with a lay, like with a depth of one- Yeah ... effectively. Yeah. So there's a generalization there that you need to be very mindful. When you design protocols, you need to think about like, does this generalize? Does it solve a problem?

David Soria Parra: It's never, it's, it's often the case [00:37:00] is yes, but the problem is like, is it solving all my problems in the future? And the answer seems sometimes no. And I need to s- that's the, the hard one, and then you need to go back and like no- The-- Let's wait. Let's see if the model gets better. Let's see if- Mm. Um, how the world looks like in six months.

David Soria Parra: Let's see how we're, um, you know, if we find a more general mechanism, even though it takes us three months more, and it's painful for the people who wait for the solution in the meantime, and I understand that. But that's the, that's the kind of role that you would where forward-looking is super important.

David Soria Parra: Um, and I think these things, um, are super interesting. But there are, like, things that are pretty clear that they are on the horizon, right? Like, we know that the task, the length of a task that a, a model does is increasing, right? And it's kinda like linear increasing, and we know this, and we have-- that, that hasn't stopped really since the years this increased.

David Soria Parra: So we know that, um, like more long-running tasks will just increasingly more important because we will [00:38:00] increasingly less interact with a model in a chat environment because the model will-- like, because the task the model will do will be more asynchronously how we interact with humans, and they go away and do something and come back.

David Soria Parra: And so in that scenario, you're like, "Okay, I will need something like tasks," right? Like, I will need something like that. I need to design it minimal and make sure it can be extended, um, because I need to look in the future that I cannot predict, right? Nobody knows what's happening six months from now in the AI field.

David Soria Parra: Um, so I will not know it either. So the all know-- I know is I need to do the minimal thing that I know I can extend so I can solve future problems additively rather than, um, subtract, like with a subtraction or removing things.

Demetrios: What do you think people don't understand about tasks?

David Soria Parra: Uh, I think Taas is a poor name.

David Soria Parra: Um, it's a poor name and a good name. It's a good name because it's accurate in what it's trying to do. It's poor- Naming is so hard ... in that it does agent-to-agent communication effectively. Um, and um, I think there's some [00:39:00] open questions there, um, around is this the right format? What are some of the shapes?

David Soria Parra: So we're experimenting around with it, but I think people really don't understand that what it effectively allows you to do is have, uh, an agent talk to another agent. That, that's effectively what it's doing. I think it's doing it quite effectively. Um, but again, um, in the long tradition of MCP, we are in- poor at naming things.

David Soria Parra: So we, we shall continue, uh, in our old tradition of continuing to name things poorly.

Demetrios: Dude, you know, so I'm not gonna comment on the naming of MCP 'cause that's your business, and I will let you have your fun with it. But I will say the folks who name things the best by far in my experience are the damn researchers.

Demetrios: Follow me on this one. Hallucinations with models, right? Yeah. And then you have... What else do you have? Catastrophic forgetting of models. Give me a [00:40:00] better name.

David Soria Parra: Uh, maybe we need a bit more drama.

Demetrios: Exactly. A bit more- Catastrophically forgotten- Yeah, yeah ... that one. Oh my God.

David Soria Parra: Yeah, yeah. So- Yeah, yeah. Maybe, maybe.

David Soria Parra: Maybe we should go and, uh-

Demetrios: Next time you have a big naming conundrum on your hands, just go walk around the office and find a few of the researchers and say like, "What would you call this?"

David Soria Parra: I love it. I love it. I, I should do that. That's a good idea. We are, we are certainly... We are not, we are good at a lot of things.

David Soria Parra: Naming is not one of them, that's for sure.

Demetrios: But it is hard, dude. It is so... I- It

David Soria Parra: is hard. It is

Demetrios: hard ... vibe with you 100% on that because you're like, "Yeah, that sounds good." And then somebody comes up and they're like, "Well, does that mean this?" And you're like, "That is not at all what I thought it was gonna mean."

Demetrios: Yeah. Uh, how did you get that? Also

David Soria Parra: think where you start and where you end up- Yeah ... these are two different things. Yes. And the name ha- does not change while the expectation has. Yeah. And so there's a, there's a fundamental issue there as well.

Demetrios: Oh. Well, all right, so I wanna shift towards JSON and token usage a little bit because- Yeah

Demetrios: a question [00:41:00] came through about JSON, how verbose it is, how many tokens it uses. Yeah. Do you feel like there's a world, and of course this may and is probably stepping outside of MCP world potentially, but is there a world where JSON is not there, or there's something more efficient?

David Soria Parra: I, I do not know. I think the, the...

David Soria Parra: So there's two things from the, on the, um, what is, what is MCP using as a transport mechanism?

Demetrios: Mm-hmm.

David Soria Parra: And that's independent from what you give the model, right? You can take a JSON blob, and w- if the model accepts something else than JSON, you can transform them as long as it's a bidirectional mapping, which it should be.

David Soria Parra: Um, it should be fine. So what you get from MCP must, must not necessarily what you give to the model.

Demetrios: Yeah.

David Soria Parra: What you give to the model, I do, I do not know what the, what the most recent research about this is. Um, I do think JSON is- Has a lot of benefits. It is actually quite, um, [00:42:00] quite, um, compact by itself.

David Soria Parra: Like, people forgot the world where XML was a big thing. That was verbose in comparison. So, um, I think JSON's actually quite compact. Um, I do think that, um, I'm, that I'm usually looking at things about less about, like, token usage, but, like, how much tokens did I need to get a task done, like, that I, that I wanted to do.

David Soria Parra: Um, and I do think, like, while tool usage might be, you know, might be not the perfect most efficient thing to do as, as we do it now, I think it's probably very good enough given that the, that the in- that the cost per task done is, is incr- is decreasing as the models are getting better, right? Right. Like, um, and so, um, I'm less worried about it.

David Soria Parra: I don't honestly really see it as a, as a massive issue. Um, I think actually that, you know what, what, what you can foresee a world where instead of, [00:43:00] like, using JSON as a, a tool-calling mechanism, you just write code that you exec- that the model just writes code that you execute. That's another idea in that field, and then your effective format becomes, I don't know, JavaScript or whatever you might choose.

David Soria Parra: Um, and that is also maybe not the most efficient thing, but it is the thing that the, the model is really pretty good at because it turns out there's a lot of JSON- On the world ... in the world. Yeah. Um, and so- Again, this is a good research question. I think it's, it does not feel to me like a particularly high priority in the savings you're getting from-

Demetrios: Hmm

David Soria Parra: being smart about your context selection, about being smart about, um, the way you execute things, the things using tool searches and tool and programmatic tool calling. I think those in general are probably bigger savings in my mind intuitively, um, than a specific formatting questions, to be honest.

Demetrios: And I do like this idea of, hey, just because the transport layer uses it doesn't mean you can't use something else.

David Soria Parra: Oh, yeah, yeah, yeah. But this is the thing, right? [00:44:00] MCP trans- like that's maybe one of the biggest misunderstandings for MCP. People think it's like that's what the model interacts with. That's not true. It's what the application around interact with. It's a pure, it's a very, um, old school program A talks to program B in some form of information, and that information does not need to be the same thing that goes into, um, the tool calling.

David Soria Parra: It turns out that the format that we chose is clearly the same format that like an OpenAI or Anthropic API would expect to do, which is very convenient for people to do, but it does not stop you to experiment with other things. Um, so anyway.

Demetrios: Let's hit on observability, because this question came through and I, I really liked it.

Demetrios: As far as like as agents start using multiple MCP servers and then they're chaining the different tool calls, debugging becomes difficult. [00:45:00] You had mentioned earlier about the ID session. That potentially feels like it could be a little bit of traces happening, but incorrect tool selection might be there also.

Demetrios: You have MCP server failures. Uh, do you see a world where you're standardized tracing, or is this again something that you're like, "That shouldn't be on us. That needs to be something else"?

David Soria Parra: Two things to that. Um- First of all, I think in a, in a more general tracing mechanics, I don't think that's an MCP problem, I think because there are aspects within an, the execution of a, of an, um, of a model that are not MCP that you still wanna trace.

Demetrios: Yeah.

David Soria Parra: Um, and so those would be different things. But at the same time, um, within MCP, there's OpenTelemetry standards, um, that you should be using. And so there are some of the, the things that, that we can do and, and should do. But again, they're on the MCP layer, and the MCP layer is just part of the general [00:46:00] execution of, of like a model loop and not the full loop.

David Soria Parra: And for that, um, you know, different model providers offer, uh, different abilities to introspect this or write trans- transcriptions or something like that. And that really depends. I... If... I'm not sure if there will be an open standard. I don't know if there, there should be. If anything, I think should be something out of something like an OpenTelemetry-

Demetrios: Mm.

David Soria Parra: To some degree. I'm not sure if that's enough, if there might be more additional information needed. There's always a challenge here, particularly when you're dealing with raw, unstructured text, which effectively models are, is Being careful about privacy and, um, data retention implications, um, while being useful at the same time, right?

David Soria Parra: And it, the, the most useful tracing you can do is trace everything, right? Yep. That, that, that would of course include things that you probably don't wanna trace for the sake of your users. Yeah. And so there's trade-offs to be done. Again, on the, on [00:47:00] the MCP side, um, there's a lot of OpenTelemetry work around.

David Soria Parra: The OpenTelemetry people have some form of standards around how you trace MCP servers and, and a lot of the, the, the typical server, uh, and client SDKs do have traceability functionality in it. On the general thing, I do not know if there should be an open standard or not. It's something that's not top of my mind.

David Soria Parra: It could be.

Demetrios: Yeah. It's outside of your scope. You-

David Soria Parra: Yeah ...

Demetrios: you just need to focus on naming properly, and that- Exactly. Yeah ... is gonna get you way farther.

David Soria Parra: I have enough issues ahead of me.

Demetrios: So when will MCP be done? Is there ever a finished state? How's the vision and mission set for that type of thing, or is it continuously going to go?

Demetrios: Do you feel like you're gonna be able to put your feet up and relax at any point if it gets to certain place?

David Soria Parra: I, I think there, there's a world where the core part of it will be kinda done. [00:48:00] I, I think it's actually not that far away. I think there, the, there's things we need to do, but, um, the general core pieces seem to get into a reasonable good shape.

David Soria Parra: Mm-hmm. The, it feels to me it might relate similar to how you see a lot of, um, um, standards, like for example around authorization, others happen where there will be a, um, a small use case that someone has and they will wanna build an extension, uh, to the protocol, and then they will make this a thing, and then that's an important thing to do.

David Soria Parra: So there's a lot of, there's a lot of like in the last 20% of use cases, there's a lot there that still needs to be done. Those will likely be done via extensions.

Demetrios: Hmm.

David Soria Parra: Um, and I think that has-- I do not see that to slow down. There are so many open questions like how do you deal around guardrails? How do you deal with the compliance problems and regula-regulatory problems?

David Soria Parra: There's, there's, um, [00:49:00] there's a extension called Interceptors that is, uh, proposed by, by the financial interest group, uh, the MCP project, which is mostly con-- like consists of the Bloomberg, the Capital One of the world, who all have, um, very interesting, very valid problems. Hmm. Um, and I think that will-- these kind of senses will continue.

David Soria Parra: There will be, you know, you can s- you can for-- I do not know, but like for example, you can think about like an extension that will do something like A to UI in MCP where you are allowed to do k- uh, um, AI components that can be rendered in a terminal. So there's a lot of these use cases-

Demetrios: That's awesome

David Soria Parra: that I think can happen, but, uh, I think they will happen in an ex-extension type of way where the core hopefully should be stable.

Demetrios: Mm-hmm.

David Soria Parra: That's the goal. Like, uh, protocols like MCP thrive from, um From stability and most importantly protocols like MCP have a budget of how willing the ecosystem is to [00:50:00] change, and that budget is pretty much directly correlated with the amount of attention the project has.

David Soria Parra: And the project has a lot of attention year one, it has more, some attention on year two. It will have less attention year three because it will just be a standard part of the stack. And that's fine, and that means that people do not wanna change the standard part of their stack because they're thinking about something else.

Demetrios: Incredible, man. Well, this has been great chatting with you. I will say that you're gonna be giving talks at the MCP Dev Summit- Yeah ... that we're doing, right? You're coming to Japan.

David Soria Parra: I'm coming to Japan, I'm coming to Amsterdam I think the week after or a few days later, and then I'm gonna be in October in San, uh, San Jose at the, the big AsianCon.

Demetrios: Well, it's great- Are you gonna be there? ... chatting with you. I'm gonna be at all of those. I'm gonna be, I'll try and corral you when we're there and-

David Soria Parra: Are you gonna to even go to even more? Are you gonna go to the SEO- the Seoul one or the

Demetrios: Nairobi one? Korea, yep. I am going to Korea. I'm very excited for that- Korea's good, yeah

Demetrios: 'cause I've never been to Korea.

David Soria Parra: I'm pretty sad I missed out on Korea. I'd really like, I'd really like myself some, some Korean food and, like, watching some, [00:51:00] some esports live and stuff like

Demetrios: that. Exactly. I wanna watch robots. Literally just saw as I was boarding the plane to Lisbon that robot soccer has become very popular now because- Oh, really?

Demetrios: of the World Cup, but Korea- That's awesome ... is not playing anymore in it, so they're enjoying robot football. So, um- That's

David Soria Parra: pretty good.

Demetrios: Yeah.

David Soria Parra: I've been to Korea before. It's a great place, man. I'm, I'm hoping you're gonna enjoy it.

Demetrios: Yeah, that should be fun, and we'll hear how people are using MCP over there, so that should be fun.

Demetrios: And then the big release is coming out, so folks, let us know what you think, and we will, uh-

David Soria Parra: 20, 28th of, uh- Of July ... of July comes the release, and then with that the, the version two of all the SDKs, which I think is the, the thing that for developers matter the most.

Demetrios: That's

David Soria Parra: a big one. Um, yeah, and I really hope that everybody's gonna operate their MCP servers, give us feedback, come to the community.

David Soria Parra: We're, we're actually I think a quite a friendly bunch. Uh, we're busy, but we're a friendly bunch and trying to listen to everyone and understand where they're coming from. Um, so [00:52:00] bring the problems, bring the ideas. A lot of the things that are in this protocol revision- Uh-huh ... are not things that came out of my head.

David Soria Parra: They are problems that other people brought to us, um, and that we're working on. So yeah, bring the back the feedback, uh, update your server, update your clients, um, and, uh, yeah, I'm super ex- super stoked to see where the ecosystem is going, man.

Demetrios: That's the beauty of open source.