Can AI Agents Be Trusted in Healthcare?

Demetrios: [00:00:00] All right, so at the beginning of April, I went to New York for the MCP Dev Summit, and while I was there, I got the chance to record a few podcasts with attendees that were at the event. This is one of those conversations. Hope you enjoy.

Alex Salkever: How do you make sure that your agents behave the right way

Kingsley Madikaegbu: The AI is learning from the queries you're giving, then it starts making its own interpretation.

Kingsley Madikaegbu: So we try to eliminate that by restricting the amount of tools

Alex Salkever: So say there's a non-compliant patient.

Kingsley Madikaegbu: that's tricky because

Alex Salkever: And I don't mean to put you on the spot, but this is actually- No, I'm just saying. This is like a huge problem because like there's what's good for them and there's what they're doing

Kingsley Madikaegbu: when an agent acts on following a specific workflow that is designed, who takes the liability?

Kingsley Madikaegbu: Does the patient, does the system that's creating that, that's hosting the agent takes the liability or does the provider take that liability? So that's the line where it's like, where the industry still needs to [00:01:00] define exactly

Alex Salkever: I'm Alex Alcufer with the Agentic AI Foundation, and I'm here today with Kingsley Madukeba.

Kingsley Madikaegbu: Yes.

Alex Salkever: Did I correct that the right way? Yeah, that's correct. We're at MCP Dev Summit North America in New York City. Uh, Kingsley's company, HealID, is exploring novel use cases for MCP and Agentic AI in the healthcare system.

Alex Salkever: Kingsley, welcome.

Kingsley Madikaegbu: Thank you.

Alex Salkever: Uh, tell me a little bit about HealID and what you're thinking about doing with- with- with the technology there.

Kingsley Madikaegbu: Cool. Yeah, well, uh, I think in healthcare, um, there was a lot of, um... comes into multiple, the multiple use cases around it, you know, from your provider, um, you know, their workflow, the clinical workflow, um, the actual patients on how they actually interact with their medical records.

Kingsley Madikaegbu: Um, and everything in mind, everything that's currently developed, [00:02:00] used, has a patient in- in, you know, in- in points of view, where, um, after a hospital is discharged, um, doctor usually provide some kind of with a summary of their visits, which is then given to the patient, and the patient has to kind of like find a way to kind of, um, you know, digest that information and kind of follow, follow through on- on- on the, uh, recommendation that was provided by clinicians.

Kingsley Madikaegbu: And what HealID does is HealID makes the connection between not just the patients and the, uh, and the- the doctors, but also a lot of the intermediary, like, you know, your caregivers, your family network. And one thing we f- find out is, like, a lot of those outflow visits, especially for patient with, um, chronic condition, the person that actually keeps the mo- most burden of putting that pieces together from the clinical re- records perspective from the, that a clinician has, um, the infor- the symptoms the patient are experiencing and, you know, the caregiver that's actually observing in [00:03:00] between is the family.

Kingsley Madikaegbu: So they tend to end up holding that burden of like- Piercing all this fragmented data and trying to make sense of what is going on. And even w- they also have their own point of view into, like, how the whole, um, um, um, process can be more efficient. And that's where MCP comes in. Um, MCP kind of like looks at this from an agentic workflow perspective where, you know, each of those individuals, like the, the patients, the provider, and the caregiver, and the family all have their own a-agent accessing the same data and doing the piecemeal between, um, what they need to use, how they need to use the data and what they need to see.

Alex Salkever: So give me a, like a hypothetical. Say someone has open heart surgery, they're returned to home. They probably have a pa- like 25 pages of care instructions. There's an occupational therapist, there's pharmacy 'cause they have a whole bunch of medicines that they have to take after surgery. There's the physical therapist, there's the nurses [00:04:00] who still need to be informed about blood pressure and wound healing, and then there's the, the, the, the surgeon who cares about things, and then there's, you know, the, the caregiver, the spouse or whatever.

Alex Salkever: How-- Tell me how that would all work together.

Kingsley Madikaegbu: Yeah. So, uh, I think, uh, from the fundamental layers is the data layer. So in how we kind of like, uh, you know, segmented it, our activities at four different pillar, where we have the data layer and we h- have the raw data in the raw form, like unstructured form, where we have, um, we take in that medical record that was prescribed by the doctor, parse that information into a, a, a data, a data store.

Kingsley Madikaegbu: The data is just dumb data. No logic, nothing built in.

Alex Salkever: It's often paper. Blank- Usually you're at a hospital and they give you a piece of, like a stack of paper.

Kingsley Madikaegbu: Exactly, exactly. And I, and I think what our system does, like we, we start out with personal information into a, a, um, a, a structured form, and just, but with no actual logic, nothing built in.

Kingsley Madikaegbu: And on top of that, that's where we actually set a context [00:05:00] permission on top of the da- data layer that, um, identified authenticated users what access they have to what. So in the use, in the example you gave of, um, your, uh, provider that's making a medication, um, request, they have a specific workflow and specific permission that they need to do on that particular patient's medical record.

Kingsley Madikaegbu: Um, and we map that, on top of that, we layer on top the MCP. And the MCP server has its own set of permission, uh, on what tools it has access to, what specific, uh, permission it's able to do for that given, um, workflow. And on top of that MCP, w- then we have the agents. Now, the agent has, uh, uh, um, are more focused on the reason, the capability.

Kingsley Madikaegbu: And so the, the doctor, um, has a specific set of workflow, but is not able to actually visualize the data in a different workflow, um, of, uh, [00:06:00] because they don't have ac- although they have the same database, but are not able to actually see those particular records. An example would be like workers around like, um, uh, mental health if they're not a mental health is- specialist.

Kingsley Madikaegbu: So only w- workers that's pertaining to their permission, um, that's what they're able to have access to and make, or make those requests. And it's also time-bound. So we're able to set time bound on each of this different workflow of how long are they able to live and, and, and to what extents of permission, to much extent of which they're able to have.

Alex Salkever: So I, I have a long list of questions, which of course I, I basically dropped your talk description into, uh, Claude and said, you know, "Prompt me." "What, what should I ask Kingsley?" Um, so why do you say MCP is the right architectural use case for, I mean, choice for this use case?

Kingsley Madikaegbu: Yeah. So initially, the initial architecture was, um, a, a API REST-based architecture where we had-

Alex Salkever: REST-based.

Alex Salkever: Yes. So you didn't even think of Graph. You were thinking REST.

Kingsley Madikaegbu: Yes.

Alex Salkever: Okay.

Kingsley Madikaegbu: So I mean, but how, you know, without MCP, [00:07:00] what, how that architecture would look like would be you making a request to a specific endpoint and say, you know, "I want to update this record." Then you layer it on top of an agent. That a- agent kind of queries that, uh- Right

Kingsley Madikaegbu: that, that REST piece, um, URI and say, says, "Okay, this is the, this is the result," and then returns it back. Um, I think from a-- in order to enforce compliance, um, data restrictions, it requires a lot of, um, encoding, a lot of logic, not just in the, in the agent level, um, but also the data level and also the security level, and which makes it very complex, and you have to hold a whole list of permissions.

Kingsley Madikaegbu: For example, if someone is a caregiver and they're-- all they want to-- the, the information they really want to see is, um, "Has my parents taken their medication?" Versus a doctor wants to see a little bit more than just have they taken their medication. They want to know how much dose have they taken. [00:08:00] And, um, as a patient, you co-concerned if the medication that is being prescribed ha- conflict with a medication you're already taking.

Kingsley Madikaegbu: So this, this kind of like complex workflow makes it very difficult to implement, um, on AI. And also, um, there's a huge risk of violating, um, a lot of, um, um, policy there. But I think MCP makes it cleanly, uh, be able to set specific rules and permission on your MCP server that segregates the amount of, um, permission they're able to access in, in, in regards for, in the data la- layer.

Kingsley Madikaegbu: And by doing so qui- uh, um, easily, we're able to kind of audit every single, um, step an agent is taking. So the agent being able to access the, a particular record, and they have, they have, they have four different checkpoints that they have to go through before they'll be able to prove that, you know, that permission is requested.

Kingsley Madikaegbu: But also, we're able to see that [00:09:00] log in that, in, in, uh, uh, that traceability and, um, which is easier for compliance, uh, and easier for y- for you to be able to demonstrate not just compliance, but also auditability for HIPAA and other, um, um, providers access, because that's something they really care about, just being able to say, you know, this, uh, particular person accesses this record, and it was not just an agent, like just doing this thing, doing random, uh, snooping around into, into my database.

Kingsley Madikaegbu: So being able to prove that into, in a, in a, in a, um- In a health setting is something that's, is, is, is, is critical for a lot of organizations operating in a re- regulated space.

Alex Salkever: So, so like for example, I've heard from, and my doctor friends working with the EMRs is very challenging because you literally get permission fatigue.

Alex Salkever: It's like Claude Code except like every five seconds you accept this, you acknowledge that or whatever. Does this help potentially streamline, uh, some of the navigations of all of these very complex regulations under the covers, uh, that [00:10:00] you'd see in EMRs?

Kingsley Madikaegbu: Yes. I think if it's adopted at a wide scale, um, there's a potential ability to be able to create that segregation later and, and, and the audits which is ment- which is mentioned.

Kingsley Madikaegbu: So, um, I think the, it, there's all this question about like liability.

Alex Salkever: Right.

Kingsley Madikaegbu: So like who is liable, um, for- For being

Alex Salkever: able, uh, being HIPAA is essentially like- Exactly ... that's what it's all about liability, so.

Kingsley Madikaegbu: Exactly. So I, I think, um, yes it simplifies it, but again, a, a, a proper, um- I guess, uh, we haven't seen it actually applied into a large scale, um, system, but still being able to recognize like, you know, when a, an agent acts on following a specific workflow that is designed, who takes the liability?

Kingsley Madikaegbu: Does the patient, does the system that's creating the, that, uh, that's hosting the agent take the liability, or does the provider who take that liability? So that's the line where it's like, [00:11:00] uh, where the industry still needs to define exactly.

Alex Salkever: Well, you probably saw, uh, some guys out of Stanford, uh, and someplace else just launched an, uh, an insurance company for AI agents.

Alex Salkever: You d- did you see that?

Demetrios: No. I'm

Kingsley Madikaegbu: curious.

Alex Salkever: Now, they will insure specific AI agents. Uh, you know, I think they're initially starting, uh, with 11 labs for voice agents.

Kingsley Madikaegbu: Oh, nice.

Alex Salkever: But I mean, like, the, the whole goal is, like, we will underwrite your agent if you, if you can demonstrate sufficiently that you have the guardrails in place.

Kingsley Madikaegbu: Yeah.

Alex Salkever: Anyway, just, you know, I, I can send you a link to that later.

Kingsley Madikaegbu: Yeah, that'll

Alex Salkever: be cool. Um, so what healthcare integration problems specific-- 'cause we talked a little bit about what you're doing and then also why MCP, but, and we talked a bit about healthcare, but what healthcare integration problems specifically do you think MCP is better for?

Alex Salkever: You talked a bit about HIPAA, uh, but let's, let's sort of take it up a level almost, 'cause, you know, like, healthcare is its u- own unique beast and, you know, strip away, uh, the, the jargon to just, like, normal problems that people think about that they get solved by that.

Kingsley Madikaegbu: Yeah. So I think, um, [00:12:00] a good analogy to think about is, um, you can think of it as, uh, MCP as kind of like a, like a bouncer in a nightclub.

Alex Salkever: Good analogy.

Kingsley Madikaegbu: So, like, each-- And, and the agent will be more like, you know, a guest coming into, into, into the club. MCP acts as, um, a-- Rather than just, um, a, a tagging the guest as, or, uh, as, you know, a member and checking up a VIP list, MCP is actually, um, reorganizing the whole, uh, club for that one particular guest.

Kingsley Madikaegbu: So when one guest comes in, the whole, it's a whole different club setup versus another guest. And that's how, um, um, in-- When you put that in the, in the hospital setting is how a patient interacts with, um, their actual record, it's a lot different from how a provider would interact w-with, with the record.

Kingsley Madikaegbu: So things doctors care about is more of like, uh, am I prescribing a [00:13:00] medication that conflict with another med-medicine that a patient is taking? A lot of time it's just a guesswork. Problem providers also struggle with is, is the medication that the hospital that was prescribed, has it been fully exhausted?

Kingsley Madikaegbu: Is the patient actually adherent to the medication? Um, so that's another, um, things that they do care about. And as a family member, um, y- um, you want to see if there is, um, if your, your parents or involved, they're actually improving in, in the care that they're receiving Um, and which is a whole different experience of what you wanna do.

Kingsley Madikaegbu: And, and, uh, also the workflow you als- when you think about it's like, how do I reach out to my parents if I notice something that's concerning? So, um, but also from the patient's standpoint, you also want us to know if there's a correlation between the medication you're taking and your b- your, your heart rate.

Kingsley Madikaegbu: Uh, am I, you know, is that, is that, is, is my blood pressure increasing? Or, you know, when, uh, I'm taking some [00:14:00] medication, is it actually working? Um, um, is that disrupting my sleep patterns? So all that insightful information is how each different stakeholder has a different, um, way of interacting with the same particular data.

Alex Salkever: Yeah.

Kingsley Madikaegbu: And, um, MCP provides that abstraction and allows us to be able to customize different agent for different workflow and customize different reasoning capability and patterns on how they should approach the same data and the information that's, is, is forwarded, um, and, um, the action also that is taken.

Kingsley Madikaegbu: So-

Alex Salkever: So, like, real world example, uh, I mean, 'cause a lot of medicine that's exactly what you're s- describing now, which is super interesting, uh, is extensibility out into... I mean, I think Johnson & Johnson actually calls it, like, real world data analytics or something like that, where, you know, for example, uh, someone that has had a, a serie- a history of arrhythmia, you know, their Apple Watch is actually pretty decent as a first-line indicator.

Kingsley Madikaegbu: Yeah.

Alex Salkever: So how do you then extend into [00:15:00] additional data sources with MCP? Uh, you know, and is that the right way to do it? Or, or sort of explain a little bit how that might work. Yeah, yeah.

Kingsley Madikaegbu: So it, the, the examp- example you, you, you, um, you gave is you experience that symptoms. The, the first person is allowed to also, the patient, they will receive a notification in their phone.

Kingsley Madikaegbu: And also if someone within the care network, you know, um, like their family- Right ... or someone that is, like, you know, authorized to be, uh, I say authorized caregiver, are notified, like, you know, your f- um, your parent just experienced a sudden change in their arrhythmia. Now, the question is, the agent is, is trained, it's like, what is the next step into that, in that- Right

Kingsley Madikaegbu: in, in that scenario? One, alert the phy- the physician how critical it is. So we have to, um, train what is the severity? Is this something that needs to be, um, addressed right away? Should you call, um, uh, a, a ambulance care? Or it's not some- it's, it's something that it could be scheduled. So those are the kind of, um, areas where we, we have to kind of really tread [00:16:00] quickly and, and it's also we're working with doctors to actually decide what is the right threshold to be able to set on a given condition, given their past history.

Alex Salkever: Yeah.

Kingsley Madikaegbu: So that's kind of, that's still la- layer where we're kind of, like, really working with providers to understand, you know, a, a, a given patient's condition, what area, what condition is considered severe, and what actions should the system take versus what it shouldn't take. And that's how we're d- how we're, we're currently solving that issue.

Alex Salkever: And how do you bake the context in at that level? So, like, as an example, um, 'cause this is actually someone I know has done this, uh- If you, if you're ca- if you worry a lot about arrhythmia, there's actually an app that, uh, will-- You push a button on your, on, you know, on your wrist, on your watch, and it will send it to a, um, an EKG review team.

Alex Salkever: I think it's, it's remote somewhere- Mm-hmm ... in the Philippines or something like that, and they'll do a real-time read, you know, on like, "Okay, does this look bad? How mu- what do your arrhythmia look like?" And, and send you back a report. And now it's actually like an AI is doing the read. Yeah. So essentially, you can create these workflows where, "Oh, [00:17:00] okay, this looks not too bad.

Alex Salkever: I'll send it there for context," or, "We'll send it there for context, and then if it looks bad, automate, like kick it up to..." So how do you, like, inject- Yeah ... context to create these fluid workflows?

Kingsley Madikaegbu: Yeah, exactly. And I think th-th-that's a good use case where, you know, where we send the AI chart, does the review and scanning, and what we just look at, like, you know, the first time maybe indication, maybe it could be a false positive.

Kingsley Madikaegbu: But if we start to see a pattern, then the next action will be, um, we provide the provider, uh, the actual, the patient, the authorities say, you know, "This has repeated for the past three, um, uh, three times in the past, um, two months. Um, we recommend scheduling an appointment. Would you like to, to, to schedule it?"

Kingsley Madikaegbu: And now the agent that does the scheduling, which is separate from the agent that's don- do-- sending the alerts, um, actually makes the scheduling, work with this provider, look into the availability, and actually book the appointment for the patient to come in for an actual, um, um, screening with the, with the primary care.[00:18:00]

Kingsley Madikaegbu: But also, uh, another workflow we're also looking into is like even in a- in advance of actual the doctor's visit, the doctor already has a summary of what is going on. Um, they're able to look at things that like, um, what is the sleep pattern, um, and also be able to rule out other things in advance of their meeting.

Kingsley Madikaegbu: So they're, they're finding out they're able to be more efficient in providing care, and also they're able to provide the quality of care is also improving- Right ... because they have a little bit more holistic information rather than just the information that we see during the actual visit.

Alex Salkever: So let me ask, like a, a harder question in this, and I actually think this is quite relevant.

Alex Salkever: Oh, yeah. Um, so say there's a non-compliant patient And the data shows, you know, the, uh... I- I- 'cause I have a lot of doctor friends who are- they, they work in, you know, in, in the, um, in, uh, public health clinics. Yeah. That's a huge problem with a, a large part. Um, obviously, you know, like, with MCP, it's the perfect vehicle to run the agents.

Alex Salkever: Yeah. How do we think about that, like [00:19:00] permissioning and those... I mean, I know this is like out on the edge, but I'm just kind of curious.

Kingsley Madikaegbu: Okay. Um, so let me make sure I understand. You m- mean like a p- a, a patient that they have been, received a, a recommendation from their doctor, but they're now ad- adherent to it?

Kingsley Madikaegbu: Is that-

Alex Salkever: Exactly.

Kingsley Madikaegbu: Uh, yeah. Yeah. So that's, that's tricky because w-

Alex Salkever: And I don't mean to put you on the spot. No, no. But this is like a, it's like a huge problem because like there's- Yeah ... what's good for them and there's what they're doing, and you know.

Kingsley Madikaegbu: Yeah, yeah. But I, I... the way we structure it is like we leave, uh, t- two things.

Kingsley Madikaegbu: We, a design principle one, the patient are in charge, uh, of the decision. Yeah. Yeah. So whatever suggestion, whether a provider can recommend something, uh-

Alex Salkever: Or, or could a patient like have their caregiver basically like a f- a full award like, "Okay, my caregiver can see this and can understand whether to forward this to doctors or whatever," and-

Kingsley Madikaegbu: Yeah, um, that permission, we, it, it depends.

Kingsley Madikaegbu: We, we keep, we separated caregivers into two different categories. Yeah. So there's care- caregivers that is actually, you know, um, one, they have, one, legal rights, but also they've been given explicit permission. So someone like a minor, for example, [00:20:00] that, you know, they're still under the care of their parents.

Alex Salkever: Well, i- like in California, actually, it's a huge problem 'cause like if your kid's in college and they're over 18, you can no longer, like see their health records. Yeah. Even though you, often you need to see their health records.

Kingsley Madikaegbu: Yeah, yeah. So yeah, so it is tricky. So I think... But there's also that, that one piece, but there's also a caregiver that you kind of assigned as, you know, someone to, uh, maybe to look after you.

Kingsley Madikaegbu: Yeah. And some of them are maybe, they could also be a paid caregiver. So that's how we separated this piece. And, um, I, I forgot, what's the, the o- the original question you asked there?

Alex Salkever: I'm sorry. No, it, it was essentially like these, there are these cases where, you know, um, we saw, we, we see it where, uh, the- The patient or, or a person that would be potentially benefit from the agentic system may not want, you know, their- Yeah

Alex Salkever: non-compliant patients, like I said- Yeah,

Kingsley Madikaegbu: yeah, yeah ... which

Alex Salkever: is a large, large, large problem.

Kingsley Madikaegbu: Yeah. So yeah, exactly. I mean, it w- the ultimate thing is that we, we leave the decision u- up to either the [00:21:00] patient, um, to kind of like, you know, make the decision, whatever they want to do about their life. Um, but also the, um, caregiver that are, like, required, authorized caregivers that are authorized to make that decision.

Kingsley Madikaegbu: So in the case of, like, a minor, um, that falls into the legal category. The main thing is, like, we're, we're, we're very focused on we do not want the system to breach any, um, regulatory or safety requirements. Right. Um, so that's the, the co-, uh, design principle around. And also, uh, the second thing I also look into is also, we also want patients to feel, um, um, you know, have control over their health, and that's something we see in doing our research is- Yeah

Kingsley Madikaegbu: um, everyone wants, um, autonomy over their health. They want to be able to make decisions and sometimes even, um, they might, some recommendation that's preve- presented, they might not even want to disclose.

Alex Salkever: Right.

Kingsley Madikaegbu: Um, so a classic example was, uh, even with my dad, he's one of our, [00:22:00] our, um, test- Oh, awesome.

Alex Salkever: Yeah.

Kingsley Madikaegbu: And, um, you know, he-- w- we're having a meeting with the doctor, and he's like telling the doctor, "Yeah, so I've been, you know, taking this medication and, you know, for some reason my, my blood pressure is still, uh, trending high." But I'm like, "The doctor's looking at you also exercising," and he's like, "Yeah, I'm, I'm exercising."

Kingsley Madikaegbu: But you know, you could look at the data and you're like, "Okay, well, the data is showing that you're not really walk- you're not putting in enough steps."

Alex Salkever: Yeah.

Kingsley Madikaegbu: So, um, but so, but the data is like, uh, is that source of truth that could kind of, um, be accurate. And sometimes the doctor might know, has that information, able to-- they know that this person is not exercising.

Kingsley Madikaegbu: But i- regardless, they needs to kind of be able to work with them in, you know- Yeah ... in, in, in their capabilities and say, you know, "Well, maybe next time you should, you should exercise a little bit more." And, um, you see that also even in some settings where, um, you know, someone goes to the dentist and the dentist is like, "Oh, are you brushing, are you flossing da- daily?"

Kingsley Madikaegbu: And you're like, "Yeah, I'm flossing daily." And you know, the dentist can actually clearly see a lot of plaque and you-- they could tell that you're not [00:23:00] flossing daily. But they give the encouragement to adhere to the processes. But regardless, it co- it falls down to the actual patients in taking the ownership and actually doing the recommendations.

Kingsley Madikaegbu: And, um, where our, our platform provides, kind of provides that, um, um, accountability and that auditability to be able to say, "Oh, um, yes, this patient is actually following my recommendations," and they're not seeing an improvement from the provider perspective, so they can actually assess if the, if the recommendations they're providing is the right recommendation they should be giving.

Kingsley Madikaegbu: So,

Alex Salkever: like for GLP prescription or something like that- Exactly. Exactly ... or if they're adhering to best practices-

Kingsley Madikaegbu: Exactly ...

Alex Salkever: and they're exercising or whatever. Yeah. Yeah. So, yeah.

Kingsley Madikaegbu: And

Alex Salkever: I- No, it's interesting 'cause it's almost like an intermediate. So, so like for example, I mean, it gives you these intermediate options that weren't there before.

Alex Salkever: I mean, some of it's verification, but also like, uh, you know, if, if there's a non-compliant patient, you could have an option that's like, "Okay, I'm gonna tell their spouse to talk to them about what's going on with this." You know, where you don't disclose- Ex- yeah ... but you just, so you [00:24:00] push or you, you tell the doctor, you know, without actually violating privacy, which is like, you know, that wasn't an option before- Mm-hmm

Alex Salkever: 'cause they couldn't even see that.

Kingsley Madikaegbu: That's true. That's true. So you're able to kind of nudge the family in the right way to kind of- Right ... care.

Alex Salkever: Right. Right. Yeah.

Kingsley Madikaegbu: Yeah. That's true.

Alex Salkever: So, so which parts of your workflows are deterministic? Which are, which are, you know, rule-based versus model driven, right? And how do you determine which parts should be which?

Kingsley Madikaegbu: Gotcha. Yeah. That's a good question. So for, um, I think even what I mentioned earlier, anything that is regulatory, um, and anything that is clinical and, um, like life-saving is very deterministic.

Alex Salkever: So binary.

Kingsley Madikaegbu: Binary. It's more like, um, so we do not want to be in a, a process that violates any government, uh, um, um, regulations.

Kingsley Madikaegbu: And the way we, we structure is, and MCP allows us to do that is like, you know, from the data layer where we kind of like don't embed any logic in the [00:25:00] data layer. The access layer is where we kind of like set our permissions.

Alex Salkever: Right.

Kingsley Madikaegbu: Then on top of that, we layer on top of the MCP layer. So the MCP only use specific tools that it has permissions to do so.

Kingsley Madikaegbu: Um, and those tools are directly assigned to a specific sets of data. So a given example would be like, you know, if someone has a high, uh, their blood pressure had been trending high for a while, we don't say, you know, "You're at risk of a heart attack." We kind of like share that information with the provider, and the provider comes into the workflow and determine, is this severe enough for me to request for the patients to schedule an appointment?

Kingsley Madikaegbu: And if the provider identifies that, um, their behavior is, is concerning, then the patients, the provider themself have to actually approve of the agent to actually make that dec- that decision of actually request for [00:26:00] the, for the patients to come in for an appointment. So the provider's also able to set that criticality.

Kingsley Madikaegbu: So, um, that's how we're able to make that distinction, is where it's like, where it's clinical, direct that to the provider, have them make the decision of what the patient should do. Things that are regulatory with, is a strict, uh, for, for error, or like no, um, or, you know, we just, um-

Alex Salkever: That's, that's absolutely prescriptive and-

Kingsley Madikaegbu: Exactly, and no access to the data, um, um, given, um, their, their particular workflow.

Kingsley Madikaegbu: And they're then, their, their workf- the agent also doesn't have even the, the, um, the visibility to be able to access the data even if it wanted to.

Alex Salkever: So I mean, we've seen instances where agents, uh, you know, they take on a life of their own. I mean, you probably saw like what happened with Meta, with the OpenCall instance that-

Kingsley Madikaegbu: Yeah

Alex Salkever: got, got loose on, on the Meta security head's re- uh, email, um, you know, which was, and she posted about it. It was fascinating. Uh, how do you make sure that your agents behave the [00:27:00] right way? You know, how do you keep them on the leash? And, uh, s- I mean, obviously sandboxing, but like in, in terms of how you're building this.

Kingsley Madikaegbu: That is, uh, that is a good question, and I, I think the, the, uh, four layers segregation layer is something that we've, we've seen, we've seen good results on where, um, uh, you know, what I mentioned before, the whole API, uh, based, um, context, like when the AI is making a lot of all these calls and making all these query, the AI is learning from the queries you're, you're giving, then it starts making its own interpretation.

Kingsley Madikaegbu: Um, so we try to eliminate that by restricting the amount of tools, um, the, each particular agent we assign. So your provider agent, the, um, the, um, the patient's, um, workflow to kind of like its own sets of, um- Uh, permission behavior, and we'll also do something that is a little bit different, is where we kind of, like, create, like, a metadata of, uh, of sorts for [00:28:00] each of the agent's behavior that kind of records their behavior, and we're able to kind of correlate it into, um, to identify there's some drift in- So you're

Alex Salkever: looking for anomaly or drift.

Alex Salkever: Exactly. Okay. And I'm curious, like, on the, in the same vein, um, so one of the magic that I've seen at least with LLMs i- in this area is that it can put patients almost on par with physicians in having conversations. Obviously they are, you know, humans are, you know, they're, they're doctors, but they're also, they're human.

Alex Salkever: You know, so, like, an example I know, uh, from, uh, from someone's life in, in my, you know, around me was they looked in ChatGPT, were like, "Hey, I'm, you know, my blood sugar's not going down. Uh, you know, is that related to my statin?" And ChatGPT actually, like, explained to them that the statin they were on was a good statin, but they could switch to a different one that might actually, you know, not impact A1C as much.

Alex Salkever: And so they had the conversation with their provider and they switched, and it helped. And it wasn't like the provider didn't know, it was just more the provider had so many things going on that that's not what they think about. [00:29:00] How do you, uh, inject, like, use MCP to inject from the models, and I'm assuming you're using a couple different models- Mm-hmm

Alex Salkever: uh, you know, that type of context and also take advantage of the way MCP works to, you know, to create that sort of capability? I'm not sure if that's what you do at all, but-

Kingsley Madikaegbu: Yeah. So we- we're not doing that. Uh, I think there's- Okay. Sorry, sorry, sorry. But I mean, I think that has a lot of, like, risk, and I, I think we're seeing that, I think, was it in, in Utah of where, um, they kind of created a lo- regulations around the use of AI in, in the medical- Yeah.

Alex Salkever: Oh, no, they're, they're getting pretty serious

Kingsley Madikaegbu: about it. Yeah, yeah, yeah. So it, it, yeah, um, so, uh, how we, we, we kind of circle it is, like, the agent is more of like, uh, the agent, the different agent has different workflow, different personality, um, and the agent that interacts with the patient is more of a, a coach and less of a clinical, uh, decision ma-maker.

Kingsley Madikaegbu: Okay. And it's more like [00:30:00] looking at, okay, based on the recommendation that your doctor- Right ... uh, prescribed, um, you should do X, Y, and Z. So what we did is, like, we always a- um, assign each of the action or the recommendations the, uh, agent is providing specific to a, a particular either clinical note that the doctor actually, um, created or, like, you know, the biometrics information that we receive from the patient's wearable information to be able to say, you know, "This is where the, sort of, the information came from."

Kingsley Madikaegbu: So, um, we could say stuff like, "Last Tuesday-" In your meeting with the doctor, uh, doctor recommended to do A, B, and C. You're able to look into that and see the raw data of exactly the notes that you captured and what the doctor's recommendation. So you're able to kind of identify there's some information drift.

Kingsley Madikaegbu: So we're-- that's how we're kind of, our workflow kind of works. But from the clinical side, it's a whole different workflow, and we have a more medically trained model that is able to kind of [00:31:00] provide the doctors like, you know, this is some information we are seeing based on, um, um, you know, the patient's data, and just more, some more information for, um, for the providers to kind of consider.

Kingsley Madikaegbu: Um, I- and during our research, we noticed there's some provider that interested in knowing some of that information because they also, they want to be ahead of like, you know, what is the latest, um, training and understanding of like any particular disease or an infection. But there are some provider that are very skeptical about AI giving them recommendation.

Kingsley Madikaegbu: Um, so we're-- that's kind of how the clinical workflow usually exists, is where, where we're providing information to the clinicians, and clinicians kind of making their judgment on whether that's something they should consider or, or like validate versus, um, information going to the patient that, um, or that use case you presented about a patient using ChatGPT to kind of read a health summary.

Alex Salkever: What kind of failure modes have you seen, and how have you mitigated that?

Kingsley Madikaegbu: Um, I think on a, that's a good question. The als- the, [00:32:00] the, the first is, um, a scenario where, um, you know, the particular patient has like a relationship drift with their family or the caregiver, and all of a sudden they're like, "Hey, well, I want to kind of restrict access.

Kingsley Madikaegbu: I don't want them to be able to kind of like, uh, um, to check on me anymore." And I think that area is like i- initially is kind of like, um Although they have access, but the information, maybe there was restricted access, but the information is still transmitted. Um, so they're able to receive some form of response that is on, uh, in, that's not completed.

Kingsley Madikaegbu: So what we did, uh, is to kind of like treat the permission as almost like a shut-off switch, where as soon as the request permission is revoked, instantly, um, it returns a 404 error. So it doesn't show any partial information or partially generated information. It's just, [00:33:00] um, the request just returns like, "I'm, I'm sorry, we can't, uh, the, the agent is not working," or, you know-

Alex Salkever: Yeah

Kingsley Madikaegbu: yeah. So that's how we

Alex Salkever: kind of- Very abrupt

Kingsley Madikaegbu: Very abrupt. Like, um, so we try to, um, do that quite some on the permission layer where we, uh, the, the patient has that almost kill switch, um, to that permission. So that's, that's the first one. Um, another thing we also, um, see is also kind of like the whole liability question comes into play of like who takes liability on, you know, if issued does occur in a production setting.

Kingsley Madikaegbu: So, and those are the areas where I think, you know, we're, uh, design around like, you know, action a, a action level where, you know, we're able to assign providers', um, action specific to the agent's behavior. So we're able to say if it, a, a medical professional recommend a medication for a patient to be taken, the medical and, you know, the agent needs to be adhered [00:34:00] strictly to what the recommendation was provided by the professional, and so they're able to adopt that as, you know, I mean, they're able to take that wa- um, liability if there was a drift in that.

Kingsley Madikaegbu: But also, if there was a drift into the recommendation that the provider provided, but the agent made a mistake, then obviously the agent, um, takes that liability. So that's how we kind of segregate it. So those are some of the challenges where we're still addressing, and also even a, a big industry, um, type, you know, challenge.

Alex Salkever: Are, are you also layering additional pro- like A to A or to sort of designing so that your agents can interact with other agents too? Or what else are you putting, are you thinking about on top of MCP to extend the functionality?

Kingsley Madikaegbu: Ah, that's interesting. We have, we haven't really explored it. I think, um, our focus right now is just trying to get the agents consi- providing consistent results, and, um, we're, uh, we're seeing, uh, progress in that, in that, in that, in that area where it's like, you know, we're able to provide that, you know, [00:35:00] that traceability within, uh, in audits.

Kingsley Madikaegbu: So yeah, but maybe in the future, and I, I, I think that's probably where the, the industry's heading towards, where y- um, agents will be interacting with other agents. There was an agent that your agent goes into the provider's agent and does a whole bunch of other workflow, kicks off all the other workflow processes.

Kingsley Madikaegbu: So that's something I, I could see, you know, the industry going towards. But I think in the health, uh, or the regulated industry in general, I think it will be much slower. Uh- Of course. Yeah. Much slower. Yeah. I mean, there are hospitals that still use paper. Uh, so-

Alex Salkever: Oh, no, I know. Discharge instructions are still- Yeah, exactly.

Alex Salkever: Still very- You know, but I, I was not joking 'cause I've seen discharge instructions,

Kingsley Madikaegbu: so. Yeah. Yeah. I think there's long ways, and I think, uh, the, in the past couple of years there's been a lot of easing some of the, the restrictions around data, so I, I'm seeing a little bit more innovation in that space, and a lot of, um, startups are coming with creative solutions to address the challenge.

Kingsley Madikaegbu: And I think there's some more features I think, you know, um, the MCP community in general I think [00:36:00] can, can invest in, um, especially for regulated industry. And y- it can be applied not just in healthcare or say in finance or, or other, or legal, other areas that requires a lot of, um, strict enforcement of data.

Alex Salkever: So, so what we'll-- Let's round it out on that point and that specific question, which is if you had a wishlist for the technical committee of here's the things I would love to see in MCP to help me or help other people in the healthcare sector better use it as a, you know, as, as, as a platform for agentic AI and building, you know, highly functional app.

Alex Salkever: What, what would you say? Like, what's my wishlist?

Kingsley Madikaegbu: So the first one is that, um, I don't know the right word to put it, but is being able to kind of identify On, on your MCP server, the enforcement's, um, um, traceability. So right now, like what we're doing is we're created, we created, you know, I mentioned we created a, a static, uh, caching layer that ident- that [00:37:00] addresses each agent, what specific tools they're using and how they're accessing it.

Kingsley Madikaegbu: So right now there's no-- Uh, I haven't seen any particular MCP server that actually has that also already embedded.

Alex Salkever: So there's like no OTEL spans or anything like that.

Kingsley Madikaegbu: Exactly. So there's always this weird workaround, like for example, like if a given agent, you only need it to be, um, live for maybe two weeks.

Kingsley Madikaegbu: It's not something you can kind of like enforce on the, on the NCP layer where you're able to shut off the access on a time bound as setting. Um, how, you know, some of the things we do, we do as a workaround is like created that caching layer of, uh, of logs that kind of like creates that audit of like, you know, um, based on this setting that you're writing to this, uh, into this, uh, log, it has this time bound, and we kind of make an hour request that to shut off that service.

Kingsley Madikaegbu: Um, so if that was something that was embedded by default in a lot of, um, MCP servers, I think that would be something that I, um, will help in the regulated industry perspective, and I [00:38:00] think it will be more, we see a little bit more adoption in that, in that area.

Alex Salkever: Anything else that jumps to mind or?

Kingsley Madikaegbu: What, uh, I think another thing is just more around, um, the old security related, um, guardrails that, um, prevents, um, you know, do, do, how, uh, prevent agents from communicating with each other and giving them siloed data access.

Kingsley Madikaegbu: So there's, I mean, we have a lot of, you know, I mentioned the three lay-layers stacked up, how we kind of address that, that situ- uh, that, that issue. But, um, these are the two things I think, um, I think MCP needs to be really more robust in to be able to attract more enterprise level, um, implementation.

Alex Salkever: Very cool.

Alex Salkever: Well, thank you so much for coming and sharing all this. I learned a lot. Um, and, um, hopefully I will be able to use HealID in the-- Well, not, hopefully not. I don't like curing the sick people, but if I have to, uh, you know- Yeah ... I will be able to use HealID in the future.

Kingsley Madikaegbu: Awesome.

Alex Salkever: Um, hope you have a good summit, and hope to see you at another event.

Kingsley Madikaegbu: Thank you. Thank you. I appreciate you interviewing [00:39:00] me and just giving me the opportunity just to share what we're doing, and also being part of this community of just a lot of, you know, eight engineers just building and just learning the, the next level of, you know, agentic, uh, deployment, which is really exciting to just be part of.

Kingsley Madikaegbu: So thank you for what you do, and um, I, I really enjoyed this conversation and just, um, giving us the opportunity just to just present what we're doing. Thanks.

Alex Salkever: Thanks, Kingsley.