The Infrastructure Imperative: What It Takes to Run Multi-Agent Systems // Dipanwita Mallick // Agents in Production 2025

Dipanwita Mallick [00:00:00]: Okay, so thank you for the kind introduction. I am super excited to be here. I'm Deepan with the product manager at HP currently. And you know, this is my first time presenting basically on agentic AI and super excited. As you know, everybody's excited about this whole, whole Agent Ki thing and I'm here to share some learnings from our journey, both my team and both mine and my team's journey, that when we talk to enterprise customers, what we have been hearing about what their enterprise or their infrastructure preferences have changed over time and why suddenly this foundational layer is super important to make your agentic AI system successful. So I'm here to share some of the learnings because I think that it's a. This knowledge would be super helpful for everyone in the community whether you're building an app or whether you're building a complete orchestration platform.

Dipanwita Mallick [00:01:06]: I think this might be super helpful for everyone. Let's dive in. There is no denying that we are now in the age of agentic AI. From single agent to multi agent, things are getting super complicated. But. But in terms of what agentic AI or AI agents can do, we are just still scratching the surface. There is so much to do. Now if you look around, you will see that enterprises are launching AI projects or AI prototype projects at a record scale.

Dipanwita Mallick [00:01:40]: But here's a catch that very few of these projects are actually unlocking the real business value or delivering the right business value to the enterprises. And even in IDC study that was recently published, it's noted that 33 AI prototypes a company might build, out of which only four of them go into the production, which means there is 88% failure rate when it comes to scaling the AI effectively. So that's kind of concerning. Right? But what's even more concerning is the cost of not scaling AI again. Another BCG study that's been done recently, it showed that 11% of the companies are able to unlock the business value from the AI at scale. They are successfully able to convert their prototypes into the production. Not all of them, but all these companies who are able to scale their AI successfully, they have seen dramatic improvement. They have seen massive improvement, even like 30% higher a bit compared to the enterprises who are stuck in the, in their prototype or in their pilot phases.

Dipanwita Mallick [00:02:50]: So there must be something which are like holding us back from moving from, you know, production pilot to the production. Sorry, from the pilot and the prototype to the production. Right. And the whole point of seeing this is the takeaway here is scaling AI today is not optional. It's super critical. If you want to really unlock the real value of your AI, deliver the real business impact, then you need to know how to scale the AI effectively. Now with that said, what is stopping us from, you know, scaling the AI? We have figured out so many things, we know the importance of agentic AI, but there must be something that's holding us back. Now at HPA we talk to a lot of our enterprise customers, even in house AI teams so we understand their pain points firsthand.

Dipanwita Mallick [00:03:38]: So we, we took a lot of interviews internally, understand why, what's stopping them, what are their pain points. Two of the things or two of the main roadblocks that have consistently stood out is one is cost concern, another is a privacy and security concern. So I'll start with the cost concern. Now there is no denying of the fact that today most of the enterprises are heavily relying on the cloud. It's totally understandable because cloud provides you all the right tools, it provides you the right foundation to get started very quickly. It helps you with a rapid innovation. But it also has certain downsides. One of the examples would be this multi agent workflows can consume millions of tokens per session, which means every API call the cost going to add up.

Dipanwita Mallick [00:04:25]: If you're running this entire AI stack fully, completely on cloud then it's going to be super expensive. The alternative is a lot of enterprises think that okay, let me move everything to this dyi on prem which means do it yourself which is like let me build up any infrastructure on my, let's say in my office location or in my preferred location and I'll move everything there. But with that the cost becomes more expensive because now it's like capex intensive right now you have to invest. So one of the big causes I think I already mentioned is a cost rise. So whether it's cloud or you build your own infrastructure, cost is one of the biggest concerns that people have said like eventually it's easy to get started but then when you think long term sometimes it gets very impractical and it's not sustainable. So that's one thing, the other thing is privacy and security risks. We all know that agentic AI there is nothing like agent. There is no agentic AI without the data.

Dipanwita Mallick [00:05:28]: So the data is constantly moving. Your APIs, your agents, your memory stores and when I say data, this data is basically sensitive data. It's the enterprise's secret source data. So right now because of that, your data is constantly moving and it's getting uploaded to the chat interfaces and what enterprises really need is a system where they have more visibility, where they have more control. They want to see where the data, how the data is handled and where the data is resides. So privacy and security concerns have become super critical for the enterprises. And unless the enterprises or the organizations figured these two pieces like, unless they figure out like how do we tackle this cost rise, cost concern and how do I tackle this privacy security concern, they're really hesitant to move everything from, you know, from the product, from the prototype phase into the production phase. Now the second part is what is infrastructure layer has to do any do with this like whole concerns and the scaling issues that is happening.

Dipanwita Mallick [00:06:36]: We all know that AI is not a monolith, it's a landscape. It's a super complicated convoluted landscape with thousands of use cases, tools, software, hardware, so many pieces you have to think about everything. If you want to build AI system early on your journey. I think it is important to figure out the right or the solid foundation because that will determine whether AI would scale effectively, securely, smoothly or it will hit the roadblocks. And two of the roadblocks that I already mentioned is the cost and also the privacy concern. So thinking about the future, thinking long term, this infrastructure which is a foundational layer becomes super important and enterprises are now thinking okay, do we need to have our, do we need to continue with our traditional infrastructure that we did earlier or agent AI is going to completely change or we have to think a strategy around the infrastructure. Now when we spoke to a lot of people, our enterprise customers, few of the trend that came out when they are starting to think about infrastructure, a few of those trends I'm going to share with the team. Number one I already mentioned, already touched on about the data privacy and IP concern.

Dipanwita Mallick [00:07:49]: The second is AI doesn't live in isolation, Agent doesn't live in isolation, it needs to talk to other agents. Everything is happening real time. Whether you are sending the data to, exchanging the information, everything is real time. Minimizing the latency at the source is now important. You need to have a system where you don't have to move your data constantly back and forth to the cloud or to other locations. What happening is your AI is coming closer to where the data resides and not the other way around. With all of that trend, two important things are coming into the picture. One is enterprises are preferring to have their private AI infrastructure which means they want to have more control, more visibility into what's happening, how the data is moving and who is using the system.

Dipanwita Mallick [00:08:45]: But they also don't want to give up the cloud because cloud gives the flexibility, cloud gives the right tools, it helps you with the rapid innovation. What they want today is like a hybrid AI infrastructure. An infrastructure that will help you, will give you the right control you need, but will also help you speed up your experimentation. It will help you quickly build something, test something and deploy something. They need a hybrid AI infrastructure and that is what it's becoming a norm or standard for all the enterprises that want to scale their AI securely and also responsibly. That's the trend now with the moving this trend towards on prem infrastructure. Let me build my on prem infrastructure or let me have our private AI infrastructure. A lot of organizations, they have this urge, like I mentioned, to figure out, let's say let me build infrastructure in the office or in a preferred location.

Dipanwita Mallick [00:09:51]: It could be a colocation as well. But the thing is, even though it sounds appealing, but when you think long term, it becomes very impractical because there are so many complexities that come up. Because you want to build your private infrastructure now you have to figure out your security protocols, you have to maintain, you have to procure the hardware. So it becomes capex intensive, but also it becomes opex intensive as well. Right? So the dyi or we call like do it yourself approach is not something that you, we recommend for your private AI infrastructure. But also there is this need and also you want to have that hybrid AI architecture or the hybrid AI workflow where the workflows are constantly moving between your private and your cloud. So that's unlocking your full potential. And because we talk to these customers, because we see these problems firsthand in HP as well.

Dipanwita Mallick [00:10:46]: So I think we understand the pain points. That's why recently we came up with, we brainstorm what we can do. We want to help these enterprises and that's, I think our mission is we want to help enterprises with their private AI strategy. What that means is we say that this private AI infrastructure, we understand what you need for the private AI infrastructure. I'm going to quickly get into the solution or talk a little bit about the product, about hp. But the idea is if we can help the customers with a private AI infrastructure by helping them build this private AI without the burden of figuring out or doing it all alone, then they can totally unlock the hybrid AI. We work with the cloud, we build the gap or the bridge the gap between the cloud and their on prem or their private AI infrastructure so they can use, they can unlock the total, the full potential of the agentic AI workflow. So we Want to be that infrastructure partner with them.

Dipanwita Mallick [00:11:51]: Now I'll take few minutes to talk about hp the solution I just talked about like how we can help with the private AI infrastructure here. So we call it HP Edge internally. Now when people hear about Edge they instantly think about edge computing. But for us age is different and the reason why we call it Edge is because we think that Edge or HP Edge or solution is an extension of your on prem infrastructure. Now it sounds a bit complicated so let me explain here a little bit. What that means is even if your infrastructure is in our managed data center, it is securely connected with zero trust super secured connections with your network. Which essentially means that even if your infrastructure is within our managed data center, it is extension of your internal network which means that it is basically within your network now you don't have to worry about the security and everything. On top of that we offer the fully managed services with security, with networking, storage, data services, everything.

Dipanwita Mallick [00:12:57]: It can be co managed as well. That's the HP Edge or the solution which is a managed data center today. But I think when we speak to customers we understand that the enterprise needs are constantly evolving. So we believe that there is a huge or a great opportunity to do more to help customers deploy their infrastructure where they need it and the format they want. That's on the left hand side. On the right hand side I'm going to quickly touch on few of the very strategic differentiators like how HP is going to handle this or HP is handling this whole private AI infrastructure for our customers is number one. I already mentioned we have the zero trust secure connections, you know from the, from, from HP to the enterprise network. Which means like it's like a secure pipeline that connects where your data can flow in and out and all taken care of by us.

Dipanwita Mallick [00:13:56]: The second is we don't charge based on the usage. It's a monthly flat fee which means budgeting becomes super easy. And I think customers love that fact that we don't charge based on the usage like every others do. Finally the thing that is not mentioned here is we have the hardware that is needed. With Nvidia GPUs under the hood we can support the super high workloads, AI demanding AI workloads and of course super complex agentic AI workflows. Having multiple agents working together. That's the strategic advantages we offer with our HP Edge solutions. I know it's a, it's a 20 minute call so I wanted to keep it very, you know, at a very high level, super focused and fast paced But I understand a lot of people might have questions.

Dipanwita Mallick [00:14:48]: So I'm willing to take those questions here or offline. So feel free to reach out if you want. And I think few last few slides. So my whole point was that there is this. People right now probably have understood how enterprises are changing. Previously it was all cloud, it's not all cloud anymore. Now people are trying to move away from the cloud, but they also need the benefits of the cloud. So they want to create their private AI infrastructure.

Dipanwita Mallick [00:15:20]: So how the ecosystem or the space has evolved is now there is a huge hybrid AI layer which connects your cloud and also your on prem where you can have workflows moving completely seamlessly between the two environments. But with that I think the underlying factor is we all understand going back to why this session is wanted to keep his infrastructure focus is to bring the attention back to the infrastructure layer. Because for AI to be successful long term, we are thinking long term because this agentic AI is going to change everything. So we have to think long term. When we think long term, I think have the solid foundation is so important for all the enterprises. Especially when you're starting this journey. Figuring out that infrastructure structure along the way is super important. And if we can figure out the right pieces in the right time, then I think enterprises are well positioned to, you know, unlock the true value of the AI.

Dipanwita Mallick [00:16:18]: And I think with that I want to say that, you know, even though there was a glitch in the beginning, but I'm so happy to and excited to share my learnings with you and if any things that I shared resonated with you and you want to continue the conversation with me, then you know, feel free to reach out to me or I can connect you to one of my team members to have the conversation. And thank you all for listening.

Adam Becker [00:16:47]: Thank you very much. Let's open up the floor to questions. We have a couple of minutes to see if any come through and if so I'll post them to you. What are you working on next then? You guys are building this, this environment. Where is your head at in terms of next steps?

Dipanwita Mallick [00:17:09]: Yeah, I think like I mentioned, our mission is to help the enterprises be successful in their Gentica and whatever they need. So we started out as an infrastructure provider like we we provide the right infrastructure and given HP we have this line of very powerful workstations. So what my product is or my team does is offer this high end workstations as a service to the customers. VR workstation as a service, but it comes with the high perform GPUs like I mentioned Blackwell GPUs and all the right best in class hardware that you would need to run this AI workloads then we work with the customers to understand what they need. If we have to integrate some platform over it for orchestration we do that for them. But our point is to make this entire experience seamless for the enterprises. That's, that's what we want to do. Like we don't want to offer fragmented solution to their enterprises right like things are already so complicated we don't want to complicate more stuff to them right so the enterprises need to have the seamless experience.

Dipanwita Mallick [00:18:13]: So that's, that's our mission.

Adam Becker [00:18:14]: So yeah wonderful. I believe that you had your email address. We can drop it in the chat below for people to follow up with you if they have questions or just want to continue the conversation about the environment and infrastructure and how it all fits together. Thank you very much for joining us today.

Dipanwita Mallick [00:18:34]: Thank you.