AI Hype vs. Real Value

Demetrios: [00:00:00] What's going on everyone? Today, we are talking AI hype versus real business value with Manish Dasaur. He's working at PwC. He's been in the data and AI game for over 20 years, helping clients, over 100 different clients, navigate the AI disruption field, and also evaluate and deliver business value through the strategic adoption and application of data, AI, and now agentic AI efforts.

Demetrios: Huge shout out to PwC for supporting this episode. Let's get into it.

Demetrios: I'll tell you what I was building real fast because I think this is in line with what we're gonna talk about right now as far as value and looking at costs. I built a little widget that can track the spend, my API spend [00:01:00] across different providers. Mm-hmm. And I gamified it a little bit, so every X amount of money, you get to choose if it's $20 increments or $100 or $1,000, it will go cha-ching and-

Demetrios: so that you know you just spent some cash, and- Yeah ... you wanna make sure that you have eyes on that. A lot of times we don't even realize how much we're spending, especially if it's going through the API. Yeah. And next thing you know, maybe it's your boss or your boss's boss that's trying to drill down and say, "Hey, what's going on here?

Demetrios: What are you getting from this?"

Manish Dasaur: Yeah. Yeah, I couldn't, I couldn't agree more. I mean, look, I'm glad you're tracking it. I think it's something that's also surprising a few organizations, right? Like all of the AI programs they launched, they were all predicated on a ROI, ROI equation that they, they got themselves aligned to and approved internally, and that ROI equation starts to really get some pressure and stress [00:02:00] if you're not careful about the compute cost and the tokenization cost and all those things that can add up.

Manish Dasaur: So I, I've got a lot of clients right now thinking about how do I apply FinOps to that discipline? And to your point, they haven't gotten quite to the Vegas cha-ching sound, but, uh, certainly are thinking about it in the same way. How do you really mitigate that cost?

Demetrios: Yeah. I've heard the term tokenomics become- Yeah

Demetrios: more and more popular.

Manish Dasaur: Yeah. I

Demetrios: like that. And it's really just looking at your spend and recognizing where's the value from it. I was talking actually at length last week with a bunch of FinOps folks on how they're classifying it and how they're deciding what is valuable spend versus what is a little bit superflous.

Demetrios: Yeah. I think that you have some cool thoughts on where and how to justify your different initiatives with AI. And so maybe- Right ... we can even just like start with What does [00:03:00] getting value from AI look like in your opinion?

Manish Dasaur: Yeah. Well, let's first, let's first set a target, right? When I think about clients who are doing this well, I think they're reporting about like a 30% improvement on value, right?

Manish Dasaur: So if they were, for example, if they were measuring, uh, code production or software development lifecycle production for their teams, what they're reporting is, "Hey, previous to giving my team a series of AI tools to help them through SDLC, prior to that, we were seeing about 100 sprint points come out of that sprint," right?

Manish Dasaur: So I gave my team a task, they're gonna go develop a bunch of code, and in the past, they're kind of averaging 100 sprint points of output per sprint. Now I've enabled them with a series of tools, arming them to accelerate through the software development lifecycle. These tools are often writing user stories, they're often writing test scenarios and cases.

Manish Dasaur: Obviously, they're doing code assist and code generation and debugging and things like that. And what they're kind of reporting back is [00:04:00] now that team is able to get through 130 points per sprint, right? So they're kind of measuring it that way. They're measuring the output that way. If in the past it was 100 points, now it's 130 points.

Manish Dasaur: So that's one example of what good looks like, right? It's kind of a 30% efficiency gain. If I think about some other clients, they're applying AI to, to some, some more aggressive workflows in finance or HR and supply chain, also reporting about a 30% automation efficiency in those cases. So when we think about how do organizations get value from AI, let's first start with what do we mean?

Manish Dasaur: What is value? And I think 30% roughly is a good target for organizations to think about, uh, when they think about value from AI, from automation, and what it can drive. Now, how to get to that value is always the challenge, right? And, and there's a few points I would share, Dmitry. One Focus on the workflow, not the use case, right?

Manish Dasaur: In, in, in client examples that I get to work with, clients that have-- clients that describe their AI [00:05:00] initiative as, you know, we're targeting this pilot case here and this proof of concept here, and we're working on this use case here, oftentimes I feel like they don't see the end-to-end, the EBITDA value that they would want to see.

Manish Dasaur: Whereas when clients are saying, "Hey, look, I'm gonna focus on a workflow. I'm gonna embed AI into that workflow, and if the workflow previously took me 20 steps and 100 humans, now it'll take me five steps, five AI agents, and 15 humans." And that delta is the value in EBITDA value that they can take home.

Manish Dasaur: So if you think about it like I've got an accounts payable workflow, or I've got an accounts receivable workflow, and I'm gonna embed AI into that workflow, right? So I know in step one a human's gonna do that. At step two an AI agent's gonna do that. Step three the invoice is gonna get created. So when I start applying AI to the actual workflow and I can look at the workflow end-to-end and compare what my old one was to what my new one [00:06:00] was, that's how I'm seeing clients really get EBITDA value.

Manish Dasaur: Not disconnected use cases, connected workflow redesign with AI generating the value. So that would be kind of point one. So let's not get into isolated tools. Let's think about a connected workflow and how AI really changes that workflow

Demetrios: I, I wanna double-click on the idea there real fast about how you were saying that 30% increase.

Demetrios: And one thing that I've heard folks talk about is how they will look back a year ago and recognize certain big pushes that they've had to do and what productivity looked like around there, how much they-- code they were shipping, whatever these- Yeah ... metrics were that they were tracking, and they can then compare that to now.

Demetrios: And it's, it's not necessarily apples to apples, but it's a much better way of deciding, "Hey, how much are we spending [00:07:00] on AI versus how much- Yeah ... more are we producing?"

Manish Dasaur: Yeah.

Demetrios: And then you can see that in a way that isn't just asking your employees to tell me how much more value do you think AI is providing to you?

Demetrios: How much more productive are you? That self-reporting metric, I think, is a little bit biased, and I'm always weary when I see those. Uh, but I just want-- I wanted to talk about that, and there's so many other things on this workflow piece that I wanna dive into, but I knew you were gonna say something else.

Manish Dasaur: Well, I mean, look, I think your point's a good one. In fact, every transformation initiative that I've seen really be successful is always challenged to prove value ongoing basis, right? I wanna-- Every transformation, if within the first quarter you wanna be able to start to demonstrate value, you wanna be able to update or increase that value quarter over quarter.

Manish Dasaur: To your point, it's gotta be very, very quantifiable, right? So the other maybe point I'll add to how do [00:08:00] organizations make sure they get value, uh, what I've seen a lot is organizations will seek to prove the ROI, and they'll set up a champion, challenger kinda test, right? So as they roll out AI automation, they'll have a control group working in the old process, and they'll have a test group working in the new AI automated process.

Manish Dasaur: And at least for a couple of cycles, they'll compare the outputs. They'll compare the results. Depending on the use case, the comparison might be different, but they'll compare the results so they can get themselves to a quantifiable value statement, and then find a way to tweak it, improve it over time, right?

Manish Dasaur: So I think that's super important to, to have that measure of whether you're measuring cost or speed or revenue uplift, whatever it is that you're measuring, but have that champion, challenger kinda mindset set up. That way, to your point, you're not guessing it, you're not talking about it, you're actually demonstrating with some numbers, and that I think is a, is a winning formula for organizations.

Manish Dasaur: Yeah.

Demetrios: Yeah. You know, I saw that, and I think these days [00:09:00] there is no software engineer that I know that doesn't wanna use some kind of a coding agent. Yeah. So it's not necessarily What-- Are we gonna do AI-

Manish Dasaur: Yeah ...

Demetrios: versus no AI with, uh, especially software engineering and even product development, that type of thing, those internal type tools when- Yeah

Demetrios: we're dealing with software? I think the really difficult part here is you've got some software engineers that are burning a ton and getting a ton done, and then you've got other software engineers, AKA me, who are burning a ton and not getting a lot done.

Demetrios: And so how do you distinguish between that efficient burning of tokens versus the non-efficient ones?

Manish Dasaur: Yeah. No, that's a great point, and I think we're gonna talk about a little bit of the, the FinOps angle as well. And look, I, I don't think a lot of organizations have found that answer quite yet, right? I do think, um, a lot of organizations have been pushing AI [00:10:00] everywhere, AI for everyone, right?

Manish Dasaur: And, and while that's fantastic and the right thing to do, the, the cost of that can be quite high, right? But I do think now organizations are starting to shift a little bit. They're thinking about, "Do I use the right model for the right use case? I don't have to use the same model for all use cases, so I can kinda right-size the model to the use case I'm driving.

Manish Dasaur: That can help me with some cost. I can do workflow engineering or prompt engineering in a way that can help me manage my cost," right? So those-- And then, of course, I've got the FinOps angle talking about, to your point, the cha-ching thing that you were describing, kinda- ... every time I use a, every time I use that compute, use that token, I've got a measurement that way.

Manish Dasaur: And like I said, I, I think all of that has to be baked into the ROI equation of the program, of the transformation that we're seeking to drive. Um, but I will tell you, overall, I do think organizations are reporting good benefits. 30% benefits are hard to ignore. When you think about the size of these organizations, if you can drive a significant reduction in operating cost or a significant growth [00:11:00] in revenue generation, then it, it does, at the large, AI benefits will exceed the value associated to it.

Manish Dasaur: Yeah.

Demetrios: Yeah. So you gave us the point number one, which was these workflows and trying to figure out where you're plugging AI into. Yeah. I wanted to mention that I have a theory that the reason you saw so many successful use cases with support chatbots and support agents-

Manish Dasaur: Yeah ...

Demetrios: was because that is a field where there's very clear SOPs, and because of that, it's all mapped out.

Demetrios: It's easier to create these workflows like you're talking about and go end to end. And if you really need to, you can escalate up to a human when there's something that will stump the agent or it's not in the books. But as a friend of mine told me, especially in call centers, you routinely get folks who are very new at their jobs, [00:12:00] and all they're doing is looking through the SOPs.

Demetrios: Yeah. And they're doing a poor job at that, and an agent can do it much better. So that idea of like having the workflows defined and then being able to plug in the agents or the LLMs where you can makes a ton of sense to me.

Manish Dasaur: I-- Look, and customer care is an excellent example, so let's double-click on that for a second 'cause I gotta tell you a fun story.

Manish Dasaur: As, as, as important it is-- as, as it is to architect it into the workflow, and I agree with you, customer care is a fantastic example for that, it is still freaking hard. And let me tell you-- Let me, let me give you a couple of things that, in my opinion, stump. Uh, I had a, I had an opportunity to work with a large telecommunications company, and they were, they were seeking to do exactly what you just described.

Manish Dasaur: Eighty percent, seventy-five percent of the calls coming into my call center, I want an AI agent to be able to handle them and only take it into a human when I need to for that remaining twenty, twenty-five percent. So that's the, that's the, that's the attempt. That's the workflow automation. So I, I would say [00:13:00] the two hardest challenges we found in that work was, one, as you know, when you start with a model, it's relatively generic.

Manish Dasaur: You have to give it enough context and training to make it specific and useful in the workflow that we're talking about, right? So for example, I've got a generic customer call center type of model. It needs to be trained in my company's policies, inventory, plans, things like that, so that when a customer calls, I've got relevant information to give it.

Manish Dasaur: So when you think about training a model like that, um, a customer could be calling about anything. They could be calling about my invoice, my trip, my phone, what's wrong with it, et cetera. So the context, the amount of training you have to give that model is quite unique, and some of that training comes from previous call logs that the customer care reps have written down and written down.

Manish Dasaur: So of course, we're gonna feed that into the model, right? But the model also needs to know about, um, when Dmitri calls in. Let's, let's use an example. Dmitri calls in and says, "Hey, I'm traveling through Asia [00:14:00] next month. Mr. Cellphone Provider, would you plead a-- please add roaming to my plan," right? "So I can go around Asia and use my phone."

Manish Dasaur: Now, for an AI agent to be able to handle that call, first it needs to know who Dmitri is. So it's gonna need a way to connect into this... I'll call it a CDP, a, a customer data platform, customer 360, whatever terminology you use that so the, so the model knows who Dmitri is. How many lines does he have? Has he paid his bill on time?

Manish Dasaur: What phone does he have? Does he- Yes. How often does he travel? What plan does he have today, right? So it needs to know who Dmitri is. Then once it knows who Dmitri is, it's gonna ask you some clarifying questions. What countries are you traveling to? What dates are you gonna be traveling in? Based on the answers you give it, it needs to update the algorithm and go into the inventory of plans I could offer you and select the one that we think is the right one for you, right?

Manish Dasaur: Then I've gotta make that offer to Dmitri on the phone. Dmitri's gonna say, "Yeah, okay, I like that plan. Give me [00:15:00] that one." He's gonna confirm the cost is, I don't know, an extra $10 a month, and he's gonna say, "Good, add it for that month." Then the model needs to have the authority to go back into your invoice and confirm that you are now gonna pay $10 more, charge your credit card, and update your plan.

Manish Dasaur: That's quite a complicated transaction. It needs to connect to-- Your billing system is wr- likely written in, in one, uh, like one platform. Your, your, your plan system is likely in another platform. Your customer system is likely another platform. So there's a bunch of API connectivity, a bunch of integration that you've gotta solve for, bunch of context and training you've gotta solve for.

Manish Dasaur: So it, you know, for those kinds of things, it can be quite complicated, but that's why I think having the right governance approach, right structure approach, and, and a plan to attack that workflow end to end is super critical.

Demetrios: Don't forget, you, h- if w- I'm calling you on the phone, you have to be doing this with very low latency.

Manish Dasaur: That's right. That's right. And you remember, like the models of the [00:16:00] past, they were taking voice, going to text, generating a response in text, and bringing it back to voice, which created a lot of latency, right? But now in the new world, we can go voice to voice. We don't have to go to voice to text to text back to voice.

Manish Dasaur: So we can go to voice to voice, so the latency is certainly starting to get better. Um, but I still think training models, contextualizing models, and then applying all the integration you need with all the enterprise corporate systems that models need to pull from and interact with in order to satisfy the customer query is quite a tall order, right?

Manish Dasaur: And that's why I think organizations need a, a practical approach to solving that problem.

Demetrios: Yeah. So there we go. That's the workflows on point number one.

Manish Dasaur: Yeah.

Demetrios: You had more points, and I remember I cut you off, so keep it rocking on the different pieces that you wanted to hit on.

Manish Dasaur: Well, the last thing I-- Look, we've talked about focus on workflows.

Manish Dasaur: We've talked about, you know, use the champion challenger model to really, to really prove out the value. The third point I would [00:17:00] just tell you is I do think it's very important for leaders to build the culture of innovation, right? Uh, the, the only places where I've really seen AI be adopted is when leaders are rewarding innovation and pushing the organization to reimagine how they do it.

Manish Dasaur: It's a hard thing to do, right? People have been running these workflows for two decades in their jobs, in their roles, and now we're saying, "Hey, that workflow is going to change." So if you can in a, in, in a leadership way, in an organizational way, find a way to reward a culture of innovation, reward a culture of let's try something new, let's adopt something new, I think that's super critical to your success as well.

Manish Dasaur: And I'll just tell you, man, like I think our-- Uh, we did some surveys on this stuff. I think we saw like eighty-eight percent of companies say they're doing AI, but only thirty-three percent of companies say they're scaling AI, right? So I still think there's quite a barrier here, and as organizations go from, "I have 10 agents," to soon, "I'll have [00:18:00] 100," to soon, "I'll have 1,000," right?

Manish Dasaur: These are the kinds of things that are gonna help them get there

Demetrios: How have you seen this idea of rewarding AI and creating that culture of innovation being successful across organizations? Especially the larger organizations that I can imagine you have folks that are set in their ways and they've been doing something for a while, and it's very hard to change.

Manish Dasaur: Yeah. It's a combination of things, I think, right? But I, I do think I see organizations seeking to reward, promote those who are adopting innovation. I've even started to see organizations who are measuring how often you use AI, right? If you're a regular consumer, you're doing your job, organizations are starting to measure how often do you prompt, uh, your, your local LLM or your local agentic solution?

Manish Dasaur: How effectively are you actually using AI? And that's becoming one of the criterias for, for your evaluation, for your performance reviews, right? And I think [00:19:00] adopting a culture of innovation and, and a culture that pushes the boundary, I think is a good thing to do

Demetrios: Yeah, I've seen that at certain companies that can remain unnamed.

Demetrios: Yeah. But it's also almost like the double-edged sword is that it becomes toxic, and then you- Yeah ... have token maxing or just prompting to prompt. Yeah. And, uh, if you only are going off of that, then it can be dangerous, too. So it, it's like everything, it's quite nuanced in that regard.

Manish Dasaur: Yeah, absolutely.

Manish Dasaur: Absolutely. But I think, you know, if you've got-- I, I think the best thing you can do as a leader is embrace the change yourself, right? Be the example that you're seeking for your team to be. Use AI in your everyday. Use it for the right use cases. Be mindful of how much you're spending on it. But also, you know, don't hesitate to drive top-line growth or bottom-line growth with AI, and be the best example of that.

Manish Dasaur: And if you can do that as a leader, then hopefully the rest of the organization can, can use that and follow.

Demetrios: Yeah, bring them with you. Yeah. And where have you [00:20:00] seen-- It feels like you-- this is a nice segue into where you can fail and where you've seen these efforts fall flat.

Manish Dasaur: Those who get it right versus those who stumble, right, is always a, is always a good question to kind of think about.

Manish Dasaur: And, and, and it helps clients think about this too, because then they avoid those same mistakes that others have failed at. The first one I would tell you is, uh, the concept of AI everywhere I don't think is the right concept for most clients to think about. When you think about AI everywhere, uh, you end up incurring a lot of cost for not a lot of value.

Manish Dasaur: So I-- the first kind of point of guidance I would give clients there is you don't wanna think about it like AI everywhere. Of course, we want AI to be pervasive. That's not what-- Of course that we, we want that, right? But we also want AI to be very focused on those high-value use cases that you think are gonna be game-changing for your organization.

Manish Dasaur: You also might wanna get AI focused and start with some low-risk use cases that won't be super sensitive to the market or to your brand. So I think you-- The first, the first [00:21:00] point of guidance I would share is don't think about applying AI everywhere. Instead, think about it as, "I've got this fantastic tool.

Manish Dasaur: Of course, eventually it will be pervasive in my organization. But if I'm starting out or if I'm in those stages going from 10 implementations to 100 implementations, I wanna be strategic about where I use it. I wanna be prescriptive about where I use it. I wanna measure the value that it creates," right?

Manish Dasaur: So the first thing I would tell you is it's not AI everywhere, it's AI applied and utilized in those workflows and those use cases that are gonna generate the biggest bang for my buck, right? So that's the first point maybe I would share. Second point I would share is, look, you have to watch out for the compute costs, right?

Manish Dasaur: I think, um, I think consumption-driven cost models, uh, sometimes create challenges. I think it's very, very important that organizations have the appropriate governance set up so they're, so they're-- so they've got some oversight, and it's not a surprise. The surprise is what I worry about, right? A lot of clients have been enamored with what AI can do.[00:22:00]

Manish Dasaur: They haven't really focused on what do the costs look like as it starts to scale, and often they can get surprised, right? So I think there you've gotta-- first of all, you've gotta architect for efficiency. Um, one, it's not one model that fits all use cases, right? So it's very important that you think about what am I trying to do, and I select the right model for that.

Manish Dasaur: If I'm doing image creation, I might pick a certain model. If I'm doing deep thinking, I might pick a different model. If I'm doing PowerPoint creation, then I might do a different model, right? So using the right model for the right use case will help clients mitigate cost. Two, I-- you know, the prompt engineering discipline is, is one that I, you know, I have to tell you is gonna become increasingly important.

Manish Dasaur: The type of prompt you put into a model can generate different types of cost as well. So prompt engineering and making sure we've got the model architected, the workflow design architected to minimize token usage, I think is, is the second thing that I would kinda think about there. And then lastly, I-- you, [00:23:00] you started the conversation with FinOps, right?

Manish Dasaur: But having a financial way of tracking cost per workflow and enforcing that budget, enforcing that governance, uh, I would certainly say is a, is a key thing for clients to do. Um, so yeah, what did I hit on? I hit on, uh, I hit on not AI everywhere. The cost model is super important. And maybe the last point I'll st-- share here, share here is, uh, as AI becomes smarter, as the reasoning capability of these models becomes better, we can apply them to more complex use cases.

Manish Dasaur: And as the autonomy gets better, as they're able to execute actions on their own behalf better, now I can have them make complex decisions and execute actions against those complex decisions. But what I also see happening as those two factors are improving is clients are seeking to use AI in more sensitive use cases.

Manish Dasaur: I want an AI-- we talked about customer care. I want an AI agent handling my customer care call. Well, that's very important [00:24:00] because your brand, your policies all need to be-- is how your AI agent needs to behave in a way that's consistent with your brand, in a way that's consistent with your policy, so your customer has a consistent experience.

Manish Dasaur: I want an AI agent approving purchase orders, and now in this case, I've got an AI agent making a financial decision on your behalf and approving purchase orders. Money's actually going out the door based on a decision that an AI agent has made. So as I think organizations seek to use AI in that way, seek to put AI in more sensitive use cases, really having a robust framework and methodology for testing accuracy, for testing hallucination, for testing responsible behavior, right?

Manish Dasaur: For essentially auditing agentic behavior is gonna become super important for our clients. That's something at PwC we're pretty excited about 'cause obviously we do a lot of risk and auditing and, and, you know, we'd like to extend that kind of thinking to your AI world as well, right? And I think in the workforce of tomorrow, if I [00:25:00] imagine it to be humans and AI working together, we've got to be able to really review, validate, audit that work product to make sure it's consistent with what we're doing.

Manish Dasaur: And as we gain more trust there, that'll be the unlock of allowing AI to participate and really make decisions in sensitive use cases. And just like, just like the cost point, I think the testing point, the responsibility point is still an area that a lot of our clients haven't explored just because it's relatively new, and I think that will be a key unlock for the, for the industry as well

Demetrios: Have you thought through how that could potentially look?

Manish Dasaur: We, uh, absolutely. We're starting to build capabilities now that can test, audit, validate results that are coming out of an LLM model. As you know, that's not easy to do. Um, in the past systems we defi- we designed was one input has a series of finite outputs.

Manish Dasaur: Let's say it has five. I can test for those outputs relatively simple. I can put the input in. I [00:26:00] can validate that I got the one of the five outputs, and if I did, then the test succeeds. Much, much harder to do in the agentic world, in the LLM world. One input could have a million different outputs. So we are now building frameworks and testing methodologies that essentially use AI to test AI.

Manish Dasaur: Um, but to be able to do that at scale, that's really the only way to do it, right? So yes, we are absolutely starting to, starting to build frameworks and capabilities that we can take to clients and offer them some help in being able to test and validate AI results. I do agree with you, however, like today, when you go buy a car, there's a J.D.

Manish Dasaur: Power associate certification, so you know you're buying a car that you can trust. I do agree with you. The, the, I, I don't know what the answer will be yet. I don't think the marketer industry has, has aligned on what the answer will be yet. But I do agree, I think clients and organizations are gonna want a way to essentially certify or some, some sort of validate that this agent is good to go.

Manish Dasaur: I can use it in a sensitive use case. I can put it in front of my customers. I've got a way to mitigate [00:27:00] the risk to my brand and to my policy. And I certainly think that once, once the industry has aligned on the right solution, that'll allow us to take a leap forward. In the meantime, I think some of the frameworks and the agentic work that we're doing is exactly how clients should be thinking about that answer.

Demetrios: Yeah, I really wonder how it would look and what it would shape out to be. I've, uh, joked around about how you potentially would have verified MCP servers, but it's not necessarily... What's so weird about this is that you can't quite say like, well, maybe you can. Companies are SOC 2 certified, right? And that means that they've done a bunch of stuff and they've gotten that checklist of things, uh, and they've passed their SOC 2 certifications, and there's more stringent versions of that and less stringent.

Demetrios: Potentially every agent that's out there you can ask what you can just like [00:28:00] have a SOC 2 type certification for the agents that are out in the wild, but it feels much harder to police and it also feels like that might not be the right abstraction. So I'm a little bit at a loss for what and how you would audit it.

Demetrios: What I do know is that the idea of just evaling the output, like you were saying, that's not gonna get you anywhere.

Manish Dasaur: Yeah. And I, you know, I, I, I completely agree. I think we-- I think the answer to exactly how it's gonna be done is still being formulated and it, and it is very nuanced still. It's still very use case dependent, right?

Manish Dasaur: But I do think responsible AI, trusting AI, uh, is still, is still quite a challenge, right? When I speak with organizations and we're talking about a complex use case, I usually get a mix of opinion of executives on the table. There are some executives that are like, "Yeah, good to go. Love it. Why wouldn't we do it?

Manish Dasaur: Let's go." Right? There are other executives who are, who are focused on the financial picture and they can't ignore the benefits that AI can drive. [00:29:00] But then there's always a, you know, executive or two that has a lo- a higher trust quotient that feels like we've gotta be able to demonstrate the, the clarity, the accuracy, the, the compliance in order to move forward and I think that's gonna be an important lock for the industry.

Manish Dasaur: Absolutely.

Demetrios: Another piece that I wanted to hit on that you mentioned and I tend to ponder about quite a bit is in these workflow type scenarios, how some of the optimization efforts can be just figuring out where to have that human sit in the loop.

Manish Dasaur: Yeah.

Demetrios: If at all. Yeah. Because maybe the ultimate optimization is that you can let the agent just go wild.

Demetrios: Yeah. Uh, and depending on the use case, like you were saying, if it is this low stakes, um, kind of high value use case, well, let it run wild and see if it works. But then if an, if a human's getting pinged [00:30:00] consistently about something that an agent is doing, uh, is the value really there? I don't know. I guess if, if I don't have to do it, I'm gonna be happy to have a computer do it.

Demetrios: I'm not doing math by hand, right? Even if it... So I do understand that, but, uh, the whole idea, if you look at a workflow and you have these graphs and you have these nodes and you're thinking, "Well, an agent can take over this process, and then we're gonna have a human in here when it comes to approving invoices or anything that has to do with money."

Demetrios: What happens with me, I don't know if you've had this happen, is that when I'm asked to verify stuff that the agent will throw at me, I just like automatically say, "Yeah, accept. Let's see what happens," type thing. And so I'm worried that that type of a habit will get instilled in us as we start trusting [00:31:00] the AI more and more, and next thing you know, we don't even look at it, and we're there as a box that you have to check.

Manish Dasaur: Well, I mean, I will tell you it's an evolving scale as, as the way I see it. Most clients who are implementing use cases, implementing workflows with AI today certainly are having a human in the loop, right? Especially for, for the ones that I would classify anything as medium or high sensitivity, they're certainly keeping a human in the loop.

Manish Dasaur: But it's also with an evolving scale, right? The intention is these models get smarter, they learn over time, they get more effective over time. So the intention is that, hey, when you first start with that automation, maybe you have the human look at 50% of what the model's per- performing, right? And as you start to gain more trust, as you start to gain more confidence that the mado- model is behaving accurately, and as the model becomes smarter and more efficient at what it's trying to do, maybe you can get that 50% down to 20%, down to 10%, right?

Manish Dasaur: And that's how you kind of move up the chain. Eventually, I do think just like when you [00:32:00] punch in a number to your calculator today, you trust the answer. You're not double-checking that work today, right? But eventually, I do think we get there. But I would advise clients and organizations to think about that as an evolving scale.

Manish Dasaur: For the... If you've got a sensitive use case or a medium sensitive use case, you wanna keep a human in the loop. You wanna validate that you're getting the right answer. And once you really start to prove yourself that over a few quarters, then you can start to reduce the amount of human in the loop that you need.

Demetrios: Okay, so I wanted to also talk to you about this idea of now you have AI working within your company, you're having success, you can do more with less is what we're hearing a lot of folks say. And I was literally just talking to a friend this morning about how I don't buy into the idea of, "Oh, we're all going to be able to do more with less.

Demetrios: That means that we're going to not need as many people to work." Because [00:33:00] if we have All of this free cash flow that's being spit off now-

Manish Dasaur: Yeah ...

Demetrios: we're assuming that companies are not going to reinvest that into their companies and either hire more people or potentially lower prices.

Manish Dasaur: Yeah. Right. I agree with you.

Manish Dasaur: I think our research indicates the same, right? We don't-- We, we certainly see, I think s- more than 75% of jobs will require reskilling or upskilling, but that doesn't mean those jobs are going away or that hiring will stop. It just means they get reskilled and upskilled, right? So we certainly see the trend heading that way.

Manish Dasaur: Look, I, I think about it as the future of the workforce is humans plus agents working together, right? The benefits are too great to ignore. I think we will see that. But largely, I, I-- my hypothesis is more closely aligned to yours. Uh, I think the job reduction, the job displacement is relatively modest at this point.

Manish Dasaur: I think the job reskilling, upskilling is much more [00:34:00] relevant. But I al- always tell folks that, look, I... This might be a controversial thing to say, but I, I will, I will say that if you're in a job function, regardless of kinda what your function is, if you're an accountant or if you're a salesperson or if you're a supply chain person, and if you, if, if you're, if you're kind of at the bottom 25% of other accountants and you're the bottom 25% of other sales leaders, then you probably will be challenged with AI.

Manish Dasaur: Because if you're, if you're, if you're a low performer in that way, you're likely doing a series of mundane tasks that AI can replace. But if you're in the top 50% of those roles, if you're the top 50% of accountants, if you're in the top 50% of sales leaders, then AI is gonna assist you in being able to do a lot more.

Manish Dasaur: So I kinda think about it as for those who are good at their jobs, th- they're gonna think about AI as a massive tool set that they've just gotten, and now they can go from being good to great at their jobs. And, and that'll allow them to create more value. Organizations will hire more for those skills.

Manish Dasaur: We [00:35:00] might see a little bit of rotation from back office to front office, just like you're talking about. As the back office gets automated, I can dedicate more time to, to revenue growth. So yeah, I think we'll see all of those. But I think if I'm a, if I'm a person in the labor force, I always say this too, I don't think AI is gonna take your job, but I think another human that's using AI, if you're not using AI, might take your job, right?

Manish Dasaur: So I kinda-- The way I kinda think about it as whatever your function is, you need to be thinking about how can I do more in my role? How can I do more in my function w- with using AI? Uh, and that's gonna not mean more hours. That's not gonna mean more work. It's actually might mean less hours and less work, but it's gonna mean more output, and that's how I think, that's how I think humans should be thinking about that.

Demetrios: Yeah. In every job, I have yet to come across-

Manish Dasaur: Yeah ...

Demetrios: a job that doesn't have just a, an absurd amount of rote work- Yeah ... that I would happily automate away- Exactly ... any day of the week. And I [00:36:00] guess I'm very privileged to be talking about, like, this kind of blue collar work. Uh, it's a much different story than if we're...

Demetrios: Or blue collar is it white collar? I'm getting confused here. On, basically, I'm sitting behind a keyboard- Yeah ... and doing it. So I'm much more privileged to be able to sit behind a keyboard and do my work in that way, and I know that there's a lot of clicking around that I can automate away. Yep.

Demetrios: I will say that I am fully on board with a ton of rote work that can just get out of what I need to do- Yeah ... and allow me to focus on what is important. And the real key here is this discernment of knowing what's important and knowing where I can add that extra value [00:37:00] But I still, I still will, like, say this until I'm blue in the face that what I see with companies is that they're not going to just say, "All right, cool.

Demetrios: We now can do the work of 100 people with 50 people. We're only going to have 50 people now." Right. And then just continue to do the work of 100 people. No. What I'm seeing is they're going to take 100 people, use the 100 people, but just have the output now of double that. So now they have maybe the output of 200 people.

Manish Dasaur: Exactly.

Demetrios: And it, it's, uh, it's one of those ones... Actually, I was just seeing, a friend sent me something on one of these big corporations that did exactly that, where they realized, "Oh, because of AI, we are now going to not need these positions." [00:38:00] But instead of this trope that you're hearing thrown around a lot of, "All right, we got rid of 20% of our workforce because we're able to do more with less, and AI is the reason for these firings," they said, "No.

Demetrios: What we're gonna do is we're going to purposefully re-skill these folks," like you were talking about.

Manish Dasaur: Yeah. I mean, I... It's spot on. Spot on, right? Like, I, I think about it as AI is gonna reset your operating model. That's how you should think about it as an organization. It is gonna reset your operating model.

Manish Dasaur: W- as we automate the workflows, that means the operating model changes, and when the operating model changes, that means people's jobs change. Not that those jobs go away, those jobs change. The operating model has changed. I might need less people doing one function, but I need more people doing a different function.

Manish Dasaur: Exactly. And that other function creates more value, and the people that I have in that function can use a bunch of tools for the automated mundane tasks, right? So I, I completely agree. I think you've gotta think [00:39:00] about the operating model is changing. I'm essentially now an organization that has humans and agents working together, and when I think about that equation, when I think about that peanut butter and jelly sandwich, the operating model of that should be different than the operating model today, and that will f- that will create the job change, the job re-skilling that I think we're gonna see in the market.

Demetrios: Yeah, I was just on a podcast a few weeks ago, and the title of the podcast was "We're All Managers Now," because we've all... We now have these- Yeah ... agents that we can go and spin off, and if we manage them correctly, hopefully it makes our lives 10 times easier, and we're- Exactly ... still able to get everything done and more.

Manish Dasaur: Exactly. Exactly right.

Demetrios: So, I mean, anything that I should be asking you that I haven't?

Manish Dasaur: No, I think, um, I think we've hit on some... I've, I think we've hit on the right topics, right? Um, I, [00:40:00] I guess I would just part with transformation over experimentation, right? Um, I think the first topic we talked about, how to get really good value from AI, the best examples of that, the most value I'm seeing clients generate is when they're applying it to an end-to-end workflow transformation, not an experiment here, not a use case here and a POC there.

Manish Dasaur: So that, I would kind of wrap us up with that would be point number one. Point number two, I would say governance. Cost governance, risk governance, all those g- those governance is, is a key lever of success, right? So, and it's separating leaders from laggards, right? So if you've got the right governance set up, FinOps set up, it, then, then I think you're gonna have some success.

Manish Dasaur: Uh, and then, you know, lastly, I think mindset matters, especially early on. A lot of clients are seeking to make a shift. They feel like it's gonna be an organizational shift, an operating model shift, and it is. I think it's very important for leaders, but all the way up and down [00:41:00] the corporate work pyramid to really adopt this mindset of, of course, we're gonna innovate, of course we're gonna relearn.

Manish Dasaur: You know, I think the, the, this is the age of continuous learning, right? If you went to school one year ago or if you went to school 20 years ago, all good, but this is the age of continuous learning. Everyone, whether you're 20 years in the game or one years in the game, is gonna need to find a way to continue to learn new capabilities, new tools, new work automations that are coming.

Manish Dasaur: Being able to learn those things, apply those things to my job and, and make my work output even stronger and better, that's the secret to success now.

Demetrios: And the beauty is that you can use AI to learn about AI and how to better- It's never been

Manish Dasaur: easier.

Demetrios: Yeah.

Manish Dasaur: Yeah. Look, I joke around about this stuff. It has never been easier to learn.

Manish Dasaur: This is the age of information. In the past, people had to go travel and, and walk to the top of a mountain to sit down with a guru and learn something. Then people had to pay thousands of thousands of [00:42:00] dollars to attend a university to go learn something. It has never been easier than today to learn, right?

Manish Dasaur: There's so much information available, so it's never been easier to learn, but the onus is on all of us to always be learning and always be applying, and if we can do that, then, then we're gonna be successful in the workforce.

Demetrios: All right, Manish. I appreciate you, man.

Manish Dasaur: So great talking to you. So much fun.

Manish Dasaur: Thank you for having me.