Sign in or Join the community to continue

How Predictive Analytics Stops Budget Overruns Before They Happen?

Posted Aug 24, 2026 | Views 22
Share

Speakers

user's Avatar
Brent Eubanks
FinOps Architect @ Wayfair

Brent is a hands-on leader with a passion for innovating in technologies, building effective teams, and a focus on delivering competitively superior FinOps technology solutions to the business while growing people in the organization and quantifying IT transformation. Design, implement, and operate platforms using AI/ML, a data lake, and local LLMs with prompt chaining and superprompts to optimize the engineer's efficiency.

+ Read More
user's Avatar
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

SUMMARY

Every engineer at Wayfair can now see, in real time, exactly what their code costs, and that's on purpose. Brent Eubanks, FinOps Architect at Wayfair, walks us through what happens when you stop treating AI spend as a finance problem and start treating it as an engineering one.

The story that sticks with you: a team was burning $400k a month on an LLM-driven workflow, until they flipped the whole thing on its head - hard-coded logic doing the heavy lifting, AI called in only when it's actually needed. New spend: $80k. It's the kind of move that only makes sense once you realize more agentic autonomy isn't always the win it's sold as.

From there we get into the machinery Wayfair has built to make cost ownership feel less like a punishment and more like a superpower: guardrails baked straight into coding copilots, predictive alerts that catch a team blowing its budget before the month even ends, a "spend captain" system that pushes budget decisions down to the people closest to the work, and a genuinely strange new question every tech leader is starting to ask: if you were staffing a company with a team of AI agents instead of people, what would that org chart even look like?

+ Read More

TRANSCRIPT

Brent Eubanks: [00:00:00] You should be aware of your quota. So we're providing predictive analytics by team of saying, "Hey, your team is fine. These two people are 400% gonna be over their monthly budget." So we have predictive analytics. So just telling them, "Hey, it's the 6th of the month or 9th of the month today," they'll already know they're way over budget for the month.

Demetrios: Brett Eubanks is a FinOps architect at Wayfair, preparing for a world where AI usage could grow 10X, 50X, or even 1000X. His focus isn't on limiting that growth, but figuring out how to make it economically sustainable. From predictive AI budgets to agents that actively optimize infrastructure, he's exploring what FinOps looks like when software isn't just used by humans, but increasingly works alongside them.

Brent Eubanks: I wanna enable, you know, uh, 10X to 1000X more token use over the next two years. That's not what we're worried about. I think it's this explosion in AI over the next two years. I think that's the bigger risk. If this thing is so smart or it's smarter than all [00:01:00] of us combined, what do we want it to do?

Demetrios: Now we're here with Brent, a FinOps architect at Wayfair. I'm excited to talk to you because you've been driving into where the value is and how to look at what is happening with AI spend for the engineering persona. Can you break this down a little bit more for me?

Brent Eubanks: Sure. So when we're associating, like, the FinOps value message across the business, we're really trying to enable the engineers to do a lot more work, so build more features faster.

Demetrios: Mm-hmm.

Brent Eubanks: We're really trying to amplify and enable our engineers, and we've already seen great benefits, so the productivity gains, uh, let's call it developer quality of life. Mm-hmm. Uh, and then we're not really tracking so much about if it's human code or [00:02:00] machine-generated code. It's more about the business objective that we set, so we're trying to establish objective and outcomes first, and then really enable the engineer to make the right choice with tools so that they can achieve that business outcome.

Brent Eubanks: Hmm. So the tools are changing very quickly. The models are changing, so we're always evaluating different options and engineering patterns. But really, it's about letting them use what they need to. So as the technology advances, they know their, um, apps and services best. So how do we make it so that it feels like a downhill, right?

Brent Eubanks: Management is trying to make it downhill for our developers, right? We want it to be very easy to be a, a AI-enabled developer at Wayfair. Uh-huh. We don't want it to be an uphill climb, so we're trying to enable them with lots of different tools and investments. And then as we track that, we're tracking measures that make sense for management on productivity, quality, uh, business results.

Brent Eubanks: But then as an engineer, what's the quality of life, right? Uh, are you, uh, feeling like it's, uh, [00:03:00] making your job better? Are we communicating better? Uh, are we repurposing things across the whole tech org, not just a single department? So those are some of the things we're kind of looking at from tech leadership.

Demetrios: For me, if you tell me, "Hey, I'm gonna make the path to using AI in your coding As easy, as simple as possible. I love that. That value props is very in line with what I wanna do. How does that work though? Because it feels like it could get very expensive very fast.

Brent Eubanks: Absolutely. And, uh, it does. The piece that we're really focusing on is, uh, let, let's call it, like, three areas.

Brent Eubanks: So you have the, the human developer, so at build time or plan time, those coding copilots. Uh, we have governance and system prompts and checks in there. So we establish a budget, and as we purchase those software, the developer environments, we have seat and token usage budgets. Mm. If they need more budget, they need to go through a human approval step.

Brent Eubanks: So we are kind of gating it. Mm-hmm. It, it... There is a [00:04:00] risk of, uh, a little bit of blocking, but, uh, for cost control, that's, uh, warranted. The other one is, uh, we're not really saying go token max. What we're saying is be intentful. Have good intent. Uh, be in, uh, extreme accountability, as our CEO Fiona would say.

Brent Eubanks: And if they're feeling like owners in a company, and we're all shareholders, then be responsible. And so part of my role at FinOps is making sure they're aware of can you run it cheaper, better, faster? Can you get the work done faster? And as we look at staffing, um, we can do pull requests. So there's a lot of other checks and balances that maybe are being sent in by security or governance- Mm

Brent Eubanks: uh, labeling, cost optimization. They don't really have to memorize every new policy or every new check. So as a pull request comes in, uh, we are doing automated, uh, FinOps guardrails and checks in there, so it logs. And then they can go back, and either a [00:05:00] manager or another agent can start reviewing what, what did we see when they did that pull request?

Brent Eubanks: Is there levels of things that we need to start checking on or preventing? So that's really how we're making it easy for them to use, and then still controlling costs so that we don't blow up the budget.

Demetrios: And what are some of these different input metrics that you're tracking for this quality of life for the developers?

Demetrios: I understand that it's probably like, uh, if I have to go and change one thing and then manually change it, yeah, that, that's one, like, the manual aspect. But there's probably a lot of other pieces, I would imagine.

Brent Eubanks: Yeah, there's a lot. So we're really trying to not focus on, uh Quantity or volume. Every app could have, uh, a different team.

Brent Eubanks: And as the team member is closing out, uh, sprint tickets, uh, there is a section that we're tracking on what code was [00:06:00] generated by machine or service accounts or collaborated on or co-built, uh, and then who's doing the reviewing and approving. So even though we're tracking it, what we're mostly after is, uh, general productivity of responsiveness.

Brent Eubanks: So, uh, how many days did it used to take to do something similar? Mm-hmm. Or what was the cost of that version of the app? Or what does it cost to run my department now? So- What

Demetrios: was that? Sorry. The cost of that version of

Brent Eubanks: the app? The cost of that version. So let's say they make a code change, and now that they've made a code change, if they're using a different model, for example, are you aware that that was a greater than 20% increase in cost?

Brent Eubanks: Right?

Demetrios: So- And is it their own productivity-

Brent Eubanks: Correct ... not allowing them? Correct. So, so as we're triggering in system prompts- Yeah ... they don't know if pricing changed or went down or they switched from a flash response model to a reasoning model. Now the token count went up 8X. Mm-hmm. They may not know that, and so how would we service it back to them?

Brent Eubanks: Or [00:07:00] now you're, uh, not allowed to use that particular open source, uh, model, and you're trying to call it. Or we just implemented a new guardrail policy. Well, he wasn't aware of that, so how would he know that that blocked his deploy? Oh. So there's all these things that go on in production where the engineer doesn't control and, but they need to surface it because why didn't it go or what failed?

Brent Eubanks: And so that's been a lot of the work we're working on now is making sure that they are aware of things that are going in as we're trying to put the fence around the playground, but still balance agility so they can still use the quota they have as best they can. So there's not unlimited budget, as we're seeing recently, but be intentful that you should be aware of your quota.

Brent Eubanks: So we're providing predictive analytics by team of saying, "Hey, your team is fine. These two people are 400% gonna be over their monthly budget." So we have predictive analytics by team, by [00:08:00] service account. So just telling them, "Hey, it's the 6th of the month or 9th of the month today," they'll already know they're way over budget for the month.

Brent Eubanks: So we don't have this huge issue of, "I didn't know. Um, sorry." So we're trying to give them the, um, cost awareness and analytics as a predictive measure, not just, "I burned through my whole budget on the 9th, and now I'm gonna sit here for the rest of the month and do nothing." Right? We don't want that either.

Demetrios: Some folks look at that 400% over developer as an asset, and others look at it as a liability. Do you have a certain view? Because I could see that you go to that person who's saying I'm just burning through tokens. I'm 400% over. And y- you could have a whole spectrum of scenarios where, wow, they're getting a lot done-

Brent Eubanks: Mm-hmm

Demetrios: and they need to teach the rest of the company what they're doing. Or on the other side of the extreme, uh, [00:09:00] they're burning a lot of money, but they're not really doing much. Mm-hmm. So how do you, when you surface these folks that are the high consumers Take them and either enable them more or look at, like, is there the value there?

Demetrios: Mm-hmm. Again, going back to, like, what are the metrics? How do you decide if th- that's good token spend or bad token spend?

Brent Eubanks: Yeah, no, great question. I think it's, you know, two different parts. One is, uh, I, I created this, uh, custom gem so that as an app is getting triggered, instead of me trying to reply as a human, I created a gem so that they could drop in and say, "Hey, this thing's gonna ask you a bunch of questions," and I've already got templates about what we've done, uh, in the last two years.

Brent Eubanks: Mm-hmm. How do you optimize your app? And it's very complicated. Like, tokenomics is a very design-heavy thing, and there's things that, uh, they would know, and there's things they would not know, and then, uh, just keeping up with, uh, current documentation. So it's an advisor companion. The second one [00:10:00] is that it's really up to the department owner.

Brent Eubanks: So we have this concept of, like, a, a spend captain who owns the, the whole budget for the whole department.

Demetrios: Mm.

Brent Eubanks: So if he has, uh, super pod teams of... And they each have engineers, then he could be over in one super pod, but he's still under for the month on his total department. So he now has to go have a conversation.

Brent Eubanks: Does he wanna let this team, uh, use some of the department's budget, or does he have to grow the pie?

Demetrios: Yeah.

Brent Eubanks: And so each month they can come back and say, "We need to grow the pie," or, "I'm going to manage within my budget. D- I don't need any more money." So that's really where we're trying to educate our tech leaders to you need to run your department, uh, efficiently.

Brent Eubanks: So if one person is using it, you need to determine is it proper use of the tokens. Uh, or would you rather give them a lower quality model at one-tenth of the cost, and now he can burn through 10 times more tokens, uh, at the same cost. Yeah. So that's a lot of the work we did in the [00:11:00] FinOps for AI working group.

Brent Eubanks: We were telling the, and educating these team leaders that you don't really need to make him stop. He just needs to use a more, uh, a couple more tactics, uh, on how to use and buy, uh, the, the intelligence he needs. And there's all sorts of different techniques, and those are more emergent. Mm-hmm. And so as we're seeing those patterns come in, uh, there's things we can do on the backend, there's things I can do by purchasing, and then there's things we can do with, um, like, incentives.

Brent Eubanks: So other vendors, they kind of subsidized- Mm ... saying, "Hey, well, come over here. We want you to use this model," or, "Use us code copilot instead of the one you're on." So, so there's incentives that come in through procurement contract negotiations. So my goal is to surface those back to the departments to say, "We can save a lot of money.

Brent Eubanks: Here's a free bucket of credits," or a highly subsidized use of credits. Why don't you direct that engineer over there? So I'm trying to orchestrate, uh, procurement financial contract [00:12:00] insights or competitive credits, uh, things like that. So that's, uh, another part of our job is being able to communicate to tech leaders, say, "Hey, we have so much of this, uh, you know, but it's up to you to figure out if you're gonna use it or not."

Demetrios: I really like how you're bringing it down to the tech leaders to say, "It's your job to deem if this spend is worth it."

Brent Eubanks: Mm-hmm.

Demetrios: Because you probably know, as a leader, if the code that this person is committing, who's spending a whole lot more than anybody else, is it quality? Are they getting more done?

Demetrios: Mm-hmm. Like, they're-- If you're really close to that problem and that issue, you have, like... It will pass the smell test in a way that for you being so removed and just looking at the numbers, it, it's probably a lot harder to deem whether or not that's valuable spend or not.

Brent Eubanks: Correct. And, and that, that's the goal.

Brent Eubanks: We don't wanna limit it. And as we are all starting to roll out, you know, I wanna enable, you know, uh, 10X [00:13:00] to 1000X more token use over the next two years. If we're really not even fully deployed, um, you know, 10%, 30%, that's not what we're worried about. I think it's just, um explosion in AI over the next two years.

Brent Eubanks: I think that's the bigger risk I'm more, more, more w- worried about.

Demetrios: Tell me

Brent Eubanks: more about that. How would we enable 50 times or 100 times more token use by all employees, not just tech, but our non-coder community, right? So as we're trying to have, um, agent swarms, uh, with a mission as a service, um, so if this guy writes this product or service, how would you staff and run it with what AI, um, team of agents?

Brent Eubanks: And that, that's really what I think's coming in the future is if this thing is so smart or it's smarter than all of us combined, what do we want it to do?

Demetrios: Yeah.

Brent Eubanks: And how are we gonna enable that, right? So it's a very different thought model of we only had a, uh, engineer, uh, as the labor. Now, with this [00:14:00] new, uh, RACI matrix, we can start inserting agents next to humans in the RACI matrix- Mm-hmm

Brent Eubanks: of how do you run, uh, day two ops, all right? Observability, like all the stuff JR showed in the slide this morning, that stuff under the glacier part that you can see, that, that's really where it gets meaty, like a lot of indexing and vector stores, KV cache. It gets very heavy, and that's foundational, and that's not the end user story.

Brent Eubanks: The last one is, uh, a lot of our engineers that are, uh, resolving internal use cases, m- most of it could just be a machine learning model- Mm-hmm ... uh, or a script writer. There's no gen AI at all, so there's no token costs. So that's a lot of the work we've seen as architects, uh, as we're deploying these orchestrator tools or agent tool flows.

Brent Eubanks: Uh, some of them don't even need AI. They just need a, a conversational data analytics, uh, layer, and it's nothing generative. It's, it's purely retrieval or it's just an index, uh, search. So I think that's the other thing too is [00:15:00] people are excited to use AI, um, but sometimes, uh, you don't really need to generate stuff if it's just a, a database, uh, conversation.

Demetrios: Yeah, it's that hammer looking for a nail-

Brent Eubanks: Mm-hmm ...

Demetrios: scenario. And I recently saw a post. I can't remember who posted it. I'm trying. For the life of me, I don't remember what it was or the person that posted it. But it was along the lines of we had AI, we had this whole workflow that LLMs or agents were executing, and then we realized we were spending, like, 400 grand a month on this workflow At the scale that we were at and whatever the steps that needed to happen.

Demetrios: They then re-implemented it with a hard-based, you know, hard-coded rule-based solution that would fall back to an [00:16:00] LLM if needed.

Brent Eubanks: Correct.

Demetrios: And they were like, "It brought our spend down to 80 grand a month."

Brent Eubanks: Correct.

Demetrios: And you recognize that that was the first time that I had heard someone using an LLM as a secondary solution- Mm-hmm

Demetrios: versus the first option- Right ... and then the rule-based as the secondary, 'cause I think we automatically go to, oh, we'll just let the agent figure it out. It'll go around and whatever. It has its loops, and it'll retry, and then if it doesn't get it this way, it'll go that way. But each time that it's doing that, you're incurring costs.

Demetrios: And if you already know what you want to get done, then just, like, add a few nodes of intelligence in that graph if you need to, but make sure that you have as much hard-coded as possible, which is a, a little bit of a mind bender for me because we've been going in the opposite direction for the past three, four years, you know?

Brent Eubanks: Yeah. The, the piece that I've always had to do is build data products, [00:17:00] and in, uh, FinOps, uh, you're doing budgets, forecasts, um, savings counts. Uh, you cannot have a generative AI miss a zero or a comma- Mm ... right? It's a failure. So in these, like, high trust, high- uh, highly specific data centric things, that was probably where I spent the most amount of my time watching it go.

Brent Eubanks: Uh, the data analytics is getting pretty good. It's... I, I have these benchmarks of tests over the years that I run, and, uh, I can score, um, the different plug-ins or tools. So, uh, in about the last year, it got really, really, uh, mature. And so, but they're also, um, you can see the models are getting more specific to, um, like video or text or audio.

Brent Eubanks: Th- there's also data analytics. Hmm. And so coming out of the, the big providers, they're open sourcing, but really the, um The structured data that we call it, that wasn't really ever an issue. Like, we've been doing neural networks and, um, reinforcement learning, uh, [00:18:00] algorithm development, time series forecasting, so we never really had a problem with it.

Brent Eubanks: That was the easy part. The, the hard part was you give someone a basic agent, and they instantly start asking it, uh, questions, uh, that it wasn't meant to solve.

Demetrios: Hmm.

Brent Eubanks: So if you give someone a SQL agent, like ask it a SQL question of our product database, uh, it's, it wants to run a SQL query and provide the result answer in a table.

Brent Eubanks: That's it. And they're like, "Well, give me a chart." And they're like, "I don't know what you mean by that," right? So that's where as soon as you give someone this FinOps agent, they try to make it be a digital employee.

Demetrios: Mm-hmm.

Brent Eubanks: And most people are only doing standard operating procedure, like task following runbook stuff today.

Brent Eubanks: So that's the one thing I noticed when I started deploying some of my items. You'd watch them, you know, just hand it and say, "Use it." They would instantly think it was a digital coworker-

Demetrios: Hmm ...

Brent Eubanks: not just a, uh, a single task runbook, uh, thing. So I think people were kind of confused on where the limits are. And in our world of [00:19:00] finance, you know, you just can't, you can't be wrong.

Brent Eubanks: Yeah. So that's why, you know, it's kind of one of the last things that we're gonna see. Uh, but now with, um, auto-generated forecasts, uh, it, we can do bottoms-up and top-down forecasting by all the different scopes and realms. And so now we can have the forecast coming in from the top down at the department level is very accurate, but the tagging is all over the place and various things, especially with AI.

Brent Eubanks: So if they want a different view or cut or filter, now we can go grab a auto-generated bottoms-up forecast. And so now we can arm our individual teams that they've never forecasted before, but now he's accountable.

Demetrios: Hmm.

Brent Eubanks: So it's a view forecast for him to manage a forecast versus actual, but the formal ask for money happens two layers up Right?

Brent Eubanks: So now they've never been accountable to... You own it, right? You own the budget, you own the tech stack. So if you write the app, you own the whole cost, uh, labor, [00:20:00] software, and infra, and that's very new to people, right? That was only a leadership function. So that's part of the cool, um, agentic experience that we're trying to offer, is that advisory companion that sits over their shoulder, and it's a perf eng, it's a solution architect, it's a cost-value unit economics person, and its job is to help that code writer deliver better value apps.

Brent Eubanks: Mm. You know, they call it well-architected framework, or FinOps to me means financially aware DevOps. So I, I come from that realm. So this companion or confidant that now sits over their shoulder and watches their screen, or at night it can go task, and I can task it or anyone. "Go see if there's any more value.

Brent Eubanks: I think they're over-allocating CPU. I think they need more memory, and I think they need to shift to the new model, and it'll, it'll cost this much." And so I can just task it or anybody can, and then it will produce a small recipe, and it'll think all night, [00:21:00] and then it'll go look at a code, and it'll produce code with due diligence charts with a savings estimate.

Brent Eubanks: So we're really trying to reduce the cognitive load on these engineers so that FinOps happens, but it doesn't hurt the performance or latency or errors of the app. And it helps them achieve FinOps excellence, but the AI is working with them or for them now. Before, we would send tons of recommendations, and they had to do all of that.

Brent Eubanks: Now we're trying- Yeah ... to do this, like, with you and for you, and that's really where I think the, the future's going and what we're trying to enable. Uh, at least me, uh, at FinOps working groups and at Wayfair. So that's really what I'm after.

Demetrios: There's so much that you just said there. I, I want to break down so many different pieces of it because the idea of pushing ownership onto the different developers so they understand what their feature or- Mm

Demetrios: whatever it is, the code that they're pushing, how much that's gonna cost in a holistic view- Mm-hmm ... is [00:22:00] incredible. Um, I wanna go down that path in a moment of the difference between me writing on my laptop and then actually shipping it and getting it onto a site like Wayfair or where, on an app, whatever it may be, and the difference in cost there.

Demetrios: But also, what did you need to do in the code base or with the engineering team to enable these FinOps agents to have that kind of visibility in going and looking around all of the different repos and being able to say, "All right, I think we can optimize here," or, "I have this way..." 'Cause I would constantly, if I were a developer, at night when I'm kicking off my 20 different agents, one agent is always gonna be like, "How can I make this cheaper?"

Brent Eubanks: Right. So the way we did it, uh, we got into the developer environments, and with those code copilots, they have system prompts. And so what I did was I went back and said, "Hey, here's the system prompt of, [00:23:00] uh, the libraries and build pipelines and the guardrails and the tagging and all the requirements in there, uh, the automation security items."

Brent Eubanks: I said, "Can we add in some FinOps, uh, prompts into that?" Mm-hmm. So that it knows that as they're writing it, and then there's tools and plug-ins and function calls, so we-- The, the goal is to, as they open up a configuration, let's say they're not allowed to use these, uh, N1s, these old machines on, on Google. So as it sees that show up in the config, if he's making a new code, uh, recommendation, as it's generating the code for him to say, "Hey, I wanna modernize the machine," or, "I'm considering modernizing," it's already doing it in the background saying, "I'm not allowed to propose N1s anymore."

Brent Eubanks: So there's like an exclude list or a desired list, so we can give the desired list or the exclude list, like disallowed. So instead of the engineer having to rem-remember all that or go do another check and, "Oh, well, I heard we weren't allowed to use N1s, but I'm not sure if it's [00:24:00] enforced." So there's this whole thing of knowledge and then enforcing and then code in.

Brent Eubanks: So as the code goes in, if it's pasted or generated, you could either do it as code's being generated or as it's doing a check. So if it's not written in there, then as they do the pull request, then they have, um, like Cursor, for example, has like a plan mode or dry run- Mm-hmm ... or, uh, execute a recipe after.

Brent Eubanks: So there's like a task plan. So every pull request has a task plan. We're just injecting some FinOps checks in there, make sure they're, uh, not doing anything horrible. Mm-hmm. We can go do anything. We can look at cost yesterday versus today or, uh, no sandbox accounts over $500 a month, uh, no GPUs in, uh, non-prod projects.

Brent Eubanks: So there's all these different checks we can do to kind of like govern, o-o-optimize, govern, control, where we We putting a fence around the playground, but they, those aren't things where, um, we're not making the [00:25:00] machine selection for them. We're not telling them what they can and can't do, right? So we're, we're trying to give them all the flexibility to use anything, but don't do dumb stuff.

Brent Eubanks: Don't do disallowed things. And then there are cost controls, and we get to either, uh, let it go to a human loop. So you can do governance your own way. So some departments may wanna enforce it. Some may say, "I just wanna know," or just let a human in the loop check. So it really depends on how they wanna do the governance.

Brent Eubanks: And then the last is the reporting So once we have these systems in place, now that it's logging, now we have a different set of agents that can look at the BigQuery tables or the dashboards and then ask questions. So I don't need to go in and produce the weekly report anymore. I would have the agent go in, I can screenshot everything, and then dump it all in a Notebook LM and then say, "Produce an app out of this."

Brent Eubanks: Or now I'm trying to do a skill. So anything I do every week, just go make a skill for it.

Demetrios: Yeah.

Brent Eubanks: Right? So that's where I'm trying to get out of the analyst work [00:26:00] and get back to the more heavy engineering architect- Mm-hmm ... uh, work. So that's what I think a lot of the engineers are gonna do as well, is anything they do more than twice a week, go make it a skill, uh-

Demetrios: Yeah, and then you set it up on a schedule and it- Correct

Demetrios: triggers and it runs and boom, you have it.

Brent Eubanks: Exactly.

Demetrios: You're getting your daily briefing, your

Brent Eubanks: FinOps. Exactly. Yeah, and then if, if something's going weird, you got a very small blast radius instead of a large blast radius, right? Mm-hmm. It's the big blast radius things that FinOps has to watch out for. But when we're doing, uh, extreme accountability, now, uh, each person, uh, needs to manage their own blast radius.

Brent Eubanks: So that's the, that's the transition of what's really going on in FinOps tooling now is it's not just at the department or the whole service, you know, cloud provider spend level per day. We really want, uh, extreme accountability at the individual engineer level. So how do we give them that, uh, and help them, right?

Brent Eubanks: Mm-hmm. Not, not do everything every day. Uh, we're gonna, we're gonna give you some AI companions, confidants. And the way we work, the charter, the governance, you know, [00:27:00] the way we expect you to work and adhere, we can code it in and all these, uh, background checks are working with them or for them to do it.

Demetrios: You make it really hard for folks to do dumb things, like leave a GPU on for a week

Brent Eubanks: without realizing. Oh, they'll, they'll still do it. But, uh, we're trying to make the not so delayed... Like, we, we had the billing alerts set up, but billing alerts are, like, a day or two behind. So now we're trying to get into, like, real time, uh, alerts or configurations.

Brent Eubanks: Like, don't even let it run. Don't even let the pull request go in first.

Demetrios: Yeah, block that.

Brent Eubanks: Or, uh, scaling checks, uh, that happen from the observability logs, not from the billing data. So I'm doing stuff from the billing data. Primarily that's where I work. Uh, but we'll recommend to the observability team, "You need to be doing these checks."

Brent Eubanks: And I think that's the next thing is have you ever seen, like, a Sankey diagram where it kind of, like, flows in?

Demetrios: Mm-hmm.

Brent Eubanks: Now you can have an agent and then its task and all the different steps it does. Each one has a cost. So now watch that. If any one step changes [00:28:00] by more than 10, 15% or a dollar, uh, notify someone.

Demetrios: Hmm.

Brent Eubanks: And so that's really where you're in these agentic, like, loops and retries is really bad. That would catch it immediately.

Demetrios: Hmm.

Brent Eubanks: And that's an observability thing happening in real time per session. So it may not be systemic, it's just someone's bad computer, uh, is causing it, or some API from someone's causing it.

Brent Eubanks: So it's those types of things where we're really trying to get on the, on the front end of it as, um, a lot of our, you know, revenues and partners are all API, so we have to do a lot of API management and services at scale. Yeah. You know? So that, that's really the, the part that I think is kinda going on now where I think we're getting a lot of value.

Brent Eubanks: It, it's not gl- it's not glamorous work, uh, but it, it keeps the, uh, you know, keeps the costs in check. Um, and you know, we're trying to grow, grow margin and sh- uh, shareholder. So tho- those are some of the things like cost governance, controls, uh, with real meaningful output. Like the good part is the bill didn't blow [00:29:00] up.

Demetrios: Yeah.

Brent Eubanks: Right? And well, that didn't happen by accident. Doing nothing by accident, your bill's gonna blow up. And I think a lot of these things is, you know, you don't really get recognized for it because the bill's just kinda growing like normal- Mm-hmm ... forecast is good. And then

Demetrios: a

Brent Eubanks: few months later- But, but there's no accident.

Brent Eubanks: There's been a lot of work for like two years to really kinda, you know, prepare us for that.

Demetrios: Yeah.

+ Read More

Watch More

ML Drift - How to Identify Issues Before They Become Problems
Posted Dec 08, 2021 | Views 561
# Monitoring
# Presentation
# ML Drift
# ML
# Fiddler
# Fiddler.ai
Inside Uber’s AI Revolution - Everything about how they use AI/ML
Posted Jul 04, 2025 | Views 2K
# Uber
# AI
# Machine Learning
Real LLM Success Stories: How They Actually Work
Posted Jan 31, 2025 | Views 378
# ChatBot
# LLM
# ZenML