In this episode of Agentic Conversations, we sit down with Ambud, Principal Engineer at Pinterest responsible for general technology efficiency, fresh off delivering a controversial keynote on AI infrastructure optimization at scale. Ambud walks us through his Five Layer Cake framework - a structured approach to driving efficiency across every level of the AI stack, from silicon and hardware procurement to model selection, inference engine design, and governance. We explore how decisions compound across layers to unlock real business growth, and how the wrong choices can lock you into expensive commitments for years.

As Large Language Models evolve, the real challenge isn't just generating text—it's remembering context, maintaining state, and managing historical data effectively. How do you provide LLMs with persistent, long-term memory without overwhelming context limits or skyrocketing latency and costs? ​Whether you're building stateful AI agents, implementing advanced Retrieval-Augmented Generation (RAG), or managing enterprise-grade vector and relational data, this meetup covers the practical architectures and trade-offs behind modern LLM memory systems.

Preparing an MLOps repository for AI coding agents requires eliminating friction points that cause AI hallucinations, such as fragmented toolchains and conflicting CI/local scripts. To create an "agent-ready" environment, teams should adopt a "single-gate pattern" that unifies execution commands across all environments, while ensuring that standard AGENTS.md instructions are strictly enforced by automated, machine-readable rules. Additionally, optimizing test performance for isolated MLflow environments—such as using a SQLite template database pattern—and managing dependency vulnerabilities through explicit, tested overrides are crucial. Ultimately, building a deterministic, rigorously checked repository allows AI assistants to bypass tooling drift and efficiently produce correct, production-ready code.