MLOps Community
+00:00 GMT
MLOps Community Podcast
# Generative AI
# LLMs
# Humanloop

Product Metrics are LLM Evals // Raza Habib CEO of Humanloop

Raza Habib, the CEO of LLM Eval platform Humanloop, talks to us about how to make your AI products more accurate and reliable by shortening the feedback loop of your evals. Quickly iterating on prompts and testing what works, along with some of his favorite Dario from Anthropic AI Quotes.
Raza Habib
Demetrios Brinkmann
Raza Habib & Demetrios Brinkmann · Jun 3rd, 2025
Popular topics
# AI Agents
# AI
# RAG
# Generative AI
# LLMs
# Kubernetes
# Synthetic Data
# Google Cloud
# AWS
# LlamaIndex
# MLOps
# Machine Learning
# AI infrastructure,
# DSPy
# ML
# Google
# Innovation
# Microsoft
# Continual.ai
# Robotics
All
Vaibhav Gupta
Demetrios Brinkmann
Vaibhav Gupta & Demetrios Brinkmann · May 30th, 2025
It's been two years, and we still seem to see AI disproportionately more in demos than production features. Why? And how can we apply engineering practices we've all learned in the past decades to our advantage here?
# Programming Language
# LLM
# BAML
50:30
Prithviraj Ammanabrolu
Demetrios Brinkmann
Prithviraj Ammanabrolu & Demetrios Brinkmann · May 27th, 2025
Prithviraj Ammanabrolu drops by to break down Tao fine-tuning—a clever way to train models without labeled data. Using reinforcement learning and synthetic data, Tao teaches models to evaluate and improve themselves. Raj explains how this works, where it shines (think small models punching above their weight), and why it could be a game-changer for efficient deployment.
# Fine Tuning
# Synthetic Data
# Databricks
54:02
Mohan Atreya
Demetrios Brinkmann
Mohan Atreya & Demetrios Brinkmann · May 23rd, 2025
Demetrios and Mohan Atreya break down the GPU madness behind AI — from supply headaches and sky-high prices to the rise of nimble GPU clouds trying to outsmart the giants. They cover power-hungry hardware, failed experiments, and how new cloud models are shaking things up with smarter provisioning, tokenized access, and a whole lotta hustle. It's a wild ride through the guts of AI infrastructure — fun, fast, and full of sparks!
# GPUs
# AI infrastructure
# Rafay
47:50
Samuel Partee
Rahul Parundekar
Demetrios Brinkmann
Samuel Partee, Rahul Parundekar & Demetrios Brinkmann · May 21st, 2025
Demetrios, Sam Partee, and Rahul Parundekar unpack the chaos of AI agent tools and the evolving world of MCP (Machine Control Protocol). With sharp insights and plenty of laughs, they dig into tool permissions, security quirks, agent memory, and the messy path to making agents actually useful.
# MCP
# A2A
# AI Agent
1:04:43
Kison Patel
Demetrios Brinkmann
Kison Patel & Demetrios Brinkmann · May 16th, 2025
The intersection of M&A and AI, exploring how the DealRoom team developed AI capabilities and the practical use cases of AI in dealmaking. Discuss the evolving landscape of AI-driven M&A, the factors that make AI companies attractive acquisition targets, and the key indicators of success in this space.
# AI
# M&A
# Dealmaking
# DealRoom
55:33
Maria Vechtomova
Maria Vechtomova · May 13th, 2025
The world of MLOps is very complex as there is an endless amount of tools serving its purpose, and it is very hard to get your head around it. Instead of combining various tools and managing them, it may make sense to opt for a platform instead. Databricks is a leading platform for MLOps. In this discussion, I will explain why it is the case, and walk you through Databricks MLOps features.
# MLOps
# Databricks
# Marvelous MLOps
52:44
Fausto Albers
Demetrios Brinkmann
Fausto Albers & Demetrios Brinkmann · May 9th, 2025
Demetrios and Fausto Albers explore how generative AI transforms creative work, decision-making, and human connection, highlighting both the promise of automation and the risks of losing critical thinking and social nuance.
# Generative AI
# MCP
# AI Builders Club
49:41
Alon Bochman
Demetrios Brinkmann
Alon Bochman & Demetrios Brinkmann · May 6th, 2025
Demetrios talks with Alon Bochman, CEO of RagMetrics, about testing in machine learning systems. Alon stresses the value of empirical evaluation over influencer advice, highlights the need for evolving benchmarks, and shares how to effectively involve subject matter experts without technical barriers. They also discuss using LLMs as judges and measuring their alignment with human evaluators.
# AI
# Machine Learning
# RagMetrics
1:01:38
Devansh Devansh
Demetrios Brinkmann
Devansh Devansh & Demetrios Brinkmann · May 2nd, 2025
Open-source AI researcher Devansh Devansh joins Demetrios to discuss grounded AI research, jailbreaking risks, Nvidia’s Gretel AI acquisition, and the role of synthetic data in reducing bias. They explore why deterministic systems may outperform autonomous agents and urge listeners to challenge power structures and rethink how intelligence is built into data infrastructure.
# Open source
# Jailbreaking
# Synthetic data
1:01:36
Existing BI and big data solutions depend largely on structured data, which makes up only about 20% of all available information, leaving the vast majority untapped. In this talk, we introduce GraphBI, which aims to address this challenge by combining GenAI, graph technology, and visual analytics to unlock the full potential of enterprise data. Recent technologies like RAG (Retrieval-Augmented Generation) and GraphRAG leverage GenAI for tasks such as summarization and Q&A, but they often function as black boxes, making verification challenging. In contrast, GraphBI uses GenAI for data pre-processing—converting unstructured data into a graph-based format—enabling a transparent, step-by-step analytics process that ensures reliability. We will walk through the GraphBI workflow, exploring best practices and challenges in each step of the process: managing both structured and unstructured data, data pre-processing with GenAI, iterative analytics using a BI-focused graph grammar, and final insight presentation. This approach uniquely surfaces business insights by effectively incorporating all types of data.
# GraphBI
# Gen AI
# Visual Analytics
# Kineviz
# Senzing
1:12:38
Popular