In this MLOps Community Podcast episode, Willem Pienaar, CTO of Cleric, breaks down how they built an autonomous AI SRE that helps engineering teams diagnose production issues. We explore how Cleric builds knowledge graphs for system understanding, and uses existing tools/systems during investigations. We also get into some gnarly challenges around memory, tool integration, and evaluation frameworks, and some lessons learned from deploying to engineering teams.

Recent rapid advancements in Artificial Intelligence (AI) have made it widely applicable across various domains, from autonomous systems to multimodal content generation. However, these models remain susceptible to significant security and safety vulnerabilities. Such weaknesses can enable attackers to jailbreak systems, allowing them to perform harmful tasks or leak sensitive information. As AI becomes increasingly integrated into critical applications like autonomous robotics and healthcare, the importance of ensuring AI safety is growing. Understanding the vulnerabilities in today’s AI systems is crucial to addressing these concerns.

Alex Strick van Linschoten, a machine learning engineer at ZenML, joins the MLOps Community podcast to discuss his comprehensive database of real-world LLM use cases. Drawing inspiration from Evidently AI, Alex created the database to organize fragmented information on LLM usage, covering everything from common chatbot implementations to innovative applications across sectors. They discuss the technical challenges and successes in deploying LLMs, emphasizing the importance of foundational MLOps practices. The episode concludes with a call for community contributions to further enrich the database and collective knowledge of LLM applications.

Ilya Reznik's insights into machine learning and career development within the field. With over 13 years of experience at leading tech companies such as Meta, Adobe, and Twitter, Ilya emphasizes the limitations of traditional model fine-tuning methods. He advocates for alternatives like prompt engineering and knowledge retrieval, highlighting their potential to enhance AI performance without the drawbacks associated with fine-tuning. Ilya's recent discussions at the NeurIPS conference reflect a shift towards practical applications of Transformer models and innovative strategies like curriculum learning. Additionally, he shares valuable perspectives on navigating career progression in tech, offering guidance for aspiring ML engineers aiming for senior roles. His narrative serves as a blend of technical expertise and practical career advice, making it a significant resource for professionals in the AI domain.

Qualcomm® AI Hub helps to optimize, validate, and deploy machine learning models on-device for vision, audio, and speech use cases. With Qualcomm® AI Hub, you can: Convert trained models from frameworks like PyTorch and ONNX for optimized on-device performance on Qualcomm® devices. Profile models on-device to obtain detailed metrics including runtime, load time, and compute unit utilization. Verify numerical correctness by performing on-device inference. Easily deploy models using Qualcomm® AI Engine Direct, TensorFlow Lite, or ONNX Runtime. The Qualcomm® AI Hub Models repository contains a collection of example models that use Qualcomm® AI Hub to optimize, validate, and deploy models on Qualcomm® devices. Qualcomm® AI Hub automatically handles model translation from source framework to device runtime, applying hardware-aware optimizations, and performs physical performance/numerical validation. The system automatically provisions devices in the cloud for on-device profiling and inference. The following image shows the steps taken to analyze a model using Qualcomm® AI Hub.

Demetrios chats with Zach Wallace, engineering manager at Nearpod, about integrating AI agents in e-commerce and edtech. They discuss using agents for personalized user targeting, adapting AI models with real-time data, and ensuring efficiency through clear task definitions. Zach shares how Nearpod streamlined data integration with tools like Redshift and DBT, enabling real-time updates. The conversation covers challenges like maintaining AI in production, handling high-quality data, and meeting regulatory standards. Zach also highlights the cost-efficiency framework for deploying and decommissioning agents and the transformative potential of LLMs in education.

Demetrios chats with Egor Kraev, principal AI scientist at Wise, about integrating LLMs to enhance ML pipelines and humanize data interactions. Egor discusses his open-source MotleyCrew framework, career journey, and insights into AI's role in fintech, highlighting its potential to streamline operations and transform organizations.

Lloyds Banking Group is on a mission to embrace the power of cloud and unlock the opportunities that it provides. Andrew, Michelle, and their MLOps team have been on a journey over the last 12 months to take their portfolio of circa 10 Machine Learning models in production and migrate them from an on-prem solution to a cloud-based environment. During the podcast, Michelle and Andrew share their reflections as well as some dos (and don’ts!) of managing the migration of an established portfolio.