In the December Reading Group session, we explored A Taxonomy of Agents for Enabling Observability of Foundation Model-Based Agents. Key participants discussed the challenges of building agentic AI systems, focusing on four key capabilities: perception, planning, action, and adaptation. The paper highlighted issues like lack of controllability, complex inputs/outputs, and the difficulty of monitoring AI systems. Early-stage insights drew on DevOps and MLOps practices, and the need for improved tools and evaluation strategies for agent observability. The session fostered a collaborative exchange of ideas and practical solutions.

This November Reading Group conversation covers advanced retrieval techniques, strategies like iter-drag and hyper-drag for complex queries, and the impact of larger context windows on model performance. The Reading Group also examines challenges in generalizing these methods.

October 2024 MLOps reading group session explores the role and relevance of small language models in an era dominated by large language models (LLMs). The author of a recent survey paper on small models joins to discuss motivations for using smaller models, including resource constraints, efficiency, and unique capabilities they bring to certain tasks. Key discussion points include the advantages of small models in specific contexts (e.g., edge devices and specialized tasks), their role in complementing large models, and emerging techniques for leveraging small models to enhance model efficiency and mitigate issues like out-of-vocabulary words. The group also touches on methods for compressing models and the challenges in balancing model size with generalization and task-specific performance.

In our September 12th MLOps Community Reading Group session, live-streamed at the Data Engineering for AI/ML Virtual Conference, we covered the paper "Hybrid RAG: Integrating Knowledge Graphs and Vector Retrieval for Information Extraction." The panel discussed using hybrid RAGs to pull information from unstructured financial documents. The paper's approach combines knowledge graphs with vector-based retrieval for better results. We critiqued the context-mixing methods and lack of graph pruning techniques. Overall, it was a solid session with great insights on improving financial data extraction using hybrid models.

MLOps Reading Group meeting Reading Group Session about Ben Wilson's Machine Learning Engineering in Action book. https://www.manning.com/books/machine-learning-engineering-in-action https://www.amazon.com/Machine-Learning-Engineering-Action-Wilson/dp/1617298719