MLOps Reading Group

This paper challenges the LLM-dominant narrative and makes the case that small language models (SLMs) are not only sufficient for many agentic AI tasks—they’re often better. ​🧠 As agentic AI systems become more common—handling repetitive, task-specific operations—giant models may be overkill. The authors argue that: ​SLMs are faster, cheaper, and easier to deploy ​Most agentic tasks don't require broad general intelligence ​SLMs can be specialized and scaled with greater control ​Heterogeneous agents (using both LLMs and SLMs) offer the best of both worlds ​They even propose an LLM-to-SLM conversion framework, paving the way for more efficient agent design.

October 2024 MLOps reading group session explores the role and relevance of small language models in an era dominated by large language models (LLMs). The author of a recent survey paper on small models joins to discuss motivations for using smaller models, including resource constraints, efficiency, and unique capabilities they bring to certain tasks. Key discussion points include the advantages of small models in specific contexts (e.g., edge devices and specialized tasks), their role in complementing large models, and emerging techniques for leveraging small models to enhance model efficiency and mitigate issues like out-of-vocabulary words. The group also touches on methods for compressing models and the challenges in balancing model size with generalization and task-specific performance.