Productionalizing AI: Driving Innovation with Cost-Effective Strategies

The way companies build AI is at an inflection point. Over the last decade, millions of developers and organizations have cultivated the knowledge and skills to build AI systems. Still, on the other hand, many of the most exciting models of late exist behind APIs other companies control with unappetizing cost and data privacy profiles. In this talk, we will explore how the AI/ML infrastructure stack is being affected by vendor APIs and the increasing desire and capacity for reasonable-scale companies to build powerful models in-house.

After the initial excitement caused by the launch of closed large language models (LLMs), many organizations struggled to reach the quality and ROI targets required to deliver production-grade AI projects. Fortunately, the fantastic pace of innovation of the open-source community quickly made it possible to match and even exceed the accuracy of the best closed LLMs with nimble and cost-effective small language models (SLMs). This session will discuss the latest techniques to tailor SLMs to specific domains and company knowledge, reassuring the audience about the feasibility of real-life AI projects. You will learn about the end-to-end model adaptation process with continuous pre-training, model merging, instruction fine-tuning, quantization, and inference.

In this talk, we will walk through the history of AWS-designed ML chips, the hardware & software stacks, and some of the most popular integrations and use cases that our customers care about. We will also look at recent case studies to see how AWS customers have derived amazing price performance for their GenAI workloads using AWS Trainium & Inferentia.