LLMs in Production Conference

In this talk, we explore the thriving ecosystem of tools and technologies emerging around large language models (LLMs) such as GPT-3. As the LLM landscape enters the "Holy $#@!" phase of exponential growth, a surge of developers are building remarkable product experiences on top of these models, giving rise to a rich collection of DevTools. We delve into the current state of LLM DevTools, their significance, and future prospects. We also examine the challenges and opportunities involved in building intelligent features using LLMs, discussing the role of experimentation, prompting, knowledge retrieval, and vector databases. Moreover, we consider the next set of challenges faced by teams looking to scale their LLM features, such as data labeling, fine-tuning, monitoring, observability, and testing. Drawing parallels with previous waves of machine learning DevTools, we predict the trajectory of this rapidly maturing market and the potential impact on the broader AI landscape. Join us in this exciting discussion to learn about the future of AI-driven applications and the tools that will enable their success.

The rise of LLMs means we're entering an era where intelligent agents with natural language will invade every kind of software on Earth. But how do we fix them when they hallucinate? How do we put guardrails around them? How do we protect them from giving away our secrets of falling prey to social engineering? We're on the cusp of a brand new era of incredibly capabilities but we've also got new attack vectors and problems that will change how we build and defend our systems. We'll talk about how we can solve some of these problems now and what we can do in the future to solve them better.

LLMs have garnered immense attention in a short span of time - with their capabilities usually being conveyed to the world in low-precision demanding scenarios like demos and MVPs, but as we all know, deploying to prod is a whole other ballgame. In this talk, we'll discuss some pitfalls expected in deploying LLMs to production use-cases both at the terminal layer (direct-to-user) as well as intermediate layers. We'll approach this topic from both infrastructural and output-focused lenses and explore potential solutions to challenges ranging from foundational model downtime and latency concerns to output variability and prompt injections.

As the landscape of large language models (LLMs) advances at an unprecedented rate, novel techniques are constantly emerging to make LLMs faster, safer, and more reliable in production. This talk explores some of the latest patterns that builders have adopted when integrating LLMs into their products.

This talk will cover everything related to getting LLMs to use tools. It will discuss why enabling tool use is important, different types of tools, popular prompting strategies for using tools, and what difficulties still exist.

Chat-based interfaces to LLMs are the command-line interfaces for interfacing with modern generative AI systems, and we should be more imaginative and ambitious when thinking about how we'll interact with them in the future. I'll share 5 concrete big ideas for how to design good interactions to LLMs and other generative AI models that I've found helpful in my own work, and hopefully in the process enable you to start to think outside of the chat box (hah, literally!) when building your own products.

Prizes and swag for the person who can get the nasty prompt injections

Today, large language or “foundation” models (FMs) represent one of the most powerful new ways to build AI models; however, they still struggle to achieve production-level accuracy out of the box on complex, high-value, and/or dynamic use cases, often “hallucinating” facts, propagating data biases, and misclassifying domain-specific edge cases. This “last mile” problem is always the hardest part of shipping real AI applications, especially in the enterprise- and while FMs provide powerful foundations, they do not “build the house”.

Building LLMs that work well in production, at scale, can be a slow, iterative, costly and unpredictable process. While new LLMs emerge each day, similar to what we saw with the Transformers era, models are getting increasingly commoditized – the differentiator and key ingredient for high performing models will be the data you feed it with.

This talk focuses on the criticality of ensuring data scientists work with high quality data across the ML workflow, the importance of pre-training and the common gotchas to avoid in the process.

As a small business, competing with large incumbents can be a daunting challenge. They have more money, more people, and more data, but they can also be inflexible and slow to adopt new technologies. In this talk, we will explore how small businesses can use the power of large language models (LLMs) to compete with large incumbents, particularly in industries like insurance. We will present two examples of how we are using LLMs at Anzen to streamline insurance underwriting and analyze employment agreements and discuss ideas for future applications. By harnessing the power of LLMs, small businesses can level the playing field and compete more effectively with larger companies.