In this talk, we explore the thriving ecosystem of tools and technologies emerging around large language models (LLMs) such as GPT-3. As the LLM landscape enters the "Holy $#@!" phase of exponential growth, a surge of developers are building remarkable product experiences on top of these models, giving rise to a rich collection of DevTools. We delve into the current state of LLM DevTools, their significance, and future prospects. We also examine the challenges and opportunities involved in building intelligent features using LLMs, discussing the role of experimentation, prompting, knowledge retrieval, and vector databases. Moreover, we consider the next set of challenges faced by teams looking to scale their LLM features, such as data labeling, fine-tuning, monitoring, observability, and testing. Drawing parallels with previous waves of machine learning DevTools, we predict the trajectory of this rapidly maturing market and the potential impact on the broader AI landscape. Join us in this exciting discussion to learn about the future of AI-driven applications and the tools that will enable their success.

The rise of LLMs means we're entering an era where intelligent agents with natural language will invade every kind of software on Earth. But how do we fix them when they hallucinate? How do we put guardrails around them? How do we protect them from giving away our secrets of falling prey to social engineering? We're on the cusp of a brand new era of incredibly capabilities but we've also got new attack vectors and problems that will change how we build and defend our systems. We'll talk about how we can solve some of these problems now and what we can do in the future to solve them better.

A. FMs/LLMs are best thought of as reasoning machines NOT sources of truth. So they need to be incorporated into a broader stack that contains sources of truth.

B. Drawing out the broader technical stack highlights opportunities to differentiate at 4 different levels:

More efficient training and inference of proprietary models with deployment optimization, data tooling, and specialized inference and training hardware Constructing and managing ensembles of multiple hosted models Retrieval from external data sources Reacting on external APIs

C. Challenges of caching dynamic data, security and privacy considerations, latency requirements, and business models, all impact the types of differentiation that can apply to each use case.

LLM's are incredible at different tasks (reasoning, generation, summarization, question answering). Trying to apply LLM's to your data (unstructured, semi-structured, structured), brings a host of different challenges. LlamaIndex provides some of the tools to help solve that

This event is for those who want to learn how to quickly prototype and ship AI apps. We're going to share a few cheat codes: you'll learn how to use pre-trained ML models, how to build generative AI apps, chatbots, and conversational agents, and how to use GPUs for maximum productivity. You don't want to miss this!

There are major challenges to deploying and maintaining LLMs in production. They come in two forms --> infrastructural (incorrect buy vs build decisioning; high costs; high latency; upstream downtimes) and output-linked (output format variability; trust and safety). There are ways around this --> infrastructural(buy while you build; multi-platform redundancy; batching/offline processing) and output-linked (example-based prompting; strong post-processing).

Chat-based interfaces to LLMs are the command-line interfaces for interfacing with modern generative AI systems, and we should be more imaginative and ambitious when thinking about how we'll interact with them in the future. I'll share 5 concrete big ideas for how to design good interactions to LLMs and other generative AI models that I've found helpful in my own work, and hopefully in the process enable you to start to think outside of the chat box (hah, literally!) when building your own products.

GPT and other large models (LMs) are excellent at working with general knowledge, but fall short when it comes to other kinds of data, for example private data like email, or new information like world news. LMs also sometimes hallucinate information when they don't have it available.

In this talk, I'll show you how to use an embeddings store like Chroma, you can turn any data into 'pluggable knowledge' for LM's, and use this powerful capability in your AI application.

It is possible for an early-stage Engineering team to put together a really compelling product that uses LLM and other ML techniques. Advances in tooling in the space make it possible to build ML-based features at a similar pace to application development. Making models perform well in production still requires strong understanding of system and infra fundamentals. Sometimes a single large model is not the large tool for the job, but that doesn't mean you can't use ML as a component of the solution.