MLOps Community
timezone
+00:00 GMT
SIGN IN
  • Home
  • Events
  • Content
  • Tools
  • Help
Sign In
Livestream
LLMs in Production Conference
# Large Language Models

Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them?

In this event, you will hear from practitioners about how they are dealing with things such as cost optimization, latency requirements, trust of output and debugging.

You will also get the opportunity to join workshops that will teach you how to set up your use cases and skip over all the headaches.

Speakers
Meryem Arik
Meryem Arik
Co-founder @ TyTN AI
 Linus Lee
Linus Lee
Research Engineer @ Notion
Lina Weichbrodt
Lina Weichbrodt
Freelance Machine Learning Development + Consulting @ Taktile
Daniel Jeffries
Daniel Jeffries
Managing Director @ AI Infrastructure Alliance
Raza Habib
Raza Habib
CEO and Co-founder @ Humanloop
Harrison Chase
Harrison Chase
CEO @ LangChain
Saahil Jain
Saahil Jain
Engineer @ You.com
Jared Zoneraich
Jared Zoneraich
Founder @ PromptLayer
Cameron Feenstra
Cameron Feenstra
Principal Engineer @ Anzen
Hanlin Tang
Hanlin Tang
CTO @ MosaicML
 Mario Kostelac
Mario Kostelac
Staff Machine Learning Engineer @ Intercom
Vin Vashishta
Vin Vashishta
CEO @ Data By V-Squared
Jerry Liu
Jerry Liu
Co-Founder/CEO @ LlamaIndex
Willem Pienaar
Willem Pienaar
Founder @ In-Stealth
Luis Ceze
Luis Ceze
CEO and Co-founder @ OctoML
 Alex Ratner
Alex Ratner
CEO and Co-founder @ Snorkel AI
Eli Mernit
Eli Mernit
CEO / Founder @ Beam
Justin Uberti
Justin Uberti
CTO and Co-founder @ Fixie
Diego Oppenheimer
Diego Oppenheimer
Partner @ Factory
Gevorg Karapetyan
Gevorg Karapetyan
Co-founder and CTO @ ZERO Systems
Demetrios Brinkmann
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community
 Tanmay Chopra
Tanmay Chopra
Machine Learning Engineer @ Neeva
Jon Turow
Jon Turow
Partner @ Madrona
Daniel Campos
Daniel Campos
Software Engineer - AI @ Neeva
 Anton Troynikov
Anton Troynikov
Head of Technology @ Chroma
Meryem Arik
Meryem Arik
Co-founder @ TyTN AI
 Linus Lee
Linus Lee
Research Engineer @ Notion
Lina Weichbrodt
Lina Weichbrodt
Freelance Machine Learning Development + Consulting @ Taktile
Daniel Jeffries
Daniel Jeffries
Managing Director @ AI Infrastructure Alliance
Raza Habib
Raza Habib
CEO and Co-founder @ Humanloop
Harrison Chase
Harrison Chase
CEO @ LangChain
Saahil Jain
Saahil Jain
Engineer @ You.com
Jared Zoneraich
Jared Zoneraich
Founder @ PromptLayer
Cameron Feenstra
Cameron Feenstra
Principal Engineer @ Anzen
Hanlin Tang
Hanlin Tang
CTO @ MosaicML
 Mario Kostelac
Mario Kostelac
Staff Machine Learning Engineer @ Intercom
Vin Vashishta
Vin Vashishta
CEO @ Data By V-Squared
Jerry Liu
Jerry Liu
Co-Founder/CEO @ LlamaIndex
Willem Pienaar
Willem Pienaar
Founder @ In-Stealth
Luis Ceze
Luis Ceze
CEO and Co-founder @ OctoML
 Alex Ratner
Alex Ratner
CEO and Co-founder @ Snorkel AI
Eli Mernit
Eli Mernit
CEO / Founder @ Beam
Justin Uberti
Justin Uberti
CTO and Co-founder @ Fixie
Diego Oppenheimer
Diego Oppenheimer
Partner @ Factory
Gevorg Karapetyan
Gevorg Karapetyan
Co-founder and CTO @ ZERO Systems
Demetrios Brinkmann
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community
 Tanmay Chopra
Tanmay Chopra
Machine Learning Engineer @ Neeva
Jon Turow
Jon Turow
Partner @ Madrona
Daniel Campos
Daniel Campos
Software Engineer - AI @ Neeva
 Anton Troynikov
Anton Troynikov
Head of Technology @ Chroma
Agenda
3:00 PM
3:10 PM
Opening / Closing

Welcome

Demetrios Brinkmann
3:10 PM
3:40 PM
Keynote

DevTools for Language Models: Unlocking the Future of AI-Driven Applications

In this talk, we explore the thriving ecosystem of tools and technologies emerging around large language models (LLMs) such as GPT-3. As the LLM landscape enters the "Holy $#@!" phase of exponential growth, a surge of developers are building remarkable product experiences on top of these models, giving rise to a rich collection of DevTools. We delve into the current state of LLM DevTools, their significance, and future prospects. We also examine the challenges and opportunities involved in building intelligent features using LLMs, discussing the role of experimentation, prompting, knowledge retrieval, and vector databases. Moreover, we consider the next set of challenges faced by teams looking to scale their LLM features, such as data labeling, fine-tuning, monitoring, observability, and testing. Drawing parallels with previous waves of machine learning DevTools, we predict the trajectory of this rapidly maturing market and the potential impact on the broader AI landscape. Join us in this exciting discussion to learn about the future of AI-driven applications and the tools that will enable their success.

+ Read More
Diego Oppenheimer
Demetrios Brinkmann
3:40 PM
4:10 PM
Keynote

Age of Industrialized AI

The rise of LLMs means we're entering an era where intelligent agents with natural language will invade every kind of software on Earth. But how do we fix them when they hallucinate? How do we put guardrails around them? How do we protect them from giving away our secrets of falling prey to social engineering? We're on the cusp of a brand new era of incredibly capabilities but we've also got new attack vectors and problems that will change how we build and defend our systems. We'll talk about how we can solve some of these problems now and what we can do in the future to solve them better.

+ Read More
Daniel Jeffries
4:10 PM
4:20 PM
Lightning Talk

Reasoning Machines: Differentiating LLM Apps with Ops at 4 Levels of the Stack

A. FMs/LLMs are best thought of as reasoning machines NOT sources of truth. So they need to be incorporated into a broader stack that contains sources of truth.

B. Drawing out the broader technical stack highlights opportunities to differentiate at 4 different levels:

More efficient training and inference of proprietary models with deployment optimization, data tooling, and specialized inference and training hardware Constructing and managing ensembles of multiple hosted models Retrieval from external data sources Reacting on external APIs

C. Challenges of caching dynamic data, security and privacy considerations, latency requirements, and business models, all impact the types of differentiation that can apply to each use case.

+ Read More
Justin Uberti
Jon Turow
4:10 PM
4:40 PM
Panel Discussion

Data Privacy and Security

Diego Oppenheimer
Gevorg Karapetyan
Vin Vashishta
Saahil Jain
4:20 PM
4:30 PM
Lightning Talk

TBC: Why specialised NLP models might be the secret to easier LLM deployment

Meryem Arik
4:20 PM
4:40 PM
Lightning Talk

How LlamaIndex can bring the power of LLM's to your data

LLM's are incredible at different tasks (reasoning, generation, summarization, question answering). Trying to apply LLM's to your data (unstructured, semi-structured, structured), brings a host of different challenges. LlamaIndex provides some of the tools to help solve that

+ Read More
Jerry Liu
4:40 PM
4:50 PM
1:1 networking

Improvised Musical Break

Demetrios Brinkmann
4:50 PM
5:20 PM
Presentation

Efficiently Scaling and Deploying LLMs

Hanlin Tang
4:50 PM
6:20 PM
Workshop

Building Fast AI Prototypes

This event is for those who want to learn how to quickly prototype and ship AI apps. We're going to share a few cheat codes: you'll learn how to use pre-trained ML models, how to build generative AI apps, chatbots, and conversational agents, and how to use GPUs for maximum productivity. You don't want to miss this!

+ Read More
Eli Mernit
5:20 PM
5:30 PM
Lightning Talk

No rose without a thorn - Obstacles to Successful LLM Deployments

There are major challenges to deploying and maintaining LLMs in production. They come in two forms --> infrastructural (incorrect buy vs build decisioning; high costs; high latency; upstream downtimes) and output-linked (output format variability; trust and safety). There are ways around this --> infrastructural(buy while you build; multi-platform redundancy; batching/offline processing) and output-linked (example-based prompting; strong post-processing).

+ Read More
 Tanmay Chopra
5:30 PM
5:40 PM
Lightning Talk

Lighting Talk

Willem Pienaar
5:40 PM
5:50 PM
Lightning Talk

Lighting Talk

5:50 PM
6:20 PM
Presentation

Generative Interfaces beyond Chat

Chat-based interfaces to LLMs are the command-line interfaces for interfacing with modern generative AI systems, and we should be more imaginative and ambitious when thinking about how we'll interact with them in the future. I'll share 5 concrete big ideas for how to design good interactions to LLMs and other generative AI models that I've found helpful in my own work, and hopefully in the process enable you to start to think outside of the chat box (hah, literally!) when building your own products.

+ Read More
 Linus Lee
6:20 PM
6:40 PM
1:1 networking

Guided Meditation Break

Demetrios Brinkmann
6:40 PM
7:10 PM
Presentation

Building Defensible Products with LLMs

Raza Habib
6:40 PM
8:10 PM
Workshop

Pluggable knowledge for AI, using embeddings

GPT and other large models (LMs) are excellent at working with general knowledge, but fall short when it comes to other kinds of data, for example private data like email, or new information like world news. LMs also sometimes hallucinate information when they don't have it available.

In this talk, I'll show you how to use an embeddings store like Chroma, you can turn any data into 'pluggable knowledge' for LM's, and use this powerful capability in your AI application.

+ Read More
 Anton Troynikov
7:10 PM
7:40 PM
Panel Discussion

Cost Optimization and Performance

Lina Weichbrodt
Luis Ceze
Jared Zoneraich
Daniel Campos
 Mario Kostelac
7:50 PM
8:20 PM
Presentation

Using LLMs to Punch Above your Weight!

It is possible for an early-stage Engineering team to put together a really compelling product that uses LLM and other ML techniques. Advances in tooling in the space make it possible to build ML-based features at a similar pace to application development. Making models perform well in production still requires strong understanding of system and infra fundamentals. Sometimes a single large model is not the large tool for the job, but that doesn't mean you can't use ML as a component of the solution.

+ Read More
Cameron Feenstra
8:20 PM
8:30 PM
Lightning Talk

Lightning Talk

8:20 PM
8:40 PM
Presentation

Talk

8:30 PM
8:40 PM
Lightning Talk

Lightning Talk

8:30 PM
8:40 PM
Lightning Talk

Lightning Talk

Sponsors
Tecton
Wallaroo
Petuum
Live in 21 days
April 13, 3:00 PM, GMT
Online
Organized by
MLOps Community
MLOps Community
Live in 21 days
April 13, 3:00 PM, GMT
Online
Organized by
MLOps Community
MLOps Community