LLMs in Production Conference Part III
# Open Source AI
# LLMs
# Prem AI
The State of Open Source AI: Deployment Engines, Licences, & Hardware
Following the LLMs in Production survey results, we picked the 3 hottest topics to bring some clarity in the current fast-paced mess of Open Source innovation: deployment (starting & maintaining a project is an ongoing unsolved problem) licenses (recently "solved"/permissive for a lot of good architectures, but will remain a mess for data for years) hardware (cost-vs-capability ratio is decreasing, but far more importantly, software portability is finally actually being solved!) Find out what, why, and how in this talk.
![Casper da Costa-Luis](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/da-Costa-Luis-Casper-Photo-sq-0e82d564-fcc5-4e0a-9834-62064e134bd4-1695922723500.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Casper da Costa-Luis & Adam Becker · Nov 1st, 2023
Popular topics
# MLops
# LLMs
# AI
# Generative AI
# Machine Learning
# Machine learning
# Interview
# Observability
# MLOps
# GetYourGuide
# Presentation
# Monitoring
# Model Serving
# Cultural Side
# Scaling
# Data Science
# ML Platform
# Responsible AI
# Run:AI
# Python
![Alex Cabrera](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Alex-Cabrera-5335e443-00a0-4974-8cfe-14803011f976-1692792773278.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Alex Cabrera & Adam Becker · Nov 1st, 2023
LLMs and foundation models are unlocking thousands of possibilities for AI-driven products, from writing assistants to art platforms. Despite their abilities, these AI systems are complex and can fail in significant ways, such as producing hallucinations, biased outputs, and more. In this talk, I’ll introduce Zeno, an interactive platform for creating and sharing in-depth evaluations of complex AI systems. Zeno lets users explore the inputs and outputs of any AI system, from text to audio and image models, and create interactive reports. We envision Zeno being the go-to tool both AI developers and auditors use to share reproducible evaluations of AI systems.
# AI Evaluation Reports
# Foundation Models
# Zeno
![Yuze Ma](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Yuze-Ma-15dc1075-3b9b-4f3e-9a12-6ebb0bcc528c-1692797287414.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Yuze Ma & Adam Becker · Oct 31st, 2023
As foundational models move fast in size and capability, the gap between research to production is yet to be completed. To iterate through the models and implementations faster, the toolchain matters a lot. This talk will cover the techniques and experience during this journey from the training of the vanilla foundational models, and optimization comparisons, to user-side system experience optimization.
# AI Applications
# Foundational Models
# Lepton AI
![Finn Howell](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Finn-Howell-a69bd9c1-34f5-458f-86d2-fa89f64720fd-1694535649329.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Finn Howell & Adam Becker · Oct 31st, 2023
How do you write a stress test for an LLM? This talk explores cutting-edge techniques to red-team generative AI and build validation engines that algorithmically probe models for security, ethics, and safety issues. Attendees will learn a framework to manage AI risk spanning the model lifecycle, from data collection through production.
# LLMs Evaluation
# AI Risk
# Robust Intelligence
![Azul Garza](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Azul-b595d5ae-2e30-461a-8697-c746a0d3f3af-1692978362122.png?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Azul Garza & Adam Becker · Oct 26th, 2023
Time series—data ordered chronologically—constitutes the underlying fabric of systems, enterprises, and institutions. Its impact spans from measuring ocean tides to tracking the daily closing value of the Dow Jones. This type of data representation is indispensable in sectors such as finance, healthcare, meteorology, and social sciences. However, the current theoretical and practical understanding of time series hasn't yet achieved a consensus among practitioners that mirrors the widespread acclaim for generative models in other fundamental domains of the human condition, like language and perception. Our field is still divided and highly specialized. Efforts in forecasting science have fallen short of fulfilling the promises of genuinely universal pre-trained models. In this talk, we will introduce TimeGPT, the first pre-trained foundation model for time series forecasting that can produce accurate predictions across various domains and applications without additional training. A general pre-trained model constitutes a groundbreaking innovation that opens the path to a new paradigm for the forecasting practice that is more accessible and accurate, less time-consuming, and drastically reduces computational complexity. We will show how to use TimeGPT in a live demo.
# TimeGPT
# Time Series
# Nixtla
![Noble Ackerson](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Noble-Ackerson-e50c6012-8e36-42d1-b9d9-5bcf8695d7b1-1694513479755.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Noble Ackerson & Adam Becker · Oct 26th, 2023
Embark on an enlightening journey with Noble as he tackles the challenges of integrating Large Language Models (LLMs) into enterprise environments. Understand the inherent unreliability of these models and explore innovative solutions, ranging from when to use vector databases to when to use retrieval augmented generation, that aim to enhance the trustworthiness of LLMs in crucial applications.
# Large Language Models
# Generative AI
# VENTERA CORPORATION
![Greg Diamos](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Greg-D-45568de5-8162-408f-9148-349b47168706-1693319975436.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Greg Diamos & Adam Becker · Oct 26th, 2023
An opinionated view of how to build production LLMs.
# Fine-tuning LLMs
# Building Production
# PowerML, Inc
![Benjamin Harvey](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Benjamin-Harvey-85e337e9-92cc-438f-a08a-fb3fe69abc02-1692796966383.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Benjamin Harvey & Adam Becker · Oct 26th, 2023
AI Squared is an AI platform designed for product owners, data scientists, and enterprise leaders. We empower you to accelerate both predictive and generative AI projects, measure their benefits, and drive significant revenue growth and cost reduction. The largest gap in the LLM developer stack is creating different experiences for users to leverage LLM results. Many companies have only utilized generative AI inside of chat applications. AI Squared has developed a framework that empowers our customers to harvest context and connect to additional content as well as other AI models from across the organization while integrating these insights directly into currently existing tools and applications.
# AI Platform
# LLM Developer Stack
# AI Squared
![Harini Kannan](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Harini-Kannan-5a14d075-41e7-429a-9627-1e00a79d61e9-1694088436338.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Harini Kannan & Adam Becker · Oct 26th, 2023
The rapid adoption of Large Language Models (LLMs) has swept across various industries, inspiring companies to incorporate them into their products, regardless of the industry's domain. Unlike organizations like OpenAI, Google, or Microsoft, most entities view LLMs as powerful tools rather than standalone products. In this context, it remains paramount to uphold core product principles where customer satisfaction reigns supreme. In this talk, we will see how to pick practical use cases for using LLMs, where they are not essentially user-facing chatbots but valuable tools to build new features into existing products - with examples from the cybersecurity domain. We will also see a sample product strategy for how to plan and roadmap LLM features, along with key performance metrics to gauge success.
# LLM features
# Product Strategy
# LLMs
![Charles Frye](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Charles-Frye-1d971231-dc72-4279-80d6-538c22d98fb8-1692785552927.jpeg?fit=scale-down&width=52)
![Sahar Mor](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Sahar-Mor-91d25565-1c68-481d-8db7-13d725b7f3e6-1683043439507.jpeg?fit=scale-down&width=52)
![Sarah Guo](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Sarah-Guo-bd092f5d-4197-430a-a9d0-fe1908da328a-1692803157948.jpeg?fit=scale-down&width=52)
+3
Charles Frye, Sahar Mor, Sarah Guo & 3 more speakers · Oct 26th, 2023
A product-minded engineering perspective on UX/design patterns, product evaluation, and building with AI.
# UX
# Product-minded Engineering
# Building with AI
![Julia Kroll](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Julia-Kroll-5bb72def-e590-4a5c-b91f-647a12c63101-1693326539731.jpeg?fit=scale-down&width=52)
![Adam Becker](https://cdn.gradual.com/images/https://d2xo500swnpgl1.cloudfront.net/uploads/mlops/Adam-Becker-a68a8273-2a35-4474-b321-d637d3188fc0-1707977582187.png?fit=scale-down&width=52)
Julia Kroll & Adam Becker · Oct 26th, 2023
Conversational AI demands low latency for a seamless dialogue between humans and AI. However, engineers face the dilemma that some latency is inherently required in order to process human speech and craft a response. Some incremental wins to shave off milliseconds involve trade-offs against how the AI response could be enriched during the additional processing time. Others simply refactor out inefficiency to obtain more performant results from AI devtools. This talk presents best practices of designing streaming speech-to-text applications, as well as reasons to accept extra latency for the sake of an enhanced product experience.
# Conversational AI
# Humans and AI
# Deepgram
Popular