Fireside Chat - The Future of LLMs

Name: Fireside Chat - The Future of LLMs
Uploaded: 2023-07-17

Posted Jul 17, 2023 | Views 508

# Future of LLMs

# LLM in Production

# LLM Applications

speakers

David Hershey

Vice President @ Unusual Ventures

David Hershey is an investor at Unusual Ventures, where he invests in machine learning and data infrastructure. David started his career at Ford Motor Company, where he started their ML infrastructure team. Recently, he worked at Tecton and Determined AI, helping MLOps teams adopt feature stores and scale deep learning workloads.

+ Read More

Daniel Jeffries

Managing Director @ AI Infrastructure Alliance

Dan Jeffries is the Managing Director of the AI Infrastructure Alliance and the former CIO of Stability AI. He’s also an author, engineer, futurist, pro blogger, and he’s given talks all over the world on AI and cryptographic platforms. With more than 50K followers on Medium and a rapidly growing following on Substack, his articles have been read by more than 5 million people worldwide.

+ Read More

Demetrios Brinkmann

Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

SUMMARY

Evaluating the performance of language models (LLMs) is a pressing issue for companies working with generative AI. Defining what makes a model "good" and measuring its performance are challenging due to the diverse range of LLM applications. Existing evaluation methods, including benchmarks and user preference comparisons, have limitations in scalability and objectivity. The future of LLM evaluation lies in scaling testing with machine learning systems, such as reward models that capture user preferences, and simulating user sessions to generate comprehensive test cases. These approaches will help developers select models, create effective prompts, ensure compliance, and enhance LLM quality.

+ Read More