Fireside Chat - The Future of LLMs

Name: Fireside%20Chat%20-%20The%20Future%20of%20LLMs
Uploaded: 2023-07-17

Posted Jul 17, 2023 | Views 605

# Future of LLMs

# LLM in Production

# LLM Applications

speakers

David Hershey

Vice President @ Unusual Ventures

David Hershey is an investor at Unusual Ventures, where he invests in machine learning and data infrastructure. David started his career at Ford Motor Company, where he started their ML infrastructure team. Recently, he worked at Tecton and Determined AI, helping MLOps teams adopt feature stores and scale deep learning workloads.

+ Read More

Daniel Jeffries

Chief Executive Officer @ Kentauros AI

I'm the Managing Director of the AI Infrastructure Alliance, CEO of Kentauros AI, and the former Chief Intelligence Officer at Stability AI. I've also spent time at wonderful companies like Red Hat and Pachyderm. Over the years, I've worn many hats in business, from IT infrastructure engineering, to evangelism, to solutions architecture, to marketing and management.

But primarily, I think of myself as an author, engineer, and futurist who's current obsession is Artificial Intelligence and Machine Learning. More than anything I'm relentlessly curious. I love to learn and think about how things work and how things can be done better.

I've given talks all over the world and virtually on AI and cryptographic platforms. With more than 50K followers on Medium and a rapidly growing following on Substack, my articles have been read by more than 5 million people worldwide.

+ Read More

Demetrios Brinkmann

Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

SUMMARY

Evaluating the performance of language models (LLMs) is a pressing issue for companies working with generative AI. Defining what makes a model "good" and measuring its performance are challenging due to the diverse range of LLM applications. Existing evaluation methods, including benchmarks and user preference comparisons, have limitations in scalability and objectivity. The future of LLM evaluation lies in scaling testing with machine learning systems, such as reward models that capture user preferences, and simulating user sessions to generate comprehensive test cases. These approaches will help developers select models, create effective prompts, ensure compliance, and enhance LLM quality.

+ Read More