Fireside Chat - The Future of LLMs

Name: Fireside%20Chat%20-%20The%20Future%20of%20LLMs
Uploaded: 2023-07-17T12:30:09.681Z

Posted Jul 17, 2023 | Views 642

# Future of LLMs

# LLM in Production

# LLM Applications

David Hershey

Member of Technical Staff @ Anthropic

David Hershey devoted most of his career to machine learning infrastructure and trying to abstract away the hairy systems complexity that gets in the way of people building amazing ML applications.

+ Read More

Daniel Jeffries

Chief Executive Officer @ Kentauros AI

I'm the Managing Director of the AI Infrastructure Alliance, CEO of Kentauros AI, and the former Chief Intelligence Officer at Stability AI. I've also spent time at wonderful companies like Red Hat and Pachyderm. Over the years, I've worn many hats in business, from IT infrastructure engineering, to evangelism, to solutions architecture, to marketing and management.

But primarily, I think of myself as an author, engineer, and futurist who's current obsession is Artificial Intelligence and Machine Learning. More than anything I'm relentlessly curious. I love to learn and think about how things work and how things can be done better.

I've given talks all over the world and virtually on AI and cryptographic platforms. With more than 50K followers on Medium and a rapidly growing following on Substack, my articles have been read by more than 5 million people worldwide.

+ Read More

Demetrios Brinkmann

Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

SUMMARY

Evaluating the performance of language models (LLMs) is a pressing issue for companies working with generative AI. Defining what makes a model "good" and measuring its performance are challenging due to the diverse range of LLM applications. Existing evaluation methods, including benchmarks and user preference comparisons, have limitations in scalability and objectivity. The future of LLM evaluation lies in scaling testing with machine learning systems, such as reward models that capture user preferences, and simulating user sessions to generate comprehensive test cases. These approaches will help developers select models, create effective prompts, ensure compliance, and enhance LLM quality.

+ Read More