This competition challenged participants to submit a model capable of answering science-related multiple-choice questions. In doing so it provided a fruitful environment for exploring most of the key techniques and approaches being applied today by anyone building with LLMs. In this talk, we'll look at some key lessons that this competition can teach us.

AI models have become orders of magnitude larger in the last few years.

Training such large models presents new challenges, and has been mainly practiced in large companies.

In this talk, we tackle best practices for training large models, from early prototype to production.

In his session Thomas will focus on understanding the ins and outs of fine-tuning LLMs. We all have a lot of questions during the fine-tuning process. How do you prepare your data? How much data do you need? Do you need to use a high-level API, or can you do this in PyTorch? During this talk, we will try to answer these questions. Thomas will share some tips and tricks on his journey in the LLM fine-tuning landscape. What worked and what did not, and hopefully, you will learn from his experience and the mistakes he made.

Leap Labs demonstrates how data-independent model evaluations represent a paradigm shift in the model development process. All through our dashboard’s beautiful Weights & Biases Weave integration.