AI in Production

Generative AI is not going anywhere, but many organizations are struggling to translate a very active research and development activity from POC to production solutions. In this brief talk I'll highlight some of the challenges I think we need to overcome if we want to deploy GenAI solutions at scale and I'll also talk about some of the opportunities this presents.

Data labelling is a key part to fine tuning open-source LLMs. However, poor labelling practices can hurt your LLM's performance. This lightning talk will cover data labelling best practices from hiring, preparing your data and managing your data labelers

Model merging has recently become extremely popular in the open-source community. The idea of merging several fine-tuned models, or combining them into a Mixture of Experts (MoE), led to new state-of-the-art LLMs. This talk introduces the main concepts around model merging and how to implement it using the mergekit library. It provides a notebook to create your own models and directly upload them on the Hugging Face Hub.

In this talk, Matt will walk through Tecton's journey to build a platform that can reliably power large-scale real-time AI applications while requiring nothing more than Python.

"I will walk through some of the key things that I have noticed in the space of Applied AI - what it takes to build AI into products and surfaces that do not contain it. How do you persuade partners of value? How do you get things done? What pitfalls might you run into, and how do you solve them?"

LLM deployment is notoriously tricky, leaving ML teams with little time left to focus on driving business value. So what can we do? If you run or are a part of a data science team working with LLMs, this one’s for you.

The intersection of graphs and Large Language Models (LLMs).

I intend to explore the benefits of combining graphs with LLMs, delving into the engineering aspects while also touching on the practical applications from my startup's perspective. This talk will highlight my recent work and findings on the superiority of Retrieval Augmented Generation (RAG) Knowledge Graphs over traditional RAG with vector databases, underlining the profound implications of their interaction.

Within robotics, we have learned how to train AIs in simulation and leverage simulation to test functionality in a harmless environment. Simulation and AI techniques combined with multi modal LLMs are now used for games. In Convai’s case, we are using it to power the mind of the Non-Player-Character NPC, enabling it to perform actions and react to its environment. Through this talk, I will discuss how we do this and what the learnings from robotics we are productizing are.

The goal of the talk will be to learn how to harness Gen. AI to build the right products for your users, efficiently. It'll cover learnings from different stages of a product, from the idea exploration stage, to hardware capacity planning, iterating on early versions, building early trust with your users, and finally measuring success over the long term.

New LLMs are constantly appearing in the AI landscape, and retrieval augmented generation (RAG) has become a dominant LLM design pattern. What will the future bring? Join Contextual AI VP Product Aditya Bindal for a deep dive into the next generation of foundation models that prioritize customization and privacy.

"I'll tip you $100". "Don't be lazy". Have you caught yourself adding these phrases to your prompts? Prompt engineering is central to developing modern AI systems, but it often devolves into an ad-hoc process that requires tribal knowledge and countless iterations of guess-and-check. We believe prompt engineering can be improved significantly and explore how we can use AI itself to guide prompt writing. Learn how intelligent "prompt editors" and synthetic data generation can supercharge your AI development workflow.

Knowing what to say is often the easy part - actually writing it is hard part. Ghostwriter will learn from your past, mimic your style and tone, include relevant content and write emails that you would have written. This talk discusses the core ideas behind Ghostwriter and how we are able to make an AI not just sound like you, but write exactly what you would have written.

Hallucination detection is a critical step toward understanding the trustworthiness of modern language models (LMs). To achieve this goal, we re-examine existing detection approaches based on the self-consistency of LMs and uncover two types of hallucinations resulting from 1) question-level and 2) model-level, which cannot be effectively identified through self-consistency check alone. Building upon this discovery, we propose a novel sampling-based method, i.e., semantic-aware cross-check consistency (SAC3) that expands on the principle of self-consistency checking. Our SAC3 approach incorporates additional mechanisms to detect both question-level and model-level hallucinations by leveraging advances including semantically equivalent question perturbation and cross-model response consistency checking. Through extensive and systematic empirical analysis, we demonstrate that SAC3 outperforms the state of the art in detecting both non-factual and factual statements across multiple question-answering and open-domain generation benchmarks.