In this podcast episode, Luke Marsden explores practical approaches to building Generative AI applications using open-source models and modern tools. Through real-world examples, Luke breaks down the key components of GenAI development, from model selection to knowledge and API integrations, while highlighting the data privacy advantages of open-source solutions.

In our journey from concept to production, we focused on delivering consistent behaviors to build user trust. This talk will cover the design and refinement of AI systems using agentic frameworks and deterministic components, emphasizing the integration of continuous learning and human oversight.

The rapid development of Large Language Models (LLMs) has led to a surge in applications that facilitate collaboration among multiple agents, assisting humans in their daily tasks. However, a significant gap remains in assessing to what extent LLM-powered applications genuinely enhance user experience and task execution efficiency. This highlights the need to verify utility of LLM-powered applications, particularly by ensuring alignment between the application's functionality and end-user needs. We introduce AgentEval, a novel framework designed to simplify the utility verification process by automatically proposing a set of criteria tailored to the unique purpose of any given application. This allows for a comprehensive assessment, quantifying the utility of an application against the suggested criteria.

The field of agentic system design is an exciting and quickly developing area with significant implications for software use across various sectors. These systems differ from conventional software by employing decision-making processes that are non-deterministic and often unpredictable. By drawing insights from diverse fields such as robotics, cybernetics, and biology, we can begin to cultivate an understanding of how to construct systems that are more dependable than their individual probabilistic components would suggest.

This panel speaks about the diverse landscape of AI agents, focusing on how they integrate voice interfaces, GUIs, and small language models to enhance user experiences. They'll also examine the roles of these agents in various industries, highlighting their impact on productivity, creativity, and user experience and how these empower developers to build better solutions while addressing challenges like ensuring consistent performance and reliability across different modalities when deploying AI agents in production.

Why can't my agent perform actions on my behalf? How is tool calling different from retrieval? How do I test my LLM tools? How do I evaluate different model's ability to call tools? These and many more questions were some of the main points we've been figuring out as we've been building Arcade AI. This talk will focus on the approaches we tried and the approaches we found that worked. If you want to or are building an Agent that can call tools, this will be a good talk for you.

The internet and its business models are changing. For the last 16 years, Stack Overflow has been at the forefront, helping to shape the future of the web. In this time of disruption comes a necessary time to reflect, change, and break norms. With the expansion of generative AI, many LLM providers are not allowing the world to operate as it has previously. Instead, a new paradigm has emerged: content created by thousands of creators across the Open Web is now being used to train models without respect or attribution for the original creator.Consequently, in the current transformation, human-centered sources of knowledge are obscured but a key component of the AI stack. Companies and organizations lucky enough to host these engines of knowledge production are at a decision point; how do they continue to grow and invest in their communities when the technological landscapes have changed. A change in strategy will allow these companies, AI tools, and the communities and data sets that power them to thrive.

If you think AGI is just around the corner, try building an agent. Today's frontier models often feel like idiot children, capable of brilliant superhuman replies and at the same time making absurd logical errors, while lacking any common sense. Even worse, those errors pile up, making them unreliable. AI Agents hold the incredible potential to change how we work, learn and play. But any company working in agents quickly realizes they're incredibly hard to build, to make reliable and to generalize. Instead, most teams have shifted to building narrow agents that they can scope to a particular problem with lots of glue code, heuristics and prompt engineering. In this talk we'll take a look at why agents are so challenging to build, what we can do about it and whether just when they'll live up to the hype of doing complex, open ended tasks in the real world.