Full Stack Multimodal Multi AI Agents in open ended world exploration // Xiaomei Song // Agent Hour

This talk explores the cutting-edge development of a multi-agent AI system designed for open-ended game environments, with a focus on Minecraft. The presenter will discuss the creation of specialized AI agents, including a Vision Agent for spatial analysis, a Curriculum Agent for dynamic task generation, an Action Agent for behavior execution, a Critic Agent for performance evaluation, and a Skill Manager for knowledge retention. The presentation will highlight how these agents work together, leveraging advanced techniques like Reinforcement Learning, Chain of Thought, and Tree of Thought reasoning to enhance decision-making and adaptability. A key feature of this project is the multi-bot system, which enables multiple AI agents to operate concurrently in the same game world, fostering collaboration and skill sharing. By demonstrating the potential of AI-driven automation and adaptability in complex, dynamic environments, this research opens up exciting possibilities for applications beyond gaming.