Optimising Routing and Caching ML Models on Serverless GPU Infra

Paul discusses their transition from a hardware to a software company and the importance of MLOps in machine learning deployments. He introduced "Radish," a research project addressing deployment challenges using Redis for communication and websockets for fast GPU processing. The system allows for multi-environment deployment on a single GPU using libraries like ZeroMQ. Paul demonstrated the deployment process and highlighted the ease of use, scalability, and cost optimization offered by Mystic.ai's infrastructure.