Why You Can’t Ignore Foundational Data Systems When Building Reliable Multimodal AI Solutions

Other than vector databases, the data systems have largely remained the same for the past few decades, and are now being repurposed to serve a fundamentally different workload, namely multimodal AI. Even if we ignore the technical debt and inefficiencies, which is not advisable, these data solutions do not provide easy ways to visualize and inspect data, track its provenance, or keep it consistent, even though this data now serves as a foundation for increasingly critical AI use cases. With years of experience studying various multimodal data types across various industries, and building a database to cater to these requirements, we have learnt some best practices and how to avoid pitfalls. These lessons would be the premise of this talk.