Sign in
Sign in or Join the community to continue

Serving ML Models at a High Scale with Low Latency

Posted Jan 20, 2021 | Views 311
# Presentation
# Model Serving
# Salesforce.com
Share

Speakers

user's Avatar
Manoj Agarwal
Software Architect @ Salesforce

Manoj Agarwal is a Software Architect in the Einstein Platform team at Salesforce. Salesforce Einstein was released back in 2016, integrated with all the major Salesforce clouds. Fast forward to today and Einstein is delivering 80+ billion predictions across Sales, Service, Marketing & Commerce Clouds per day.

+ Read More
user's Avatar
Demetrios Brinkmann
Chief Happiness Engineer @ MLOps Community

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

+ Read More

SUMMARY

Serving machine learning models is a scalability challenge at many companies. Most applications require a small number of machine learning models (often <100) to serve predictions. On the other hand, cloud platforms that support model serving, though they support hundreds of thousands of models, provision separate hardware for different customers. Salesforce has a unique challenge that only very few companies deal with; Salesforce needs to run hundreds of thousands of models sharing the underlying infrastructure for multiple tenants for cost-effectiveness.

+ Read More

Watch More

Deploying Machine Learning Models at Scale in Cloud
Posted Apr 14, 2021 | Views 564
# Data Science
# Notebook
# Machine Learning
# Autodesk.com
Building Data Infrastructure at Scale for AI/ML with Open Data Lakehouses // Vinoth Chandar // DE4AI
Posted Sep 17, 2024 | Views 1.3K
Trust at Scale: Security and Governance for Open Source Models
Posted Sep 12, 2025 | Views 229
# Generative AI
# Security and Governance
# JFrog