Mercado Libre Gateway: Challenges in GenAI Adoption and a Use Case

Ai in Production

Slides: https://drive.google.com/file/d/1JE-4RzB8DXlNSi-OHD4gSU4Ni4Jvv-g5/view?usp=drive_link

Adam Becker [00:00:05]: It's so useful to actually talk to people who've actually solved actual problems. And Lina Paola, I believe that you're here to tell us about a very specific use case that you had been working on. So we're stoked to have you. I'm going to have you share your screen. Let's see. Okay. Hey, the stage is yours. I'll be back in ten minutes.

Lina Paola Chaparro [00:00:31]: Thank you. Well, hi, my name is Lina Chaparo. I'm in machine learning project leader at Mercado Libre. I have been working leading teams to offer a set of tools for the entire life cycle of machine learning systems. Currently, I'm focusing on making available the use of generative artificial intelligence in Mercado libre. I'll be talk about challenge in this adoption and our gateway as a solution and finally a use case and our days artificial intelligence has become one of the most important advance of the digital era. It has revolutionated way we interact with technology and has opened up a world of possibilities in all areas of our daily life. The use of generative artificial intelligence has the potential to take mercadolibre to a high level.

Lina Paola Chaparro [00:01:24]: We are in an exciting and challenging area where the combination of our human skills with the capabilities of generative artificial intelligence allows us to innovate and create unique solutions for our clients. Our mission as a company is democratize commerce and financial services to transform the lives of millions of people in Latin America. Imagine the variety of possibilities and applications for using this technology. Infinity solutions with high potential and impact. Well, how to take advantage of Gen AI to do it? And what challenge do we face? First of all, think about using generative artificial intelligence that not sound complex, but thinking about using it on a large scale guarantee multiple interactions at the same moment, large amounts of requests per minute, which even exceeds the limits of the models. Secondly, think about all the possible users between the company who can take advantage of the use of this technology. We are more than 50,000 employees who do not necessarily have computer knowledge. How to make it available and even worse, how to be aware with the large branch of models and the changing market day by day.

Lina Paola Chaparro [00:02:43]: Additionally, how to have traceability of the use, how to obtain metrics to monitor quality, response times and cost, how to make it visible at different levels to understand the impact, and finally, how to use it safely where information security is our priority. These and other questions led us to build a gateway to make the use of Genai available. In Mercado Libre, we built a system that provides management, monitoring and control to consume Gen AI implementation of this gateway architecture was possible quickly thanks to our internal platform fury, which allow us to scale the solutions without worrying about some benefits of having gateway architecture. It allows centralized management and control of communications between different systems and applications. This is very important for us because we have a big ecosystem of elements, flexibility and adaptability. The Gerpoy can easily adapt to different protocols, standards and communication technologies. This provides flexibility to integrate systems and applications, resulting in a more scalable and adaptable infrastructure. Increased security the gateway acts as a security barrier between different systems and networks.

Lina Paola Chaparro [00:04:07]: It can provide encryptation, authentication and authorization functions to protect sensitive data and communications. Performance optimization we can optimize performance by performing tasks such as intelligent routing and rate limits. This help us to reduce latency and improve application response times, leading to a better end user experience. Facilitates integrations and interoperability the gateway facilitates integrations of systems and applications from different vendors and technologies. Currently, we have available four of the most important llms providers that facilitate and unify the user experience. In addition, we are in capacity to have a fallback system with a logic to guarantee the continued service. Simplify architecture the gateway can simplify the architecture of our computing solutions by acting as a single entry point for different systems and applications. This reduce complexity and direct dependency between systems, making the architecture easier to maintain on scale and finally centralized billing given the different supply of providers, models and therefore cost our gateway centralized consumption and the associate cost.

Lina Paola Chaparro [00:05:25]: In addition, we develop our own playground as a tool hub to centralize solutions based on Gen AI that can be used by any one employee. In Mercadoly in 2023, we have more than 16,000 of Unix users interacting with that and for more coder users, we built SDK toolkit to facilitate and simplify the user experience. In summary, having a gateway, we can offer benefits such as centralized management, flexibility, enhanced security, optimized performance, facilitation of integrations, multiproider and with option to have a fallback system, centralized billing and simplified architecture. These benefits helps improve efficiency and effectiveness of communications in a complex technology environment. It is aligned with our internal ML Ops practices to provide a series of practice and tools that improve the quality, performance, scalability and security of the model. Now I'm going to share with you a productive use case. The problem was drive recommendations in products or items related to the user's interest. The objective was enhance the user experience by creating customized wardings with high shipping products and of course take advantage the power of an LLMs to improve the algorithm of notifications and increased pushes in our marketplace.

Lina Paola Chaparro [00:07:00]: All these based on a user interaction with an item. For example, reviews, questions and fiber. We send a push notification encouraging them to push us the item, always leading them to a landing page with recommendations based on that item. That is when you are seeing a publication of a product, a review asking questions. We are using the LLMs to answer the question about what are the top ten items that you will be interesting in buying basic in your preference and the idea that the user at the end finish buying them. Some of the challenge that we face in this case was we have a real time application, then we need to give the LLM some time to generate the wording. Then we have the challenge of limits and latency. We need to be able to generate wardings in the ascent or language of each country.

Lina Paola Chaparro [00:08:00]: In Latin America we can speak in Spanish, but different Asians such as neutral Spanish or vioto platinum Spanish or brazilian Portuguese have multiple prompts and made it dynamic with prompt versions. We are able to experiment with new prompts to give a style when communicating with the customers throughout notifications be able to offer benefits such as free shipping or interest free installments or any new benefit that arise on Mercado library. Like a loyalty experiment. We see improvements in the recommendations and we expect a positive impact in core metrics like nps with the use of these new models and we continue improvement the generation of recommendations and benefits in the marketplace. I finish by telling you that today we have more than 150 user case using our gateway. We are in continuous improvement and our gateway allow us to make these technologies available in our company and generate the transformation and impact that we are looking. Feel free to contact me by LinkedIn if you have any questions.

Adam Becker [00:09:13]: Wonderful. Thank you so much and thank you for walking us through that use case. It's always very good to see how people are actually deploying these things and how they end up touching users. Please stick around for the chat. I'm sure people will have questions so stay there and also if people can find you on the slack afterwards, that would be excellent. Lena Palace Tavara thank you so much for joining us.

Lina Paola Chaparro [00:09:37]: Thank you.