Traditional vs LLM Recommender Systems: Are They Worth It?

Arpita Vats [00:00:00]: So I'm working on mostly how we can recommend the right content.

Demetrios [00:00:04]: Why is it different than traditional recommender systems?

Arpita Vats [00:00:07]: I have to explicitly go and ask it like take this signal, take this signal. And when I'm using LLMs I don't have to explicitly go, okay, take this signal. Okay. See what this person has interacted to.

Demetrios [00:00:18]: So basically you're abstracting away the feature engineering that you should be doing.

Arpita Vats [00:00:22]: Yeah.

Demetrios [00:00:25]: I like posts on LinkedIn just to show support to friends. Doesn't mean I want to see 50, 50 more posts like that.

Arpita Vats [00:00:33]: Yeah, but the model doesn't know that. Right? So you're liking it just because how.

Demetrios [00:00:38]: Do I tell the model that? How do I say this is a support like, not a interest like?

Arpita Vats [00:00:44]: Yeah, because it counts. Any reaction that you make to the pose, that counts as a plus for you. So this is something that you're interested to know more about. Because if you're not interested, you might not like or comment on it. Right. Imagine it as like a cluster.

Demetrios [00:00:56]: Yeah.

Arpita Vats [00:00:57]: So if you have two clusters and then one cluster has all the IT industry people and other cluster has like all finance people. So if one of the finance guy likes his post, it's pretty much sure the model is pretty much sure that other person is going to like it. So they will show that post to the other person and then once you like it, then it gives the signal for them and it goes on like that. So it's basically two different cluster, every different profession or every different people based on their activities or interaction. It's being in one cluster and they belong to the cluster where they have similar interests. So that's how if one of the user likes it, they are pretty sure in that cluster the other people will like it as well.

Demetrios [00:01:36]: And then occasionally there's probably one that crosses over clusters.

Arpita Vats [00:01:41]: I think it's very rare that I would cross over your cluster because if you see based on how it decides, the cluster is based on whatever you have entered in your profile information and whatever type of content you have been liking so far.

Demetrios [00:02:00]: So even it takes as a feature what is on my profile.

Arpita Vats [00:02:04]: Your information is basically the most key point for the model to understand what actually you're going to like. So that's why whatever you enter in your profile, it's the first entry point in order to make any recommendation.

Demetrios [00:02:16]: Yeah, my profile is pretty satire these days. I am the Vibe manager at Vibe Ops community. So I don't know how well the models get.

Arpita Vats [00:02:28]: It gets confused and it does weird things. Right.

Demetrios [00:02:32]: It's going to Default back to the old stuff.

Arpita Vats [00:02:35]: Yeah, it's just like you. And if we get confused, we do whatever we can think of.

Demetrios [00:02:39]: Right. So that's the user facing side that I'm seeing. You're working on the back end of it. Right.

Arpita Vats [00:02:46]: So I'm working on mostly how we can recommend the right content. That user may have more and more reaction to it or they like those content. And then also on the recommendation side, we want to make sure that whatever we are recommending is adding value to you. It's not some random recommendation that we are doing here and there. And to get those things we need to know what are your interest. What time period is the one where you like to see most of the content? Maybe in the morning you're in hurry and you don't want to see too many contents, you just scroll and just you leave. So maybe that's not the ideal time for us to give you the best content. And then maybe after work you like to scroll more and go into the deep of it.

Arpita Vats [00:03:24]: So that's the time we should recommend you more important content or more valuable content that you might be interested in. So this is like one of the factor. And then there are a lot of features that we consider such as like what are in your recent interaction with the people or recent interaction with the feed basically and that we consider like okay, so this person likes more video content, the person doesn't like more text posts. So okay, that's something that we need to consider that we shouldn't be recommending more video posts to this person. And then also we also look like okay, so maybe you like AI content but you like more infra content. So it's more specific. Right. It's not just GPT models and it's also about how GPT models are doing those things.

Arpita Vats [00:04:01]: So it takes all this very, it looks like very minute signal but it makes a major difference on the feed side of it. And since with LLM being coming it's getting much more and more easier to get all those tiny information and then get the most out of it. So I think every action or everything that you're doing on LinkedIn it might seem sound like very nonsensical to you, but on the other side it's some signal that you are sending that we need to consider in order to give you right pose that you're looking for.

Demetrios [00:04:34]: Why is it different than traditional recommender systems or how has it changed the LLMs?

Arpita Vats [00:04:40]: Yeah, so it depends. Like I think in overall industry LLMs is definitely something everyone is Trying to use in recommendation. One reason is like eventually, earlier in the traditional recommender system, you have these signals that you need to create and you need to give it to the model. So these are the signals that you need to look and then recommend the post. But with LLM income, I don't have to give it like, look at the signal, look at the signal. I just give it like, okay, this is a post and this is a user and they like this post. In the past few features, other people have liked this. So just tell me whether I should recommend this post to user or not.

Arpita Vats [00:05:19]: So this LLM is now already capable enough to understand like, okay, these are the signals that I should think about. And then it recommends you the next post or it creates the next post if you want to use LLM for creating the post as well. So I think what LLM is capable is since it's being trained on huge amount of resources, it's able to cache those tiny signals much easier as compared to a traditional recommender system where I have to explicitly go and ask it like, take this signal, take this signal. And when I'm using LLMs, I don't have to explicitly go, okay, take this signal, okay, see what this person has interacted to. I don't have to look on those tiny minute things anymore, which LLM can handle because it has all those understanding that it needs to know that whether I should recommend it. Because if you think on the backend side, GPT is already for an example, let's take a GPT. If we ask it something, it gives the answer based on that, right? And now maybe a month back, if I want to know, okay, give me my biography or tell me if I will be interested in this or not. It knows based on all the data that whether you will be interested in this or not.

Arpita Vats [00:06:25]: So it's kind of already doing the recommendation for you. So it knows what signals to catch if you have to make a recommendation. So that makes it much easier if you use LLM as a recommender compared to the traditional recommender system.

Demetrios [00:06:38]: So basically you're abstracting away the feature engineering that you should be doing.

Arpita Vats [00:06:42]: Yeah, because it's making it much easier for LLMs to do that.

Demetrios [00:06:46]: But I think the thing that I've heard folks talk about, which is hard, is if you're using the LLM for the inference, it takes so long to actually give you that recommendation. And then how do you do it or think about it if you're now having to populate a feed so it's not just One LLM call.

Arpita Vats [00:07:09]: So I think that's the most challenge part when you're using LLM. So there are a few things that we can think of. The one way is maybe if I have to handle this problem, one way to create a light handed model. I think Google is currently working on making some light or lightweight LLM models.

Demetrios [00:07:28]: Yeah, like the Gemma models, right?

Arpita Vats [00:07:30]: Yeah. So which you can directly use during the inference and it won't add those latency. So you can still serve all those candidates like whatever candidates recommended you from the last phase and then you can still serve it on the feed of the user. So that's one way to handle, is to make the model more and more lighter. The other thing also is something known as knowledge distillation, which is something you have this big model which you already have, fine tune and everything and then you create a student model out of it. So the, let's call the earlier model as a teacher model. So do you have the student model which mimics this teacher model, but it's much lightweighted, it's much smaller. So you can still use this to populate your complete feed.

Arpita Vats [00:08:12]: So that's to handle the infra case and other thing. If you can't do all these two options, the other option would be to not use LLM directly on the inference side. You do it in the offline training or whatever training you're doing and then you do some other normal traditional ML models to do the final recommendation on your feed.

Demetrios [00:08:31]: Yeah.

Arpita Vats [00:08:32]: So in that case you still leverage the LIMS input and then you do the final ranking on the feed side.

Demetrios [00:08:37]: So you're just using the LLM to help you generate features.

Arpita Vats [00:08:41]: You can use it either to generate the feature because obviously the post is already there. Right. So you're not using LLMs to create those posts for you. You use existing post and ask who should I recommend this post to? So basically all these things can be done in offline, but if, if you want to use an inference, you have to consider the lightweight models. Otherwise I think it's not possible to use until we have such a huge collection of GPUs and all those things that we can serve the whole feed with the LLAMA or whatever model we have so far, which is very huge weight.

Demetrios [00:09:14]: Yeah, traditional recommender systems have very tried and true eval systems, but with LLMs the eval systems are a bit different. Are you still going back and using the traditional ML or predictive ML eval systems when you're even plugging in the LLM?

Arpita Vats [00:09:37]: So I Think if you are, even though you're using the new LLMs, right, you can still keep the eval system as same because the base the criteria that you are counting if this. So for say I have ABC post, right? And I asked my LLM like okay, out of this abc, what should I recommend to Arpita? Should I recommend A, B or C? And LLM says okay, recommend her C. And I recommended C to that user, right? But my eval still earlier, even then traditionally recommended the eval was based on will the user like it or not. If the user likes it, that's a positive signal. Like my recommendation is correct. So even with the traditional recommended, that was the same case, right? We were still evaluating if I recommended this, did the user like it or they spent or commented it or whatever reaction they made on it. So I think the EVA system still remains the same because we still need people to show like, okay, this is a positive post because that's how the LLM will learn in future, right? That okay, I recommended C to Arpita and Arpita liked it, so means this is something valuable to her. So I should recommend similar post to her.

Arpita Vats [00:10:40]: So I think it doesn't change as much on the eval side of it, it just changes like how you use the output of your model. In the eval side. You still need to see if user making reaction to the post that you're recommending either be a traditional recommender system or by LLM. But it still has to be are the user liking it or is it being interest to it? Or they're just disinterested on this post. So that's still a negative signal. So I don't think much changed on the eval side of it yet.

Demetrios [00:11:08]: So I guess where I'm trying to wrap my head around is which parts do it change and it feels like it's that that model that you're using. Obviously if you're swapping it out for an LLM, that's a big change. But then on the data side, like the data you're feeding the LLM changes too because now you're not having to feed it so much, you're just giving it a post. The LLM understands the post and so you don't have to do much there.

Arpita Vats [00:11:38]: So I think earlier in the traditional recommender system as well, it needs to know the member information and it needs to know the post information. And still, even with the LLMs, it still needs to know the member information and it still needs to know what are the items that you want to recommend. So I think not much changes in that area. What changes here is in the traditional recommender system, I have this member information, I have this item and item, I mean post informations. And then we also give it like what are the specific signals you need to look for it? Like features, features to the model. Like look for the last interaction of the user or look like what he liked in the last seven days or eight days, all those information and what the things that they don't like or what time they like to see the post more. So you used to give this signal as well to the model. But right now with LLMs we only need member information and the post that I want to ask LLMs for, I don't need to give LLMs the information like oh, whether this person will like it or not, the person didn't like this post in last seven days.

Arpita Vats [00:12:45]: So do you think I should recommend this post or not? So I think these features, the features that we are giving to the model is not required anymore because LLM already has those inbuilt feature which it can incorporate in order to make the final judgment for the recommendation.

Demetrios [00:12:59]: Also with recommender systems, a lot of times you're not just using one model, right?

Arpita Vats [00:13:03]: Yeah.

Demetrios [00:13:04]: Are you still bringing that idea to this type of architecture?

Arpita Vats [00:13:09]: So I think if you're using LLM, it's very hard to add more model on top of it because itself is. And also honestly I don't think you need multiple models because it's capable enough to handle all those scenarios. You just need to use it in a certain way to get out what you want. I think with LLMs the most key factor is the prompt. How better you prompt your question. That's what will get you the right answer out of it. So I think earlier in the traditional models the key factor was the features. What feature should I give? What more feature can I give to my model to make it more better, to make it more precise Recommendation.

Arpita Vats [00:13:48]: But now with LLM, how better can I prompt it? How better can I get prompted in such a way that I get the exact information that I'm looking for? So I think the shift has moved from feature to being more prompt engineering parts because we know LLM has those features, it can extract those features. Just need to know what I'm looking for.

Demetrios [00:14:06]: Yeah, you have to be specific in those prompts and those prompts will get you that feature engineering in a way.

Arpita Vats [00:14:11]: Yeah, but it's like, I mean it's more trickier compared to the traditional because in traditional System your data science team tells you, or it seems like these are the feature it's very important or you look even for Google or Meta, you see what are the things that been happening so far. And these are the feature that's important, the model. You just give those features to your model. But right now with the prompt you don't know what's the right prompt. How do I get the model to use those feature and give me the right answer? So you just play around and then you fake a new D, like some literature survey and then you figure out like, okay, this might work. So I think the LLM side is still lot in progress. It has lot of potential, but I don't think from the scientific side we have actually explored all the potential of LLMC yet.

Demetrios [00:14:58]: So you're not using LLMs in production on the feed right now or sometimes you're trying or you can't answer?

Arpita Vats [00:15:07]: I think I can't answer them.

Demetrios [00:15:08]: Plead the fifth. Okay, so. Well, I guess, I think once we.

Arpita Vats [00:15:13]: Have more details and it's open source, maybe then I can share.

Demetrios [00:15:17]: Okay. That's a very diplomatic way of saying it. I like it. Now how do you measure that? This is actually better than the traditional or predictive ML way of doing it?

Arpita Vats [00:15:29]: Yeah, so I think that's. I think there is no right or wrong answer to it. So one way would be definitely. So say like if I have an example like a feed that I want to recommend to and I use LLM to Recommend out of 10, just recommend 5 to this user and then I compare what my traditional LLM recommended 5 and then we have the same eval system for both of them. Right. So then we just compare the evaluation is similar or not like. So if I committed 5 posts using LLMs, did user like all of 5 or did user had only paid attention to 2 of it. But rather than when I use a traditional, they liked all five of it.

Arpita Vats [00:16:10]: So in that case LLM is not performing as good as traditional recommender because in traditional I gave the user five posts and user tend to like all of it. But here I'm not having that case. Right. So I think it's the. When we do the evaluation it mostly is like how is it compared to traditional model? Do you, did you recommend a post which add more value or do you recommend post which had more interaction with the user? So if you don't satisfy all these two conditions or any like there are a lot more. So if you don't satisfy all those conditions, then obviously that's not the right model for you to use. Because I think right now it's become so much like everyone wants to use LLM. But the point is maybe you don't need an LLM to do this.

Arpita Vats [00:16:52]: Maybe a traditional model is good enough and you don't need to worry about using LLMs because I understand that it's a new technology and it's a sota, so everyone wants to use it. But what if you're able to achieve pretty well or better results or at least not forget about the better it's in parity with LLM or why do you need LLM? Why do you need to spend so much on the infra part of the LLM when you can have the same parity result with the traditional recommender system? So I think it always comes down to what is your interest and what you want to use your model for. So if my traditional recommendation system is working good enough, like Facebook, if you look Instagram. Right, it's working good enough. So I don't know what they're using, but whatever they're using, it's working for us. So I don't think they need to change it right now. Right. Because whatever is working, I don't think they want to plan or if they have to move to LLM or they want to do it, they need to because right now it's good enough until.

Arpita Vats [00:17:50]: Unless there's something that I don't know. But I think it's we need to make the decision in more sensible way rather than just running around it like LLM. LLM, LLM. It's not the only solution I believe.

Demetrios [00:18:04]: Just doing it just because we.

Arpita Vats [00:18:06]: Just for sake of doing it. Yeah, because everyone is doing it.

Demetrios [00:18:09]: And then how do you look at some of the pros and cons you mentioned? Like obviously it's a headache on the infra side because it's a much bigger model. So you have to figure that out. What are other ways that you're thinking? Well, the pros are that we now don't have to worry about this feature engineering and we can just give the data to the model.

Arpita Vats [00:18:29]: Yeah, I think there are some more pros to reduce a lot of burden that we had earlier. In the traditional model you have to make those clusters and then pick like okay, this post goes to this cluster so we should recommend it to all the user in this cluster, not in this cluster. We don't have to handle all those issue anymore. So there are multiple models in traditional recommender. One is taking care of those clustering all things and then we are doing the recommendation. But in LLMs you don't have to worry. Right. So even though adding an extra infra cost, but you're getting rid of all those forward and backward things.

Arpita Vats [00:19:05]: So you just have one model which is taking care of everything that you need to do. So if you think in the pros and cons, so yeah, you do add infra cost to it, but you're also reducing the cost. You still were having some costs when you're using a traditional recommender system. Obviously it might not be as high as LLM, but if you're getting rid of all the other steps that you are doing, you can still use those resources in your LLM and use that as your potential recommender system. But I think there's a trade off. I believe if you don't want to spend on infra, then obviously traditional LLMs add more features, keep adding more feature, whatever you think is working. And then the trade off is like instead of having just one model, you will have to still have multiple models to take care of it. And then the trade off is if you just need one model, you need to have more infra to handle those things.

Demetrios [00:19:56]: Yeah, yeah, you want many models that then you have to babysit and continuously be retraining and trying to optimize. Or you have this one model that potentially doesn't need as much retraining, but it's bigger and you have to figure out that inference side of things.

Arpita Vats [00:20:14]: Yeah, I think it's basically a trade off that we do in normal ML models. Right. There are so many trade offs that we do like, okay, we want more precision, we want less recall or less recall. So this is a trade off I think you have to make in every part of ML, like, okay, I can take extra infra cost, but I don't want to take care of the multiple models that I have to handle.

Demetrios [00:20:34]: You bring in the LLM just to certain parts of the puzzle and you say, all right, well just the LLM here, but then we're going to have traditional ll, traditional ML be the final. Yeah, the final inference piece, because we still need that speed. Have you seen the LLMs be put in other ways? Because I've heard of folks using LLMs to suggest features and this feels like a bit of a variation of that, but not necessarily the same thing. If you're just giving it to the LLM and saying you got the features inside of you, let's prompt it so that it uses them and we can find those features or we can find the best performance through Prompting not through featuring.

Arpita Vats [00:21:22]: Yeah, I think I did see some research work that it was more on sequential recommendation. And they have used this LLM to do the sequential recommendation. So what I mean by sequential recommendation here is like, say, okay, if I go to a new city for say if I go to Tennessee, the next step is like, want to get a hotel? So now it recommends you. Okay, so even though when you first say, when you open a book, go and book for a flight, right. The next thing it recommends you to rent a car. And then once you rent a car, where do you want to stay?

Demetrios [00:21:54]: Yeah, your hotel.

Arpita Vats [00:21:55]: Yeah, so the complete package. So this is sequential recommendation, insurance, etc. Yeah. So I think one of the model that I was reading the paper about is like they're using LLMs to do this. So they see like, okay, if the user has booked a flight for what should be the next thing that the person is looking for. And then it recommends like, okay, once you book the flight, you'd need a car or you need to book an Uber or reserve it beforehand to go and then book a hotel and all those like attraction spots that you need to go to. So I think in this, an earlier traditional recommended system, you have to explain the model. So okay, it was like, given this, what should be the next part that you should do? And then again you, okay, this is done, then what's the next that it has to do? But LLM can do all the sequential in one go.

Arpita Vats [00:22:39]: You don't have to ask LLM every time it say like, okay, person booked the flight, what are the next five step that the person is going to do? And then it will tell you like once they have booked the flight, they will book the car, they will book this, this, this. So that's what you need to basically recommend it. So I think when we are doing the LLM part of it, we are getting rid on the part where you mentioned for using how different that we can use LLM. So I think this can give you the whole structure after this step. One step to step three. And then you can use your normal models if you don't want to do to okay, out of this car, like Hertz or Enterprise or whatever. One of this you can recommend to the user, whatever you have association with. So you don't have to have a model which is a very heavy weight to understand all these signals.

Arpita Vats [00:23:26]: You have this LLM which gives you offline results like, okay, these are the things that I want that a person will ideally do. And then you use your model to just recommend those things. So I think this still in my understanding it still reduces your multiple models to do these things and it still reduces your. So if you think like if you have multi model like multiple models you are also adding the risk of the accuracy of each of the model because if one of them my model is not performing well, the rest won't be able to perform either. But with the LLMs, since I only have one model and just one model folder, just fetching the results and just ranking it. So in that case like I don't have to take care of the multi model and accuracy, I only care the accuracy for my LLM. Right. Because the other is just ranking based on whatever score that LLM has given it.

Arpita Vats [00:24:18]: So I think this helps reduce us and even I agree with the part you mentioned like we fetch the feature and give it to LLMs. That's another way to do it as well that you use your traditional models like extract the features and then you give it to the LLMs. Like these are the feature I'm looking for and just give me the information about it. So that's one way to do it as well because in this way I think even though you are using LLMs but you still need. But you're still saving the time and the computation because it doesn't have to do all those fine tuning and everything for the model because you're just asking given this text post, give me the features from it. So I think that's one way to. Also we can probably use LLMs.

Demetrios [00:24:59]: Yeah, and I like this. It's almost like a reasoning model is going to tell you. Yeah and you might not even need to have a reasoning model because generally when you go on vacation you're going to do the same five things. It's not like unless you're doing some wild stuff which I can only imagine there are people out there doing it. Then depending on where you go also you're probably going to get different recommendations. But for the most part you need somewhere to. You need somewhere to go, you need some way to get around when you're there. You need somewhere to sleep when you're there.

Demetrios [00:25:36]: So those are going to be pretty.

Arpita Vats [00:25:39]: Same for all the time.

Demetrios [00:25:40]: Yeah. And then while you're there you're going to want to do experiences or you're going to want to have experiences. So maybe that's where it gets very personal to each person.

Arpita Vats [00:25:49]: And I think, I don't know if they're still being using it, but one thing they can definitely do with LLM is like if it has. So if I have been booking my things from the same website. Right. So they can feed this information to LLM. So this person doesn't like for say hiking. They don't like hiking so don't recommend them those things for the attraction spot. Probably recommend them more sightseeing.

Demetrios [00:26:12]: They get off of the site as soon as we recommend any hiking activities.

Arpita Vats [00:26:17]: So I think they can use those information as well. I think. I don't know if they're still doing it or not, but that's a very good signal to use LLM and customize your preferences. Because we know if we book a flight, we take a car, we take a hotel. These are very generic, everyone will do it. But. But what after that is very specific to every person. Maybe some person likes to.

Arpita Vats [00:26:36]: Okay, I don't want to roam around the city. I just want to go more suburb and use more site like Nature Beauty. So you can't recommend that person. Okay, go and watch like if I'm in D.C. go and see Abraham Lincoln Tower. They are not interested in those things. So I think they can also the other part would be where after this three part they have this LLM model being more centric toward the user interest, not like a generic thing that you will do in that city or town. Yeah, I think that's something very much easier to do using LLMs if we are using.

Demetrios [00:27:09]: Because with traditional ML models you can't say what people don't like.

Arpita Vats [00:27:14]: Yeah. So I think with the traditional model the problem will be they will have to first get these features of the user and it's like a cold start problem. Yeah. So if I'm a first time user won't be able to handle that. So I think LLM does pretty good with the cold start problem because they just learn it like very quickly. So even though I ask LLM today that I am more a laid back person, what are the things that you will recommend? And it knows. Okay, this is what you need to know. And since we are already using GPT so much and it has all the data honestly so it's much easier for them to understand.

Arpita Vats [00:27:51]: Okay, this kind of person might like these things.

Demetrios [00:27:54]: And then the question becomes that data sharing. Because ChatGPT or whatever, anthropic or Gemini, they have that data because I talk with it all the time. Yeah but then is the service that I'm using to book my flights or book my whatever is it going to be through ChatGPT so it has that data on me.

Arpita Vats [00:28:19]: But I think even though if they're using any LLMs they again still can understand. Like okay, if the person is more toward this nature, they might like something around it. Because even in general as a human, if we see some person like more introverted person, we know what things obviously we know they won't like to socialize, right? So how we know that is just. We just know one data point about them that the person is introvert. And so the same thing is with LLMs, right? I think they just need few data points. They don't need all the data points like how do I look what I do in every day. It just need one.

Demetrios [00:28:52]: I see what you're saying. So it doesn't have to be so rich. It doesn't have to. You don't have to give it a million features. You just give it one and it'll give you. Yeah and it will give you at least put you in the ballpark so you can say hey we've got cool movie theaters. But then those super specific ones you'll need special data for.

Arpita Vats [00:29:14]: So I think even for the any, even for your feed or anything that you're using LLMs, right? It's very good in doing a generic recommendation based on your past interest or something. But if you want your post to be very, very like granular towards you, it has to learn all those feature eventually and that it learns over the time, right? Even how does the GPT currently know so much information is because it has learned about us over the time that we have been chatting with it. Like when we ask something about a model or something it knows like okay, this person might be working on certain areas. So when you ask it it gives you similar ideas like right? So I think that's what LLM will learn over the time. So I think as a, that's trying to do similar to humans, right? So as a human we talk and then we get to learn more and more and more about the person and we can recommend more things about that person. So I think that's what the LLM is trying to do here. But when you compare traditional recommender system or any traditional models, they are only learning whatever we are giving it to. We are not doing any active learning while they are doing.

Arpita Vats [00:30:18]: So if I only give you three features, you are only going to learn about those three features. You don't have any idea about that they can't write the current model. I think there are some improvement that's been going on, but it's not there yet. That okay, if the person is introvert then the person can also with this they don't know that yet, but LLMs knows that. Right. So if the person is say autistic, they might be introverted autistic or they might be extra. There's only two ways that they can be categorized. But the informational model they won't know.

Arpita Vats [00:30:47]: They just know. Okay, this person is autistic. That's the only information it has. So it doesn't know the features of autism basically. And in LLMs they know the feature of features. So I think that's what is making them more powerful compared to the traditional LLMs.

Demetrios [00:31:02]: Yeah, it almost gives you a much more rich feature or that kind of like the 360 view as opposed to a very two dimensional view of. All right, there's this feature, yes or no. Yeah, okay, there's this feature, yes or no. Boom.

Arpita Vats [00:31:18]: Yeah, there are some improvement that they're making in the that side as well where you do cross featuring. So if you have this feature then probably the feature. If the person has feature A, that person will have feature B. So then they do cross featuring. But I haven't seen any successful research paper out of it yet. So I can't say 100% it's going to. It's working or it's. It will work.

Arpita Vats [00:31:40]: But I think as the AI is evolving obviously I'm pretty sure that a lot of research is actually working how we can do the same thing without LLMs as well.

Demetrios [00:31:47]: Oh, without.

Arpita Vats [00:31:49]: So I think I recently did a work on ssms so which is state space models. So I use this model to basically do question answering and obviously we used LLM is much better to do it. But what I did, it was just for the indic languages like language that's being spoken in India over. So there's very much less data that you have and there's much more change in the scripting part of it. So I didn't use LLM for two reasons. The first it's very heavy to use so it's hard to fine tune it if you are doing it from academic purpose. The second part is I wanted to see if we can use some lightweight model because it's just a question answering task. I'm not asking you to recommend after this or after about this anything, just giving a question.

Arpita Vats [00:32:39]: Give me the answer. I think for those things I did see that SIM is able to perform much better than a lot of other Transformers model and Transform models are. So if you consider like Bert or Roberta models, those are also large language models and it did perform better than Roberta. So there Are some models that's being developed currently which are more like kind of equally well as LLM or slightly underperforming. But then the problem is they are very specific. Like it's only going to work for question answering but in LLMs it works for everything. It summarizes it question answer, it gives like summary of whatever you have given it to it. So I think there are people can develop or eventually there will be some development where we can do so many things with smaller different models rather than having one huge LLMs.

Arpita Vats [00:33:33]: Yeah, but that's like another area that probably will have some research later on.

Demetrios [00:33:38]: Yeah. And you see that being different than the way that we're doing recommender systems.

Arpita Vats [00:33:43]: Now I don't think that will change anything with the recommender system because in the recommender system right now we are doing the LLMs. Right. Because it's making sense. But for right now I don't see anything like that. But in future if there is it might change but based on this SSM evaluation that I did, I don't think it can replace the recommender system LLMs because it doesn't have that much.

Demetrios [00:34:07]: So it's more for different use cases.

Arpita Vats [00:34:09]: Yeah, is more. I would say it's more very specific. If I want to just do like one task out of it, that model is going to be just made for that task. So you have to create multiple models for each of the tasks that you want to do. So it won't be able to replace the whole LLM for sure. But what I'm saying is like maybe in future if since people have already put some foot on this direction they are also probably be researching on how we can do a more better recommended model without using LLMs as well. We never know what's the future.

Demetrios [00:34:38]: If it's really good at one task then you can potentially plug it into your DAG and you have that as the one task that it does. And you know that in my workflow I need this done. So we have that done but. Or you plug it in as a tool for an agent to use.

Arpita Vats [00:34:57]: Yeah, I think that's the next wave that's coming. Right. Agentic AI. And I think with the agentic AI being coming since you bought it up. So I think it will change a lot of things basically if it actually starts performing that well because based on the research work that I have seen so far I don't see lights outperforming drastically anything. But maybe over the time it will evolve because I think right now agenda is at not that Phase that we can count on it yet. But I think if that is there then it's definitely going to change the recommender system as well. Maybe I don't need LLMs as well.

Arpita Vats [00:35:35]: I just ask the agent. So this is the person and this person likes this, Just recommend it, whatever. And you put some layer on, I put some more inside the agentic AI to do certain tasks. You just add some flows to it. So even the agentic I could be for say if I want to a research paper and I give like okay, so this is the agent which writes research papers and I give you this is topic that I want to write about. So do the literature survey, do related work, search and just write a paper for me. I want to submit it to say New Ribs or CVPR and it does that for you. So you just need to review it and that's it.

Arpita Vats [00:36:15]: And if the agents feel like they're lacking some information or they want some validation, they can obviously ask back. So that's the least effort you have to do. Right. It does the experimentation, it does the literature survey and it just writes a paper for you. So then what's the future for the researchers? If the agent AI is able to do that, you just have to give idea like this is something I think might work and then it will do all the different layers of work. So you can have agent for each of the tasks and it will do that.

Demetrios [00:36:44]: I'm ready for when the agents can go out and also collect the data too. They're calling people.

Arpita Vats [00:36:50]: Yeah, I think it's evolving, but I don't know if it's going to evolve that fast because no one, I think five years back no one thought about LLMs or anything such that. But it came and it changed everything drastically. So we never know maybe the agent is going to be hugely reliable by tomorrow and then everyone is moving toward it.

Demetrios [00:37:12]: Yeah, you bring up a great point with the reliability especially on agents and how right now it's not good enough to actually be using for a lot of different pieces of this puzzle. And especially if you're at a big company like LinkedIn I imagine it's very hard to figure out can this work in the manner and fashion that we need it to be working at like holding it to this high standard so that it's able to complete the tasks nine times out of 10 and there's so much work that goes into that. Yeah, you think about it for a sec and you go, well actually our recommender, maybe there's lower hanging fruit that we can Tackle before we need to find out if we can recreate a recommender system with agents as opposed to just the LLMs versus the traditional ML.

Arpita Vats [00:38:11]: Yeah, I think I agree with that because I think the agents. So first of the thing is like at LinkedIn or any other companies, you're sharing those information with agents. So first you need to be very sure that the agent is being used correctly. And so that's like the first part, like privacy and everything is being taken care and then still you don't know if the. So it becomes very hard to debug an agent, like if something goes wrong. So I got five output, but maybe all of them are irrelevant. But I don't know what should I fix in the agent to get those to be relevant. Because even with LLMs or even with the traditional.

Arpita Vats [00:38:51]: So I think one thing that's very easier for the traditional recommender system, it's very easy to debug, you know, what is coming and which one do you need to tune to get the right output. Even with the LLMs you are only 50% sure that why it might be doing it, but you're not 100% sure. But I think with agentic you are 80% not sure how it's doing, what it's doing. So it's very hard to debug it if you don't know what's happening inside the black box. So the agent becomes a kind of black box for us. Right. So we know like we are giving it this information, we are asking it to do it, but we don't know like how it finally ended up doing it.

Demetrios [00:39:26]: Totally.

Arpita Vats [00:39:26]: So I think as more advanced we go for Agentic or any other thing, it becomes more and more black box going backwards. So I think that's where the traditional ML models or any traditional models brings more sense because you know, like, okay, this layer is taking care of this. That's why we are getting this. So it's simple maths that you do and then you get it. Yeah, but with LLM, I know it's still math, but like you don't know exactly which math is put, where to get you what you want. Yeah, so I think that's, that's going to be very challenging with agentic AI, I guess.

Demetrios [00:40:02]: Yeah, right. Total random question. Why have I been seeing older posts on my feed lately in LinkedIn?

Arpita Vats [00:40:12]: Do you? Okay, that's a very good question that I don't know why you're seeing older.

Demetrios [00:40:19]: But when I say older, I mean like a week old, two weeks old, three weeks old, I remember seeing.

Arpita Vats [00:40:24]: And you're not seeing any new content?

Demetrios [00:40:26]: No, I am seeing, but every once in a while I'll get a two week old post put in or a three week old post put in and it's like, what is this doing here?

Arpita Vats [00:40:36]: Maybe they don't have any new posts for you to.

Demetrios [00:40:39]: I've looked at it. I'm that much of a power user. No, I have more friends. I got more than like five friends. And I think they post about.

Arpita Vats [00:40:48]: Yeah, I think sometimes it's also like, maybe the model doesn't have the new data, so it's not recommending with a new post.

Demetrios [00:40:55]: I thought for sure it was some kind of a weight. Got tuned to show more. Yeah. Older posts or like your post has a longer lifetime now because there's certain posts that have more exposure or. But it's not like I was thinking about it too, because I was looking and it's not like these posts have been performing wildly. It's not like they're like those thousand likes posts. It's like a 30 like post or 100 like post, but it's two weeks old or three weeks old.

Arpita Vats [00:41:28]: Maybe there is some bug that you should definitely report because that's not. That's not what we expect.

Demetrios [00:41:33]: That's not normal.

Arpita Vats [00:41:34]: Yeah, that's not normal actually. Yeah, you shouldn't be seeing like older and older posts.

Demetrios [00:41:39]: Yeah, I think three weeks is the oldest that I've seen, but it's pretty common, honestly, on my feed. I'll show you in a sec.

Arpita Vats [00:41:46]: Okay.

Demetrios [00:41:47]: Yeah. And I thought it was. For some reason, I thought it was your, your change in the model to give the posts a longer lifetime. And it made me think now because when I first started using LinkedIn back in whatever, 2019, the posts would have a long lifespan. Like it would take a few days for the post to get going and if it got going, then it would stay for a bit longer. Like a week and a half, two weeks type thing. When I say longer. And then I saw it shrink and it was like, okay, now it's more like it happens for a couple days.

Demetrios [00:42:31]: But it's really hard on LinkedIn to share things that are happening today or tomorrow. Not like, you know, like with Twitter. You usually, oh, I'm going here, who's coming? Or I'm free in the airport for an hour. Ask me anything. That kind of stuff is impossible because it wouldn't.

Arpita Vats [00:42:50]: Even though you post it, your user won't see it.

Demetrios [00:42:53]: Exactly.

Arpita Vats [00:42:53]: So it won't make any sense.

Demetrios [00:42:55]: Exactly.

Arpita Vats [00:42:56]: That's It, Yeah, I think that's the area that I don't think LinkedIn is currently working on. Like, but I think that's going to be. I think they will be working on that very soon. Because that's very important. Because if you think of a LinkedIn, it's more kind of professional networking. Right. And for professional networking you might not use such things like, okay, I'm at airport or let's meet or let's chat. You probably plan ahead, right.

Arpita Vats [00:43:21]: So I'm going to this workshop and I plan to do this, this. So feel free to connect or feel free to come and talk. So that's like a professional thing that you will do on LinkedIn. Maybe that's why that's not a criteria that they want to focus on. Because on Facebook or Instagram you can just go and write, okay, I'm here, just meet me or. But because that's their purpose, right. They want to be social networking. But I think that's where LinkedIn is separated from Facebook or LinkedIn or Instagram or TikTok is that LinkedIn is very specific to professional networking.

Arpita Vats [00:43:50]: What they call is like, you connect based on your profession. So you basically, okay, someone is in industry, is doing very well and you want to connect to that person. Yeah, this is a platform for you. So basically I think that's probably why they don't have this feature that you can just post it and everyone can see and come. It's more about pre planning. So I'm going to be there, you can come and we can have a chat, explore and talk, whatever. So I think that's the one direction. And regarding the older post, did you also tend to see this post earlier and then you saw it again or is it just the first time you saw and it's three week old.

Demetrios [00:44:25]: Sometimes I would see so both. But sometimes I would see it once and then I would see it again. Oh, a few weeks later. Why is that something now it makes sense.

Arpita Vats [00:44:35]: No, I. It still doesn't make sense. I'm just trying to see what are the things.

Demetrios [00:44:39]: Yeah, yeah, because I see that. I liked it already.

Arpita Vats [00:44:42]: Okay. And you still see it?

Demetrios [00:44:43]: Yeah, and I still see it.

Arpita Vats [00:44:45]: Okay, maybe, I don't know.

Demetrios [00:44:47]: We're debugging live on air. I like it. What is going on. So is it then that my cluster has a finite amount of people and they're on when they're on and then you show it to them when they're on and if they're not on, you don't show it? Obviously you don't show it to them. But it's like, oh, there's 2,000 or 2 million people in your cluster and outside of that cluster we're not going to ever show it outside of that cluster unless it really.

Arpita Vats [00:45:17]: So I think if some post becomes very viral that's being shown to everyone, it doesn't matter what cluster you are.

Demetrios [00:45:23]: But I guess the question I have.

Arpita Vats [00:45:25]: Is then so the cluster is basically on the offline side of it just to get the features from there. But on the online side there is no such thing as cluster. Everyone is the same user. So when the post is there, the model just knows based on whatever it has learned from the cluster or whatever feature it has been given. It just knows like okay, this is Arpita's AI post. Maybe someone from LinkedIn or someone from AI will like this post post or is interested in this post. Even though that person is not in my network, it will still recommend that post to that user. So I think what it does during the inference time, it doesn't know any clustering, it just know whatever it has learned so far.

Arpita Vats [00:46:04]: So when I use example of clustering it's just for model to learn like how it's learning those features. So once it has learned it like okay, so Arpita is not from finance and there's her friend who posted something about finance. Maybe she might not be interested in it because it's not the area she works or something. And then it might not show up that day on my feed, but later on since we are still friends, right? So it might show you okay after even three weeks your friend did finish her finance education or whatever. But then so I think it based on the model at that time they already have say it has to rank top 5 posts for my feed and it already has the best top five. Then it won't take that finance post in. But after 10 days or 20, they don't have enough post or there's not enough being posted about, then it will pick that post and will put like the first position that this is a post. And also this answers to that question where you have multiple posts event showing up again and again.

Arpita Vats [00:47:01]: Probably there's no they couldn't find any relatable content that's available currently that could be shown to your feed. So they just show that post exposure again like this is all we have for now.