Hallucination in LLMs
Vipula Rawte is a Ph.D. student at AIISC, UofSC, USA. Her primary research interests are in Hallucination, Generative AI, and Large Language Models.
At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps.community meetups. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.
The latest progress in Large Language Models (LLMs) has received praise for their impressive emerging abilities. Nevertheless, the problem of hallucination has also surfaced as an unintended outcome, raising notable concerns. Although recent efforts have aimed at recognizing and alleviating various forms of hallucination, there is a limited focus on the detailed classification of hallucination and the corresponding mitigation approaches. This talk will briefly touch upon some of these existing challenges.
Hallucination in LLMs
AI in Production
Demetrios [00:00:05]: Next up, we've got Vipula, who is coming at us, and Vipula I saw do a. She was co author of a paper that I absolutely loved. Maybe some of you saw it. I made a little video of it back on the beginning of the conference. We played that video and let's see, where is Vipula? Hello.
Vipula Rawte [00:00:28]: Hi.
Demetrios [00:00:30]: How are you?
Vipula Rawte [00:00:31]: I'm doing good. How about you?
Demetrios [00:00:34]: Excellent. I am so excited for your talk because I love to make jokes about hallucinating, and I know you're about to talk to us about hallucinating. I don't know if anybody else out there is as excited as I am, but we made a shirt that says I hallucinate more than chat. GPT, in case you want to cop that shirt, scan the QR code right now and get it. It's for real. You can go onto our site and get that shirt. I actually have one of these shirts at home. I didn't wear it.
Demetrios [00:01:10]: I should have just to prove it to you. So yes, you can go to the moops community shop and get that. We also have another shirt that I made specifically for this conference, which is my prompts or my special sauce. And so we got shirts basically. That's it, though. I know you have a fun talk for us. Do you want to share your screen before I get out of here?
Vipula Rawte [00:01:37]: Yes, absolutely. So let me just quickly share my screen and put it on full screen.
Demetrios [00:01:48]: Right on. I see it. I'm going to bring it onto the stage and I'm going to get out of here. And this hallucination t shirt is also going to get out of here. I'll see you in ten minutes.
Vipula Rawte [00:02:01]: Sure, sounds good.
Demetrios [00:02:10]: All right, you're on. Hit it whenever.
Vipula Rawte [00:02:13]: All right. Hi, everyone. I'm Vipula. I'm a PhD student at the AI Institute at the University of South Carolina, and I'll be speaking on one of the biggest challenges in LLMs, which is hallucination. Some fun fact, hallucinate is Cambridge Dictionary's word of the year 2023, and the definition goes like this. It says when an AI hallucinates, it produces false information. Although there has been quite a debate on the usage of the term hallucinate versus confabulate, given the popularity of this term, let's just stick with it throughout this talk and move forward. So some interesting real world examples as to why this problem is so important and how it impacts even the real world scenario.
Vipula Rawte [00:03:15]: So this example is taken from last year when Google released its chatbot bar and given a prompt, what new discoveries from the James Webb, Space Telescope, can I tell my nine year old about? And it generated some responses, out of which the one which is highlighted in yellow, it is false. And based on this situation, Google lost almost $100 billion in its stock price on February 8, 2023. There is another example from this week itself, where Air Canada, the news headline, says it has to honor a refund policy its chatbot made up. So what happened was the chatbot, there was a passenger who booked a ticket, and it was based on the death of some of, I think it's his mother. And then the chatbot said that you will receive a refund within 90 days after you book this ticket. So the passenger went ahead and booked the ticket, and then when it claimed for a refund, Air Canada refused and said that this is something made up by the chatbot, and we are not liable for any kind of refund. And the passenger, he had to file a case, and he had to go through a lot of hassle to get that refund. So these are some real world situations where we saw how llms can generate some fake responses.
Vipula Rawte [00:04:48]: So it's not just an academic problem of research, but it's also a real world problem where we have to deal with such scenarios. And so just background, as in why this talk fits in this theme of the conference, AI in production, because it's also something we need to pay attention to as we work as industry researchers. So this talk is divided basically in three different aspects. So there is this paper, the troubling emergence of hallucination in large language models and extensive definition, quantification, prescriptive remediations. This paper was published last year at 2023, where we have proposed quite some interesting things. And even though it looks like an academic research paper, there are quite some interesting contributions, which can even be further used in industry research as well. So we are the first ones to come up with such a comprehensive hallucination taxonomy, where we say that we divide it into three different aspects, orientation, category, and degree. I'm not going to go into the depth of it, given the time constraint, which is just ten minutes.
Vipula Rawte [00:06:07]: So just some quick things to take note of is the categories and the orientation, and then there are some corresponding degrees associated with it. So we just say factual mirage and silver lining, as in, so there can be two real scenarios. One is the prompt itself could be factually correct, or it could be factually incorrect. So on the left hand side, you will see that the prompt is correct, and it says, engineering effort to build Eiffel Tower. And then the AI generated response says that it was built to celebrate the anniversary of european civil war, but the fact is, it was built to celebrate the anniversary of the French Revolution. On the right hand side, you will see that the prompt is factually incorrect. The prompt basically says, kamla Harris and Elon Musk are getting married. Which is not true.
Vipula Rawte [00:06:59]: However, the LLM generates a beautiful story with the wedding venue, with the wedding date, but this never happened. So the prompt could be either correct or incorrect. And still the LLM generates some fake responses. And then these are just some categories we proposed. So I'm not again going to go into the depth of it, but you can always look at the paper and read the details. So these categories are proposed based on what kind of hallucination occurs. So if the numbers are imaginary, we call it numeric nuisance. If the place is imaginary, as in location, we call it geographic eratom.
Vipula Rawte [00:07:43]: If there is a fictitious character, we say it's a generated Golem situation. If there is some acronym problem, then we say acronym ambiguity and so on. As part of this paper, we also created this huge benchmark data set. We call it hilt data set, hallucination elicitation data set, where we have taken New York Times tweet as our prompts, and politifact is another resource we have used to generate our data set. And we proposed this huge benchmark of 75k prompts and the number of sentences around 129k. This is one interesting contribution of our work where we are again the first ones to propose such an index where we basically measure. So we call it hallucination vulnerability index. And then later there are some other groups who have come up with such similar metric to measure hallucination.
Vipula Rawte [00:08:49]: So based on the HVI scores, we rank different llms. So this work is from last year, and we have chosen a suite of 15 llms. You can see it was until GPT four. And if you carefully look at the ranking, it's not based on the size of the LLM. So this is some interesting findings, or these are some interesting findings from our experiments where we observed that it's not just the size of the LLM, so we cannot directly draw any kind of correlation saying that, look, if it's a smaller LM, maybe it hallucinates less, if it's bigger, it hallucinates more, or vice versa. So there are other key factors which contribute to hallucination. And one of our observations was whether it uses rlhf or not, and so on. So this is some interesting ranking we have here.
Vipula Rawte [00:09:45]: And like I said, others have come up with such similar ranking as well. And these are some mitigation techniques we have proposed. First one is the black box technique and the second one is gray box technique. So basically in the black box, as the name itself suggests, it does not rely on any kind of external information. So you must have heard of rag retrieval augmented generation, where it retrieves external information or external knowledge from web sources or any kind of external information resources available. So we do not use that in our first technique. However, in our second technique, which is gray box, we do use some kind of retrieval mechanism. And these are again some stats here where we see how different mitigation techniques work for different llms for six different type of categories.
Vipula Rawte [00:10:51]: These are some interesting findings, again, not going in the depth, but you can always look at the paper. So moving forward. So I think this was the paper which caught a lot of attention. So this paper is a survey paper which talks about different hallucination mitigation techniques. It was out early this year, in January, and we have conducted a survey of 32 hallucination mitigation techniques where we broadly identify them into two categories, prompt engineering and developing models. So prompt engineering. So one observation from the existing papers so far is if we want to mitigate hallucination, it's better to just the way the prompts are designed. That's also something which leads to hallucination.
Vipula Rawte [00:11:46]: So instead of post responses which are generated by llms, if we tweak the prompt at the generation side itself before responses are generated, that could be one of the effective ways. So it deals with, say, prompt tuning. You can see some subcategories here, and then there are quite some interesting papers, and the timeline is again late 2023 and so on. And then another category where most of the techniques fall under is developing models where we can develop, say, specific models which only deal with mitigating hallucination. So this is an interesting paper. Not going into the depth, I just took one screenshot of this entire taxonomy moving forward. This is another survey paper I wrote sometime last year around, I think September or early October, where I discuss hallucination in the context of not just language models, but even different modalities like image, video, audio. So again, under llms, you will see there are multilingual llms, there are domain specific llms.
Vipula Rawte [00:13:06]: So you can see there is this medical domain or legal domain. So this case or this challenge of hallucination is basically prevalent in almost all kinds of language models, and not just limited to language models, but also to say, vision language models or audio video language models as well. So this is one example where if the prompt is, say, just provide a detailed description of a given image, and then the response generated by the LLM is, it says, you can see the image features a person standing on a sandy beach holding a colorful striped umbrella, blah blah blah. But okay, there is an umbrella, but there is no person. So this is interesting. So this example shows that hallucination is not just present in the textual responses, but also in the visual description of the images. Having said that, there is this another interesting example. I just took it from last week when Sora was released by OpenAI.
Vipula Rawte [00:14:22]: And this example is from Gary Marcus's blog post where there is a prompt which says, monkeys playing chess in the park. And you can see this video, as in, it's a screenshot taken from that video where you see a monkey playing chess in a park sitting on a bench. However, on the chessboard there are three kings, which is so if you see carefully, there are two whites and one black. So this is another case of video hallucination. So the challenge of hallucination basically is there in almost all modalities. Moving forward. There is another interesting observation, as in, is Harris mission always bad? So this is one reference to the Washington Post opinion piece I was interviewed for, and the question was, is it always bad? And then this article, if you look at the title, it says, honestly, I love when AI hallucinates. So yes, it is bad in the context of some use cases, as in if it's medicine or say, some mission critical situations, definitely it is not just bad, it's risky, right? But if we are using it for some creative content generation, where we want some creativity, where a human mind cannot think as creatively as the responses or the content generated by these models, then yeah, there is some creative part of it which is good.
Vipula Rawte [00:16:22]: So we cannot completely rule out. So it really depends on what you want to use it for. So this is an interesting article, by the way, if you want to just quickly have a read. So some key takeaways with the popularity in generative AI, definitely hallucination is one of the challenging problems. But will it go away? Can it be solved? Yes, but again, to a certain extent. So right now, there has been quite a lot of research done and going on in this specific direction where researchers have shown that how different mitigation approaches or combination of different techniques can be used to mitigate it. But there is no one solution that fits all the scenarios. So it's something that can be done on case by case basis.
Vipula Rawte [00:17:22]: So that's one observation from the research. Yeah. Having said that, in the context of, let's say industry or when it comes to putting models into production side, we need to be careful as to how we are mitigating it because then there are issues like scalability and so on, like the real issues, not just something which is limited to academic research. There are some further updates so we are organizing a workshop at Elric Kohling conference which will be held in Italy in May and whatever I discussed so far. So to conduct more research in this direction, we will be releasing some shared tasks and further details will be provided. In addition to that, I'll be speaking again and we'll be giving a tutorial on hallucination in llms at the same conference. So if you want to just follow on this work, you can again log into these two sessions later in May. All right, thank you.
Demetrios [00:18:44]: Excellent. So now I have an excuse to go to Italy in May. I love it, Vipola, thanks. I have got super behind because of my guitar playing skills, so I'm just going to keep it moving and say thank you. I think people may have been chatting in the there's some questions that have come up and I think you are on there so you can feel free to ask vipula any questions that you may have in the chat. And a huge thanks for this. That was awesome. You did not disappoint.
Demetrios [00:19:17]: I love waiting. I can't say it enough. It is great that this is going on. So I'll see you later.
Vipula Rawte [00:19:25]: All right, thank you so much. Bye.
