By writing instructions to insist that it double verifies every (non obvious) claim with a minimum of two independent sources. I also told mine to always assume that the initial prompt is missing crucial context, and to ask as many follow-up questions as necessary until it has enough information to provide the answer to the question I’m really asking. (For speed and efficiency you can even make it give you multiple choice options to click on.) Because sometimes the problem isn’t with the LLM, but with the user asking the wrong questions.
Using those two instructions alone, I’ve encountered considerably fewer hallucinations, and when I’m still not certain, I can simply click on the sources linked next to every single claim the AI makes.
Right, but that sounds like you implicitly trust the LLM and only verify statements when they seem off. So your intuition is the final arbiter if truth?
No I’m saying that I don’t have to trust it because I instructed it to put two independent sources next to every claim it makes. I just check the sources.
Gemini does a pretty good job of this because it doesn’t seem to have much built-in knowledge. Instead, it just searches the Internet on your behalf and returns summarized results with links to where it got that specific information.
I use it to search for scientific research all the time and the summaries often aren’t detailed enough so I actually click on those links. I’ve yet to encounter a situation where it fucked that up (invented links that don’t exist) but I have heard about it happening.
So far, the summaries have seemed to be pretty spot-on when it comes to biology papers 🤷
How do you verify that everything it tells you is correct?
By writing instructions to insist that it double verifies every (non obvious) claim with a minimum of two independent sources. I also told mine to always assume that the initial prompt is missing crucial context, and to ask as many follow-up questions as necessary until it has enough information to provide the answer to the question I’m really asking. (For speed and efficiency you can even make it give you multiple choice options to click on.) Because sometimes the problem isn’t with the LLM, but with the user asking the wrong questions.
Using those two instructions alone, I’ve encountered considerably fewer hallucinations, and when I’m still not certain, I can simply click on the sources linked next to every single claim the AI makes.
Right, but that sounds like you implicitly trust the LLM and only verify statements when they seem off. So your intuition is the final arbiter if truth?
No I’m saying that I don’t have to trust it because I instructed it to put two independent sources next to every claim it makes. I just check the sources.
It definitely has an element of garbage in, garbage out.
You check the links/references it gives you.
Gemini does a pretty good job of this because it doesn’t seem to have much built-in knowledge. Instead, it just searches the Internet on your behalf and returns summarized results with links to where it got that specific information.
I use it to search for scientific research all the time and the summaries often aren’t detailed enough so I actually click on those links. I’ve yet to encounter a situation where it fucked that up (invented links that don’t exist) but I have heard about it happening.
So far, the summaries have seemed to be pretty spot-on when it comes to biology papers 🤷