I have asked various LLMs questions and receive an oh-so-certain definitive answer that is plainly, blatantly WRONG.
This is as annoying as a person who is incapable of admitting making an error, or at least knowing enough to think they are right, but not knowing enough to know they are wrong (to echo Neil deGrasse Tyson).
I include an example below…
This inability to address ambiguity and to be “so certain yet wrong” will become dangerous when we start trusting AI with critical tasks. A “so certain yet wrong” medical diagnosis will kill patients with the wrong treatment plan. AI-powered weapons will kill innocent people that it is “so certain yet wrong” about supposed enemy combatants.
Of course, I might be wrong. This may just be me amusing myself and feeling so superior in my (self-perceived) ability to discern “truth” or at least admit I’m not certain.
And with AI, I intend to remain very uncertain and always skeptical… I am totally not sure.
Not just “a good possibility” that I’m capable of being wrong. I certainly am wrong. Often. and I’m not sure even more often.
Example
The latest, though not my best, example of this unwarranted certainty came from ChatGPT when I asked it,
Who wrote, “One of the urgent tasks of theology today has become the critique of idolatry within theology itself”?
ChatGPT replied,
The quotation is generally attributed to Dorothee Sölle.
It is usually cited as:
… and it went on to quote the sentence back to me and gave a biographical outline of Dorothee Sölle with a quick overview of her theology, only providing a caveat buried in the second paragraph,
…while this sentence is widely attributed to Sölle in secondary literature and collections of quotations, I could not verify the exact wording in a primary source. It may be a translation or paraphrase rather than a verbatim sentence from one of her books.
So, I asked,
Are you sure it wasn’t Rosemary Radford Ruether?
To which, ChatGPT replied,
That’s a good possibility, and I wouldn’t say I’m certain it was Dorothee Sölle.
After checking more carefully, I can’t verify the quotation in a primary source for either Sölle or Rosemary Radford Ruether. The exact sentence appears in secondary literature and quotation collections, but those sources often don’t provide a citation to the original work.
It is a step in the right direction that it admitted a good possibility it was someone else (since it actually was someone else!). Note that I am not picking on ChatGPT with this example of convenience. Gemini and other LLMs have made similarly “so certain yet wrong” answers.
To be fair, I did not get that quote from a book – it was from a PowerPoint slide in a seminary class, taught by a colleague of Dr. Ruether. Also, ChatGPT used the phrase “generally attributed to,” which is an insidious lie in itself. The included reference provided no general attribution for a quote, translation or a paraphrase, just the general idea.
And to follow-up, I’m not the first to recognize this issue….