No one should say that a chatbot "hallucinates". Chatbots do not have minds, they manipulate text. Hallucination requires not only consciousness but the physical brain to falsely perceive a sensation as real. Machine learning models have neither consciousness or physical form, and they never will.
@annedrewhu It’s definitely an odd choice of language that implies some kind of victim status for the model and is considerably less clear than saying “outputs false information “
@louiseadennis @annedrewhu I really think we should use Frankfurter's terminology: Bullshit
@annedrewhu @louiseadennis @rrb I hadn’t heard the concept that LLM AI is like a machine hallucinating before, and I quite like it. (This is of course the problem with trying to suppress an idea by discussing it)
Why I like it: it makes clear that the machine isn’t lying, it just has no idea. It’s constructing a description of reality out of scraps of description, and each part fits into the next but there’s no consistency or holistic logic.
@rrb @annedrewhu In other contexts bullshit is clearly a good term for communicating the phenomenon since it succinctly communicates something about the process by which the output is generated beyond simply noting that the output is incorrect.
@louiseadennis @annedrewhu I like the term for two reasons. 1 it is accurate using Frankfurter's definition and 2 it gives the tech the gravitas it deserves.
A tool that spouts n'importe quoi just because it sounds credible is of minimal utility really. A bullshit generator
@rrb @annedrewhu I'd say it depends upon context. Use of of bullshit in some contexts will encourage people to take you less seriously than the person calling it a hallucination. One of the clever things about using "hallucinate" about this phenomenon is it sounds both technical and mysterious and encourages the listener to view the speaker as cleverer/more knowledgeable than them.