In our book Calling Bullshit, Jevin West and I provide the following definition.
Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade by impressing and overwhelming a reader or listener, with a blatant disregard for truth and logical coherence.
Notice that this is what large language models are designed to do. They are designed with a blatant disregard for logical coherence in that they lack any underling logic model whatsoever.
@ct_bergstrom But it's very insecure and quick to change the answer if challenged!