For @emilymbender , but also anyone else who has an opinion on the subject. :)

I've been rereading "Thought experiment in the National Library of Thailand" and thinking about my own early (long pre-LLM) thoughts about Searle's Chinese Room, and wondering where I actually stand on various questions at the moment.

And I realize that it's not clear to me exactly what Bender is saying, in "Thought experiment".

In particular, on the claim that "it is possible to manipulate forms well enough to be indistinguishable from a system that understands the meaning of the forms, reasons about it, and responds appropriately", is Bender denying that that is possible?

Or is it just that, even if a system does that form-manipulating well enough to be externally indistinguishable, the form-manipulating system will still not, in the deeper sense, be understanding and reasoning and responding?

That is, will the person who attempts to learn Thai solely by frowning at lots of Thai, in fact fail to be able to fool a Thai speaker? Or is the point that, even if a Thai speaker is convinced, they will have been fooled?

#AI #LLM #Linguistics #philosophy