'Dr. Gopnik compared the theory of mind of large language models to her own understanding of general relativity. “I have read enough to know what the words are,” she said. “But if you asked me to make a new prediction or to say what Einstein’s theory tells us about a new phenomenon, I’d be stumped because I don’t really have the theory in my head.”

https://www.nytimes.com/2023/03/27/science/ai-machine-learning-chatbots.html