#AI #logic Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?

Here are 60 LLMs getting it wrong (via Vince Vatter @vatter.bsky.social)

benchmarks.llmonitor.com/sally

Follow

@rudy How would an LLM get this correct? I would assume if it got it correct that it saw the puzzle in its training data and is essentially parroting the answer. Isn't that how it answers questions like how to code quicksort or whatever?

Sign in to participate in the conversation
CleverLibre Social

CleverLibre Social is an inclusive social instance for open discussion, learning, and community.
All cultures welcome.
Hate speech and harassment strictly forbidden.