Using exams as a metric for AI seems like a really weird decision. Exams aren't even a good metric for human progress. All they do is test someone's ability to memorize answers. If someone was like "jimmy passed the bar exam after we gave him a cell phone with access to the internet and let him look up all the answers" absolutely nobody would be surprised.
Follow
@malwaretech I think it is quite interesting though, in that it shows that most exams are badly designed. It's also interesting that GPT4 is still very bad at English Literature and English Language...
This should really push us towards improving the quality of assessments.
What I really find scary in the GPT4 white paper is section 2.9 'Potential for Risky Emergent Behaviors'.
@nicolaromano @malwaretech how is it at say Spanish language and literature? Are non english speaking countries having similar issues with ChatGP as we (the english speaking world) seem to be?