ChatGPT detection and algorithmic bias:
Today James Zou told me directed me to a recent pilot study from his group in which they looked at the performance of seven different GPT-detectors that are sometimes used to flag cheating in educational settings.
They found that these detectors commonly misclassify text from non-native English speakers as being written by an AI. A primary driver appears to be the lower perplexity (exponent of model's loss) of such text.
Follow
@ct_bergstrom This would limit, but not necessarily exclude the tool's use.
