@Benhm3 @pluralistic This rests on an assumption that we know what causes consciousness and so know that the differences between "the next-word-predictor program" and humans fall on different sides of that divide. But that is both massively exaggerating what we know about consciousness and massively downplay current AI. To start with, the "next-word-predictor programs" are Turing complete with just a basic loop around them.
Follow
@vidar @Benhm3 @pluralistic The other problem with the argument is that consciousness isn't the point. The e/acc vs. doomer debate about AI risk of human extinction has nothing to do with consciousness. There is no sense in which the AI doom argument requires the super-intelligent AI to be conscious.
