Yesterday I had a number of conversations with people working in the scholarly publishing sphere about what happens when AI chatbots pollute our information environment and then start feeding on this pollution.
As it so often, the case, we didn’t have to wait long to get some hint of the kind of mess we could be looking at.
https://www.theverge.com/2023/3/22/23651564/google-microsoft-bard-bing-chatbots-misinformation
My fear is that we’ve created an information ecosystem that is uniquely susceptible to the perversions of these AI tools. Fifty years ago, had they existed, they would’ve been mere curiosities because we lacked the information infrastructure for their output to swamp more trusted forms of information. Even twenty years ago there would have been substantially less opportunity for them to have cause harm.
The confluence of this technology with the information ecosystem that we described in our paper from a couple of years could be an epistemic catastrophe.
https://www.pnas.org/doi/10.1073/pnas.2025764118
I’m coming to think that releasing these tools was a reckless act with the potential to generate negative externalities we have barely started to imagine.
The threat isn’t rogue superintelligence. It’s bullshit at unprecedented scale, reflected back upon itself and iteratively amplified.
@ct_bergstrom Neal Stephenson predicted this too: read his discussion of "bogons" in Anathem.
