This is the best piece I’ve read yet on the dangers of AI.
“Language is the operating system of human culture. From language emerges myth and law, gods and money, art and science, friendships and nations and computer code. A.I.’s new mastery of language means it can now hack and manipulate the operating system of civilization. By gaining mastery of language, A.I. is seizing the master key to civilization, from bank vaults to holy sepulchers.“
The essay adroitly puts together pieces I’ve been trying to assemble in my own mind and put on paper myself.
We don’t have to wait for AGI to go rogue, gain access to physical means of coercion, and enslave the human race.
The current online technologies for information exchange, and large language models that can generate convincing text at scale— these together can be a binary weapon that destroys a huge amount of what we hold dear, including democracy.
The problem with the piece, which I didn't acknowledge in my original post, is that it ascribes an unrealistic and unnecessary degree of agency to LLMs and AI.
They aren't necessarily capable of learning how to manipulate humans to anything like the extent imagined in that essay.
The thing is, they don't have to be to cause dreadful harm.
The model in my head is not LLMs "deliberately" doing anything, but rather us poisoning ourselves with their output while they just blindly churn.
@ct_bergstrom They certainly don't have the ability to manipulate humans directly.
But they do have incredible (dangerous) potential as a tool to allow humans to manipulate other humans. People don't seem to have discussed this yet, but anyone (or group) could just build a new #LLM trained with emphasis on different data sources - qanon material, fascist texts, etc. Such new flavors of LLMs could be very compelling and very dangerous to a lot of people.
