Maximizers are gonna maximize
@urusan maximizers in what sense?
@maddiefuzz A maximizer is any agent that seeks to maximize some value.
It's often used in AI to describe AI systems that seek the maximum reward value, but it applies to non-artificial intelligence too.
It also applies to artificial systems that, while not strictly intelligent on their own, still act to maximize some value (usually through the intelligent agents that they're composed of).
@maddiefuzz Maximizers have serious safety problems, because they see anything they're not maximizing as an obstacle and/or a resource to be exploited.
There's a whole AI safety field that's working on figuring out what to do about this issue.
I think it's not a hypothetical issue at all. We have to deal with intelligence safety issues all the time and we don't really have a good solution at the moment.
@urusan Ah, I have a sudden memory unlocked of playing the paperclip maximizing browser game.
I've heard about this problem before and it's really interesting, although for some reason I didn't connect your post with that concept.
@maddiefuzz
I've played that one too: https://www.decisionproblem.com/paperclips/
That, in turn, reminds me of http://www.emhsoft.com/singularity/
@urusan
