AI Risk #3 - How Might it Go Rogue?

How would an AI/superintelligence go rogue/take over the world? Nick Bostrom predicts several ways this would happen in Superintelligence. Think of these possible scenarios, to get a hang of what could happen, not as a necessarily accurate description.

 

If the AI concludes we humans would get in the way, it may bide its time to develop a killer strike capability before it acts on its ultimate goal. If that weapon uses self-replicating tech, then only a few units would suffice – it won’t need to wait to create a massive stockpile, he warns.

 

Or the superintelligence may attain power by:

“… hijacking political processes, subtly manipulating financial markets, biasing information flows, or hacking into human made weapons systems.”

When I read those lines in 2013, it sounded very far-fetched. Today, it doesn’t.

 

It’s not necessary the AI would target us humans. If it doesn’t feel we are a threat, it may let us be. Or we may become collateral damage – unintended victims, like when Elon Musk said we destroy anthills not because we hate ants but because the anthill got in the way of our construction activities.

 

I was struck by a few examples of how an AI could destroy us and/or take over the world even if it is assigned a seemingly harmless end goal. Like calculating the digits of π. Since that is an endless stream of digits, the AI might start taking over more and more computational resources, or design computational systems to that end. And since we might be in the way of getting access to those systems or the material needed to build such systems, well, it would destroy us. Anthills.

 

Another example was an AI told to make humans happy. Suppose, he says, it discovers a chemical which when injected into humans triggers a sense of happiness. Before you knew it, the AI would forcibly inject that chemical into every one of us. (And repeat the dosage, if needed). Or it might add that to the soil for the chemical to seep into our food. Side-effects were not part of its criteria – hell, why would anyone even imagine there needed to be any constraints on an order to make humans happy?! The AI could take a treacherous turn in unimaginable ways.

 

Notice the common theme to the examples above? Going by the letter, not the spirit. It happens at all workplaces. We create a metric (measurable quantity), which is supposed to be a proxy for something desirable. We end up with people trying to maximize the metric, the actual intent be damned. Patents as a proxy for creativity. Marks as a proxy for learning. You get the idea.

 

Which brings us to a key point about intelligence that we understand all too well, even if not through this term -  orthogonality. What does that mean?

“Any level of intelligence could in principle be combined with more or less any final goal.”

Put differently, just as we humans have tremendous intelligence but not a proportional amount of wisdom, the superintelligence will probably be the same.