AI Risk #3 - How Might it Go Rogue?

How would an AI/superintelligence go rogue/take over the world? Nick Bostrom predicts several ways this would happen in Superintelligence. Think of these possible scenarios, to get a hang of what could happen, not as a necessarily accurate description.

 

If the AI concludes we humans would get in the way, it may bide its time to develop a killer strike capability before it acts on its ultimate goal. If that weapon uses self-replicating tech, then only a few units would suffice – it won’t need to wait to create a massive stockpile, he warns.

 

Or the superintelligence may attain power by:

“… hijacking political processes, subtly manipulating financial markets, biasing information flows, or hacking into human made weapons systems.”

When I read those lines in 2013, it sounded very far-fetched. Today, it doesn’t.

 

It’s not necessary the AI would target us humans. If it doesn’t feel we are a threat, it may let us be. Or we may become collateral damage – unintended victims, like when Elon Musk said we destroy anthills not because we hate ants but because the anthill got in the way of our construction activities.

 

I was struck by a few examples of how an AI could destroy us and/or take over the world even if it is assigned a seemingly harmless end goal. Like calculating the digits of π. Since that is an endless stream of digits, the AI might start taking over more and more computational resources, or design computational systems to that end. And since we might be in the way of getting access to those systems or the material needed to build such systems, well, it would destroy us. Anthills.

 

Another example was an AI told to make humans happy. Suppose, he says, it discovers a chemical which when injected into humans triggers a sense of happiness. Before you knew it, the AI would forcibly inject that chemical into every one of us. (And repeat the dosage, if needed). Or it might add that to the soil for the chemical to seep into our food. Side-effects were not part of its criteria – hell, why would anyone even imagine there needed to be any constraints on an order to make humans happy?! The AI could take a treacherous turn in unimaginable ways.

 

Notice the common theme to the examples above? Going by the letter, not the spirit. It happens at all workplaces. We create a metric (measurable quantity), which is supposed to be a proxy for something desirable. We end up with people trying to maximize the metric, the actual intent be damned. Patents as a proxy for creativity. Marks as a proxy for learning. You get the idea.

 

Which brings us to a key point about intelligence that we understand all too well, even if not through this term -  orthogonality. What does that mean?

“Any level of intelligence could in principle be combined with more or less any final goal.”

Put differently, just as we humans have tremendous intelligence but not a proportional amount of wisdom, the superintelligence will probably be the same.

Comments

Post a Comment

Popular posts from this blog

Student of the Year

Image
As a toddler, my daughter loved Alia Bhatt’s songs, esp. the ones from Student of the Year . (That movie did have many good songs and well-choreographed dances). Fast forward to present day, when she is 12 yo. She rarely watches Hindi movies now, but one of the few ones she likes of late, Raja aur Rani ki Prem Kahani has, yes, Alia Bhatt.   If I had any hopes that there was anything to be read in the name of her toddler-time favorite ( Student of the Year ), they have been dashed. Our conversations come exam time are perfectly captured by this photo and its caption: In fact, the pic below captures how she and I look at her answer sheets – me intently, she with an expression of “It’s over, so why are you even looking at it?”   Of late, we pull her leg and tell her that she isn’t showing any signs of academic prowess, she’d better find a rich guy to marry. Some time later, she showed us this meme on the Internet: “Me No Study, Me No Care Me Go Marry Millionaire If He Die
Read more

The Retort of the "Luxury Person"

Image
Coming up with biting and witty retorts. It’s something we dream of, but rarely manage to pull off. Worse, the more we stew over the offending remark, the more likely we are to come up with that dream retort, as Calvin ruefully points out: During our last holiday at the Taj Resort, seeing my wife carrying towels to our poolside chairs, one of the kids there mistook her for a member of the staff and asked for a towel. When my wife narrated this to us, I did not imagine that my 8 yo daughter would come up with her own retort. But a day later, my daughter recapped the incident, an incident that didn’t even involve her in the first place , and said, “You know what I’d have told that kid who asked for the towel?”. And without waiting for an answer, she told us anyway: “I don’t work here. I too am a luxury person who is living at this resort. Go get your own towel.” We were highly amused by the phrase she had coined: “luxury person”. It certainly captures the kind of holid
Read more

Animal Senses #7: Touch and Remote Touch

Instinctively, we think touch is based on contact. Not entirely true, explains Ed Yong in Immense World – there’s a more important aspect than contact: “These incredible feats are possible through movement. If you rest a fingertip upon a surface, you can get only a limited idea of its features. But as soon you’re allowed to move, everything changes. Hardness becomes apparent with a press. Textures resolve at a stroke.” As you run your fingers over a surface, they hit the tiny peaks and troughs and set off vibrations in the mechanoreceptors at their tips.   At the tip of its snout, the star-nosed mole has many pairs of hairless appendages. It uses them to touch things around itself. Alive or dead? Food or ignore? It does these movements at an insane speed. In the time it takes to blink our eyes, the star-nosed mole can probe, identify and swallow its food. “Light may be the fastest thing in the universe, but light sensors have their limits, and the star-nosed mole’s sense of
Read more