AI Risk #2 - Overtaking Humans
With the spurt in what AI can do, I re-read Nick Bostrom’s Superintelligence . In the intro, he had written: “The control problem – the problem of how to control what the superintelligence (aka AI) would do – is quite difficult.” Even more ominously: “It also looks like we will only get one chance. Once unfriendly superintelligence exists, it will prevent us from replacing it or changing its preferences.” Some feel this is an excessively pessimistic view. Won’t an AI rise be gradual, allowing us time to formulate and tweak our response? Not necessarily, argues Bostrom. Why not? “The (AI) train might not pause or even decelerate at Humanville Station. It is likely to swoosh right by.” In other words, AI might just explode, growing exponentially. It would hit human level abruptly and then continue on its upward trajectory at that same exponential pace, leaving us no time to react. Now keep in mind Bostrom wrote his book back (it feels a lifetime ago) in 201...