When one AI researcher publishes a personal announcement that “The people building AI earnestly believe that it could kill us all by the end of the decade” and another responds by saying, “I personally think it is >10% within the next decade”, even sober deliberative types ought to take a moment to digest what is being said aloud. A 10% extinction risk from a wayward asteroid would surely be enough to mobilize an all-hands-on deck response from the world’s scientific and engineering communities. So why isn’t this purported 10% risk being taken with equal concern?
■ The incentives right now are almost incomprehensible in their complexity. It’s like we’re in a large casino, where every slot machine has a Powerball-scale jackpot, a handful of whales have hired everyone in Nevada to keep pressing “SPIN” on every machine in sight, no matter what, and the whales don’t actually know their account balances. Meanwhile, the roof is on fire and could cave in at any moment.
■ As has so often been the case throughout history, what we have is not a technical problem but a human one: We live and breathe and feel, and we can do those things with or without these tools. The only moral thing to do with the tools is to make life (specifically, human life) better off. We have never held the capacity to forecast all of the outcomes of a new technology (nobody hanging around with James Watt looked down the road and foresaw global warming), but we have nearly always known that human endeavors come with social responsibilities not to harm others: “Do not plot evil against your neighbors, when they live at peace with you” (Proverbs 3:29).
■ Navigating a strange new world ought to be done with intention and due caution not to do harm. Whether the right people with the right sensibilities are in the right places making the right decisions may not be known for some time. But we definitely know enough already to realize that if anyone close to the action is sounding alarms as dire as “10% extinction risk for humanity”, then it’s worth placing the problem on the front burner.