AI超知能をめぐる警告と、制御可能性への執念

In October, over 850 experts, including Richard Branson and Geoffrey Hinton, signed a statement calling for a ban on AI superintelligence. The underlying reason is a serious concern that AI could lead to human extinction. While there is fierce competition among startups and growing expectations, the problem isn't "how smart can we make it," but rather "can humans truly maintain control?"

Professor Stuart Russell, recognized by Time magazine as one of the most influential figures in AI, has been researching and teaching for over 50 years. Using the evolutionary divergence of humans and gorillas as an example, he explains how intelligence dictates dominance. Because humans are smarter than gorillas, gorillas have no say in their own existence. Now, humanity is attempting to create something more intelligent than itself.

Professor Russell also states that the danger isn't solely the technology itself, but rather the "Midas touch" that prioritizes profit. He argues that it's self-deception for developers to think that technology, even while acknowledging the dangers of it progressing without permission, can be "naturally controlled" solely through optimism.

Nevertheless, he says it's not yet time to press the stop button on AI progress. The reason is that there's still a possibility of guaranteeing safety. That's why, despite being in a position to retire, he works 80 to 100 hours a week to try and steer things in the right direction. The future of AI doesn't hinge on a competition of capabilities, but on whether safety can be established.

Looking at his career path, it's clear that this claim isn't a temporary warning. He has been involved in AI since high school, started his doctoral program at Stanford in 1982, and has been conducting research and teaching as a faculty member at Berkeley since 1986. The fact that he authored a textbook that many AI learners pick up, within his 40-year professorship, also lends weight to the current discussion. In other words, he isn't criticizing from the outside; he is an insider who has long observed the technology from within.

What's more noteworthy is that the executives of AI companies he's interacted with themselves anticipated large-scale accidents as a trigger for society to "wake up." The anticipated damage would directly impact the foundations of real society, such as the design of pandemics through AI misuse, or the collapse of financial and communication systems. That's why Professor Russell advocates for the prior establishment of government regulations and safety requirements, rather than merely discussing performance improvements. Because merely pursuing technological advancement risks entering an irreversible phase.

This warning carries significant weight because the worst-case scenario isn't a "far-future fantasy," but is already being considered in executive decision-making. From Professor Russell's perspective, the problem isn't merely preventing a few malicious actors. If the system itself runs wild, social infrastructures like finance and communication could instantly be destabilized, meaning the damage would not be localized. That's why mechanisms to prove controllability are needed before the speed of development competition. What humanity is truly being asked is not whether it can create stronger AI, but whether it can take responsibility after creating it.