Six Worst-Case Scenarios for AI in the Real World
The worst-case scenario involving artificial intelligence (AI) is familiar to everyone from science fiction blockbusters: machines acquire human-like minds, become sentient and inevitably turn into evil overlords trying to destroy the human race. This story is based on our innate fear of technology, which reflects the profound changes that often accompany new technological developments.
However, as Malcolm Murdoch, a machine learning engineer and author of the 2019 novel "The Quantum Price," says, AI does not have to be sentient to kill us all. There are many other scenarios that will wipe us off the face of the earth before sentient AI becomes a problem.
“We are entering dangerous and uncharted territory with increased surveillance and data tracking, and we barely understand the potential consequences”
In interviews with AI experts, IEEE Spectrum identified six worst-case scenarios for artificial intelligence in the real world that are far more mundane than those shown in movies. But they are no less dystopian.
When fiction defines our reality
If we allow fiction to define our reality, an unnecessary tragedy may occur. But what choice do we have if we cannot distinguish the real from the false in the digital world?
In a frightening scenario, the rise of deepfakes—fake images, videos, audio and text created with advanced machine learning tools—could one day lead national security decision-makers to take real actions based on false information, causing a major crisis or, even worse, war.
Andrew Lohn, senior research fellow at Georgetown University's Center for Security and Emerging Technology (CSET), said that "AI-powered systems are now capable of generating disinformation at scale." By creating large volumes and variety of fake messages, these systems can conceal their true nature and optimize for success, improving the desired impact over time.
The mere idea of deepfakes during a crisis may also cause leaders to hesitate in their actions if the credibility of information cannot be verified in a timely manner.
Marina Favaro, research fellow at the Institute for Peace Research and Security Policy in Hamburg (Germany), noted that "deepfakes by default undermine our trust in information flows." Both action and inaction prompted by deepfakes could lead to catastrophic consequences for the whole world.
A dangerous race to the bottom
When it comes to AI and national security, speed is both the point and the problem. As AI-powered systems give their users major speed advantages, countries that first develop military applications will gain a strategic edge. But which principles might be sacrificed in the process?
Helen Toner, director of strategy at CSET, suggests that a crisis could "start as a harmless single point of failure that shuts down all communications, causing panic among people and a halt to economic activity. A persistent lack of information, accompanied by other miscalculations, could cause the situation to spiral out of control."
Vincent Boulanin, senior researcher at the Stockholm International Peace Research Institute (SIPRI) in Sweden, warns that major catastrophes could occur "when great powers cut corners to get ahead first. If one country prioritizes speed over safety, testing or human oversight, it will be a dangerous race to the bottom."
The end of privacy and free will
With every digital action, we generate new data—emails, text messages, downloads, purchases, posts, selfies and GPS locations. By giving companies and governments unrestricted access to this data, we hand over tools of surveillance and control.
With the addition of facial recognition, biometrics, genomic data and AI-powered predictive analytics, Lohn of CSET worries that "we are entering dangerous and uncharted territory with increased surveillance and data tracking, and we barely understand what the possible consequences are."
Michael C. Horowitz, director of Perry World House at the University of Pennsylvania, warns "about the logic of AI and what it means for domestic repression. In the past, the ability of autocrats to suppress their population depended on a large group of soldiers, some of whom could side with society and carry out a coup. AI may reduce such constraints."
The power of data, once collected and analyzed, goes far beyond monitoring and surveillance functions, enabling predictive control. Today, AI systems predict which products we will buy, which entertainment we will watch and which links we will click.
The human Skinner box
Social media users have turned into lab rats living in human Skinner boxes (a Skinner box is a laboratory apparatus used to study animal behavior), glued to their smartphone screens, forced to sacrifice more precious time and attention to platforms that profit at their expense.
Helen Toner of CSET said that "algorithms are optimized to keep users on the platform as long as possible." By offering rewards in the form of likes, comments and subscriptions, Malcolm Murdoch explains, "algorithms close the loop on our brain's wiring, making our next engagement irresistible."
To maximize advertising profits, companies divert our attention from work, family and friends, responsibilities and even hobbies. Even worse, the content often makes us feel unhappy and worse than before. Toner warns that "the more time we spend on these platforms, the less time we spend pursuing a positive, productive and fulfilling life."
The tyranny of AI design
Every day we hand over much of our daily lives to AI-powered machines. This is problematic because, as Horowitz notes, "we have yet to fully grasp the problem of bias in AI. Even with the best intentions, the design of AI-powered systems, both their training data and mathematical models, reflects the narrow experiences and interests of the biased people who program them. And all of us have our own prejudices."
Lydia Kostopoulos, senior vice president of emerging technology at information security company KnowBe4, argues that "many AI-powered systems do not account for the diverse experiences and characteristics of different people." Because AI solves problems based on biased views and data rather than the unique needs of each person, such systems impose a level of conformity that does not exist in human society.
Even before AI, the design of everyday objects often served a particular type of person. For example, studies have shown that cars, hand tools, including mobile phones, and even temperature settings in offices were designed for average-height men, putting people of different heights and builds, including women, in uncomfortable and sometimes even dangerous positions.
Fear of AI deprives humanity of its benefits
Because modern AI works with datasets, advanced statistical models and predictive algorithms, the process of creating machine intelligence ultimately centers on mathematics. According to Murdoch, "linear algebra can do crazily powerful things if we are not careful."
But what if people become so afraid of AI that governments regulate it in a way that deprives humanity of many of its benefits? For example, DeepMind's AlphaFold made a major breakthrough in predicting how amino acids fold into proteins, allowing scientists to determine the structure of 98.5% of human proteins. This milestone will lay a fruitful foundation for rapid development in the life sciences.
Consider the benefits of improved communication and cross-cultural understanding made possible by seamless translation across any combination of human languages, or using AI-powered systems to find new treatments for diseases.
Government regulatory measures to protect against AI's worst-case scenarios may also have unpleasant side effects and lead to their own unintended negative consequences, resulting in us being so frightened of this amazing technology's power that we resist using it for the good it can bring to the world.
