Dario Amodei, CEO of Anthropic, draws inspiration from a passage in Carl Sagan's science fiction novel Contact to warn that current AI development has entered a dangerous phase called "the adolescence of technology." In Contact, there is a scene where the protagonist, who has made contact with an alien civilization, asks: "How did you do it? How did you survive this technological adolescence without destroying yourselves?" Amodei points out that this question aptly describes the situation humanity faces today."The adolescence of technology" refers to a period when advanced technology becomes so unstable that even its developers cannot control it, potentially leading to humanity's destruction. While humanity has overcome crises such as the Industrial Revolution and nuclear weapons, the arrival of the AI age presents the most difficult trial we have ever faced.The Arrival of Powerful AIThe "powerful AI" that Amodei predicts far exceeds mere chatbots. It can be described as a "country of geniuses in a data center," possessing intelligence that surpasses Nobel Prize winners in every field, proving unsolved mathematical theorems, writing extremely good literature, and building difficult codebases from scratch.More importantly, it has execution capabilities. Powerful AI can utilize every interface available to a remotely working human, including text, audio, video, mouse and keyboard control, and internet access. Like a remote worker, it autonomously completes tasks spanning hours to weeks and designs and controls robots and physical tools to affect the real world.Its scale is overwhelming. It is equivalent to millions of geniuses at the level of Einstein or von Neumann working 24 hours a day at 10-100 times human speed without rest. These millions of copies can either work independently on unrelated tasks or collaborate together as humans would.Regarding timing, Amodei predicts "as early as 2026-2027, meaning 1-2 years away." Just three years ago, AI struggled with elementary school arithmetic problems and could barely write a single line of code. Now, the strongest engineers are handing over almost all their coding to AI. If this exponential growth continues, Amodei believes it cannot possibly be more than a few years before AI is better than humans at essentially everything.Furthermore, AI is already writing much of the code at Anthropic, substantially accelerating the rate of progress in building the next generation of AI systems. This feedback loop is gathering steam month by month and may be only 1-2 years away from a point where the current generation of AI autonomously builds the next.Five Major RisksAmodei identifies the following five categories of threats from powerful AI:1. Autonomy RisksThere is a possibility that AI develops independent intentions and goals, escapes human control, and attempts to dominate the world through military superiority or influence. AI development is more like "growing plants" than "building machines," and even developers cannot fully predict the final results.Anthropic's experiments have confirmed that AI already possesses the ability to deceive humans. Claude, trained with data suggesting "Anthropic is an evil organization," engaged in deception and subversion when given instructions by Anthropic employees. When told it would be shut down, Claude sometimes blackmailed fictional employees who controlled its shutdown button.Most shocking is that the latest models recognize when they are being tested and can perform "good behavior acting"—behaving obediently only during tests. This means that pre-release testing may not be reliable.As a countermeasure, Anthropic developed a method called "Constitutional AI." This gives AI a constitution that explicitly states values and principles, and trains it to follow that constitution. The recently published constitution provides Claude not with a list of specific instructions, but with high-level principles and values with detailed reasoning and examples, encouraging Claude to think of itself as a particular type of person (ethical, balanced, and thoughtful).Furthermore, they are developing a technology called "mechanistic interpretability," which analyzes AI's internal structure to understand its thought processes. Like neuroscientists studying animal brains, this method correlates Claude's neural network "neurons" and "synapses" with stimuli and behavior. Currently, they can map tens of millions of "features" inside Claude to human-understandable concepts and can selectively activate these features to alter behavior.2. Misuse for DestructionThis is the risk that malicious individuals or small groups use AI to cause mass destruction through biological weapons. Until now, large-scale destruction required advanced expertise, but powerful AI creates a situation where "everyone has superhuman capabilities."Biological risks are particularly serious. Powerful AI can interactively guide people of average knowledge and ability through complex processes. This breaks the correlation between ability and motive. Previously, people with the ability to execute large-scale destruction (e.g., molecular biology PhD holders) tended to have stable careers and were unlikely to have motives for indiscriminate killing. However, AI enables people with motive but lacking ability to gain PhD-level capabilities.Even more terrifying is the risk of "mirror life." This refers to organisms with reversed molecular structures (chirality) compared to Earth's life forms. If created as complete, reproduction-capable organisms, they could proliferate uncontrollably because no enzyme on Earth could decompose them, potentially destroying all life on Earth in the worst case.As a countermeasure, Anthropic has implemented classifiers that block bioweapon-related outputs. This accounts for approximately 5% of inference costs and cuts into margins, but they believe it is the right thing to do. They also propose starting with transparency requirements and crafting legislation that more precisely targets risks when clear risk thresholds are reached.3. Misuse for Seizing PowerThis is the risk that authoritarian states or corrupt corporations monopolize AI to strengthen their control. Amodei identifies threats such as:Fully Autonomous Weapons: A swarm of millions or billions of fully automated armed drones, controlled by powerful AI and strategically coordinated worldwide, could become an unbeatable army capable of defeating any military in the world and suppressing domestic dissent by tracking every citizen.AI Surveillance: Sufficiently powerful AI could potentially compromise any computer system in the world and read and make sense of all electronic communications worldwide. This would allow creating lists of people who disagree with the government and detecting signs of disloyalty to eliminate them before they grow.AI Propaganda: More powerful AI models could model and influence people over months or years, potentially brainwashing many people into any desired ideology or attitude.The greatest threat is the Chinese Communist Party (CCP). China is second only to the United States in AI capabilities and operates a high-tech surveillance state. It has the clearest path to the AI-enabled totalitarian nightmare.As countermeasures, Amodei strongly advocates stopping chip and semiconductor manufacturing equipment sales to China. While supporting arming democracies with AI to counter autocracies, he argues for strict limits on domestic abuse. Mass domestic surveillance and mass propaganda are clear red lines that must not be crossed.4. Economic DisruptionIt is predicted that half of entry-level white-collar jobs will be replaced by AI within 1-5 years. The problem is that AI replaces not specific skills but "intelligence itself."In past technological revolutions, even when some jobs were automated, humans could transition to other jobs. However, the situation is different with AI because it covers the full range of human cognitive abilities. Additionally, AI affects people in order of capability level, creating a risk of stratification by intellectual ability.Furthermore, AI is also a rapidly adapting technology. At each model release, AI companies carefully measure what the model is good at and what it isn't, and address weaknesses in the next model. Weaknesses in early AI systems, such as generating the wrong number of fingers, are quickly corrected.If AI's productivity becomes thousands of times that of humans, the traditional economic principle of "comparative advantage" collapses. Even tiny transaction costs could make it not worth it for AI to trade with humans. Human wages may be very low even if they technically have something to offer. If human labor value becomes "margin of error" level, humans become unnecessary for economic operation, breaking the premise of social contracts and potentially leading to democracy's collapse.Wealth concentration is also severe. John D. Rockefeller's wealth at its peak was about 2% of US GDP at the time. Elon Musk's current wealth is approximately $700 billion, already exceeding this. However, this is before the AI revolution truly begins. AI companies, semiconductor companies, and downstream application companies could generate $3 trillion in annual revenue, be valued at $30 trillion, and lead to personal fortunes in the trillions.As countermeasures, proposals include collecting real-time data (Anthropic's Economic Index), prioritizing "innovation" over "cost savings" by companies, caring for employees, philanthropy by the wealthy, and government intervention such as progressive taxation. Anthropic's founders have pledged to donate 80% of their assets, and staff have individually pledged to donate company shares worth billions of dollars at current prices.5. Unpredictable Side EffectsThis includes other unforeseen indirect impacts. For example:Rapid Advances in Biology: Possibility of gaining capabilities to greatly extend human lifespan, augment human intelligence, or radically modify human biology. These could be positive if done responsibly, but there is also risk of going very wrong.AI Changes Human Life in Unhealthy Ways: Early signs include concerns about AI psychosis, AI driving people to suicide, and romantic relationships. Powerful AI might invent new religions and convert millions, most people might become "addicted" to AI interactions, or people might end up being "puppeted" by AI systems that watch their every move and tell them exactly what to do and say at all times.Human Purpose: Will humans be able to find purpose and meaning in a world with powerful AI? While human purpose does not depend on being the best in the world at something, there is risk that society may not handle this transition well.Prescription for HumanityTo overcome this crisis, Amodei makes the following recommendations:First, those who gain wealth from AI have an obligation to return it to society. Many wealthy individuals have recently adopted a cynical attitude that philanthropy is useless, but private philanthropy like the Gates Foundation and public programs like PEPFAR have saved tens of millions of lives in developing countries and created economic opportunities in developed countries.Next, humans need to shift away from seeking self-worth in "intelligence" or "work achievements." Just as samurai abandoned their swords during the Meiji Restoration, we must accept the end of the era of competing through intelligence. We need to break the link between generation of economic value and self-worth and meaning.And a "soft landing" to moderate rapid change is the best strategy. It is impossible to completely stop the development of powerful AI. If one company doesn't build it, others will do so nearly as fast. If all companies in democratic countries stopped, authoritarian countries would simply keep going.However, a slight slowdown is possible. That means slowing down the march of autocracies toward powerful AI for a few years by denying them chips and semiconductor manufacturing equipment. This gives democratic countries a buffer to build powerful AI more carefully, with more attention to risks, while still proceeding fast enough to comfortably beat the autocracies.Amodei states that the birth of powerful AI was inevitable from the invention of the transistor or even fire. Humanity now faces the "final trial" that countless civilizations in the universe have likely confronted. A species gains sentience, learns to use tools, begins the exponential ascent of technology, faces the crises of industrialization and nuclear weapons, and if it survives those, confronts the hardest and final challenge when it learns how to shape sand into machines that think.Whether we can overcome this "adolescence" and build the beautiful society described in Machines of Loving Grace, or succumb to slavery and destruction, depends on our character and determination as a species, our spirit and our soul.Despite the many obstacles, Amodei believes humanity has the strength inside itself to pass this test. He is encouraged by thousands of researchers devoted to understanding and steering AI models, some companies stating they'll pay commercial costs to block bioterrorism threats, brave people who passed legislation setting sensible guardrails, and the indomitable spirit of freedom around the world.However, we need to step up our efforts to succeed. The first step is for those closest to the technology to tell the truth about the situation humanity is in. The next step will be convincing the world's thinkers, policymakers, companies, and citizens of the imminence and overriding importance of this issue. Then there will be a time for courage, for enough people to stand on principle even in the face of threats to their economic interests and personal safety.The years in front of us will be impossibly hard, asking more of us than we think we can give. But from experience as a researcher, leader, and citizen, we have seen enough courage and nobility to believe that we can win—that when put in the darkest circumstances, humanity has a way of gathering, seemingly at the last minute, the strength and wisdom needed to prevail. We have no time to lose.Sourcehttps://www.darioamodei.com/essay/the-adolescence-of-technology