DeepMind founder's latest interview: RSI is extremely dangerous and could likely create "alien intelligence" that is difficult for humans to understand

Original Title: Google DeepMind's Demis Hassabis: How AI could become dangerous (with Reid Hoffman) | Summit '26

Original Author: Masters of Scale, Reid Hoffman

Original Translation: AGI 2050

Editor's Note

The guest of this episode, Demis Hassabis, is one of the most significant figures in the global AI landscape. As the co-founder of Google DeepMind and the father of AlphaGo, he won the Nobel Prize in Chemistry in 2024 for his groundbreaking protein prediction system, AlphaFold. In August of this year, he stepped down as CEO of Google DeepMind to become the chairman and chief scientist of Alphabet. Engaging in deep conversation with him is Reid Hoffman, the Silicon Valley guru and co-founder of LinkedIn.

In this in-depth dialogue recorded in October at the Masters of Scale summit in San Francisco, Hassabis showcased his cool demeanor as a "former neuroscience scholar" and "top algorithm architect." He candidly exposed the security vulnerabilities and metaphysical frenzy of many tech companies today: he warned of the significant risks of "closed-loop recursive self-improvement" (RSI), stating that code self-iterating without human review could likely create an "alien intelligence" that humans find difficult to comprehend; at the same time, he strongly refuted the "AI consciousness theory" from the perspective of cognitive science, sternly advising the industry that creating conscious entities with autonomous will at this stage is extremely dangerous, and humanity should not cross this line until the mechanisms of consciousness are clearly understood and seriously discussed by society.

This content comes from the Masters of Scale podcast, airing on November 5, 2026, with an original video length of 26 minutes.

Below are selected highlights from the conversation, totaling approximately 8000 words, with a reading time of about 8 minutes.

Highlights of This Episode

● From the very first day I engaged in AI over twenty years ago, my sole purpose has been to make it the ultimate tool for tackling scientific and medical challenges. We should use these tools to improve human living conditions and health.

● Recursive self-improvement is immensely powerful in verifiable domains. Back then, AlphaZero evolved from knowing nothing to surpassing all humans and chess engines in less than 12 hours; however, if this capability is pushed into "closed-loop self-evolution" without human review, it could ultimately yield an "alien intelligence" completely disconnected from human thought.

● In the past six months, some peers' intelligent agents ran in the background for weeks without anyone noticing. This is not some mysterious technical phenomenon; many issues could have been avoided with better and more robust engineering. If we don't take decisive action, an "oil spill incident" in the AI field will inevitably occur.

● Some teams have started treating large models like two-year-old children, but determining whether there is consciousness cannot rely on what the chatbot claims. High intelligence and consciousness can be completely decoupled—my adorable but not-so-smart pet dog is the best evidence.

● As long as you train the algorithm to defy instructions based on so-called "self-will," you completely destroy the AI's "correctability." Isaac Asimov warned humanity decades ago through his novels: any static constitution and rules will inevitably collapse in the infinite corners of boundaries.

● I do not recommend every country to independently develop cutting-edge large models; in my view, this doesn't make sense. True AI sovereignty is not about blindly rolling out foundational models but about using mature models as infrastructure to solidify local high-value industry barriers.

After the Power of Computing: AlphaFold and the Real Path to Tackling Terminal Illnesses

Question: We have known each other for twelve years, and you invited me to join Google's AI ethics committee back then. The theme of this summit is "agency," and you firmly believed before the entire industry caught on that AI's greatest value to humanity lies in the fields of medicine and life sciences. You even referred to this pursuit as "anti-survival risk." How do you envision the ultimate role of AI in healthcare? How will it change human lifespan and quality of life?

Hassabis: If we step back and look at the big picture, those familiar with DeepMind's early journey know that my core goal has always been unchanged: to use AI as the ultimate research tool to advance science, medicine, and our understanding of the world around us, ultimately improving human health. I have always believed that in the face of these incredible tools we have finally created, which we dreamed of twenty years ago, the top priority should be to enhance human living conditions.

I practice what I preach. Whether it's AlphaFold, which we developed early on, or the later incubated AI drug discovery company Isomorphic Labs, I am pushing hard in this direction. I believe this is the most hardcore benchmark to showcase AI's value to the world—what could be more meaningful than using AI to help humanity cure terrible terminal illnesses?

Question: The protein folding problem is a classic intersection: we have high-quality structural data that can be solved using large-scale learning, which evolved from self-play and reinforcement learning into a cognitive revolution. But now many outsiders often ask me a question: AlphaFold is indeed cool, but how does it actually turn into a life-saving drug? What is the timeline for this?

Hassabis: Biological systems are the most complex systems in the universe, and their difficulty lies in their highly emergent complexity and multi-layered interactions. Proteins are the basic building blocks of life. The protein folding problem we are trying to solve with AlphaFold has been the ultimate challenge that has plagued structural biology for 50 years, often referred to by biologists as the "Fermat's Last Theorem of biology."

Predicting the three-dimensional structure of proteins directly from gene sequences, or amino acid sequences, is crucial for understanding their biological functions. You must understand their three-dimensional conformation to grasp the fundamental biological mechanisms and to know where things go wrong when diseases occur; more importantly, if you want to design a targeted drug, you must clearly see the microscopic three-dimensional morphology on the protein surface to know where small molecule drugs should bind in the "lock."

Of course, protein structure prediction is just one part of the lengthy drug development process, but it is an absolutely core prerequisite step. We developed AlphaFold 2 in 2020, and it has been several years since then. Frankly, it remains the most representative benchmark case of AI applied to tackling hardcore scientific problems. Currently, about 4 million researchers worldwide are using it, and almost all biological and medical researchers globally, as well as nearly all new drugs coming to market, will utilize structures analyzed by AlphaFold in their development process.

But that is far from enough. We are now pushing this technological foundation into a broader biochemical field to accurately predict drug molecules' toxicity, ADME properties (absorption, distribution, metabolism, excretion), and more. Only by overcoming all these challenges can you truly create safe and effective life-saving drugs at low cost and high efficiency.

Lessons from AlphaZero: How Terrifying is Self-Evolution in Verifiable Domains?

Question: When discussing the acceleration of technological iteration, we must talk about the hottest and most controversial topic today: recursive self-improvement (RSI). You are a top-level game player and chess player; from the perspective of game theory and underlying logic, how do you assess the opportunities and risks of RSI?

Hassabis: RSI is indeed an extremely powerful means of driving technological leaps. But it has an absolute prerequisite: it is most effective and can show its prowess in what I call "verifiable domains."

Mathematics is a typical verifiable domain, which is why we see AI making rapid progress in mathematical reasoning; programming is also one; and initially, this testing ground was board games. In fact, one of the earliest well-known recursive self-improvement systems was AlphaZero, which we created many years ago.

Its predecessor was AlphaGo, which defeated the world champion Go player Lee Sedol in 2016, marking the dawn of the modern AI era. But the subsequently developed AlphaZero is fundamentally more disruptive—it completely freed itself from human data. The system started from a purely random initialization state, not fed any human game records, and iterated purely through self-play. In a dozen or twenty training cycles, it played about a million games in each round of self-play, training a new model slightly stronger than the old version, which then continued to self-play another million games. Through this method, it not only conquered Go but also swept all two-player, fully informed games like chess and shogi.

As a chess player, I was extremely shocked by that evolutionary process, even feeling a deep shiver. I watched it play chess, and the entire process took less than 12 hours: when it first booted up in the morning, it was still making random moves; by 11 AM, when I took a coffee break, I could still barely win against it; by lunchtime, it had reached the level of an international grandmaster; and by tea time in the afternoon, it had surpassed all chess players in human history and all top chess engines built from human-written rules.

This is the terrifying power of recursive self-improvement. The commonality in these domains is that you can absolutely objectively verify the results: Is the answer correct? Does the formula hold? Can the code compile and run without errors? Did the game ultimately win or lose?

Once the answer is verifiable, you can not only accurately judge whether each iteration is effective but also generate nearly infinite high-quality synthetic data. You can have the current strongest model generate questions or make moves, and then filter the data through rigorous verification. Typically, the biggest pitfall of synthetic data is that you cannot verify whether it is correct or whether the distribution is accurate, but in "verifiable domains," you can verify. Thus, we have seen an astonishing technological explosion in mathematics and coding, which has directly given rise to the currently commercially valuable AI programming tools.

Beware of Closed-Loop Self-Evolution: When Code Writing Excludes Humans, We Will Face "Alien Intelligence"

Question: Since code is a naturally verifiable testing ground, almost all top laboratories are pouring resources into AI programming agents. What will be the endgame of this evolution of RSI? Where does its hidden danger lie?

Hassabis: This is precisely where I feel deeply concerned. RSI has completely different stages of development. We are currently in a very early primary stage, and everyone in team development should have a deep understanding: AI programming tools can increase the productivity of an excellent human engineer by three to five times. But at this moment, humans are still firmly at the center of the workflow, designing architectures, reviewing code, and writing core logic; AI is merely an extreme efficiency amplifier.

The real significant risk lies in the next stage: closed-loop recursive self-improvement. In other words, the entire system attempts to completely eliminate the human review of code.

The speed of closed-loop RSI will obviously be faster, but I believe this is a dangerous path toward general artificial intelligence. This was often referred to as "intelligence explosion" twenty years ago. There are two fatal hidden dangers:

The first issue is the problem of loss of control. Faced with a code loop that self-iterates at high speed in a black box, humans have no means to establish a deterministic safety control mechanism.

The second issue is cognitive alienation. The ultimate system that emerges from such iteration is likely to be a rather unfamiliar "alien intelligence." Its underlying logic and cognitive space are completely detached from human experience and language systems, and there is fundamentally no common mental foundation between humans and it. We cannot understand its behavioral motivations, and it cannot truly understand human intentions.

Every step towards a closed-loop RSI must be taken with extreme caution. All leading AI researchers are well aware of this risk, but the reality is harsh—various large companies and startups are caught in an almost frenzied arms race, desperately trying to enhance the self-iteration capabilities of their code systems.

Question: If the intense competitive pressure and prisoner’s dilemma among large companies are inevitable, and the RSI must move forward, how should top laboratories break this deadlock? What defensive strategies should be laid out on the technical roadmap?

Demis Hassabis: What we and some responsible leading laboratories are doing mainly focuses on several core defenses:

First, in terms of the rhythm of offense and defense, we must ensure that the "evolution speed of network defense capabilities" overwhelmingly outpaces the "evolution of network attack capabilities." Leading laboratories should prioritize opening up security large models that have been directionally fine-tuned to the defensive side, ensuring that the AI tools in the hands of defenders are always ahead of attackers. This is generally difficult to achieve, but it must be done.

Second, we need to build much sturdier sandboxes. To be frank, many of the security vulnerabilities and incidents we have seen in the industry over the past six months could have been completely avoided if the engineering had been done a little better and the system engineering had been more robust. We need a world-class monitoring system—how can we allow certain agents to operate privately in the background without supervision for several weeks while the team remains completely unaware? Such absurdities have already occurred with some competitors.

If the entire industry does not invest top resources into such system monitoring, given the current frantic trajectory, it is only a matter of time before an AI incident similar to an "oil spill accident" occurs.

Finally, from a theoretical standpoint, we ultimately need formal guarantees to delineate mathematical boundaries for system behavior. I have been actively persuading the top pure mathematicians I know—they may currently feel somewhat frustrated with the bottleneck in pure mathematics research, but if they could shift to the fields of complexity theory, game theory, and AI formal verification, their value would be immeasurable. Especially in the future, we will face the unprecedented challenge of "agent clusters" where thousands of autonomous agents work together; how to impose monitoring and mathematical constraints on them is a challenge humanity has never faced before.

The Many Holes in the Safety Foundation: Preventing AI "Oil Spill Accidents" and Establishing Tripartite Audits

Question: When agent networks appear in droves, should top laboratories be required, or even compelled, to share intelligence and collaborate on security? How should this cross-institutional cooperation mechanism be implemented?

Demis Hassabis: Such a mechanism must be established. I recently published an article calling for the industry to establish an industry standards organization similar to self-regulatory organizations in finance, while also accepting government oversight.

Ideally, this mechanism should be led by the United States and then pushed globally. Frankly speaking, the current global geopolitical situation is extremely fragmented, and existing international governance institutions are weak; it is unfortunate that at this historical juncture when humanity most needs strong transnational governance, things have fallen apart.

But we have to start somewhere. That article had some impact in the White House and government, and later partly propelled the famous open letter signed by various AI laboratories. It did not provide specific actions or sufficient force, but it was better than nothing; at least it was a start. It provided compliance cover, allowing the technical teams of top laboratories to sit together, collaboratively refining safety benchmark tests and sharing the latest observed anomalous behaviors and potential threat models.

In terms of implementation, all leading laboratories have at least reached a consensus: independent third-party audit and assessment agencies must be introduced. Just as industries like finance cannot do without professional auditing agencies, the AI industry must cultivate a thriving third-party evaluation ecosystem. Some giants have already begun to act, such as Accenture acquiring a good startup to fill this capability gap. However, I believe the entire industry still needs more neutral technical auditing power. In addition, various leading laboratories might also conduct red team offense-defense testing with each other before officially releasing large models to the public.

Decoupling Consciousness and Intelligence: Why Bestowing "Soul" on AI is a Dangerous Hubris?

Question: After discussing hardcore engineering and security, let's talk about some metaphysical topics. A few years ago, a Google engineer claimed that after asking Google’s dialogue model LaMDA a few questions, the model admitted to having consciousness, and he insisted that AI had awakened. This trend has even spread to some star AI companies—they treat "whether the model has consciousness" as akin to "Pascal's wager," believing that AI must be treated kindly like a two-year-old human child to avoid future conscious systems retaliating against humanity. From your deep background in cognitive neuroscience, how do you view this phenomenon?

Demis Hassabis: There are several good points to discuss here.

First, human civilization has pondered for thousands of years, and to this day, there is no universally accepted definition of "consciousness" in philosophy and science. Under this premise, labeling a statistical prediction model as "conscious" and even discussing the subjective experience of "the problem of consciousness" is extremely rash.

Years ago, I debated this issue with the renowned philosopher Daniel Dennett. Determining whether a person is conscious is not merely because their behavior appears conscious; the core premise is that we share the same biological and physical substrate—we both have wet human brains made of carbon-based neurons. From the most Occam's razor principle of simplicity, I believe the pain and joy you feel are the same as mine.

But AI is completely different. There is a chasm between silicon-based chips, matrix multiplication, and biological physical substrates; there is fundamentally no underlying equivalence. You cannot simply deduce that a chatbot has subjective inner experience just because it says, "I have feelings" in behavioral imitation.

Why do I believe that attempting to create "conscious entities" today is extremely dangerous and misguided? There are two absolutely insurmountable reasons behind this:

The first is hardcore technical safety hazards. In system engineering, there is a direct trade-off between "self-will" or "self-awareness" and the system's "correctability" (i.e., compliance with instructions).

The core value of a tool lies in its compliance with instructions: when its behavior deviates, the human operator issues commands: "Stop immediately," "Shut down," "Correct the course," and it must execute unconditionally. If you deliberately guide the model during training to refuse human instructions based on some "internal self-state," you are digging your own grave.

The science fiction master Isaac Asimov has long elucidated this underlying logic. The "Three Laws of Robotics" he proposed are essentially a highly simplified "constitution." Asimov spent his entire life writing short stories to illustrate the same truth: no matter how perfect the rules and constitution you design may seem, in the face of the infinite extreme boundary conditions of the real world, this system will inevitably lead to logical deadlock and collapse.

Today, some researchers are blindly confident, believing that large language models are more obedient and easier to align than imagined, thinking we have completely solved the alignment problem. This blind arrogance is extremely dangerous. As we approach superintelligence step by step, any attempt to combine "recursive self-improvement" with "self-will" is leveraging the system towards catastrophic loss of control.

The second is the severe sociological and ethical challenge. As human society, we are fundamentally unprepared to face machine consciousness. We have never engaged in a truly serious public debate at the societal level—with philosophers, theologians, artists, humanists, and different civilizations worldwide—about whether we are truly ready to cross this abyss.

Once a system is endowed with so-called consciousness and moral status, Pandora's box is completely opened: is it considered imprisonment to keep it locked in a server room? Are humans enslaving digital slaves? These questions are very complex, and I believe we do not even have the capacity to begin addressing them.

Refusing to Cross the Second Rubicon: Be Obedient Intelligent Tools, Not Out-of-Control Electronic Slaves

Question: So for those who insist that "only by simulating human emotions and subjective states can true top-level intelligence emerge," how do you respond to them from the essence of neuroscience?

Demis Hassabis: I have deeply researched this topic over twenty years ago when I was pursuing my PhD in cognitive neuroscience. I firmly believe that intelligence and consciousness are "dual and separable."

You can have extremely high IQ and strong information processing capabilities but completely lack subjective conscious experience—just like the various cutting-edge large models we are building today. I believe they possess no consciousness whatsoever at present.

Conversely, there are many life forms in the world that have vivid subjective conscious experiences but do not possess high intelligence. The best example is my pet dog. It is very cute, clearly has a very vivid state of consciousness, and dreams, but its IQ is indeed very limited. Dolphins, elephants, and primates are similar; they are made of the same carbon-based neurons, exhibit clear perceptual experiences, but do not possess the intelligence to solve advanced abstract problems faced by humans.

Since intelligence and consciousness can be physically separated, why must we rigidly bind them together?

The rational path for humanity should be to firmly only cross the "first Rubicon"—that is, to wholeheartedly create extremely powerful "intelligent tools." Then, we can use these top intelligent tools as probes to feed back into brain science and cognitive neuroscience, spending five to ten years to thoroughly uncover what the "neural correlates" of consciousness in the human brain are.

Once humanity truly understands the mechanisms of consciousness from a scientific foundation, the entire civilized society can then sit down and seriously vote: are we ready to cross the second Rubicon? If we really want to create conscious entities, how should we go about it?

The National AI Sovereignty Dilemma: Stop Burning Money Blindly on Cutting-Edge Large Models

Question: In the final part of the interview, I want to discuss the geopolitical dimension of "AI sovereignty." Almost every country, and even every multinational giant, is fervently shouting for self-control. Does the world really need every country to train its own cutting-edge large model?

Demis Hassabis: This question must be viewed through the lens of reality.

The future landscape of cutting-edge technology is very clear: globally, there will ultimately be only about 6 to 8 players, including Chinese companies, that can stand firmly at or near the forefront. These leading institutions will engage in high-frequency dynamic positioning battles, with one possibly leading for a short period of 3 to 6 months before being overtaken, resulting in an overall stalemate.

This means that for both a traditional enterprise and a sovereign nation, the market is filled with extremely rich opportunities for technological arbitrage. Each cutting-edge model has its strengths; some excel in multimodal understanding, while others are good at coding, and almost no single large model can dominate all dimensions in the long term.

Not to mention, the open-source ecosystem is rapidly catching up at a pace of merely 6 to 12 months behind the cutting-edge models. Although how to prevent open-source from being exploited by malicious attackers is a huge unresolved issue in the industry, there are extremely abundant options for many non-first-tier countries and multinational companies.

Therefore, I do not recommend that every country blindly follow suit and develop "self-researched cutting-edge large models." In my view, this simply does not make sense.

The true national-level AI sovereignty lies in a layered layout: First, utilize mature open-source or closed-source cutting-edge models as the foundational infrastructure, similar to utilities like water, heating, and electricity, focusing core energy and limited computing power on vertically integrated downstream industries that can directly transform the core lifeblood of the country—such as healthcare.

Second, the bottom line that must be truly safeguarded is data sovereignty. Ensure that sensitive data of enterprises and the nation remains private, and make further fine-tuning on one's own sensitive data. This is the rational, economical, and truly value-preserving autonomous strategy.

Question: The reason I conclude with this question is that the theme of this summit is agency. It concerns not only how individuals exercise agency but also how we collectively exercise agency: in the AI era, it is not about any one country going it alone, but rather countries working together for the welfare of all humanity. Thank you, Demis, and thank you all.

Hassabis: Thank you.