Superintelligence Strategy: Expert Version

Authors

  • Dan Hendrycks University of California Berkeley
  • Eric Schmidt Schmidt Sciences
  • Alexandr Wang Founder, CEO at Scale AI

DOI:

https://doi.org/10.70777/si.v2i1.13961

Keywords:

agi, artificial general intelligence strategy, superintelligence strategy, agi governance, artificial general intelligence governance, mutual assured destruction MAD, mutual assured ai malfunction MAIM

Abstract

Rapid advances in AI are beginning to reshape national security. Destabilizing AI developments could rupture the balance of power and raise the odds of great-power conflict, while widespread proliferation of capable AI hackers and virologists would lower barriers for rogue actors to cause catastrophe. Superintelligence—AI vastly better than humans at nearly all cognitive tasks—is now anticipated by AI researchers. Just as nations once developed nuclear strategies to secure their survival, we now need a coherent superintelligence strategy to navigate a new period of transformative change. We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals. Given the relative ease of sabotaging a destabilizing AI project—through interventions ranging from covert cyberattacks to potential kinetic strikes on datacenters—MAIM already describes the strategic picture AI superpowers find t hemselves in. Alongside this, states can increase their competitiveness by bolstering their economies and militaries through AI, and they can engage in nonproliferation to rogue actors to keep weaponizable AI capabilities out of their hands. Taken together, the three-part framework of deterrence, nonproliferation, and competitiveness outlines a robust strategy to superintelligence in the years ahead.

Author Biographies

Dan Hendrycks, University of California Berkeley

I received my PhD from UC Berkeley where I was advised by Dawn Song and Jacob Steinhardt. I am now the director of the Center for AI Safety. I am interested in AI Safety. I received my BS from UChicago. My research is supported by the NSF GRFP and the Open Philanthropy AI Fellowship. I helped contribute the GELU activation function (the most-used activation in state-of-the-art models including BERT, GPT, Vision Transformers, etc.), the out-of-distribution detection baseline, and distribution shift benchmarks.

Eric Schmidt, Schmidt Sciences

Eric Schmidt is an accomplished technologist, entrepreneur and philanthropist. He is the co-founder of Schmidt Sciences with his wife, Wendy, and former CEO & chairman of Google.

In his roles across industry, philanthropy, academia and government, Eric works to promote social good, support innovation, and coach the next generation of exceptional people to tackle the world’s biggest challenges.

Alexandr Wang, Founder, CEO at Scale AI

Wang was born in Los Alamos, New Mexico. He is the son of Chinese immigrants who worked as physicists at the Los Alamos National Laboratory in New Mexico, where nuclear weapons were first developed. Wang was passionate about math and computer programming since childhood. He qualified for the Math Olympiad Program in 2013, the US Physics Team in 2014, and was a USACO finalist in 2012 and 2013. He was educated at Los Alamos High School after which he moved to Silicon Valley to become a software engineer at wealth management company, Addepar. During his teens, Wang worked for Quora as a software programmer. He briefly attended the Massachusetts Institute of Technology and had a stint as an algorithm developer at the high-frequency trading firm Hudson River Trading before he dropped out to co-found Scale AI in 2016.

References

Center for AI Safety. Statement on AI Risk (“Mitigating the risk of extinction from AI should be a global priority

alongside other societal-scale risks such as pandemics and nuclear war.”) 2023. URL: https://aistatement.

com.

SuperIntelligence – Robotics – Safety & Alignment 2025 2(1) Large Language Models I

Epoch AI. Key Trends and Figures in Machine Learning. Accessed: 2025-02-05. 2023. URL: https://epoch.

ai/trends.

Herman Kahn. On Thermonuclear War. Princeton, NJ: Princeton University Press, 1960, p. 668. ISBN: 978-0-691-

-1.

Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”. In: Policy Sciences 4.2 DOI: https://doi.org/10.1007/BF01405730

(1973), pp. 155–169.

Graham Allison. Destined for War: Can America and China Escape Thucydides’s Trap? Houghton Mifflin Harcourt,

Jim Mitre and Joel B. Predd. Artificial General Intelligence’s Five Hard National Security Problems. Santa Monica,

CA: RAND Corporation, 2025. DOI: 10.7249/PEA3691-4. DOI: https://doi.org/10.7249/PEA3691-4

Edward Geist. “Fog-of-War Machines”. In: Deterrence under Uncertainty:Artificial Intelligence and Nuclear

Warfare. Oxford University Press, Aug. 2023. ISBN: 9780192886323. DOI: 10.1093/oso/9780192886323.

0006. eprint: https://academic.oup.com/book/0/chapter/416162987/chapterpdf/

/oso-9780192886323-chapter-6.pdf. URL: https://doi.org/10.1093/

oso/9780192886323.003.0006.

Jr. Perkins Ray. “Bertrand Russell and Preventive War”. In: Journal of the Bertrand Russell Archives 14 (1995), DOI: https://doi.org/10.15173/russell.v14i2.1867

pp. 135–153.

William Burr and Jeffrey T. Richelson. “Whether to Strangle the Baby in the Cradle: The United States and the

Chinese Nuclear Program, 1960–64”. In: International Security 25.3 (2000), pp. 54–99. DOI: https://doi.org/10.1162/016228800560525

James D. Fearon. “Rationalist explanations for war”. In: International Organization 49.3 (1995), pp. 379–414. DOI: https://doi.org/10.1017/S0020818300033324

Jasper Gotting et al. “Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark”. In: (2025).

Katarzyna P. Adamala et al. “Confronting risks of mirror life”. In: Science 386.6728 (2024), pp. 1351–1353. DOI: https://doi.org/10.1126/science.ads9158

U.S. Department of Homeland Security. Groundbreaking Framework for Safe and Secure Deployment of AI in

Critical Infrastructure. Accessed: 2024-12-06. Nov. 2024. URL: https://www.dhs.gov/news/2024/

/ 14 / groundbreaking - framework - safe - and - secure - deployment - ai - critical -

infrastructure.

Dan Hendrycks et al. Unsolved Problems in ML Safety. 2022. arXiv: 2109.13916 [cs.LG]. URL: https:

//arxiv.org/abs/2109.13916.

Steve Newman. Cybersecurity and AI: The Evolving Security Landscape. Accessed: 2024-12-06. 2024. URL:

https : / / www . safe . ai / blog / cybersecurity - and - ai - the - evolving - security -

landscape.

Dan Hendrycks. “Natural Selection Favors AIs over Humans”. In: ArXiv abs/2303.16200 (2023).

Raffi Khatchadourian. “The Doomsday Invention”. In: The New Yorker (Nov. 23, 2015).

Richard Danzig. Technology Roulette: Managing Loss of Control as Many Militaries Pursue Technological

Superiority. Tech. rep. Center for a New American Security, June 2018.

Akash R. Wasil et al. Verification methods for international AI agreements. 2024. arXiv: 2408.16074. DOI: https://doi.org/10.2139/ssrn.4938419

2024 Report to Congress: Executive Summary and Recommendations. U.S.-China Economic and Security Review

Commission, Nov. 2024. URL: https://www.uscc.gov/sites/default/files/2024-11/2024_

Executive_Summary.pdf.

Andrew J. Coe and Jane Vaynman. “Why Arms Control Is So Rare”. In: American Political Science Review 114.2

(2020), pp. 342–355. DOI: 10.1017/S000305541900073X. DOI: https://doi.org/10.1017/S000305541900073X

Aaron Scher and Lisa Thiergart. Mechanisms to Verify International Agreements About AI Development. Tech. rep.

Machine Intelligence Research Institute, 2024.

Richard Ren et al. Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? 2024. arXiv:

21792 [cs.LG]. URL: https://arxiv.org/abs/2407.21792.

Yangjun Ruan, Chris J. Maddison, and Tatsunori Hashimoto. Observational Scaling Laws and the Predictability

of Language Model Performance. 2024. arXiv: 2405.10938 [cs.LG]. URL: https://arxiv.org/abs/

10938.

SuperIntelligence – Robotics – Safety & Alignment 2025 2(1) Large Language Models I

Vincent Fournier. “Surveying Safeguarded Material 24/7”. In: International Atomic Energy Agency (2016). URL:

https://www.iaea.org/newscenter/news/surveying-safeguarded-material-24/7.

Gabriel Kulp et al. Hardware-Enabled Governance Mechanisms: Developing Technical Solutions to Exempt

Items Otherwise Classified Under Export Control Classification Numbers 3A090 and 4A090. Tech. rep. RAND

Corporation, 2024.

Marc Andreessen. Whitepill 37. Twitter. Accessed: 2024-12-06. Jan. 2024. URL: https://x.com/pmarca/

status/1747534187597586615.

World Artificial Intelligence Conference. AI Succession. https : / / www . youtube . com / watch ? v =

NgHFMolXs3U. 2023.

Sella Nevo et al. Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models. Santa Monica, CA:

RAND Corporation, 2024. DOI: 10.7249/RRA2849-1. DOI: https://doi.org/10.7249/RRA2849-1

Mrinank Sharma et al. Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of

Hours of Red Teaming. 2025. arXiv: 2501.18837 [cs.CL]. URL: https://arxiv.org/abs/2501.

Andy Zou et al. Improving Alignment and Robustness with Circuit Breakers. 2024. arXiv: 2406.04313 [cs.LG].

URL: https://arxiv.org/abs/2406.04313.

Sarah Emerson. “This Hacker Team Is Bulletproofing AI Models For Companies Like OpenAI”. In: Forbes (Oct.

. Accessed: 2024-12-06. URL: https://www.forbes.com/sites/sarahemerson/2024/10/

/this-hacker-team-is-bulletproofing-ai-models-for-companies-like-openai/.

Kevin Roose. “A Conversation With Bing’s Chatbot Left Me Deeply Unsettled”. In: The New York Times

(Feb. 16, 2023). URL: https://www.nytimes.com/2023/02/16/technology/bing-chatbotmicrosoft-

chatgpt.html.

Mantas Mazeika et al. “Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs”. In:

(2025).

Long Ouyang et al. “Training language models to follow instructions with human feedback”. In: ArXiv (2022).

Yushi Bai et al. “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback”.

In: ArXiv (2022).

Yuntao Bai et al. Constitutional AI: Harmlessness from AI Feedback. 2022. arXiv: 2212.08073 [cs.CL]. URL:

https://arxiv.org/abs/2212.08073.

Stephen Casper et al. Open Problems and Fundamental Limitations of Reinforcement Learning from Human

Feedback. 2023. arXiv: 2307.15217 [cs.AI]. URL: https://arxiv.org/abs/2307.15217.

Andy Zou et al. Representation Engineering: A Top-Down Approach to AI Transparency. 2023. arXiv: 2310.

Michael C Horowitz. The Diffusion of Military Power: Causes and Consequences for International Politics.

Princeton University Press, 2010.

Stacie Pettyjohn et al. Swarms over the Strait: Drone Warfare in a Future Fight to Defend Taiwan. Center for a

New American Security (CNAS), June 2024.

Daniel M. Gerstein and Erin N. Leidy. Emerging Technology and Risk Analysis: Unmanned Aerial Systems

Intelligent Swarm Technology. Santa Monica, CA: RAND Corporation, 2024. DOI: 10.7249/RRA2380-1. DOI: https://doi.org/10.7249/RRA2380-1

Dan Hendrycks, Mantas Mazeika, and Thomas Woodside. An Overview of Catastrophic AI Risks. 2023. arXiv: DOI: https://doi.org/10.1201/9781003530336-1

12001 [cs.CY].

Liam Vaughan. Flash Crash: A Trading Savant, a Global Manhunt, and the Most Mysterious Market Crash in

History. New York: Doubleday, 2020. ISBN: 978-0385543651.

Reuters. “China sets up third fund with $47.5 bln to boost semiconductor sector”. In: Reuters (2024).

Catherine Aiken, James Dunham, and Remco Zwetsloot. Immigration Pathways and Plans of AI Talent. Center for

Security and Emerging Technology, 2020.

Andy Zou et al. Forecasting Future World Events with Neural Networks. 2022. arXiv: 2206.15474 [cs.LG].

SuperIntelligence – Robotics – Safety & Alignment 2025 2(1) Large Language Models I

Benjamin F. Jones and Xiaojie Liu. A Framework for Economic Growth with Capital-Embodied Technical Change.

Working Paper 30459. National Bureau of Economic Research, 2022.

Philip Zelikow et al. Defense Against the AI Dark Arts: Threat Assessment and Coalition Defense. Tech. rep.

Hoover Institution, Dec. 2024.

Erik Jones, Anca Dragan, and Jacob Steinhardt. Adversaries Can Misuse Combinations of Safe Models. 2024.

arXiv: 2406.14595 [cs.CR]. URL: https://arxiv.org/abs/2406.14595.

Leopold Aschenbrenner. “Situational Awareness: The Decade Ahead”. In: (June 2024). URL: https : / /

situational-awareness.ai/.

Eliezer Yudkowsky and Richard Ngo. “Ngo and Yudkowsky on Scientific Reasoning and Pivotal Acts”. In:

(Mar. 2022). URL: https://intelligence.org/2022/03/01/ngo- and- yudkowsky- onscientific-

reasoning-and-pivotal-acts/.

Figure 2.2: The amount of compute used to create AI models has been increasing exponentially for decades

Downloads

Published

2025-03-15

How to Cite

Hendrycks, D., Schmidt, E., & Wang, A. (2025). Superintelligence Strategy: Expert Version. SuperIntelligence - Robotics - Safety & Alignment, 2(1). https://doi.org/10.70777/si.v2i1.13961