Superintelligence Strategy: Expert Version
DOI:
https://doi.org/10.70777/si.v2i1.13961Keywords:
agi, artificial general intelligence strategy, superintelligence strategy, agi governance, artificial general intelligence governance, mutual assured destruction MAD, mutual assured ai malfunction MAIMAbstract
Rapid advances in AI are beginning to reshape national security. Destabilizing AI developments could rupture the balance of power and raise the odds of great-power conflict, while widespread proliferation of capable AI hackers and virologists would lower barriers for rogue actors to cause catastrophe. Superintelligence—AI vastly better than humans at nearly all cognitive tasks—is now anticipated by AI researchers. Just as nations once developed nuclear strategies to secure their survival, we now need a coherent superintelligence strategy to navigate a new period of transformative change. We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals. Given the relative ease of sabotaging a destabilizing AI project—through interventions ranging from covert cyberattacks to potential kinetic strikes on datacenters—MAIM already describes the strategic picture AI superpowers find t hemselves in. Alongside this, states can increase their competitiveness by bolstering their economies and militaries through AI, and they can engage in nonproliferation to rogue actors to keep weaponizable AI capabilities out of their hands. Taken together, the three-part framework of deterrence, nonproliferation, and competitiveness outlines a robust strategy to superintelligence in the years ahead.
References
Center for AI Safety. Statement on AI Risk (“Mitigating the risk of extinction from AI should be a global priority
alongside other societal-scale risks such as pandemics and nuclear war.”) 2023. URL: https://aistatement.
com.
SuperIntelligence – Robotics – Safety & Alignment 2025 2(1) Large Language Models I
Epoch AI. Key Trends and Figures in Machine Learning. Accessed: 2025-02-05. 2023. URL: https://epoch.
ai/trends.
Herman Kahn. On Thermonuclear War. Princeton, NJ: Princeton University Press, 1960, p. 668. ISBN: 978-0-691-
-1.
Horst W. J. Rittel and Melvin M. Webber. “Dilemmas in a General Theory of Planning”. In: Policy Sciences 4.2 DOI: https://doi.org/10.1007/BF01405730
(1973), pp. 155–169.
Graham Allison. Destined for War: Can America and China Escape Thucydides’s Trap? Houghton Mifflin Harcourt,
Jim Mitre and Joel B. Predd. Artificial General Intelligence’s Five Hard National Security Problems. Santa Monica,
CA: RAND Corporation, 2025. DOI: 10.7249/PEA3691-4. DOI: https://doi.org/10.7249/PEA3691-4
Edward Geist. “Fog-of-War Machines”. In: Deterrence under Uncertainty:Artificial Intelligence and Nuclear
Warfare. Oxford University Press, Aug. 2023. ISBN: 9780192886323. DOI: 10.1093/oso/9780192886323.
0006. eprint: https://academic.oup.com/book/0/chapter/416162987/chapterpdf/
/oso-9780192886323-chapter-6.pdf. URL: https://doi.org/10.1093/
oso/9780192886323.003.0006.
Jr. Perkins Ray. “Bertrand Russell and Preventive War”. In: Journal of the Bertrand Russell Archives 14 (1995), DOI: https://doi.org/10.15173/russell.v14i2.1867
pp. 135–153.
William Burr and Jeffrey T. Richelson. “Whether to Strangle the Baby in the Cradle: The United States and the
Chinese Nuclear Program, 1960–64”. In: International Security 25.3 (2000), pp. 54–99. DOI: https://doi.org/10.1162/016228800560525
James D. Fearon. “Rationalist explanations for war”. In: International Organization 49.3 (1995), pp. 379–414. DOI: https://doi.org/10.1017/S0020818300033324
Jasper Gotting et al. “Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark”. In: (2025).
Katarzyna P. Adamala et al. “Confronting risks of mirror life”. In: Science 386.6728 (2024), pp. 1351–1353. DOI: https://doi.org/10.1126/science.ads9158
U.S. Department of Homeland Security. Groundbreaking Framework for Safe and Secure Deployment of AI in
Critical Infrastructure. Accessed: 2024-12-06. Nov. 2024. URL: https://www.dhs.gov/news/2024/
/ 14 / groundbreaking - framework - safe - and - secure - deployment - ai - critical -
infrastructure.
Dan Hendrycks et al. Unsolved Problems in ML Safety. 2022. arXiv: 2109.13916 [cs.LG]. URL: https:
//arxiv.org/abs/2109.13916.
Steve Newman. Cybersecurity and AI: The Evolving Security Landscape. Accessed: 2024-12-06. 2024. URL:
https : / / www . safe . ai / blog / cybersecurity - and - ai - the - evolving - security -
landscape.
Dan Hendrycks. “Natural Selection Favors AIs over Humans”. In: ArXiv abs/2303.16200 (2023).
Raffi Khatchadourian. “The Doomsday Invention”. In: The New Yorker (Nov. 23, 2015).
Richard Danzig. Technology Roulette: Managing Loss of Control as Many Militaries Pursue Technological
Superiority. Tech. rep. Center for a New American Security, June 2018.
Akash R. Wasil et al. Verification methods for international AI agreements. 2024. arXiv: 2408.16074. DOI: https://doi.org/10.2139/ssrn.4938419
2024 Report to Congress: Executive Summary and Recommendations. U.S.-China Economic and Security Review
Commission, Nov. 2024. URL: https://www.uscc.gov/sites/default/files/2024-11/2024_
Executive_Summary.pdf.
Andrew J. Coe and Jane Vaynman. “Why Arms Control Is So Rare”. In: American Political Science Review 114.2
(2020), pp. 342–355. DOI: 10.1017/S000305541900073X. DOI: https://doi.org/10.1017/S000305541900073X
Aaron Scher and Lisa Thiergart. Mechanisms to Verify International Agreements About AI Development. Tech. rep.
Machine Intelligence Research Institute, 2024.
Richard Ren et al. Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? 2024. arXiv:
21792 [cs.LG]. URL: https://arxiv.org/abs/2407.21792.
Yangjun Ruan, Chris J. Maddison, and Tatsunori Hashimoto. Observational Scaling Laws and the Predictability
of Language Model Performance. 2024. arXiv: 2405.10938 [cs.LG]. URL: https://arxiv.org/abs/
10938.
SuperIntelligence – Robotics – Safety & Alignment 2025 2(1) Large Language Models I
Vincent Fournier. “Surveying Safeguarded Material 24/7”. In: International Atomic Energy Agency (2016). URL:
https://www.iaea.org/newscenter/news/surveying-safeguarded-material-24/7.
Gabriel Kulp et al. Hardware-Enabled Governance Mechanisms: Developing Technical Solutions to Exempt
Items Otherwise Classified Under Export Control Classification Numbers 3A090 and 4A090. Tech. rep. RAND
Corporation, 2024.
Marc Andreessen. Whitepill 37. Twitter. Accessed: 2024-12-06. Jan. 2024. URL: https://x.com/pmarca/
status/1747534187597586615.
World Artificial Intelligence Conference. AI Succession. https : / / www . youtube . com / watch ? v =
NgHFMolXs3U. 2023.
Sella Nevo et al. Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models. Santa Monica, CA:
RAND Corporation, 2024. DOI: 10.7249/RRA2849-1. DOI: https://doi.org/10.7249/RRA2849-1
Mrinank Sharma et al. Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of
Hours of Red Teaming. 2025. arXiv: 2501.18837 [cs.CL]. URL: https://arxiv.org/abs/2501.
Andy Zou et al. Improving Alignment and Robustness with Circuit Breakers. 2024. arXiv: 2406.04313 [cs.LG].
URL: https://arxiv.org/abs/2406.04313.
Sarah Emerson. “This Hacker Team Is Bulletproofing AI Models For Companies Like OpenAI”. In: Forbes (Oct.
. Accessed: 2024-12-06. URL: https://www.forbes.com/sites/sarahemerson/2024/10/
/this-hacker-team-is-bulletproofing-ai-models-for-companies-like-openai/.
Kevin Roose. “A Conversation With Bing’s Chatbot Left Me Deeply Unsettled”. In: The New York Times
(Feb. 16, 2023). URL: https://www.nytimes.com/2023/02/16/technology/bing-chatbotmicrosoft-
chatgpt.html.
Mantas Mazeika et al. “Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs”. In:
(2025).
Long Ouyang et al. “Training language models to follow instructions with human feedback”. In: ArXiv (2022).
Yushi Bai et al. “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback”.
In: ArXiv (2022).
Yuntao Bai et al. Constitutional AI: Harmlessness from AI Feedback. 2022. arXiv: 2212.08073 [cs.CL]. URL:
https://arxiv.org/abs/2212.08073.
Stephen Casper et al. Open Problems and Fundamental Limitations of Reinforcement Learning from Human
Feedback. 2023. arXiv: 2307.15217 [cs.AI]. URL: https://arxiv.org/abs/2307.15217.
Andy Zou et al. Representation Engineering: A Top-Down Approach to AI Transparency. 2023. arXiv: 2310.
Michael C Horowitz. The Diffusion of Military Power: Causes and Consequences for International Politics.
Princeton University Press, 2010.
Stacie Pettyjohn et al. Swarms over the Strait: Drone Warfare in a Future Fight to Defend Taiwan. Center for a
New American Security (CNAS), June 2024.
Daniel M. Gerstein and Erin N. Leidy. Emerging Technology and Risk Analysis: Unmanned Aerial Systems
Intelligent Swarm Technology. Santa Monica, CA: RAND Corporation, 2024. DOI: 10.7249/RRA2380-1. DOI: https://doi.org/10.7249/RRA2380-1
Dan Hendrycks, Mantas Mazeika, and Thomas Woodside. An Overview of Catastrophic AI Risks. 2023. arXiv: DOI: https://doi.org/10.1201/9781003530336-1
12001 [cs.CY].
Liam Vaughan. Flash Crash: A Trading Savant, a Global Manhunt, and the Most Mysterious Market Crash in
History. New York: Doubleday, 2020. ISBN: 978-0385543651.
Reuters. “China sets up third fund with $47.5 bln to boost semiconductor sector”. In: Reuters (2024).
Catherine Aiken, James Dunham, and Remco Zwetsloot. Immigration Pathways and Plans of AI Talent. Center for
Security and Emerging Technology, 2020.
Andy Zou et al. Forecasting Future World Events with Neural Networks. 2022. arXiv: 2206.15474 [cs.LG].
SuperIntelligence – Robotics – Safety & Alignment 2025 2(1) Large Language Models I
Benjamin F. Jones and Xiaojie Liu. A Framework for Economic Growth with Capital-Embodied Technical Change.
Working Paper 30459. National Bureau of Economic Research, 2022.
Philip Zelikow et al. Defense Against the AI Dark Arts: Threat Assessment and Coalition Defense. Tech. rep.
Hoover Institution, Dec. 2024.
Erik Jones, Anca Dragan, and Jacob Steinhardt. Adversaries Can Misuse Combinations of Safe Models. 2024.
arXiv: 2406.14595 [cs.CR]. URL: https://arxiv.org/abs/2406.14595.
Leopold Aschenbrenner. “Situational Awareness: The Decade Ahead”. In: (June 2024). URL: https : / /
situational-awareness.ai/.
Eliezer Yudkowsky and Richard Ngo. “Ngo and Yudkowsky on Scientific Reasoning and Pivotal Acts”. In:
(Mar. 2022). URL: https://intelligence.org/2022/03/01/ngo- and- yudkowsky- onscientific-
reasoning-and-pivotal-acts/.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Dan Hendrycks, Eric Schmidt, Alexandr Wang

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.