Vol. 2 No. 3 (2025): Governance, Agents, Evolutionary Search

The Self-Improving Darwin Godel Machine

Senior Editor-at-Large Gil Syswerda has constructed a provocative Timeline to Artificial General Intelligence 2025 – 2030+. He gives key AI advances, economic, social, and geopolitical effects of AI, predicting that, as early as 2028-2029, AI could replace the majority of human economic activity.

Agentic AI is advancing at a rapid pace. Two articles look at self-improving Agentic AI that use evolutionary programming techniques. The Darwin Godel Machine, Appendix F, describes a simple proof-of-concept that self-improving AI can focus on its own safety and value alignment. In a third article, Kumarage et al. describe a multi-agent architecture in which agents collaborate in chain-of-thought reasoning about LLM responses vs. policy before output is permitted, representing another paradigm of recursive safety improvement.

Governance is also changing rapidly. It is critical to get a handle on the global AI legal regime with its varied approaches as they evolve, and flag gaps in regimes.

Safety and alignment methods must stay ahead of AI advances, and the latest advances, such as reasoning, must be applied to safety and alignment - recursive safety improvement.

Published: 2025-07-20

Articles

  • Highlights of the Issue: Governance, Agents, Evolutionary Search In progress

    Kris Carlson
    DOI: https://doi.org/10.70777/si.v2i3.15417
  • Comparing Apples to Oranges: A Taxonomy for Navigating the Global Landscape of AI Regulation

    Sacha Alanoca, Shira Gur-Arieh, Tom Zick, Kevin Klyman
    DOI: https://doi.org/10.70777/si.v2i3.15137
  • Real-World Gaps in AI Governance Research

    Ilan Strauss, Isobel Moure, Tim O’Reilly, Sruly Rosenblat
    DOI: https://doi.org/10.70777/si.v2i3.15163
  • AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges

    Ranjan Sapkota, Konstantinos I. Roumeliotis, Manoj Karkee
    DOI: https://doi.org/10.70777/si.v2i3.15161
  • Measuring AI Agent Autonomy: Towards a Scalable Approach with Code Inspection

    Peter Cihon, Merlin Stein, Gagan Bansal, Sam Manning, Kevin Xu
    DOI: https://doi.org/10.70777/si.v2i3.15295
  • Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents

    Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, Jeff Clune
    DOI: https://doi.org/10.70777/si.v2i3.15063
  • DarwinLM: Evolutionary Structured Pruning of Large Language Models

    Shengkun Tang, Oliver Sieberling, Eldar Kurtic, Dan Alistarh
    DOI: https://doi.org/10.70777/si.v2i3.15171
  • Towards Safety Reasoning in LLMs: AI-agentic Deliberation for Policy-embedded CoT Data Creation

    Tharindu Kumarage, Ninareh Mehrabi, Anil Ramakrishna, Xinyan Zhao, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta, Charith Peris
    DOI: https://doi.org/10.70777/si.v2i3.15249
  • Trends in Frontier AI Model Count: A Forecast to 2028

    Iyngkarran Kumar, Sam Manning
    DOI: https://doi.org/10.70777/si.v2i3.15155
  • Hardware-Enabled Mechanisms for Verifying Responsible AI Development

    Aidan O’Gara, Gabriel, Will Hodgkins, James Petrie, Vincent Immler, Aydin Aysu, Kanad Basu, Shivam Bhasin, Stjepan Picek, Ankur Srivastava
    DOI: https://doi.org/10.70777/si.v2i3.15157
  • Deliberative Alignment: Reasoning Enables Safer Language Models

    Melody Y. Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel, Andrea Vallone, Hongyu Ren, Jason Wei, Hyung Won Chung, Sam Toyer, Johannes Heidecke, Alex, Amelia Glaese
    DOI: https://doi.org/10.70777/si.v2i3.15159

Commentary

  • Timeline to Artificial General Intelligence 2025 – 2030+

    Gil Syswerda
    DOI: https://doi.org/10.70777/si.v2i3.15119

Reviews

  • Review: Metacognition in LLMs and its Relation to Safety

    Kris Carlson
    DOI: https://doi.org/10.70777/si.v2i3.15271