Game-Theory

Prior reading: Decision Theory for AI Safety | Competitive Dynamics and Safety A version of this argument was originally posted on LessWrong. I've reworked it here with some additional context and a less formal tone. I'm a computer scientist, not an astrophysicist — corrections welcome. The Problem The Fermi paradox asks a simple question: given the size and age of the universe, where is everyone? Standard answers include: life is rare (the Great Filter), civilizations destroy themselves before going interstellar, the distances are too vast, or everyone is hiding from everyone else (the Dark Forest, from Liu Cixin's Three-Body Problem). ...

Prior reading: Game Theory for AI Safety | Decision Theory for AI Safety | Competitive Dynamics The Question If AGI is developed, what stable configurations could the world settle into? Not all equilibria are equally survivable. Possible Equilibria Many Competing Systems Multiple AGI systems operated by different actors (nations, companies). No single dominant system. Closest to today's trajectory. Stability: Moderate. Competition continues. Arms race dynamics persist. Risk of conflict, but also checks and balances. ...

This post builds up the game theory you need to reason precisely about AI safety dynamics — races, coordination failures, and regulatory design. If you already know Nash equilibria, you can skip to Games That Show Up in AI Safety. If you're here for solutions, skip to Mechanism Design. But I'd recommend reading the whole thing, because the point isn't any single definition — it's seeing why the structure of the interaction, not the intentions of the players, drives the outcome. ...