3. Why Standard Remedies Are Insufficient
ID: 3-why-standard-remedies-are-insufficient
The standard response to technological risk involves some combination of: industry self-regulation, government oversight, liability law, academic research, and international standards. All of these mechanisms exist for AI. None of them is adequate.
Industry self-regulation fails because the incentive structure points in the wrong direction. The companies racing to build more capable AI face competitive pressure to move faster, not slower. Self-imposed safety constraints are competitive disadvantages. This is not a criticism of individuals within these companies — many are genuinely committed to safety — but of the structural incentives they operate within.
Government oversight has historically lagged transformative technologies by years or decades. The institutions and expertise required to regulate AI effectively do not yet exist in most governments. And unlike previous technologies, AI development is global — no single government can regulate it unilaterally.
Liability law provides inadequate deterrence when harms are diffuse, delayed, or catastrophic. If an AI system contributes to a civilizational collapse, there is no one left to sue.
Academic research on AI safety is valuable but underfunded relative to AI capabilities research, and its findings are often not translated into deployed systems.
International standards bodies are slow, consensus-driven, and lack enforcement mechanisms. The First Committee of the United Nations General Assembly adopted a resolution on autonomous weapons systems for the third year running in November 2025, concerned by the "consequences and impact of autonomous weapon systems on international peace and security," but it stopped short of mandating further negotiations. Stop Killer Robots
The historical parallel most relevant here is not nuclear weapons — where the technology was immediately recognized as catastrophic and subject to serious international control efforts — but biological weapons before the Biological Weapons Convention of 1972: a technology whose risks were understood by specialists, ignored by policymakers, and allowed to proliferate until near-disaster forced action.
We may not have another near-miss opportunity. AIAR is potentially irreversible.
________________________________________
Industry self-regulation fails because the incentive structure points in the wrong direction. The companies racing to build more capable AI face competitive pressure to move faster, not slower. Self-imposed safety constraints are competitive disadvantages. This is not a criticism of individuals within these companies — many are genuinely committed to safety — but of the structural incentives they operate within.
Government oversight has historically lagged transformative technologies by years or decades. The institutions and expertise required to regulate AI effectively do not yet exist in most governments. And unlike previous technologies, AI development is global — no single government can regulate it unilaterally.
Liability law provides inadequate deterrence when harms are diffuse, delayed, or catastrophic. If an AI system contributes to a civilizational collapse, there is no one left to sue.
Academic research on AI safety is valuable but underfunded relative to AI capabilities research, and its findings are often not translated into deployed systems.
International standards bodies are slow, consensus-driven, and lack enforcement mechanisms. The First Committee of the United Nations General Assembly adopted a resolution on autonomous weapons systems for the third year running in November 2025, concerned by the "consequences and impact of autonomous weapon systems on international peace and security," but it stopped short of mandating further negotiations. Stop Killer Robots
The historical parallel most relevant here is not nuclear weapons — where the technology was immediately recognized as catastrophic and subject to serious international control efforts — but biological weapons before the Biological Weapons Convention of 1972: a technology whose risks were understood by specialists, ignored by policymakers, and allowed to proliferate until near-disaster forced action.
We may not have another near-miss opportunity. AIAR is potentially irreversible.
________________________________________
New to topics? Read the docs here!